QCompiler

QCompiler: A Meta-Level Compilation Framework for Enabling Ubiquitous, Distributed Component-Based QoS-Aware Programming

This work was part of the 2K and 2KQ system, and it is part of Gaia and QoS in Gaia project.

Introduction

The rapid growth of distributed component-based environments and coexistence of different application domains, such as multimedia and electronic commerce, present significant challenges to the provision of different applications’ Quality of Service (QoS). First, different domain applications have specific semantics for QoS requirements and provisions. For example, multimedia and library information retrieval applications are concerned about the service qualities of audio, video streaming (e.g., frame rate, rame size, sampling rate, end-to-end delay) and messaging (e.g., priority, response time, reliability), respectively. Developing an application to be QoS-aware during the application development cycle, and ensuring its QoS provisions during the application’s run-time are non-trivial tasks for a specific application and even harder for different applications. Second, these QoS-aware applications are becoming ubiquitous. They are expected to be executable in heterogeneous computing and communication environments with different capacities of processing power (e.g., high performance PC, PDAs), battery power, and network bandwidth (e.g., wired, wireless networks).

To ease the development of a QoS-aware application, several QoS-oriented middleware services, assisting QoS-aware applications with different aspects of QoS provisions, in specific deployment environment, have been proposed. To use them to enable QoS-awareness for an application effectively, however, the application developer needs to well understand their provisions, semantics, pre-defined interfaces, and expected parameters, and knows how to translate application-specific QoS requirements into these information properly. Moreover, a QoS-aware application deploying specific middleware services may not be deployable in different deployment environments due to availability of required services, devices, and resources. Hence, the big challenging problem is:

“What should be the systematic methodology and system support which can help an application developer to flexibly develop a component-based application to be deployable flexibly, and efficiently with the satisfactory quality of service in the ubiquitous environments?”

To solve this problem, we propose a meta-level compilation framework for enabling ubiquitous, distributed component-based QoS-aware programming, called Q-Compiler. The Q-Compiler covers the life cycle of component-based QoS-aware application development and deployment. The original philosophy of Q-Compiler is based on the integrated approach of off-line QoS configuration compilation and the on-line middleware QoS management services in [1]. It extends 2KQ [1] to 2KQ+ [2] (See Figure 2).  The Q-Compiler, assisted by  a run-time middleware, generates the “ready-to-use” QoS-enabled meta information, QoSCASpec, based on the input QoS-aware application specifications representing a QoS-aware application.

The run-time middleware (e.g.,run-time 2KQ Middleware, Gaia QoS Services) are running in the distributed machines, and representing the distributed run-time system. They are the core of application execution. They instantiate, manage and control a QoS-aware application during the application execution and adaptation(reconfguraiton), based on the (1) Q-Compiler’s compiled result, the application’s QoSCASpec, (2) incoming user QoS request, and (3) dynamic run-time constraints such as resource availability, execution environment, and mobility. They also assist the Q-Compiler to perform the efficient compilation.

QoS-Aware Application Specifications

QoS-aware application specifications are the “source code” of the Q-Compiler. They allow for flexible descriptions of different component-based QoS-aware applications. They can be entered via a visual QoS programming environment (QoSTalk [4]), available for an application developer or via a simple XML editor. The descriptions include: (a) application’s general description, (b) application’s specification, and (c) user-to-application-specific translation template (UtoA template).

Application’s general description allows the application developer to specify general information such as application name, application category, and accessibility for the application.

Application’s specification includes application functional dependency graph, its individual components’ and connections’ descriptions, and its end-to-end description. The application functional dependency graph allows an application developer to describe a QoS-aware application via the composition of QoS-aware application service components (ASCs), flexibly. A directed edge between two application service components represents an information (data or control) flow from the component at the starting point to the component at the ending point.

The end-to-end description, along the dependency graph, represents the end-to-end QoS requirement between a pair of a source node (S) and an end-user node (U), and can be described by the pre-defined QoS label. The application functional dependency graph represents the abstract QoS-aware application model.

In the application functional dependency graph, each application service component is associated with service component description, and each connection is associated with connection description. A service component description includes application service component name, component type (e.g., Specific, Generic, Composite), component model (e.g., CORBA, COM, Java class), component role (e.g., real-time streaming producer, real-time streaming consumer, real-time messaging producer, real-time messaging consumer, reliable messaging producer, and reliable messaging consumer), interfaces, category (e.g., VoDServerService, TranscoderService, VoDClientService, ), component repository, target machine(s), hardware requirement, system software requirement, state (e.g., Shared, or Exclusive), supporting QoS, and depending libraries. “supporting QoS” is pointed to the supporting QoS profile which consists of input quality vectors, each consisting of QoS categories, QoS guarantee levels, and their quantitative QoS dimensions, that the service component supports. Each input quality vector is associated with a reward profile. “depending libraries” is pointed to depending libraries profile which consists of a list of application service component’s depending libraries with their locations or pointers to their locations.

An application service component type can be Specific, Generic, or Composite. A Specific application service component represents a specific implementation of a functional unit. A Generic application service component represents the service without specific implementation. A Composite application service component represents the integrated application service components which could be specific or generic. The application functional dependency graph is partially-defined if some of its application service components are generic; otherwise, it is fully-defined. A connection description includes connection type (e.g., unicating, multicasting), communication model (e.g., CORBA RPC, DCOM, Java RMI), and security capability (e.g., enable encryption-decryption).

<pUser-to-application-specific translation template defines the mapping between different user QoS levels and corresponding application specific QoS categories, and their dimensions.  User QoS levels are quality levels which an application developer provides for a user when initiating the application.

Application-specific QoS categories and dimensions for each user QoS level, are expected as the input parameters of the Q-Compiler. UtoA template also allows the application developer to specify QoS guarantee level for each QoS category.

Q-Compiler

The Q-Compiler enables flexible and efficient development and deployment of distributed component-based QoS-aware applications in ubiquitous environments. The main concepts (See Figure 4)  of the Q-Compiler are the integrated reuse of available application and middleware services, and the separation between generic services and their specific implementations. To enable QoS provisions, the Q-Compiler is based on the translation/mapping models between application-to-application, application-to-middleware, and middleware-to-middleware, and the substitution models between generic services to specific implementations.

The Q-Compiler environment (See Figure 5) consists of a set of QoS-aware application specifications, a core translation engine, a set of tools, and a reconfigurable component-based run-time middleware.

During the pre-process environment-independent translation, the Fixed resource profiler and the Reward profiler help a service component developer developing a QoS-aware application service component or enable QoS-awareness for a component-of-the-shelf (COTS). The former helps determining the basic resource requirements of a service component (application or middleware). Basic resource requirements are fixed amounts of resources to execute the service. The latter provides the mapping from application service component’s input quality and resource allocation to its output quality. The Interface binder builder helps building an interface binder, a “glue code” between a specific implementation of application or middleware service component and a specific middleware implementation, based on the availability of semantic-specific mapping schemes.

During the development and deployment of a QoS-aware application, a QoS-aware application developer can implement a QoS-aware application by customizing the pre-defined set of QoS-aware application specifications. The translation engine of the Q-Compiler compiles

QoS-aware application specifications into two phases: (1) environment-independent translation, and (2) environment-dependent translation. The first phase helps developing a generic, portable QoS-aware application. Based on a rule-base, it compiles input specifications into possible generic QoS-aware configurations, collected together in a QoS-aware application descriptor, where each configuration represents a possible configuration of input application and its association with a configuration of generic middleware services. The second phase helps customizing QoS-aware application descriptor within a specific deployment environment. Assisted by the run-time middleware, it compiles each generic QoS-aware configuration into specific QoS-aware configurations with cost estimations, where application and middleware service components in each configuration represent a specific implementation available in specific deployment environment. Collection of specific QoS-aware configurations is passed to the post-process environment-dependent translation. The Packager helps packaging each specific QoS-aware configuration into executable codes ready to be instantiated and run in the deployment environment. It helps binding application and middleware implementations, represented in a specific QoS-aware configuration, by rebuilding each application service component with proper interface binder(s). The Verifier helps validating cost estimations for each specific QoS-aware configuration. It instantiates a specific QoS-aware configuration in the deployment environment, probes the end-to-end resource requirements of the configuration, compares these results with the estimated costs, and tunes the costs if necessary.

Post-processed specific QoS-aware configurations are collected in QoS-aware Component-based Application Specification (QoSCASpec). QoSCASpec is QoS-enabled meta information which can be used flexibly by the run-time middleware during the application execution and adaptation (reconfiguration) in a specific deployment environment.

Implementations and Documentations

QoS-aware Application Specifications:

The Q-Compiler expects the input QoS-aware application specifications in XML-based formats with some pre-defined QoS-related tags (HQML) [4]. Currently, the application functional graph and the service component descriptions can be specified via the visual QoS programming environment [4], which, then, generates these QoS specifications into a XML description. For example, a XML description of the Video-on-Demand application generated from the visual programming environment is MONETVoD.xml. The user-to-application-specific translation template (UtoA template), currently, is defined manually via an editor following the pre-defined tags. An example of the UtoA template for the Video-on-Demand application is MONETVoD-UtoA.xml.

Q-Compiler:

The Q-Compiler is implemented in Java. The new APIs for the symbolic configuration translation is in APIs. Note that the previous version of implementation and document as part of the 2K system can be found at 2KQ Implementation and document.

Publications

NSF logoNASA logoThe QoS configuration compilation system is supported by the National Science Foundation under Grants No. 9870736, 9970139, and EIA 99-72884EQ,  0086094, and NSF-CCR 9988199, and NASA under Grant No. NASA NAG 2-1406.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.