Printer Friendly

Content adaptation in distributed multimedia systems.

Abstract: New developments in computing and communication technology facilitate mobile data access to multimedia application systems such as healthcare, tourism and emergency. In these applications, users can access information with variety of devices having heterogeneous capabilities. One of the requirements in these applications is to adapt the content to the user's preferences, device capabilities and network conditions. In this paper we present a distributed content adaptation approach for distributed multimedia systems. In this approach content adaptation is performed in several steps and the adaptation tools are implemented by external services, we call them adaptation services. In order to determine the type and sequence of the adaptation services, we develop an adaptation graph based on client profile, network conditions, content profile (meta-data) and available adaptation services. Different quality criteria are used to optimize the adaptation graph.

Categories and Subject Descriptors

H.5 [Information Interfaces and Presentation]: Network communications; H.2 [Multimedia Information Systems]; E.I[Data Structures]

General Terms

Graphs and networks

Keywords: Distributed multimedia systems, content adaptation

1. Introduction

Distributed multimedia systems (DMSs) provide access to different media objects such as text, image, video and audio. By extending DMSs with wireless communication components, many new applications become feasible (Gecsei 1997), for example delivering critical information to people in the move (eg patient information to a doctor or nurse, location information to a tourist etc.). Wireless network environment and mobile devices create many challenges that have not been adequately addressed in traditional distributed multimedia systems (Jayaputera et al. 2003). One of the issues is quality of services (QoS) that depends on network conditions (delay, throughput etc.), user's preferences and device capabilities. With many mobile devices being introduced to the market (such as PDAs, Palmtop, Smart-phones, etc) and many of which are unable to display high-resolution graphics, transferring high-resolution images to those mobile devices is not feasible. Moreover, mobile data access results change of user interest with respect to used device and context (time, place etc.).

These conditions lead to create systems that provide content adaptation facilities. By content adaptation we meant that the software is able to scale the content being delivered to the target devices based on the QoS of the wireless network, devices' capability and user's preferences. The idea is, when the network is not good then there is no need to send big data. Some devices are unable to display high-resolution images due to their internal hardware capabilities such as supported color depth, buffer size, etc., in this case low-resolution image can be sent.

The process of content adaptation also referred as transformation is usually applied to the multimedia content in order to satisfy the device and network constraints and user's preferences. Content adaptation includes but not limited to: format conversion (e.g. XML to WML, JPEG to WBMP), scaling of images as well as video and audio streams, media conversion (e.g. text-to-speech), omission or substitution of document parts (e.g. substitution of an image by a textual representation), document fragmentation, language translation, etc (Lei et Georganas 2001).

Content adaptation has been actively explored by many research groups. In general these works can be categorized into two main approaches: static and dynamic (Chang et al. 2002; Lum et al. 2002). In static adaptation, contents in formats specific for each terminal or class of terminals are created and stored by the content provider. This approach has several advantages in terms of delivery performance, specificity to particular device, and full control of the content by the server; however it lacks consideration of dynamic variations of terminal's hardware or software components. Moreover, authoring and updating of the content is very costly and impractical and also requires large storage space (Lei et Georganas 2001). To address such issues dynamic content adaptation is proposed. In dynamic content adaptation, the system gathers characteristics of terminals, users and networks, call delivery context or context profile, and adapt the content accordingly. Dynamic content adaptation technologies can be classified into three categories: server-based, client-based, and proxy-based (Lei et Georganas 2001; Mohan et al. 1999).

There are ongoing standardization activities to develop an open framework of multimedia delivery and consumption. The work in progress by MPEG-21 is one of them. The aim of MPEG-21 is to enable the use of multimedia resources across a wide range of networks and devices (Burnett et al. 2003). One part of MPEG-21 is the Digital Item Adaptation. DIA defines description tools for usage environment and content format features for the purpose of transparent access to the multimedia content. These tools enable the adaptation engine to make better decision in order to provide best content to the user.

Although previous works has shown that the use of the general adaptation approaches do result in better performance and user experience compared to situations when no adaptation is provided, each of these approaches exhibits performance shortcoming in certain environments (Fu et Karamcheti 2004). Moreover, these approaches limit openness, flexibility, and interoperability of the data delivery system. The proxy-based approach can further be extended in such away that, the proxy can use adaptation tools developed externally. The idea is service providers can develop these tools for example using Web Service technology and can be accessible by data delivery applications. Our framework bases on this new approach, which is also proposed in some works such as (Buchholz et Schill 2002; Elson et Cerpa 2001; Rahman et al. 2002). As indicated by (Elson Cerpa 2001), external services enhance the performance of the adaptation process and accommodate a number of adaptation requirements which is not the case in previous approaches. The problem of complex adaptation task can also be handled by effectively combining several adaptation services. However, there is a challenge to find the appropriate chain of transformations that best fit the delivery context, given a composite adaptation task that can be carried out in a number of steps and given that there could be a number of possible configurations to adapt the content.

In this paper we present an architectural framework for distributed content adaptation approach. In this framework, adaptation tools developed externally are employed. In order to facilitate the selection of appropriate chain of transformations, an algorithm is developed to construct and optimize a chain of transformations called content adaptation graph.

The remainder of the paper is organized as follows. Section 2 describes architecture of the framework. In section 3, we present context and multimedia profile. In section 4 we present construction of content adaptation graph. Prototype implementation and experimentation of the framework is presented in section 5. In section 6 we conclude the paper and list future works.

2. Architecture of the Distributed Content Adaptation Framework (DCAF)

Before presenting, the DCAF, we describe a simple scenario that highlights the motivation of our work. This scenario is a base for our discussion and used to demonstrate the DCAF prototype.

2.1 Application Scenario

Consider a situation wherein a physician making rounds at health center in Paris. The health center is built with the state-of-art of modern health care system to serve European communities. This center is networked with other health centers throughout Europe and allows anywhere/anytime patient data access by doctors, nurses, caretakers and other hospital staffs. The physician is visiting a patient from one of the European states. The patient shows signs of a bronchial infection. Before prescribing any new medication, the doctor wants to check the patient's medical records which may consist of media data such as text, image, video or audio on his wireless handheld device (e.g. PDA). He connects to the system and gets her historical data from previous centers she has visited and notes that she is already taking medications for high blood pressure and heart problems and is also allergic to penicillin. He then uses the handheld device to check an online drug database for the most effective treatment compatible with the patient's other medications. The doctor fills out a short form on the handheld device screen to authorize and send the prescription via a wireless network to a pharmacy.

The example scenario highlights the need of content delivery framework that can provide access to multimedia information and supports heterogeneous terminals: PC, PDA, Smart phone etc. The framework should also be able to recognize terminal capabilities and contextual situations in order to apply different content adaptation processes such as image resizing (to display images on small devices), text-to-speech translation (to deliver in audio form), language translation (to deliver in different languages), summarization and so on. The adaptation processes could be part of the application service or software elements call adaptation services provided by third party and accessible through a communication interface. Security and confidentiality of data access in such scenario is also very important though it is not addressed in this work. Currently we are working with another group in our lab that has developed confidential data access mechanism for highly distributed system (e.gbySeitz[16]).

2.2 Architectural Framework

Figure 1 describes the architecture of our framework. Main components of the architecture and their function are summarized as follows:

Local proxies: access points to information delivery systems (content providers). They are in charge of retrieving and processing context profile (user, terminal, network), decide the type and number of adaptation processes, discover adaptation services and plan execution of the services. Core part of a local proxy is a Content Negotiation and Adaptation Module (CNAM). This module computes an optimal execution plan of adaptation process and invokes the adaptation services accordingly. The module consists of profile manager and adaptation planner.

Client profile repository: stores user and device profile. Users can update and modify their profiles at any time. Dynamic profiles such as user location and network condition are determined during request execution.

Adaptation service registry (ASR): it is like Universal Description, Discovery and Integration (UDDI) registry; it stores adaptation service profile (functional and non-functional) and allows for service look up. UDDI uses a Web Service Description Language (WSDL) (Web Service Definition Language 2001). A WSDL file describes the input and output of a Web Service. It also describes an interface to invoke a Web Service. However, a WSDL file provides no semantic information about what the service is used for, what its input stands for, what the effect/ consequence it will bring about, how much it costs, or its execution time. The semantic information is important for automated service discovery and composition. In order to facilitate adaptation service look up we have used a very simple and structured service description that is adapted from WSDL.

Adaptation service proxies (ASP): to keep the framework open and extensible, we consider external adaptation tools (adaptation services). The ASP hosts the adaptation tools.

Content proxies: provide access to content servers, formulate user request to source format, manage and provide content profile (meta-data).


Usage example of the architecture

Consider a physician that connects to a medical information system through the local proxy and request for patient's medical history with his handheld device (e.g. PDA). The local proxy forwards user request to content proxy and at the same time gathers client profile (user and terminal) from the client profile repository. The content proxy sends meta-data response to the request. The language and content type preference of the user is determined from the profile. In our example, we consider French as language preference and audio as content type preference of the user. One of the components of the medical record is text data. The meta-data response shows that the text data is very long and written in English n this case, we need a minimum of three adaptation processes that are text summarization, language translation and text to audio translation to deliver the data in the currentcontext.

3. Context and Multimedia Profile

Decision of the adaptation process depends on the extent and quality of profile information gathered from various sources. We categorize the profile information into context profile which includes user, device, network, etc. and multimedia profile or metadata.

3.1 Context Profile

The context profile must give a complete description of the environment and provides easy way of processing. The predominant approach for profile representation is the WSC's Composite Capability/Preference Profiles (CC/PP) language (Klyne at al. 2001). CC/PP is an XML based Resource Description Framework (RDF) (Lassila 1999) for describing device capabilities and user preferences. In the CC/PP specification, a profile is basically constructed as a strict two-level hierarchy: each profile having a number of components, and each component having a number of attributes. Arguing that the two-level restriction of CC/PP is not appropriate to capture complex profile structures, Held et al, (Held et al. 2002) proposed a multi-level profiling called CSCP (Comprehensive Structured Context Profiles). The CSCP is an RDF based meta language, hence it inherits the inter changeability, decomposability and extensibility of RDF. The CSCP permits merging profile fragments that are dynamically retrieved even from different Web sites- for example, from profile repositories of device vendors. User can specify conditional expressions, for example Content Type= "text" and Location= "meeting-room" or Content Type= "audio" and Activity= "driving". In these expressions location and activity are dynamic information that can be captured from sensors.

Partial part of a context profile is represented in Figure 2. <device>
Figure 2 XML representation of context profile

 <dev:deviceType> </dev:deviceType>

 <dev:Software/dev: UserAgent>

 </dev:DeviceProfile> </device> <user>
 <rdf:Bag> <rdf:li>
 </rdf:li> </rdf:Bag>

 <net:Bandwidth > </net:Bandwidth> <net:Delay>
 </net:Delay> </net:NetworkProfile> </network>
 </ContextProfile> </rdf:RDF>

3.2 Multimedia Profile/Metadata

The system architecture consists of content servers and content proxies. The content servers store media objects which can be text, image, audio or video. The content proxy manages the meta-data. In the context of multimedia systems, meta-data consists of content and feature of media objects. Content metadata is information about the object(s) captured in the media -the object's title, description, language, etc. and feature metadata is data about the media itself (not about the object(s) in the image). It includes information such as media type, file format, file size, media dimensions/spatial resolution (width and height), and color information, identifier (location) etc. In multimedia systems, in addition to facilitating content search, meta-data plays important role in the process of content adaptation since it guides the decision process. The meta-data can also include possible or allowable adaptation processes to preserve semantics of the content. This increases acceptance of the adapted content by the user as it maintains quality at the same time end-to-end agreement. For the multimedia profile simple, XML representation is used.

4. Constructing an Adaptation Graph

In order to decide appropriate adaptation processes/operators, find and compose the available adaptation services, decision must be made by considering all the factors. The profile manager part of the CNAM retrieves client, network and content profile and generates context profile (see Figure 2). Using the context profile and rule-based constraint mapping, the context profile processor generates a set of transformation processes/operators called transformation prescript required to meet the constraints.

Definition: The transformation prescript is a Directed Acyclic Graph (DAG) where

--The start nodes of the graph are media sources, the intermediate nodes are operators and the end nodes are media sink or adapted media.

--The edges are media/data flow from one operator (or media source) to another operator (or media sink).

For the usage scenario described in section 2, the transformation prescript graph is shown in Figure 3. The transformation prescript graph provides a list of transformations need to deliver the data in the current context. It does not specify the actual services executing the transformation. In order to generate an execution plan we need to construct the adaptation graph from the transformation prescript by replacing each transformation with appropriate adaptation service(s).

Having the set of transformations and their execution order, the adaptation planner constructs the adaptation graph. The transformation prescript is an input to the planner. Considering the transformation prescript given in Figure 3, T= {[t.sub.1]; [t.sub.2], [t.sub.3]} its adaptation graph is constructed as follows:

Step 1:

G= {<[t.sub.1], [s.sub.11], [s.sub.12]) >, <[l2.sub.2], [s.sub.21], [s.sub.22], [s.sub.23]>, <[t.sub.3], [s.sub.31], [s.sub.32]>), where t] (text summary), [t.sub.2] (language translation) and [t.sub.3] (text to audio) are realized by services {[s.sub.11]; [s.sub.12]), ([s.sub.21], [s.sub.22], [s.sub.23]) and {[s.sub.31], [s.sub.32]) respectively.

Step 2:

We assume that each service accepts input data with a quality value ([]) and generates output with a quality value ([Q.sup.out]). The quality parameters may include data format, resolution etc. To connect two services we consider quality consistency, that is, the output quality value of one service should match with the input quality value of the next service. In our prototype we limit service connectivity to data format. Figure 4 shows the adaptation graph.

Step 3:

Definition: an operator is called neutral if we can insert it anywhere in the graph but do not change the semantics of the graph (e.g. format trasncodage). Neutrality of an operator is contextual, for example if the user is interested on graphic image but not on the quality of the image then converting BMP to JPEG is neutral.

Now, let us observe the adaptation graph in Figure 4. By looking at the graph we can see services [s.sub.21] and [s.sub.33] are connected only to one service. This is because the output ([F.sub.5]) and input ([F.sub.8]) formats of services [S.sub.21] and [S.sub.33] respectively do not match with the input and output formats of the next and predecessor services in the graph. It necessary to find one or more neutral operators so that [s.sub.21] and [s.sub.33] will be connected to one of ([s.sub.31], [s.sub.32], [s.sub.33]) and {[s.sub.21], [s.sub.22], [s.sub.23]) respectively or remove them from the graph. Removing the unconnected services helps reduce the computational time of the algorithm to find the best path. But by doing this, we might exclude an interesting path through [s.sub.21] or [s.sub.33] in terms of global execution time from S to E.

Assuming that no neutral operator (s) is/are found to connect [S.sub.33] then we remove [S.sub.33] from the graph. On the other hand, suppose that the output of [S.sub.21] is Ms-Word data and the input of [S.sub.31] is rtf data. Let us say there are two services with neutral operators [sn.sub.1] (Ms-Word topdf) and [sn.sub.2] (pdfto rtf) we can thus connect service [S.sub.21] to [S.sub.31] through a chain of the two services ([sn.sub.1] -> [sn.sub.2]). By doing this we add the path {S, [s.sub.11], [s.sub.21], [s.sub.n1], [s.sub.n2], [S.sub.31], E) that can be competitive with other paths in the graph. The complete adaptation graph is depicted in Figure 5.

Step 4:

We can see from Figure 5, there are at least four possible adaptation paths. We use quality criteria to select an optimal adaptation path. We consider time and cost as quality criteria but other criteria such as availability, reliability, etc. can be easily included with out affecting the algorithm. Given time and cost as quality criteria, the quality vector of a service s for an operator t is defined as follows:

Q(S) = (Scos, (S, t), [S.sub.time] (S, t))




The quality vector of a path P is calculated as the sum of quality vectors of its component services and defined as follows:

Q(P) = ([Q.sub.cost](P), [Q.sub.time](P), where


Given [Q.sub.ij] (quality value of criteria j of path pi, the scaled quality (Qsij) is calculated as follows:


where and [Q.sup.max.sub.j] and [Q.sup.min.sub.j] the maximal and minimal quality values of criteria j. User's preferences or requirement of quality criteria such as fastest or cheapest can be incorporated using weighting values in calculating overall quality score value of each path as follows:


where [w.sub.j] [epsilon] [0,1] and [n.summation over (j=1)][w.sub.j] = 1. [w.sub.j] is weight of each criterion and n is number of quality criteria.

The adaptation planner chooses the execution path which has the maximal value of Score (pi, that is max (Score (pi). In general planning is NP-complete problem but we have learned from different scenarios that the number of transformations required is very limited. Search for optimal path is done through exhaustive computation of the adaptation graph.

5. Prototyping of the framework

We have developed a prototype implementation of the content adaptation framework in the context of medical application with some sets of adaptation services that allow us to test the viability of the ideas presented. The prototype has been developed in Java using MySQL. For the purpose of testing the prototype, remote adaptation services such as image resizing, format conversion, color reduction, text to audio translation are developed with Web Service technologies. One of the external services used in the prototype is language translation service provided by WebServiceX.Net (

5.1 Preliminary evaluation

Our preliminary evaluation of the prototype demonstrates openness, flexibility, and interoperability of the architecture. Firstly, the framework is able to use external adaptation services hosted in different platforms. Secondly new adaptation services can be incorporated easily by simply registering them on the adaptation service registry. Finally the framework can be used to seamlessly combine a set of adaptation processes in a way that guarantees conversion from arbitrary input formats to arbitrary output formats (acceptable by the end device).

It is evident that more experiment is required to measure overall performance of the architecture. However to see importance of using external adaptation services, we have measured execution time of some image adaptation services such as resizing, format conversion and color reduction. In the experiment, the server hosting the adaptation services was Intel Pentium 4 CPU 1.90 GHz running Microsoft Windows 2000 with a memory of 261664 Kb. Our results showed that the time required for executing an adaptation services is acceptable. The results are shown in Figure 6.


5.2 Status of the prototype

The necessary source program and the executable files with full documentation manual are put on the url ( Adaptation Project). The manual is well organized to simplify the installation process so that any one can install and test the prototype. The prototype is also accessible on Adaptation Project:8081.

The user-interface part of the prototype is still in progress. We are in the process of integrating our system with SEFAGI (Simple Environment for Adaptable Graphical Interfaces) (Chaari et Laforest 2004) which is developed by another research team of our laboratory. SEFAGI allows us to deploy the interface of our prototype on several devices. SEFAGI permits users to describe their interface and the system generates automatically the code required to use the interface on several devices such as PDA, Palmtop, Smart-phone etc.

6. Conclusion and future work

To deliver multimedia data in mobile environment, content adaptation is a natural solution to alleviate the problem of heterogeneity in client devices, network connectivity, content heterogeneity and users' preferences. This paper presented a framework of distributed content adaptation approach. Important part of the framework is an algorithm that constructs an adaptation graph. This graph determines the chaining of appropriate adaptation services. The selection of an optimal adaptation path from the adaptation graph is done based on QoS criteria. The framework is open, flexible and interoperable. The prototype we have developed has demonstrated these properties. Moreover, the performance measurement of different adaptation services done on the prototype indicates feasibility of utilizing the adaptation services. But several experiments must be done to study the performance of the algorithm with respect to operator and service complexity.

In addition, many enhancements can be made to the language that describes adaptation service capabilities such as input and output types. With the existing service description languages such as WSDL, there is no formal mechanism of input and output type naming. Currently each service is free to choose any name it wishes for the properties specified in its input and output formats, leading to the possibility of name space collisions. The same name may refer to different formats and vice-versa. A mechanism for avoiding such collisions is needed, such as creating a standard ontology for adaptation service description.

Received 30 Oct. 2004; Reviewed and accepted 30 Jan. 2005


[1.] Buchholz, S., Schill, A. (2002). Web Caching in a Pervasive Computing World. Proc. of the 6th World Multiconference on System's, Cybernetics and Informatics (SC12002), Orlando, FL, USA, Jul 14-18.

[2.] Burnett, I. Van de Walle, R. Hill, K. Bormans, J. Pereira, F (2003). MPEG-21: goals and achievements. Multimedia, IEEE, 4. 60-70.

[3.] Chaari, T., Laforest, F. (2004). Generation et adaptation automatiques des interfaces utilisateurs pour des environments multiterminaux. Journal of Ingenierie des Systemes d'Information, Revue des Sciences et Technologies de l'Information (RSTI) serie ISI. vol. 9. no 2. 11-38.

[4.] Chang, C. Y. and Chen, M. S. (2002). Exploring Aggregate Effect with Weighted Transcoding Graphs for Efficient Cache Replacement in Transcoding Proxies. Proc. of the 18th IEEE International Conference on Data Engineering (ICDE-O).

[5.] Elson, J., Cerpa, A. (2001). ICAP-the Internet Content Adaptation Protocol. Internet Draft draft, the ICAP Protocol Group, June. (URL: specification.fart).

[6.] Fu, X. and Karamcheti, V. (2004). Performance Implications of Different Adaptation Mechanisms for Network Content Delivery, in Proc. of the 231 IEEE International Performance Computing and Communications Conference (IPCCC 2004), Apr, 2004, Phoenix, Arizona.

[7.] Gecsei, J. (1997). Adaptation in Distributed Multimedia Systems. In IEEE Multimedia, S. 58-66, April-June.

[8.] Held, A., Buchholz, S., Schill, A. (2002). Modeling of Context Information for Pervasive Computing Applications. Proc. of the 6th World Multiconference on Systemics, Cybernetics and Informatics (SC12002), Orlando, FL, USA, Jul 14-18.

[9.] Jayaputera, G., Alahakoon, O., Cruz, L., Loke, S.W., Zaslavsky, A. (2003). Assembling Agents On-Demand for Pervasive Wireless Services. Proceedings of the 2nd International Workshop on Wireless Information Systems, Anger, France, April 23-26.

[10.] Klyne, G., Reynolds, F., Woodrow, C., Ohto, H. (2001). Composite Capability/Preference Profiles (CC/PP): Structure and Vocabularies. W3C Working Draft, Mar 15, 2001. (URL:

[11.] Lassila, O. and R. Swick (1999). Resource Description Framework (RDF): Model and Syntax Specification. W3C Recommendation, [available online].

[12.] Lei, Z., Georganas, N. D. (2001). Context-based media adaptation in pervasive computing. Proceedings of IEEE Canadian Conference on Electrical and Computer (CCECE'OI), Toronto, May 2001.

[13.] Lum, W.Y. and Lau, F.C.M. (2002). On Balancing Between Transcoding Overhead and Spatial Consumption in Content Adaptation. Mobicom 2002, Atlanta, USA, September 2002, 239-250.

[14.] Mohan, R., Smith, I, Li, C.S. (1999). Adapting Multimedia Internet Content For Universal Access. IEEE Transactions on Multimedia, March 1999, pp. 104-114.

[15.] Rahman, R., Menon, R.R. and Rafalow, L. (2002). Enabling OPES to Use Web Service for Callout Service. Draft- rrahman-web-service-OO.txt (work in progress), May 2002 ( non-wg-doc/draft-rrahman-web-service-OO.txt).

[16.] Seitz, L., Pierson, J-M., Brunie, L. (2003). Semantic Access Control for Medical Applications in Grid Environments. Euro-Par 2003: pp.374-383.

[17.] Web Service Definition Language.

Girma Berhe, Lionel Brunie, Jean-Marc Pierson

Lyon Research Center for Images and Intelligent Information Systems (LIRIS)

Institut National des Sciences Appliquees de Lyon, France

Batiment Blaise Pascal (502) 20 Avenue Albert Einstein

69621 Villeurbanne, France

{girma.berhe, lionel.brunie, jean-Marc.pierson}Ca
COPYRIGHT 2005 Digital Information Research Foundation
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2005 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Berhe, Girma; Brunie, Lionel; Pierson, Jean-Marc
Publication:Journal of Digital Information Management
Date:Jun 1, 2005
Previous Article:Content-aware segment-based video adaptation.
Next Article:Reducing communication overhead over distributed data streams by filtering frequent items.

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters