Extensible utility: implementing XML for databases. (Internet).

Many companies today can easily exchange critical data with their employees, but when a partner or customer needs the same information, delays, inaccuracies and mistakes plague the process. Solving this problem raises concerns about preserving significant investment in database systems, establishing uniform operating procedures among business associates and optimizing, rather than replacing, current resources.

Until recently, the solution was a pricey and complex Electronic Document Interchanges (EDI) network. But the development of extensible Mark-up Language (XML) and efficient tools to support and process this language offer an economic alternative to sharing data. Using XML technology, companies can enhance the way they share information among orders, inventory, and production to help partners communicate throughout the supply chain and respond directly to changing market conditions. Incor-porating XML into relational database management systems (RDBMS) makes them more versatile, robust and effective. Additionally, XML is also used for storing data or archiving transactions.

In the early 90s, EDIs offered one-to-one transaction capability, but complex integration and high price tags prevented widespread adoption. The need to effectively process volumes of semi-structured data and transactions on the Internet drove the development of XML, a nimble, robust, and effective alternative to EDI. Both machine and human readable (EDI is only machine-readable), XML adds a layer of intelligent communication, enabling disparate applications to talk to each other. Subsequently, more organizations can transact business online without large software applications or resources, constant upgrades, and/or system overhauls.

Benefits of XML

Because XML is an orderly, easy-to-use specification for describing information, it delivers a universal data interchange format, easily shared between different systems. In such an agreement, trading partners can leverage XML to issue product catalogs to suppliers, who in turn may transform the product data stored in a back-end database to XML, rapidly interpret the information, and deliver it to buyers via their Web application. The seamless integration of internal and external systems improves the process of applications and organizations interaction over the Internet.

XML also supplies the ability to embed business descriptors along with raw data, thereby providing a common format for businesses to exchange information over the Internet. For example, a financial institution's customer wants to know which stocks were sold during the last quarter in Argentina for more than $100. This should not require the financial company's IT department to write a new process to collect this information; it should be able to be done immediately. In an XML-driven environment where the trades are in XML, the IT department should be able to use the RDBMS to store and index the XML. Subsequent queries over these XML documents should use the index to answer arbitrary queries.

To ease processing, XML has several internal mechanisms that insure the structure of an XML document: document type definitions (DTDs), XML schemas, and XML namespaces. Developers use these mechanisms to define the rules that process data. XIvIL processors use DTDs to determine the validity of a document. XML schemas enable the developer to require that documents contain specific data types for tags, as well as create user-defined complex data types, data ranges, and masks. XML namespaces leverage tags created from other developers even if they have the same name. By basing the identification of the document on the context of the tag rather than the name itself, documents with identical tags can still be processed according to their appropriate functionality.

Adoption of XML

In the past year, XML has emerged as the de facto standard of encoding information for document, interchange but implementing, optimizing and extending its use seems risky and expensive to potential users. While they realize they must handle an increasing quantity of XML-formatted data, customers are comfortable with their data in an RDBMS engine and their data processing in packaged applications, investing significant resources in making the applications and the database interact effectively. The ideal XML solution will capitalize on that investment and improve its functionality, without requiring a massive conversion effort, and will integrate seamlessly with the RDBMS that contains the business information--resulting in current, accurate information, which enables informed and efficient business decision-making.

To understand the effect of real-time collaboration, consider planning a vacation to New York City. It would be quite convenient to logon to a travel website, enter a budget, travel dates, and a few preferential details (non-smoking room, late check-out, full-size rental car for the trip upstate, etc.) and have an entire customized vacation delivered via email within minutes. If the flight were cancelled, the travel service would reroute you through the most convenient location and convey the change via a previously indicated method (cell phone, PDA, email, etc.). To provide such a service, the system must not only capture the data, but also store it in a format easily accessible in real time by all affected parties and transform it into another format for interaction with the affected parties. This implies a standard data format to facilitate exchange of information between different companies, and a highly efficient store and search mechanism.

Effective Implementation of XML

To accomplish this, the state and context of the data would need to be compiled first, while the system searches for a comprehensive and intelligent examination of travel alternatives. Then booking the flight, hotel, and rental car should be completed uniformly, and the reservation confirmation and details communicated to you. Such a system meets customer demand by piecing together information from suppliers, partners and other providers, with the key to success lying in the system's ability to combine disparate sources of information, with data flowing seamlessly between applications.

There are several methods of addressing the project of processing XML: creating an inhouse solution, XML to DBMS mapping, and XML query. With an in-house solution, the travel service could require its suppliers, partners, and other providers to generate and use information in XML, while their internal IT department focuses on efficient handling of XML. However, the IT staff would need to write information extraction software for each supplier and anticipate different data sequencing. While the custom code might meet some immediate needs, special cases and changes in schema would quickly create an overwhelming writing task. In any case, the in-house solution would defeat the original aim of leveraging existing investments, without adding another application into the system. Also, for all applications and the database, the in-house code would write adapters to extract data as XML, and transform it in the custom code.

An XML Transformation Engine

Instead of writing the XML transformation engine as custom code, the travel service could buy a packaged product that handles transformations. This would reduce time-to-market for the solution and free up IT resources. The biggest issue with this solution is the creation of another stop or hub for data, where it would be extracted by the XML transformation engine, stored in main memory or on the file system, and then served to in-house and partner applications when requested. If the different packaged applications and the database do not provide data in XML, then the XML engine would be required to generate XML from proprietary formats.

A couple of years ago, the most common relational database solution to the XML storage problem was XML to DBMS mapping. This requires third-party software or features from the database to correlate XML data with tables already stored in the DBMS database. The solution requires using correlations to convert every nuance in every XML document from each service provider so that the XML documents fit into the database tables. Additionally, each time a provider updates or changes his services, the correlations would also need to be updated. This time-intensive option would also require a substantial writing task to maintain mapping files, translate searches into queries and create multiple data extractors to translate information from different formats. For real-life XML standards, conversion of the complete XMIL document into a relational representation would require an impractical number of relational tables.

Acknowledging that mapping solutions would not pass muster, most database companies have announced features to support storage and querying of XML data. Some have gone further and implemented XML indexing in their RDBMS system. Most of these querying engines also serve well in transforming relational data into XML, providing the capability to further store and search transformed data into XML. These transformation capabilities can also be accessed by existing applications to fulfill their XML requirements. Similar to relational database vendors, application vendors are also providing XML ports" so that they can serve as XML data sources.

Improving industry standards of using XML and developing innovative database servers will render information into customer service. Currently, customers expect certain online transactions to be reliable, such as online airline reservations, online banking and stock trading, but they do not expect such transactions to communicate with each other. Today, databases answer queries, but in several years, a query will evolve into a command that is immediately understood, processed, and delivered. As vendors continue to offer specific industries custom tools that meet the needs of IT customers, the adoption, development, and integration of XML will increase.

www.sybase.com

Anupam Singh is a staff software engineer with Sybase