Database strategy: active archiving.

Somewhere between database management and storage management lies a new technology, active archiving. Active archiving combines the best of the storage world and the database world to address skyrocketing database growth in mission-critical databases. Today's corporate databases are growing at unparalleled rates, as much as 40 percent annually, according to META Group. Despite tighter budgets due to the tightened economy, skyrocketing database growth remains a concern for IT and management at today's largest companies. Finding terabyte-sized databases is the norm with the largest databases now breaking into petabyte territory.

Meanwhile, database performance, availability, and limited data access have begun to take their toll on the bottom line at these companies, particularly with today's data-intensive, customer-facing applications. The hardest hit databases are connected with today's mission critical applications such as ERP, CRM, and data warehousing. What initially appears to be a pure IT dilemma becomes an enterprise problem, with a potentially huge negative impact on the bottom line.

The effects can range from minor irritation relating to inaccessible data or files to more serious consequences such as loss of customers, market share, or revenue. Time spent on routine maintenance of these behemoth databases has begun to affect the open-for-business hours window, leaving staff without access to necessary product and customer data.

Many industries, such as the health care, pharmaceutical, and financial services, face regulatory requirements mandating data retention and accessibility to data. Until now, companies have been reluctant to archive any data for fear of the sudden need to produce it on demand. As a result, companies have accumulated massive amounts of data on their production databases.

Traditional Solutions for a New Problem Don't Work

Most IT departments have been struggling with poor database performance and availability--the symptoms of database growth--deploying traditional solutions. These solutions treat the symptoms, but not the underlying cause--excessive database growth.

Hardware capacity upgrades. Most companies immediately throw hardware at the problem, upgrading processor power. However, this is just a temporary fix without a guarantee of improving performance. Inevitably, you reach a "performance wall" where upgrades become unaffordable or it becomes impossible to meet service level requirements using capacity increases alone. In addition, capacity upgrades not only include the cost of the hardware, but additional software costs since software licenses are often based on database size.

Custom archiving programs. Many companies try to write custom archiving programs. These projects demand a large investment of programming talent and personnel for application development. Many companies choose not to develop their own archiving programs since the programming talent and time can be applied to more important projects. Those companies that attempt to develop their own archiving programs often times abandon these projects because development is costly, it usually requires extensive maintenance, and the result doesn't work as effectively as an outsourced solution.

Packaged backup software. These utilities are meant for routine backup and disaster recovery purposes. They are not designed to move data back and forth from an archive on the row level. Traditional archiving saves data to a flat file and in order to access the data, one must restore the entire archive requiring time and capacity to perform the restore.

Combining the Best of Database Technology with Storage

A new data storage strategy, active archiving, combines the best of both the storage and database worlds. It helps companies safely remove infrequently used data, or "active reference data," from production databases and keep it close at hand on an inexpensive storage medium like tape, DVD, or optical disk. The mission-critical database retains important operational data and is freed of the burden of storing infrequently used data. Meanwhile, users maintain quick and easy access to the archived data through browsing, searching, and selectively restoring active reference data that retains its business context.

Active archiving allows you to remove precise sets of data. With dozens or hundreds of normalized tables in a database, active archiving preserves data from even the most complex relational data model, maintaining its relational integrity with 100 percent accuracy. Further, it handles relationships defined to the database and relationships managed by the application. On the data level, it is a more comprehensive solution for data storage management than alternatives.

Active archiving features the ability to accurately delete subsets of the data that the user wants to remove from the production database. When archived data needs to be located in an active archive, the solution allows a user to search the archives quickly and easily to find specific data without restoring a single row. In addition, active archiving features the unique capability to 'selectively restore' data on the row level from archive files. This is a major advance from traditional archiving that requires reloading entire sets of archived data to simply view one row.

In addition, active archiving also stores the meta data associated with the database to the active archive as well as information about the data model at the time the data was archived. As a data model changes, you are able to take account of these changes with your archive files.

Active archiving is a continuous solution, not a one-time fix like capacity expansion. By using an ongoing active archiving strategy, a company can safely store its active reference data and continue to "active archive" data on a scheduled basis--monthly, quarterly, or annually. For some companies, this is the first time that they have ever removed data from a database, so the initial archive can remove a substantial amount of data from the database. The result is increased database performance and availability while controlling capacity and maintenance costs. A large steel company was able to initially archive 65 percent of the data off of their order fulfilment system resulting in 20 percent faster batch processing.

Enhancing Today's Storage Environment

In today's budget conscious economy, IT executives are taking a closer look at managing their storage costs. They are evaluating new methods to improve the effectiveness of their current storage environment while preparing for their future storage needs. Many are deploying or considering new technologies to provide high-speed access to data, improve data backup, and reduce the cost of storage management.

Analysts believe a need exists for both SAN and NAS in the typical enterprise. However, the cost of these solutions can also skyrocket with explosive database growth. The efficient use of storage capacity is the key to controlling storage costs and delivering a return on investment.

Although SANs improve performance and availability and streamline system administration, they do not solve the problem of explosive database growth. With a SAN, all data from a database remains stored in the production database with the potential for greater increases in volume if multiple backup copies exist. Active archiving relieves not only the database from the burden of backing up unnecessarily bloated data, but relieves the SAN from backing up unneeded data in a database, saving critical resources. For those companies outsourcing their storage, active archiving could save money since the storage is priced based on capacity.

Active archiving also works with the storage and virtualization pooling configurations defined for the network. Archive files are saved on the available network storage where they can be easily accessed, browsed, used for reporting and even restored when needed, in all instances, file access is performed through the existing virtualization and pooling configuration.

Welcome Relief from Database Growth

Active archiving utilizes the best of database and storage realms, while providing a cost effective method to store and manage database data. It can also enhance SAN, NAS, and storage virtualization technologies by increasing the value proposition of your storage strategy. This process allows companies to:

* Increase database performance.

* Increase database availability.

* Perform database maintenance and backup more efficiently.

* Maintain quick and easy access to infrequently used data.

* Control or defer capacity upgrade costs.

Simply, active archiving complements today's emerging storage technologies and helps enhance the return on investment in these new technologies. No other storage alternative can accomplish so much.

Lisa Cash is the president and CEO of Princeton Softech (Princeton, NJ).

www.princetonsoftech.com