SenSage solves compliance worries.

Enterprise managers scrambling to gain control of their log files--searching and archiving massive amounts of data arriving daily--can find the help they desperately need from the SenSage Security Compliance Bundle featuring EMC Centera.

On most networks, log files represent a largely untapped wealth of diagnostic information from a variety of sources: data/device access logs, intrusion warnings, user lockouts, kernel panics, hardware complaints, and application errors. On the largest enterprise networks these log files can grow by hundreds of Gigabytes per day, and regulations such as SOX require it all to be not only archived for years but also digested at a moment's notice.

Figuring out what to look for is hard enough for determined managers, but the sheer immensity of collection, archive, report and investigation compliance mandates is daunting. SenSage Inc. of San Francisco, CA created the solution. SenSage Security Compliance Bundle combines a robust security event information management solution and EMC Corporation's Centera "near-line" content-addressable storage (CAS). We liked the power of the drill-down investigations for real-time correlated alerts, ad hoc queries and batch mode reports, as well as the storage optimization and performance.

What's in the Bundle?

The SenSage Security Compliance Bundle is available now, and ships in multiple configurations to medium to large enterprise customers. The lowest-priced package is designed for managing up to 15 GB daily log volume, stored for greater than 25 months in a solution managing 11.3 TB of compressed raw event data leveraging Centera. A higher-end bundle absorbs up to 50 GB of daily log volume with storage of greater than 13 months managing 23.4 TB compressed raw event data with Centera. Every bundle includes event log collection, as well as a standard set of rules and reports based on ISO 17799 best security practices; plus specialized analytics packages to satisfy sets of regularity or industry compliance guidelines. The whole system is managed through a Java-based console and provides for real-time alerts, custom queries and reporting.

Compliance analytics sets are presently available for the following systems: Sarbanes-Oxley (SOX), the Health Insurance Portability and Accountability Act (HIPAA), and the Federal Financial Institutions Examination Council (FFIEC) guidelines. Packages for government (FISMA, DCID, NISPOM) and privacy standards (SB-1386, PCI) are soon to be released.

The software side of the Bundle, SenSage ESA, could be appealing by itself to smaller organizations. The solution integrates the collection, analysis, reporting, performance and primary storage capacity, but would lack the immense near-line storage and management capabilities of the Centera hardware. SenSage ESA software scales exceptionally well from single servers to multi-member clusters.

[FIGURE 1 OMITTED]

Wondering about which RDBMS it uses? There isn't one. The SenSage Security Compliance Bundle supports SQL-compatible queries but the costs of database licenses, tuning and management do not play in to this product.

As for the hardware, the SenSage ESA software installs on hardened RedHat 3.0 Linux and SUSE Enterprise 9 machines. It is on these servers that compresses and processes incoming event data, conducts real-time correlated alerts, and executes centralized event repository analyses utilizing both primary and Centera storage.

Log data migrates from the servers into "near-line" permanent and protected storage on the Centera disk array. The data is moved automatically based on age, type, table or log source but remains available to the analysis engine. The Centera storage media slows queries by only 4.5%, according to a SenSage spokesman; since there is some overhead in looking up the stored log entry location and communicating between systems.

However, the Centera hardware offers some compelling advantages that make the mild performance hit well worth the expense. First, the Centera hardware is a high-availability system, eliminating a need for backups because data can always be recovered even when a disk fails. Second, the data cannot be read or modified except through the API from the SenSage application, so it remains securely out of reach to anyone who is not a SenSage administrator (though data can also be automatically deleted off of the disk after a specified period of time). Finally, the Centera provides a huge amount of extensible, managed storage space at a price point that may beat the cost of Storage Area Network or Network Attached Storage.

Setting up

VeriTest, an independent testing service of LionBridge Technologies Inc., has certified Centera-SenSage interoperability. We were able to gauge the performance and scalability of the SenSage components in the Bundle using the base SenSage ESA servers, from which the Centera complements the bundle with conveniently scalable and manageable storage capacity.

We set up SenSage ESA on a matched set of IBM HS20 blade servers, with dual 2.4 GHz processors (slower than today's standard processors and SenSage recommendation). Installation is not difficult, although SenSage routinely assists customers with implementation. For our tests we configured the blade servers into several test groups of 3, 5, and 10-clusters.

Part of the innovative beauty of the SenSage software is its ability to automatically load-balance and fail-over on a cluster. As log data is loaded onto the cluster before querying, each server receives its own allocation plus a copy of the next server's share. If one of the cluster members goes down, the member that has the duplicate copy automatically processes the unanswered query--all without user intervention. This would slow the query time proportionally; since one server is missing from the cluster, but it maintains availability. Therefore, large data sets remain online--users do not have to reorganize and reload the data set nor modify search mechanisms.

Outside of the server setup, interfacing with device/system event logs is likely to occupy a manager's thoughts. SenSage supplies adapters for a wide variety of log sources, and users can write their own collectors for home-grown or uncommon systems employing SenSage's SDK. Supplied adapters support both real-time and batch-mode event capture protocols (such as Syslog, SNMP, LEA, SCP, SFTP, FTP, HTTP, RDBMS).

For test data, we used actual log files from several sources, modified only as necessary to preserve privacy or increase variety. (The data was purposefully not tuned.) Sources included a Blue Coat proxy log, a CheckPoint firewall log, and a web server access log. The data files ranged in size from approximately 200M for the web server logs to 10G for Blue Coat.

Drill it

We used the Java console to launch any of the pre-built standard or compliance analytics reports, or even create new queries for ad hoc investigations. But interaction does not end with the report that displays on the screen. We could conduct immediate investigations by drilling down into the results. SenSage Security Compliance Bundle contains mechanisms that automatically correlate data from multiple sources as it is stored, enabling administrators to probe into user activities dynamically.

For example, we launched a report of user activity outside business hours, which pulled data from multiple logs. Selecting one user by date range, we could burrow in to view specific applications and times of use.

Although there is no wizard for creating reports, we had all the tools we needed to perform SQL-compatible queries. We set up search criteria for filtering the data--selecting, ranking, etc. We could also sort and graph the data, and create a "reportbook" that grouped queries together. Each report could be shown either by graph or table.

We were even happier to experience visual playback of a sequence of events. Real-time correlated data with timestamps let us step-through an incident to gain even deeper knowledge of the sources and targets of, say, a root kit attack. An Asset Manager provided definitions of servers/devices and their locations.

The console allows users to set up reports to run on a schedule and be sent to destinations as PDF, HTML or CSV files. The program also lets analysts activate email and SNMP alerts to be issued when correlations and real-time events filter through custom rules. Even from the alerts a user can summon relevant details such as the IP addresses and user IDs involved.

Role-based permissions control user access to individual components and reports, but SenSage also extends this to special care of sensitive data items. Field-level access permissions for data such as birth dates and Social Security numbers helps the analysts perform their jobs without invading the privacy of those they are charged to protect.

Not everything worked as expected--we uncovered a couple flaws with the user interface, but they were minor and easily avoided.

High performance

Speed is a primary concern to security and compliance analysts; data loading and query running could consume vast amounts of time with such a volume of data. So, we were pleased to see the speed and scalability afforded by the SenSage product.

Data loads onto the servers in a B-tree format to make it fully indexed for rapid searching. Within that B-tree the data is also compressed to make it essentially of identical size to a normal gzip of the same data--about 90% reduction of the raw data size. Queries are able to pull reports using SQL commands without uncompressing the data.

We loaded each of the log sources onto each of our cluster configurations and noted that the rate of loading varied by the type of log. For example, on the 5-node cluster (which handles up to 25GB of data per day) the CheckPoint log loaded at about 354 MB/sec (17K records/sec) while the less bulky Blue Coat log entries loaded twice as fast at 718 MB/sec (61K records/sec). The web server logs loaded somewhere in between the two. A SenSage representative related that Windows event logs (which we did not test) load at near 90K records/sec on this same hardware. Generally speaking, all log types will have varying affects on performance. Obviously, if we had used the recommended 3.2GHz processors instead of the 2.4GHz models that would have significantly increased speed for every log type.

Graphing our results, the scalability became apparent. The curves were almost straight as an arrow, the rate being dependent upon the log data being tested. On average for the three log sources we used, each extra server could handle approximately 70% of the single server capacity. But again, processing Windows event logs would raise the average considerably.

Query rates exhibited mostly similar graphs of scalability. Of course, like the difference in log file content, differences in the complexity of SQL queries can also create a wide divergence of results. For example, looking at the Blue Coat log results on the 5-node cluster showed one query running at 1.3M records/sec and a complicated query at only 1/3 of that speed.

We found that the SenSage Security Compliance Bundle quickly brings an organization up to speed on compliance--Sarbanes-Oxley (SOX), HIPAA, FFIEC, FISMA and more--in an easy to use, high performance and innovative package.

www.sensage.com