SPEC releases SPECjbb2005 for Measuring Java Virtual Machine -JVM- Server Performance.

WARRENTON, Va. -- The Standard Performance Evaluation Corp. (SPEC) has released SPECjbb2005, an updated benchmark for evaluating the performance of servers running typical Java business applications. The benchmark can be used across several versions of UNIX, Windows, Linux and other operating systems.

SPECjbb2005 was developed by SPEC's Java subcommittee, with contributions from BEA, Darmstadt University of Technology, HP, IBM, Intel and Sun. The benchmark's workload represents an order processing application for a wholesale supplier. Systems integrators and end users can use the benchmark to evaluate performance of hardware and software aspects of Java Virtual Machine (JVM) servers. Hardware vendors can use SPECjbb2005 to analyze their platform's scalability when running Java applications. Software vendors can use it to evaluate the efficiency of their JVMs, just-in-time compilers (JITs), garbage collectors, thread implementations, and operating systems.

Updates reflect current practices

The new benchmark is a major update to SPECjbb2000, and results from the two benchmarks cannot be compared. The following important changes have been made to the benchmark to reflect current practices of Java application developers and new software codes:

--The internal database is now modeled using HashMaps or TreeMaps in cases where operations on a table require sorting.

--System.gc calls are no longer included in the main part of the benchmark.

--Handling of financial data and calculations has been changed from float to BigDecimal to match current industry practice.

--Several features from Java 5.0 have been added to the benchmark.

--Parts of the code were reengineered to better reflect object-oriented styles of programming.

--Transaction logging is now done by building and writing DOM objects using the JAXP XML functionality in Java 5.0.

--The benchmark can be deployed using multiple Java Run-Time Environment (JRE) instances, each independently handling the transaction load on its own data tables.

Real-world workload

SPECjbb2005 simulates a wholesale company with warehouses that serve different districts. It mimics customer operations such as placing orders or requesting the status of an existing order, and operations within the company, such as processing orders for delivery, entering customer payments, checking stock levels, and requesting a report on recent activity by a given customer.

The benchmark measures throughput of the underlying Java platform, which is the rate at which business operations are performed per second. It steps through increasing amounts of work, providing a graphical view of scalability. Performance is assessed by two metrics: bops (business operations per second), which measures overall throughput for all of the JVMs in a benchmark run, and bops/JVM, which measures the performance and scaling of a single JVM.

"Like its predecessor, we expect that SPECjbb2005 will quickly become an industry standard," says Ricardo Morin, chair of the SPEC Java subcommittee. "It will be an important yardstick that companies can use to compare servers running business applications written in Java."

Initial results, availability

Performance results for SPECjbb2005 will be available on SPEC's web site at www.spec.org/jbb2005/results/ beginning on June 22. SPECjbb2005 licensees may publish their own results in accordance with SPEC's run and reporting rules. The minimum configuration on which the benchmark has been tested successfully (with eight warehouses) is a laptop with a 1.7Ghz Pentium M processor and 1GB of memory using a 512MB heap.

SPECjbb2005 is available now on CD-ROM from SPEC for $500. Discounts are available for eligible universities and non-profit organizations. SPECjbb2000 licensees can upgrade to the 2005 benchmark for $250.

About SPEC

SPEC (www.spec.org) is a non-profit organization that establishes, maintains and endorses standardized benchmarks to measure the performance of the newest generation of high-performance computers. Its membership comprises leading computer hardware and software vendors, universities, and research organizations worldwide. For more information, contact Dianne Rice, SPEC, 6585 Merchant Place, Ste. 100, Warrenton, VA 20187, USA; phone: [phone]; fax: [phone]; e-mail: [e-mail]; web: www.spec.org.

For additional information about SPECjbb2005:

Frequently asked questions: http://www.spec.org/jbb2005/docs/FAQ.html

Design document: http://www.spec.org/osg/jbb2005/docs/WhitePaper.html