Ice Performance

Ice is highly efficient middleware. It has excellent performance for both latency and data transfer. Furthermore, Ice provides features that make it inherently faster than other middleware technologies, such as CORBA. Examples are batched oneways, efficient event forwarding, and data compression over slow connections.

We have conducted extensive performance comparisons with TAO, which is one of the most popular ORBs and is widely considered to be among the fastest CORBA implementations available.

Test Setup

For the tests, we used the latest non-beta versions of Ice and TAO as of this writing, which are Ice 2.1.0, and TAO 1.4. We ran our tests under both Linux and Windows XP. In either case, we used a Dell Dimension 4500, with a Pentium 4 running at 2.4 Ghz (no hyper-threading) and 512MB of memory.

For the Linux tests, we used Fedora Core 3 with the GCC 3.3.5 compiler. (The GNU C++ compiler version 3.4 is not supported by TAO 1.4.) Both Ice and TAO, as well as the source code for our tests, were compiled using the options -O3 -DNDEBUG -D_REENTRANT. For Windows XP, we used the Windows XP Home Edition with Service Pack 2, and the Visual C++ 6.0 compiler with Service Pack 5. All sources were compiled with /O2 /D NDEBUG.

To get comparable results, it is very important to conduct all tests with equivalent concurrency models. We chose the thread pool model, since this is the most commonly used threading model. We configured four worker threads for both Ice and TAO. The TAO settings in svc.conf were:

static Client_Strategy_Factory "-ORBClientConnectionHandler MT"

Ice was configured as follows:

Ice.ThreadPool.Server.Size=4
Ice.ThreadPool.Server.SizeMax=4

For oneway tests, we used SYNC_WITH_TRANSPORT for TAO, which provides the same quality of service guarantees as oneway invocations in Ice.

Except where mentioned otherwise, we used the loopback network interface, and client and server were collocated on the same computer.

All test results are presented as time per request, meaning that smaller values are better (in contrast to requests per second, as some other tests do). We ran every test N times, and then used the mean of the best N/2 results, throwing away the other N/2 worst results. That way, we avoided false performance measurements due to operating system activity during some of the test runs.

Performance differences are calculated as (t_TAO - t_Ice) / t_Ice, expressed in percent. For example, a difference of 50% means that Ice is 1.5 times faster than TAO.

Performance Test Index

We performed the following tests:

• Latency Performance Tests.
• Throughput Performance Tests.
• Slow Connections Performance Tests.
• Event Distribution Performance Tests.

Summary

The following table summarizes all performance measurement results:

Summary of Results

Test	Ice Performance Benefit
	Linux	Windows XP
Latency
Twoway	145%	77%
Twoway AMI	220%	136%
Oneway	322%	83%
Batched Oneway	1609%	368%
Throughput
Byte Sequence	-9%	-39%
Struct Sequence	23%	78%
Slow Connections
Document Transfer	138%
Structured Data	341%
Event Distribution
Latency	up to 271%	not measured
Throughput Unbuffered	up to 302%	not measured
Throughput Buffered	up to 588%	not measured

As the data shows, Ice yields substantial performance benefits of TAO in all but one of our tests. Many of the performance benefits are due to Ice's quality of implementation, but some also reflect basic design differences between Ice and CORBA.

Types of Tests

Performance differences between Ice and CORBA can be separated into differences due to quality of implementation and differences due to inherent design advantages. Comparison of the quality of implementation applies where a test measures the performance of an aspect of the middleware that is functionally identical (or at least very similar) for both products. For example, a latency test falls into this category because, for both platforms, substantially the same functional activity takes place. Comparison of inherent design advantages applies where one platform offers a feature that has no equivalent in the other platform. For example, a performance test that measures throughput over a slow link with data compression falls into this category because CORBA does not offer data compression. For tests in the first category, we made the comparison as fair as possible, that is, we ran the tests on identical hardware, we compiled TAO and Ice with the same compiler options, and we used the same concurrency models for the tests (see Test Setup for details). For tests in the second category, for which Ice has inherent performance advantages due to its design, comparison favors Ice because a CORBA ORB simply cannot provide an equivalent design. Therefore, such tests are best regarded as comparing architecture and design, rather than the quality of particular implementations.