Understading Oracle Query Performance Tuning and using Indexes for optimizations

Sometime back we had a suspected issue with the performance of an Oracle query. Although the issue was found to be unrelated to the suspicion, we did investigate the issue and was able to further improve the performance based on the analysis performed.

To list indexes already existing on a table, you can login as SYSDBA and execute the following. From a Linux workstation, this would mean:

[user@host ~]$ sudo su - oracle
-bash-4.1$ sqlplus / as sysdba

From within SQLPlus, execute:
set pages 999
set lin 999

break on table_name skip 2

column table_name  format a25
column index_name  format a25
column column_name format a25

select
   table_name,
   index_name,
   column_name
from
   dba_ind_columns
where
   table_owner='APS_ESB'
order by
   table_name,
   column_position;


The output will list each table and the indexes

TABLE_NAME          INDEX_NAME            COLUMN_NAME
------------------- --------------------- -------------------------
...
DELIVERY_INFO       DELIVERY_INFO_PK      MSG_REF
...

By default, Oracle will only create an index for the primary key of a table.




To understand how a query is analyzed and performed by Oracle, use the "EXPLAIN PLAN FOR " prefix before your SQL query to analyze. Then after SQLPlus outputs "Explained", issue the query "SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());" as shown below

SQL> EXPLAIN PLAN FOR SELECT MSG_REF FROM SCHEMA.PERF_DELIVERY_INFO WHERE ST_STAGE IN (4, 6) AND ((ST_SUBSYS0_STATE  < 2 AND (ST_SUBSYS0_UPDATE_TIME  IS NULL OR ST_SUBSYS0_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS1_STATE  < 2 AND (ST_SUBSYS1_UPDATE_TIME  IS NULL OR ST_SUBSYS1_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS2_STATE  < 2 AND (ST_SUBSYS2_UPDATE_TIME  IS NULL OR ST_SUBSYS2_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS3_STATE  < 2 AND (ST_SUBSYS3_UPDATE_TIME  IS NULL OR ST_SUBSYS3_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS4_STATE  < 2 AND (ST_SUBSYS4_UPDATE_TIME  IS NULL OR ST_SUBSYS4_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS8_STATE   < 2 AND (ST_SUBSYS8_UPDATE_TIME   IS NULL OR ST_SUBSYS8_UPDATE_TIME   < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS9_STATE   < 2 AND (ST_SUBSYS9_UPDATE_TIME   IS NULL OR ST_SUBSYS9_UPDATE_TIME   < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS10_STATE   < 2 AND (ST_SUBSYS10_UPDATE_TIME   IS NULL OR ST_SUBSYS10_UPDATE_TIME   < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS11_STATE   < 2 AND (ST_SUBSYS11_UPDATE_TIME   IS NULL OR ST_SUBSYS11_UPDATE_TIME   < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS5_STATE < 2 AND (ST_SUBSYS5_UPDATE_TIME IS NULL OR ST_SUBSYS5_UPDATE_TIME < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS6_STATE < 2 AND (ST_SUBSYS6_UPDATE_TIME IS NULL OR ST_SUBSYS6_UPDATE_TIME < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF'))) OR (ST_SUBSYS7_STATE  < 2 AND (ST_SUBSYS7_UPDATE_TIME  IS NULL OR ST_SUBSYS7_UPDATE_TIME  < TO_TIMESTAMP ('19-Mar-15 10:03:10.123000', 'DD-Mon-RR HH24:MI:SS.FF')))) AND ROWNUM < 101 ORDER BY LAST_RECV_TIME;

Explained.

SQL> SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 1365086006

--------------------------------------------------------------------------------------------------
| Id  | Operation        | Name         | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |             |   100 |  6400 |     |  2622   (2)| 00:00:32 |
|   1 |  SORT ORDER BY        |             |   100 |  6400 |   968K|  2622   (2)| 00:00:32 |
|*  2 |   COUNT STOPKEY     |             |     |     |     |          |      |
|*  3 |    TABLE ACCESS FULL| PERF_DELIVERY_INFO |  7763 |   485K|     |  2499   (2)| 00:00:30 |
--------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter(ROWNUM<101)
   3 - filter(("ST_STAGE"=4 OR "ST_STAGE"=6) AND ("ST_SUBSYS0_STATE"<2 AND
          ("ST_SUBSYS0_UPDATE_TIME" IS NULL OR "ST_SUBSYS0_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS1_STATE"<2 AND
          ("ST_SUBSYS1_UPDATE_TIME" IS NULL OR "ST_SUBSYS1_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS2_STATE"<2 AND
          ("ST_SUBSYS2_UPDATE_TIME" IS NULL OR "ST_SUBSYS2_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS3_STATE"<2 AND
          ("ST_SUBSYS3_UPDATE_TIME" IS NULL OR "ST_SUBSYS3_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS4_STATE"<2 AND
          ("ST_SUBSYS4_UPDATE_TIME" IS NULL OR "ST_SUBSYS4_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS8_STATE"<2 AND
          ("ST_SUBSYS8_UPDATE_TIME" IS NULL OR "ST_SUBSYS8_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS9_STATE"<2 AND
          ("ST_SUBSYS9_UPDATE_TIME" IS NULL OR "ST_SUBSYS9_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS10_STATE"<2 AND
          ("ST_SUBSYS10_UPDATE_TIME" IS NULL OR "ST_SUBSYS10_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS11_STATE"<2 AND
          ("ST_SUBSYS11_UPDATE_TIME" IS NULL OR "ST_SUBSYS11_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS5_STATE"<2 AND
          ("ST_SUBSYS5_UPDATE_TIME" IS NULL OR "ST_SUBSYS5_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS6_STATE"<2 AND
          ("ST_SUBSYS6_UPDATE_TIME" IS NULL OR "ST_SUBSYS6_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF')) OR "ST_SUBSYS7_STATE"<2 AND
          ("ST_SUBSYS7_UPDATE_TIME" IS NULL OR "ST_SUBSYS7_UPDATE_TIME"<TO_TIMESTAMP('19-Mar-15
          10:03:10.123000','DD-Mon-RR HH24:MI:SS.FF'))))


As we can see above, a full table scan will occur if we do not have any indexes over the columns over which any rows will be filtered. After creating an index over the columns of significance, we can now analyze the performance again with "EXPLAIN PLAN FOR .." and the same query now executes with:

-------------------------------------------------------------------------------------------
| Id  | Operation     | Name           | Rows  | Bytes | Cost (%CPU)| Time      |
-------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT |              |   100 |  6400 |    10   (0)| 00:00:01 |
|*  1 |  COUNT STOPKEY     |              |      |      |           |      |
|*  2 |   INDEX FULL SCAN| PERF_DELIVERY_INFO_IDX |  7763 |   485K|    10   (0)| 00:00:01 |
-------------------------------------------------------------------------------------------

As we can see, a simple index created to correctly support a query can increase the performance many folds. In the example above, the table scan was replaced with a scan only over the index, which is much more efficient in CPU usage and time.

UltraESB 2.0.0 is released!

Yeah, its been quite sometime since we did a major release of the UltraESB! Although the last GA release was extremely stable and widely used, we've made significant improvements for the long awaited 2.0.0 release.

Since the beginning, we have been privileged to work closely with some of the best architects actually using our software in production. The suggestions and improvements proposed by them helped us implement key features that set the UltraESB apart. These were real requirements that meant a lot for large scale production deployments running 24 x 7 x 365 with zero downtime. For the 2.0.0 release, we've implemented many features that have been similarly inspired by the requirements of a top Fortune #10 company. These include a built-in metrics management and alerting functionality, and a programmatic instance management capability - coupled with annotation driven extensibility, especially for custom message interception at various stages.

Ruwan has already blogged about some of the metrics graphs generated by the UConsole in the 2.0.0 release, and you could find more documentation at http://docs.adroitlogic.org The 2.0.0 release also migrates to Apache HttpComponents/Core 4.2.4 and Spring framework / Spring security 3.1.1 and introduces a concept of a 'Deployment Unit' that can be deployed/updated or removed at runtime. Deployment units now has the ability to re-load dependency JAR files into the runtime as well, and support atomic updates as before, which allows all changes to be deployed into ESB nodes servicing client requests - without causing inconsistencies.

The Mediation API has also been cleaned up and streamlined with the introduction of 'Support Interfaces'. While being backwards compatible, the changes will help users utilize the UltraESB mediation API more easily.

Check out the full news release here, and refer to the documentation to get started!

AdroitLogic Announces API Management Solution Based On Its High Performance ESB

We've just announced the APIDirector! an API and Services management solution based on our high performance Enterprise Service Bus UltraESB.

One of the key differences of the APIDirector is that it will offer both API and Services management features for enterprises, including support for AS2 and other legacy/traditional B2B and service protocols.

We've announced results of the 6th Round of ESB Performance Benchmarking earlier in August, although I've missed blogging about it previously, as my father was ill during those few days. The benchmark results showed the extreme performance as well as the stability of the UltraESB, which are both key elements of any API management solution.

Since an API management solution will be the entry point for your trading partners, customers and users accessing your exposed APIs - it MUST be capable of withstanding extreme load, as well as deliberate security attacks, without crashing by itself. The Round 6 results and the related information shows how some of the ESBs fail to withstand even legitimate and relatively small amounts of loads, when compared to an external attack. 

The APIDirector performs functions such as credential management, service logging, auditing and performance management support, with an easy to use graphical administration interface that also provides analytical features. We also ship the AS2Gateway as an optional module of the APIDirector, and this allows users to deploy a custom AS2 trading gateway - similar to the AS2Gateway we have hosted publicly.

Next week we will be deploying the AS2Director at one of our beta customer sites in the US, and it will initially deal with defining S/FTP file exchanges and SOAP based services, as well as AS2 based B2B exchanges. The APIDirector will be generally available to the public during the first quarter of 2013, although we will be happy to work with customers with beta releases prior to the general availability. Please contact us to learn more about the APIDirector, and how you could participate in the beta!

Original News Release: http://www.prweb.com/releases/prweb9820781.htm

ESB Performance Benchmarking Round 6 is coming..

AdroitLogic has just announced the Sixth round of ESB Performance Benchmarking today!

Round 5 introduced a public Amazon EC2 AMI image of all the ESB's that were included into the benchmark, and the next round has moved further by requesting vendors and/or contributors for submission of better optimized configurations for the different ESBs to be tested.

Stability and Performance are both important to any ESB, and the last round saw half of the selected ESBs not being able to complete the benchmark successfully. Each ESB has since releases new versions, and we look forward to testing each one successfully!

For more information, visit http://esbperformance.org and email info@adroitlogic.com if you are interested to participate closely with us during this round!

ESB performance testing - Round 5 Results for 8 Open Source ESBs

AdroitLogic Private Ltd. announced today the results of the fifth round of ESB Performance testing. This round compares the WSO2 ESB, Mule ESB CE, Fuse ESB, Apache ServiceMix, Talend ESB, Petals ESB, JBoss ESB and the UltraESB. The ESB performance benchmark has been in use since June 2007 by vendors such as WSO2, AdoritLogic, Mulesoft and BEA to compare ESBs across different loads and message sizes for six common use cases. This round also publishes an EC2 AMI to easily reproduce the results by average users, and also publishes all source configuration for the 8 different ESBs in a public BitBucket repository

What's significant about this test suite is that it has now been accepted as a de-facto ESB performance test suite by vendors such as WSO2, BEA and Mulesoft in the past, who have reported results against their implementations publicly. This has also been used extensively by end users for ESB comparison, and load testing of configurations before production deployments.

Although 8 open source ESBs were considered, 4 of these failed without being able to complete the load test. One ESB had a 3-4% error rate, while three completed successfully. The test includes two variants of the free and open source UltraESB. The enhanced version is supercharged with the power of VTD XML which allows the UltraESB to perform XPath evaluation on XML payloads without XML parsing. This works nicely with the RAM Disk based file cache used by the UltraESB to provide both Zero-Copy and Java Non-Blocking IO for the extreme levels of performance.


Full information available at http://esbperformance.org

To build the best ESB, you have to select the best open source components too!

If you've ever wondered how we build the worlds best Free and Open Source ESB, in terms of performance, features and ease of use; then read about it all on Thanking great Open Source projects and teams!

To build the best Open Source ESB, we *had* to select; and only depend on; the best Open Source components - and not the old cupboards or the stinking kitchen sinks, that some vendors cannot yet entangle themselves from!

Did I mentioned that our complete distribution is just ~35MB? Compare it to some others who call 135MB upto ~1G ESB's as light. I guess after putting lipstick on the Pig, they now like to call it "light-weight" like a damsel too ;) But sadly, some are still stuck in the days of hub-and-spoke messaging, and some made the whole world around them too SOAPy and slippery too soon as they didn't take enough REST at the right time.

Being the youngest ESB in town, and to Win! - we had no choice but to select the best technologies and open source projects that we would depend on and use.

Introducing true Zero-Copy proxying, with memory mapped files and Non-Blocking IO was killer in performance. Some even tried to copy the keywords without implementing the code, and even without understanding what Zero-Copy was :)

We also introduced the concept of using Java (classes or just fragments) or any JSR 223 scripting language (such as Javascript, Groovy, Ruby) for mediation. Although many vendors falsely claim that their ESB is "configuration" only - the configuration language they use is no different from a programming language. However, someone new must learn that "XML"ish language to "program business logic" - and no one is going to tell him the equivalent of a try {} catch {} finally block that can be reliably used in that context :)

Our approach was to let the user decide the way he is going to mediate, using a language and/or technologies he is already aware of. But unlike some ESBs where you write Java code, then use some script to compile, bundle and deploy - the UltraESB just requires you to write the lines - and the compilation is hidden! You can even debug your mediation from within your favourite IDE - either IDEA, Eclipse or Netbeans.

When we wanted to introduce clustering almost an year back, we looked at Apache ZooKeeper and knew at first sight that it was the right one for us - call it link at first sight ;) !

Selecting a UI framework was the most time consuming.. and we looked at GWT based, and other frameworks, based on the servlet model but did not find the right balance. We wanted to use the best of HTML5/CSS3 and be in control always.

The combination of HTML5/CSS3/JQuery/datatables - connected via JSON using Pure - to a Wink REST application secured by Shiro and implemented via JMX with direct calls to the remote ESB was one of the main introductions in this last release! But it was all worth it! Except for a few issues on the IE browser (which statistically a lot fewer people use now) everything has been smooth. But we will follow up a 1.4.1 release with these fixes for IE too.

AdroitLogic announces v1.4.0 of the UltraESB

The latest version brings support for clustering using ZooKeeper, and introduces a web based administration console, and support for Caching, XACML and JSON among many other features.

Singapore - May 2, 2011 - AdroitLogic Private Ltd. announced today the release v1.4.0 of its free and open source Enterprise Service Bus, the UltraESB. The UltraESB is released under the OSI approved GNU Affero General Public License; as well as a zero-dollar non-GPL commercial license which allows unlimited and perpetual use free of charge. First released in January 2010, the UltraESB was the first ESB to utilize memory mapped files and zero-copy, coupled with non-blocking IO to provide extreme levels of performance.

Version v1.4.0 released today offers support for clustering and management of cluster nodes through the new Web based administration console based on JMX. Clustering support is built over Apache ZooKeeper, which is used to manage extremely large clusters of Hadoop nodes. Advanced features such as automatic restart of a complete cluster of nodes - with round-robin processing, or management of a service, or endpoint across a cluster have been made trivial operations. The new version also supports caching, and the current release ships support for ehCache. The new release also ships support for high performance XML processing, utilizing FastXML. This allows end users to integrate third party libraries such as VTD XML with the UltraESB for extremely fast XPath processing, without parsing XML payloads. XACML is supported for fine grained authorization using the PicketBox  XACML library, and automatic conversion to/from JSON to XML makes processing JSON payloads even simpler.

In addition to supporting many transports and message formats, the UltraESB also offers support to easily develop JSON based data services with just a single line of configuration, and provides full support for the HTTP transport, including true REST support, basic, digest, NTLM and Amazon S3 authentication using pre-emptive authentication or challenge response mechanisms as applicable. The UltraESB also ships a very high performance WS-Security library that allows it to perform WS-Security operations over 3X faster than solutions based on the WSS4J/Rampart libraries used by other ESBs.

JMX support has been thoroughly improved to utilize MXBeans, and allows detailed monitoring and management of a runtime using JMX consoles such as Zabbix. The UltraESB now ships with a native Zabbix agent for even easier monitoring. As the UltraESB integrates with the Spring framework and supports Java as well as JSR 223 scripting languages, integration with third party libraries and extension is simple and powerful.

WSO2 falsely claims support for Zero copy

A few minutes ago, Asanka Abeysinghe the Director of Solutions Architecture of WSO2 claimed that the WSO2 ESB supports "Zero Copy", at the WSO2Con held in Colombo, 14th September 2010

However, this statement is false, and can be easily proved by just looking at the implementation source code of the BinaryRelayBuilder, which copies the complete input stream into a byte[] in Heap memory via traditional CPU copying into user space.

In-fact, when WSO2 announced this "Binary Relay" in December 2009, even the author of this "feature" did not state that its anything near "Zero Copy". Even a search on Google for WSO2 and Zero copy returns no results!

"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another." - Wikipedia

The AdroitLogic UltraESB is the FIRST and still the ONLY Open Source ESB (or most possibly "the only ESB") that supports True Zero-Copy proxying.

Thats why its the fastest ESB out there!

AdroitLogic Open Sources the High Performance and Lightweight Enterprise Service Bus - UltraESB

UltraESB is the first Open Source Enterprise Service Bus (ESB) to utilize Zero-copy proxying with Memory Mapped files and Java Non-Blocking IO for extreme performance. 

AdroitLogic Private Ltd. announced today that it is open sourcing the code of its Enterprise Service Bus the UltraESB, under the GNU Affero General Public License. The UltraESB first announced in January, becomes the first Open Source ESB to claim support for Zero-copy proxying of requests with Java Non-blocking IO and Memory Mapped files to support extreme levels of performance. AdroitLogic also released the code of a Web Services Security library 'SoapBox', that was custom built by them for use by the UltraESB, which performs about 4 times faster than solutions based on the Apache WSS4J/Rampart frameworks. This support combined with SSL, HTTP Basic/Digest and other authentication methods makes the UltraESB an ideal platform to implement a Security Gateway for hosted services.

The UltraESB supports legacy transports such as JMS, File, S/FTP, FTPS, Email(POP3/IMAP/SMTP), Database, B2B AS2 (Applicability Statement 2), MLLP/S and TCP/S in addition to HTTP/S, and handles messages in their native formats without any conversions into an intermediate format. It supports REST, EDI, SOAP 1.1/1.2, POX/XML, Hessian, JSON, Health Level 7 (HL7), Binary, Text, CSV, HTML and many other message formats. The UltraESB supports JTA XA transactions, including asynchronous suspend and resume, and can be deployed standalone or within a JEE Servlet container such as Tomcat.


"The UltraESB is a complete new ESB written from scratch. Thus we were able to improve the performance and ease of use phenomenally by using techniques such as Java 6 support for compilation, NIO and Zero-copy proxying with Memory Mapped files. We base the core engine on a very few and stable libraries, and are extremely careful about good design and coding practices” said Asankha Perera the founder of AdroitLogic. Asankha earlier contributed most of the code of the Apache Synapse ESB which formed a basis for other open source ESBs as well as Security Appliances.

The UltraESB includes a Performance Benchmarking Framework to compare its performance against other ESBs on identical conditions. More information about this framework can be found at the ESB Performance website.
The UltraESB is configured with one or more XML files that allows mediation code to be written as fragments of Java, or JSR 223 Scripting languages - such as Javascript, Groovy, Ruby etc. Mediation logic can also be written as Java classes or POJOs, without depending on any specific interfaces. This ensures that the users do not have to learn yet another programming or XML configuration language; or compile-bundle-and-deploy artifacts for configuration changes to take effect.

"The UltraESB is very easy to use. The learning curve is minimal since it's based on Java and Scripting languages. The support is great, and the performance, exceptional!" said Shalinda Ranasinghe, Chief Software Architect of ShipXPress Inc. The configuration could be performed within an IDE of the users' choice that supports intelligent context aware auto completion, and step-through debugging. The UltraESB supports JMX management and reporting with standard JMX consoles, and is about 30MB to download inclusive of many samples.

Availability and Support
The UltraESB is available for download under the GNU Affero General Public License (AGPL); and under a Free Commercial License allowing unlimited and perpetual use. Custom paid licensing options are available for users who wish to embed the UltraESB into proprietary applications. AdroitLogic offers world-wide production support, consultancy, training, development support and custom development for the UltraESB.

A 5 Minute Screencast to get Started!

Enterprise Service Bus UltraESB challenges open source and commercial equivalents on performance and ease of use

Singapore - May 17, 2010 - AdroitLogic Private Ltd. announced today the general availability of its high performance Enterprise Service Bus (ESB) UltraESB, available free of charge for perpetual and unlimited production use. AdroitLogic also launched a website dedicated to ESB performance testing - http://esbperformance.org, that hosts a benchmark used by multiple ESB vendors since June 2007.

The UltraESB utilizes Zero-copy proxying of messages using Non-Blocking IO, to scale and support extremely high numbers of concurrent connections over HTTP/S. By moving away from a canonical message format, the UltraESB natively supports multiple types of payloads over transports such as HTTP/S, JMS, File, S/FTP, FTPS, MLLP/S, TCP/S and Email (POP,IMAP,SMTP). In addition, it can act as a B2B AS2 (Applicability Statement 2) connector to integrate backend systems securely with trading partners over the Internet, and as a Health Level 7 (HL7) systems integrator over multiple transports. By developing a custom WS-Security implementation, it offers much better performance when used as a WS-Security gateway than solutions based on the WSS4J/Rampart libraries. Fully supporting REST, SOAP, JSON, XML, Binary, Hessian, EDI, Text, HTML etc as payloads, the UltraESB ships with a load of ready-to-run samples, tools & documentation and bundles everything into a download of 30MB. The UltraESB can be deployed standalone or over a JEE application server, and supports both JTA XA and local transactions - including suspend and resume - even asynchronously.

The UltraESB is configured with an XML file that allows mediation code to be written as Java or [JSR 233] scripting language (e.g. Javascript, Groovy, Ruby..) snippets. Mediation can also be written as Java classes, or POJOs without depending on any specific interfaces. The UltraESB does not require the user to learn a new programming or configuration language; or compile-bundle-and-deploy artifacts for configuration changes. The configuration could be performed with an IDE of the users' choice that supports intelligent context aware auto completion, and allows step-through-debugging and unit testing to ensure quality. The UltraESB supports JMX management and reporting with any JMX console, and could even be used as a high performance load balancer in front of Tomcat instances etc to provide sticky loadbalancing and failover.

About AdroitLogic

AdroitLogic Private Ltd. is a privately owned, innovative technology company based in Singapore. AdroitLogic believes in working closely with real enterprise users of its software, and thus invites them to participate in defining its road map, schedule, features & releases. Becoming a parter with AdroitLogic, offers users access to its source code.

The ESB Performance Testing Framework and Execution Round 4 - explained in detail

I've just published an article that explains the ESB Performance Test Framework in detail, and how an end user could compare an ESB of his choice to any other ESB.

http://adroitlogic.org/samples-articles-and-tutorials/16-articles/50-the-esb-performance-testing-framework-and-execution-round-4-explained-in-detail.html

PS: I was also able to resurrect this blog post of the Product Manager of the BEA AquaLogic ESB - Dain Hansen - from June 2008, which states that even BEA ran the ESB Performance Test Framework back then..

The ESB Performance Testing Framework & Results of Round 4

I developed the original ESB Performance Test Framework - in June 2007 while actively working on the Apache Synapse and the WSO2 ESBs. Since then, we've run three rounds of testing, comparing both Proprietary and Open Source ESBs - including Mule, Apache ServiceMix, Apache Synapse/WSO2 ESB, a leading proprietary ESB and the proprietary version of an open source ESB.

It was interesting to see other vendors such as Mule and BEA picking up this test framework in addition to WSO2, in publishing results. However, due to different hardware configurations being used, and other advanced tuning or optimizations performed by vendors - the results were left a bit questionable and could not be compared in a fair manner.

This latest round - Round 4 - takes this framework to the Amazon EC2 - and is designed to let end-users run the tests on an EC2 node for less than $2 of computing time! This will allow differences in hardware to not make any difference, and allow the users to see how exactly the tests have been configured, tuned and run.

I hope Mule, ServiceMix, JBoss, OpenESB, Petals, BEA/Oracle, IBM, WSO2 and any other ESB vendors I've missed will make use of this opportunity and publicly share the necessary configurations so that we could all rely on an accurate and fair benchmark - which can be verified independently on demand by any end-user!

Hence - unlike in the past, I will refrain from naming any competitor - unless its vendor or the open source project team requests that it be included and the results publicly shared. Thus it is now left to the users of ESBs - to demand the configurations necessary for them to run these tests from vendors - and themselves decide which one to select after executing the test on Amazon EC2. I hope to make this process even simpler at the next round - with a custom AMI and more automation

The Round 4 results includes the three test cases conducted earlier (i.e. Direct Proxy, Content Based Routing [CBR] Proxy, and XSLT Proxy), and adds a new scenario for WS-Security processing.

http://adroitlogic.org/samples-articles-and-tutorials/15-tutorials/48-esb-performance.html

HTTP client supporting REST, SOAP, Hessian etc for testing and Load testing!

The AdroitLogic UltraESB includes the ESB and SOA ToolBox - that can be used to test any payload over HTTP/S. Supporting all methods over HTTP such as GET, POST, PUT, DELETE, HEAD and OPTIONS, it allows a payload pasted into a text area, or read as a file to be sent to any HTTP/S endpoint, optionally authenticating the request with HTTP basic or digest authentication.

The user can select if the request should use HTTP 1.0 or 1.1, or send the entity as chunked or using 'Content-Length' encoding, if the '100-Continue' expect handshake should be used-or-not, if HTTP KeepAlive should be used or not, and if response  compression (i.e. GZip) is requested. If the request is over SSL, you could also turn off SSL Trust and Hostname verification for testing purposes - such as when using test certificates. Additionally, it can also use an identity certificate if 2-way SSL is requested.

To make this even better, the ToolBox also offers a HTTP/S Load test version of this same utility - that can be used to load test REST, SOAP, Hessian or any kind of payload over HTTP/S. This Load Test utility is a clone of the popular Apache Bench load test client - but much more advanced and stable!

The ToolBox also includes a TCP Dump utility - that can be used to view requests over the wire - in both HEX and Text formats. It additionally introduces support to capture a request into a file - which allows one to capture any kind of request message - even WS-Secured messages, Hessian binary messages etc, which can then be used with the JavaBench - to run a load test against your services.

For those who really knows what they are doing, and for example if you would like to test your ESB or SOA server for its support against malicious or malformed requests - there is a Raw socket client.

This allows one to hand-craft a HTTP request as you wish! The ToolBox also hosts a module that will start one or more Jetty servers with sample applications, which could then be used to test your ESB or SOA services. Starting multiple instances over different ports allows one to easily test load balancing and fail-over options of your ESB.

Finally the ToolBox allows you to start an instance of the UltraESB - the ultimate ESB that is simple to use, but supports ultra performance! Do read the full article "Getting started with the AdroitLogic ToolBox for the UltraESB" and follow through the end-to-end sample to get started.