AdroitLogic Announces Major New Release of UltraESB with an Enterprise Integration Platform and Developer Studio

A short while ago, we announced a major update to the UltraESB Enterprise Service Bus, called the UltraESB-X release. This is a rewrite of our high performance UltraESB Enterprise Service Bus run-time, on top of a new modular framework code-named Project-X. The new release retains the performance benefits introduced by the UltraESB, supporting HTTP/S exchanges with zero-copy data transfers and Non-blocking IO (NIO). This performance edge prompted Walmart to license our technology in 2013 to power its new integration platform; and for Kuoni GTA to support over half a billion requests per day, with its B2B travel API, since 2014.
 We also released the UltraStudio, a rich integration solution development studio to increase developer productivity. It enriches the end-to-end experience of integration development with powerful extensions and plugins to the popular IntelliJ IDEA IDE. The UltraStudio allows users to develop integration flows with a drag-and-drop palette of connectors and processors, and helps to test, debug and trace integration flows through the execution path, and view properties at each stage. The palette of connectors and processors keeps increasing, and its a simple effort to develop custom connectors and message processors, by end users. The YouTube video linked from Sajiths blog shows this in action.

The new Integration Platform announced today, offers integration capabilities as a service, over private, public or hybrid cloud based Docker containers, utilizing the Kubernetes framework for managing the run-time and availability. The Integration Platform becomes available as an on-demand service to multiple organizational units within an enterprise, allowing users to target different UltraESB-X deployments with varying availability requirements, over a set of shared physical resources. The platform manages the deployments and automatically repairs failed instances, guaranteeing availability and stability. In the event of a container failure, the incarnation details of the instance is retained to allow troubleshooting of the root cause, and the container instances thus retain state in an otherwise stateless container deployment to facilitate management. The metrics of each UltraESB-X runtime are reported to ElasticSearch which then powers a consolidated management and monitoring dashboard that manages the platform.

Try out the new products from our website, and the integration resources, samples and documentation from the new Developer site at http://developer.adroitlogic.org

Happy 7th Birthday AdroitLogic and UltraESB!

Today marks 7 years since the first public announcement from AdroitLogic about its core product, the UltraESB Enterprise Service Bus!

We've completed yet another year of success, and we are very close to releasing our upcoming products for 2017, the new UltraStudio with the UltraESB-X runtime, and the Integration Platform Server (IPS). The UltraStudio is a new Integrated Development Environment for the development, testing and deployment of integration flows into the new UltraESB-X runtime. The graphical user interface based on the worlds best IDE, IntelliJ IDEA, allows users to implement integration solutions faster and easier. The Integration Platform (IPS), based on Kubernetes and Docker, is able to deploy highly available integration services across a cluster, managed as a collection of lightweight containers. The services can be deployed on private, public and hybrid clouds as well. We will soon be releasing these new products, with an update to our web site and documentation, and also introduce a new developer site to help integration engineers use our products and solutions more easily.

During the past year, we've also increased our team size, and five more engineers have confirmed employment with us within next few months. And as we planned an year back, we successfully purchased our own property in March last year, and will now embark on setting up and moving into our own new office this year.

We look forward to yet another exciting year ahead, and believe that our efforts have enhanced the solutions for Enterprise Integration challenges, to many organizations across the world!

AdroitLogic and the UltraESB is 6 years!

The 19th of January 2016 marks 6 years since the UltraESB was first announced publicly; It also marks the completion of six years of success for AdroitLogic!

The last year was an exciting one, with our team size increasing more than two fold.. As a side effect, we've now run out of space and are actively looking for alternate locations to move into; with one of the possibilities being to purchase our own property and build our office the way we want.

On the technical front, we've developed the Enterprise Middleware Framework (EMW) last year, and deployed it in production, along with the IMonitor enhanced management console with detailed statistics and management capabilities with an Elastic Search back-end. Our core products are now available with Ansible based scripts for auto installation and configuration, including the configuration of large clusters - making the deployments easier for our customers. These enhancements will be made available to more customers this year, and detailed information about the EMW Framework, IMonitor and the Ansible scripts made publicly available.

The APIDirector and the AS2Gateway too have kept gaining traction over the past year, and we've now started investing in our Integration Platform as a Service endeavor, which is based on Containers. This would be the key product to be introduced during this coming year, along with our next major update with a secret project, currently code named 'X'.

We plan to start more aggressive marketing and sales activities, with enhanced web sites and documentation, and introduce a new Customer Portal to assist our customers with more valuable content and up-to date information. It will certainly be an exciting year ahead!

Connecting to local Coherence instance with WKA instead of Multicast

I have been trying to setup a test environment for a local Coherence cache implementation for sometime, and the information found via Google was not that helpful in understanding why the client application (in this case the UltraESB) was not connecting to the already created Coherence cluster using WKA, and was trying to create its own cluster with Multicast.

The System properties I passed to the JVM included the following, but the issue was not necessarily on these as I found out with my limited knowledge of Coherence.

The key lesson was - the System properties being correct was not enough for WKA to function properly!

wrapper.java.additional.13=-Dtangosol.coherence.cluster=adrt-cluster
wrapper.java.additional.14=-Dtangosol.coherence.management=all
wrapper.java.additional.15=-Dtangosol.coherence.cacheconfig=coherence/xxxx-cache-config.xml
wrapper.java.additional.16=-Dtangosol.coherence.localhost=127.0.0.1
wrapper.java.additional.17=-Dtangosol.coherence.localport=15000
wrapper.java.additional.18=-Dtangosol.coherence.wka.address=127.0.0.1
wrapper.java.additional.19=-Dtangosol.coherence.wka.port=15000
wrapper.java.additional.20=-Dtangosol.coherence.ttl=0
wrapper.java.additional.21=-Dtangosol.coherence.distributed.localstorage=false

The solution was to create a new file "tangosol-coherence-override.xml" and place it in the classpath (make sure its at the start to get picked up before any others - if any)

<?xml version='1.0'?>
<coherence  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config"
            xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-operational-config coherence-operational-config.xsd">    

  <cluster-config>
    <unicast-listener>
            <well-known-addresses>
                <socket-address id="1">
                        <address>127.0.0.1</address>
                        <port>15000</port>
                </socket-address>
        </well-known-addresses>
        <address>127.0.0.1</address>
        <port>15000</port>
    </unicast-listener>
  </cluster-config>

  <logging-config>
    <severity-level system-property="tangosol.coherence.log.level">9</severity-level>
    <character-limit system-property="tangosol.coherence.log.limit">0</character-limit>
  </logging-config>
</coherence>

If your client "connects" to the WKA cluster you will see the following, and you can notice that it connected to an already existing cluster with a different process

WellKnownAddressList(Size=1,
  WKA{Address=127.0.0.1, Port=15000}
  )
MasterMemberSet(
  ThisMember=Member(Id=3, Timestamp=2015-08-18 12:24:32.153, Address=127.0.0.1:15002, MachineId=60314, Location=site:,machine:localhost,process:5634)
  OldestMember=Member(Id=1, Timestamp=2015-08-18 12:24:07.652, Address=127.0.0.1:15000, MachineId=60314, Location=site:,machine:localhost,process:5366, Role=CoherenceServer)

And if it didn't connect with WKA, and created its own Multicast group you would see the following, where you can also notice that the client you just started is the only member of that new cluster

Group{Address=224.12.1.0, Port=12100, TTL=0}
MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2015-08-18 10:25:08.389, Address=127.0.0.1:15002, MachineId=60314, Location=site:,machine:localhost,process:4278)
  OldestMember=Member(Id=1, Timestamp=2015-08-18 10:25:08.389, Address=127.0.0.1:15002, MachineId=60314, Location=site:,machine:localhost,process:4278)

AdroitLogic Recognized in DZone’s 2014 Guide to Enterprise Integration

We are very excited to be recognized as a featured vendor in DZone’s 2014 Guide to Enterprise Integration, a premium resource focused on enterprise integration and API management trends, strategies, and tools. The guide includes topic introductions, expert opinions, best practices, and solution comparisons. 

Readers of the guide will get an overview of enterprise integration and learn about obstacles that developers are facing to create seamless integration. Topics covered by the guide include:

• The role of message queues, middleware, and ESBs in the enterprise.
• Decomposition patterns for breaking down monolithic architecture.
• A model for understanding the maturity level of REST APIs.
• A forecast of how building a large project with multiple integrations might look in the future.

DZone’s Enterprise Integration guide also offers key insights into integration and API management practices through a survey of 500+ developers and experts, allowing readers to learn trends from practitioners in the technology professional community. Additionally, the guide’s solutions directory compares different API management platforms, integration suites, ESBs, message queues, and integration frameworks to help readers wisely choose the solutions they need.

Readers can download a free copy of the guide here:

About DZone

DZone provides expert research and learning communities for developers, tech professionals, and smart people everywhere. DZone has been a trusted, global source of content for over 15 years.

ESB Performance Testing - Round 7 Published!

We have just published the results for the long awaited Round 7 - of ESB performance benchmarking. This round compares 4 of the leading free and open source ESBs, Mule CE 3.4.0, Talend ESB SE 5.3.1, WSO2 ESB 4.7.0 and the UltraESB 2.0.0

Although at first glance the above image may indicate that the WSO2 ESB and the UltraESB have very similar performance characteristics, the Devil is indeed in the detail.

During the testing we discovered several issues with the previously published article Round 6.5 from WSO2, including a severe response corruption for messages over 16,384 bytes when the default pass-through transport is being used. However, what's most surprising is that this issue remains in versions 4.6.0 and 4.7.0 of the WSO2 ESB, as well as the latest Milestone 4 of the soon to be released version 4.8.0.

Sadly, the WSO2 engineers conducting the Round 6.5 also failed to notice a complete failure of all of the XSLT test cases, but nevertheless published numbers obtained for the failed test cases as high performance numbers over the other ESBs.

Read about these and other flaws of the Round 6.5 in the article Why the Round 6.5 results published by WSO2 is flawed

UltraESB 2.0.0 is released!

Yeah, its been quite sometime since we did a major release of the UltraESB! Although the last GA release was extremely stable and widely used, we've made significant improvements for the long awaited 2.0.0 release.

Since the beginning, we have been privileged to work closely with some of the best architects actually using our software in production. The suggestions and improvements proposed by them helped us implement key features that set the UltraESB apart. These were real requirements that meant a lot for large scale production deployments running 24 x 7 x 365 with zero downtime. For the 2.0.0 release, we've implemented many features that have been similarly inspired by the requirements of a top Fortune #10 company. These include a built-in metrics management and alerting functionality, and a programmatic instance management capability - coupled with annotation driven extensibility, especially for custom message interception at various stages.

Ruwan has already blogged about some of the metrics graphs generated by the UConsole in the 2.0.0 release, and you could find more documentation at http://docs.adroitlogic.org The 2.0.0 release also migrates to Apache HttpComponents/Core 4.2.4 and Spring framework / Spring security 3.1.1 and introduces a concept of a 'Deployment Unit' that can be deployed/updated or removed at runtime. Deployment units now has the ability to re-load dependency JAR files into the runtime as well, and support atomic updates as before, which allows all changes to be deployed into ESB nodes servicing client requests - without causing inconsistencies.

The Mediation API has also been cleaned up and streamlined with the introduction of 'Support Interfaces'. While being backwards compatible, the changes will help users utilize the UltraESB mediation API more easily.

Check out the full news release here, and refer to the documentation to get started!

Does Tomcat bite more than it can chew?

This is an interesting blog post for me, since its about the root cause for an issue we saw with Tomcat back in May 2011, which remained unresolved. Under high concurrency and load, Tomcat would reset (i.e. TCP level RST) client connections, without refusing to accept them - as one would expect. I posted this again to the Tomcat user list a few days back, but then wanted to find out the root cause for myself, since it would surely come up again in the future.

Background

This issue initially became evident when we ran high concurrency load tests at a customer location in Europe, where the customer had back-end services deployed on multiple Tomcat instances, and wanted to use the UltraESB for routing messages with load balancing and fail-over. For the ESB Performance Benchmark, we had been using an EchoService written over the Apache HttpComponents/Core NIO library that scaled extremely well and behaved well at the TCP level, even under load. However, at the client site, they wanted the test run against real services deployed on Tomcat - to analyse a more realistic scenario. We used a Java based clone of ApacheBench called the 'Java Bench' which is also a part of the Apache HttpComponents project, to generate load. The client would go up-to concurrency levels of 2560, pushing as many messages as possible through the ESB, to back end services deployed over Tomcat.

Under high load, the ESB would start to see errors while talking to Tomcat, and the cause would be IO errors such as "Connection reset by peer". Now the problem to the ESB is that it had already started to send out an HTTP request / payload over an accepted TCP connection, and thus it does not know if it can fail-over safely by default to another node, since the backend service might have  performed some processing over the request it may have already received. Of-course, the ESB could be configured to retry on such errors as well, but our default behaviour was to fail-over only on the safer connection refused or connect timeout errors (i.e. a connection could not be established within the allocated time) - which ensures correct operation, even for non-idempotent services.

Recent Observations

We recently experienced the same issue with Tomcat when a customer wanted to perform a load test scenario where a back-end service would block for 1-5 seconds randomly, to simulate realistic behaviour. Here, again we saw that Tomcat was resetting accepted TCP connections, and we were able to capture this with Wireshark as follows, using JavaBench directly against a Tomcat based servlet

As can be seen in the trace, the client initiated a TCP connection with the source port 9386, and Tomcat running on port 9000 accepted the connection - note “1”. The client kept sending packets of a 100K request, and Tomcat kept acknowledging them. The last such case is annotated with note “2”. Note that the request payload was not complete at this time from the client – note “3”. Suddenly, Tomcat resets the connection – note “4”

Understanding the root cause

After failing to locate any code in the Tomcat source code that resets established connections, I wanted to simulate the behaviour with a very simple Java program. Luckily the problem was easy to reproduce with a simple program as follows:

import java.net.ServerSocket;
import java.net.Socket;

public class TestAccept1 {

    public static void main(String[] args) throws Exception {
        ServerSocket serverSocket = new ServerSocket(8280, 0);
        Socket socket = serverSocket.accept();
        Thread.sleep(3000000); // do nothing
    }
}

We just open a server socket on port 8280, with a backlog of 0 and start listening for connections. Since the backlog is 0, one could assume that only one client connection would be allowed - BUT, I could open more than that via telnet as follows, and even send some data afterwards by typing it in and pressing the enter key.

telnet localhost 8280
hello world

A netstat command now confirms that more than one connection is opene:

netstat -na | grep 8280
tcp        0      0 127.0.0.1:34629         127.0.0.1:8280          ESTABLISHED
tcp        0      0 127.0.0.1:34630         127.0.0.1:8280          ESTABLISHED
tcp6       0      0 :::8280                 :::*                    LISTEN    
tcp6      13      0 127.0.0.1:8280          127.0.0.1:34630         ESTABLISHED
tcp6      13      0 127.0.0.1:8280          127.0.0.1:34629         ESTABLISHED

However, the Java program has only accepted ONE socket, although at the OS level, two would appear. It seems like the OS also allows more than two connections to be opened, even when the backlog is specified as 0. On Ubuntu 12.04 x64, the netstat command would not show me the actual listen queue length - but I believe it was not 0. However, before and after this test, I did not see a difference in the reported statistics for "listen queue" overflow, which I could see with the "netstat -sp tcp | fgrep listen" command

Next I used the JavaBench from the SOA ToolBox and issued a small payload at concurrency 1024, with a single iteration against the same port 8280

As expected, all requests failed, but my Wireshark trace on port 8280 did not detect any connection resets. Pushing the concurrency to 2560 and the iterations to 10 started to show tcp level RSTs - which were similar to those seen on Tomcat, though not exactly the same.

 

Can Tomcat do better?

Yes, Possibly .. What an end user would expect from Tomcat is that it refuses to accept new connections when under load, and not to accept connections and then reset them halfway through. But one would ask if that is achievable? Especially considering the behaviour seen with the simple Java example we discussed.

Well, the solution could be to perform better handling of the low level HTTP connections and the sockets, and this is already done by the free and open source high performance Enterprise Service Bus UltraESB, which utilizes the excellent Apache HttpComponents project underneath.

How does the UltraESB behave

One could easily test this by using the 'stopNewConnectionsAt' property of our NIO listener. If you set it to 2, you wont be able to even open a Telnet session to the socket beyond 2.

The first would work, the second too

But the third would see a "Connection refused"

And the UltraESB would report the following on its logs:

  INFO HttpNIOListener HTTP Listener http-8280 paused  
  WARN HttpNIOListener$EventLogger Enter maintenance mode as open connections reached : 2

Although it refuses to accept new connections, already accepted connections executes without any hindrance to completion. Thus a hardware level load balancer in front of an UltraESB cluster can safely load balance if an UltraESB node is loaded beyond its configured limits, without having to deal with any connection resets. Once a connection slot becomes free, the UltraESB will start accepting new connections as applicable.

Analysing a corresponding TCP dump

To analyse the corresponding behaviour, we wrote a simple Echo proxy service on the UltraESB, that also slept for 1 to 5 seconds before it replied, and tested this with the same JavaBench under 2560 concurrent users, each trying to push 10 messages in iteration.

Out of the 25600 requests, 7 completed successfully, while 25593 failed, as expected. We also saw many tcp level RSTs on the Wireshark dump - which must have been issued by the underlying operating system.

However, what's interesting to note is the difference - the RSTs occur immediately on receiving the SYN packet from the client - and are not established HTTP or TCP connections, but elegant "Connection Refused" errors - which would be what the client can expect. Thus the client can safely fail-over to another node without any doubt, overhead or delay.

Appendix : Supporting high concurrency in general

During testing we also saw that the Linux OS could detect the opening of many concurrent connections at the same time as a SYN flood attack, and then start using SYN cookies. You would see messages such as 
-->

Possible SYN flooding on port 9000. Sending cookies

displayed on the output of a "sudo dmesg", if this happens. Hence, for a real load, it would be better to disable SYN cookies by turning it off as follows as the root user
-->

# echo 0 > /proc/sys/net/ipv4/tcp_syncookies

To make the change persist over reboots, add the following line to your /etc/sysctl.conf
-->

net.ipv4.tcp_syncookies = 0

To allow the Linux OS to accept more connections, its also recommended that the 'net.core.somaxconn' be increased - as it usually defaults to 128 or so. This could be performed by the root user as follows,
-->

# echo 1024 > /proc/sys/net/core/somaxconn

To persist the change, append the following to the /etc/sysctl.conf
-->

net.core.somaxconn = 1024

Kudos!

The UltraESB could not have behaved gracefully without the support of the underlying Apache HttpComponents library, and the help and support received from that project community, especially by Oleg Kalnichevski - whose code and help has always fascinated me!

Electronic Invoicing Announced on the AS2Gateway

We've just announced support for electronic invoicing on the cloud based Free B2B Trading Gateway AS2Gateway! This allows users to invoice trading partners via EDIFACT INVOIC messages over AS2, using a simple and intuitive web interface

Recurrent invoicing is simplified as any invoice can be saved as a template and re-used multiple times. The AS2Gateway converts all the invoice details into an EDIFACT D93A INVOIC message right now, and soon will support the other versions, as well as X12 based messages such as the 810.

In the near future, the platform will add support for parsing and generation of more message types, which will allow users to easily generate an invoice based on a purchase order received; or an advanced shipping notification (ASN) etc.

The AS2Gateway offers a free tier to support most SMEs that are required to electronically invoice trading partners for payment. For larger users, a premium tier is available with advanced options (to be announced shortly!), and for retailers or large corporations, the AS2Gateway is available for on-premise private deployment.

UltraESB Automates AS2/EDI Based B2B Trading for SKB Europe

We've just published a new case study on how SKB Europe, a leader in high-quality aluminium and plastic cases and boxes, has deployed the UltraESB as an AS2/EDI integration platform for Business-to-Business (B2B) integration.

The system initially integrates the ERP system of SKB Europe with one of its key trading partners Amazon, via EDIFACT based EDI documents exchanged over the AS2 protocol.

Read the news release and Case study from here
http://www.prweb.com/releases/2012/7/prweb9706746.htm

ESB Performance Benchmarking Round 6 is coming..

AdroitLogic has just announced the Sixth round of ESB Performance Benchmarking today!

Round 5 introduced a public Amazon EC2 AMI image of all the ESB's that were included into the benchmark, and the next round has moved further by requesting vendors and/or contributors for submission of better optimized configurations for the different ESBs to be tested.

Stability and Performance are both important to any ESB, and the last round saw half of the selected ESBs not being able to complete the benchmark successfully. Each ESB has since releases new versions, and we look forward to testing each one successfully!

For more information, visit http://esbperformance.org and email info@adroitlogic.com if you are interested to participate closely with us during this round!

We've just launched our Free EDI/AS2 B2B Gateway - AS2Gateway.ORG

Since its launch in January 2010, the UltraESB has the only free and open source ESB that natively supported the widely used B2B trading protocol AS2 (Applicability Statement 2). AS2 is used extensively in the US, Europe and Asia for B2B integration especially in the Retail and Manufacturing industries, and the UltraESB has been used by many organizations to integrate EDI/AS2 systems with S/FTP, Web Services and XML style systems.

The UltraESB is now used to integrate with some of the largest retailers in Europe, and the US by multiple organizations and it was natural for us to see the potential in offering a Cloud hosted EDI/AS2 solution as a Service (SaaS).

The AS2Gateway is powered by our flagship product UltraESB, and offers free EDI/AS2 connectivity for SME's and certainly disrupts the traditional market place in both the features offered and the cost savings. For example, the free tier allows SMEs to trade electronically with upto 5 trading partners and offers 60 messages per month.

We've also just launched AS2Integration.ORG which is a resource site on AS2/EDI integration, as well as the documentation and user guide for the use of the AS2Gateway.

Utilizing the UltraESB underneath means that the flexibility is virtually limitless, and we will soon expand this service to include Electronic Invoicing, EDI document creation and processing as well!

The AS2Gateway begins its 'beta' testing phase today, and you can utilize all of its features for free until we move into production in a few more months! No credit cards are necessary for sign up - so get your AS2/EDI trading account from the Free AS2Gateway!

Read our Press Release about the launch for more details!

Digital Content and Service Provider Jesta Digital Migrates to the UltraESB

Zero-Copy proxying and Non-Blocking IO allows three node cluster to process ~80 million messages a day at just 5% CPU utilization and 300MB heap memory usage

AdroitLogic, developer of the high performance Open Source Enterprise Service Bus (ESB) UltraESB, today announced that Jesta Digital, a leading global provider of next generation entertainment content and services for the digital consumer, successfully migrated their ESB clusters to the UltraESB. Jesta Digital is behind Bitbop wireless subscription service to deliver on-demand commercial-free television and films to personal computers, tablets and smartphones, and home to a number of well-known and established brands including Jamba and Jamster, which deliver branded content, music, games and apps to millions of mobile consumers.

"In a very demanding and rapidly changing market it is of utmost importance to be able to quickly adjust the technical platform to support product innovation and change. The UltraESB was able to demonstrate to the Jesta Digital Technology team in Berlin that its simplicity, testability, extensibility and performance is made for a matching foundation" said Eric Hubert, the Executive Director of Strategy and Architecture at Jesta Digital.

On the main ESB cluster, Jesta Digital processes around 80 million messages a day with a peak 3,000 TPS across three nodes. The average CPU utilization has been just 5% with the heap memory usage at 300MB with very low GC overheads due to the efficient use of a RAM disk coupled with Zero-Copy and Non-Blocking IO. Prior to the migration, Jesta Digital customized the ESB Performance Benchmark (http://esbperformance.org) to compare the performance of the UltraESB for both SOAP and Hessian messages over HTTP/S for a sample scenario. Benchmark results similar to those published recently on the ESB Performance site were independently verified by Jesta Digital during this exercise. Furthermore the UltraESB showed extreme stability during stress tests executed over multiple days.

"We have a top notch technical team at Jesta Digital in Berlin and it was a great pleasure to work with subject-matter experts who combined first-hand knowledge, passionate work on their product and dedication to tackle the customer's challenges" said Eric.

During the migration process, AdroitLogic also worked with the Jesta Digital team to include some key features for even better enterprise deployment support. Among these is the ability to switch a live configuration with zero down time while messages are being processed. Previously servers had to be gracefully shutdown for updates, resulting in an unequal distribution of traffic across the nodes after a configuration switch, leading to connection related issues. In addition, AdroitLogic developed a feature to easily manage and monitor services and endpoints via automatic registration against a Zabbix monitoring server, using predefined or customized templates via JMX.

The UltraESB is clustered using the Apache ZooKeeper framework, and thus a single node, or the whole cluster can be managed from any of the nodes via a remote web based console, command line interface or via JMX/jconsole.

AdroitLogic was founded in January 2010 by Asankha Perera, the former lead contributor of the Apache Synapse ESB and the original architect of the WSO2 ESB. In September 2011, Ruwan Linton who initially succeeded Asankha at WSO2 as its next ESB architect, also joined AdroitLogic as its Director of Engineering. AdroitLogic published a new round of ESB benchmark results in October across 8 open source ESBs, which showed a very clear lead in performance against its competitors. In addition to performance, the UltraESB is also one of the simplest ESB's to use effectively, with support for IDE based step-through debugging, Unit testing and a light weight deployment with over 70 samples. AdroitLogic is also one of the very few - if not the only - ESB vendors that publishes code coverage statistics along with continuous builds and code quality reports publicly.

"It was a great pleasure to work with subject-matter experts who combined first-hand knowledge, passionate work on their product and dedication to tackle the customer's challenges"

Read the full Press Release and Download the Case Study on Migration to the Free and Open Source UltraESB

ESB Performance Testing.. Getting ready for the next round

Its been sometime since we've compared the performance of free and open source ESBs.. and now its time for the next round..

Amazon EC2 AMI to be published!
At the end of this round, we will publish an Amazon EC2 machine image (AMI) with the selected ESBs so that any end-user can simply fire up an instance of this image, and re-run the performance suite to compare the ESBs of his choice!

This will also make it easier to extend the performance test to include any custom test cases of the user etc, before selecting an ESB for use. But most of all, it will allow a true apples-to-apples comparison of the ESBs of choice, without any vendor bias, as each ESB could be tested on the same EC2 instance, and only allowed to talk to itself on the localhost - preventing a requirement for stringent network isolation.

In the past different vendors used the performance benchmark on different hardware configurations, and also it was not easy for a newbie to re-run the test suite on his own without investing some time to learn things and setup the various ESBs. However, the soon to be published EC2 AMI will contain everything required to fire-up the different ESB's of choice - preventing anyone from having to spend hours to learn and set-up the different ESBs for comparison. A vendor or another end user can also modify the configurations of the ESBs to better tune them, and republish the new image again, making the process simpler and easier for everyone!

The free and open source ESB's evaluated in this round include:

• UltraESB v1.5.1
• WSO2 ESB v4.0.0
• Mule CE v3.1.2
• ServiceMix v4.3.0

It has not been easy to set-up all these different ESB's, and we've found that its pretty difficult to get some of the ESB's to execute all test scenarios.. More on this will be published soon as we try to overcome these difficulties.. so stay tuned!!

If you would like any other ESB to be included into this round, do let us know, and share a compatible configuration.

Visit the ESB Performance site for more details..

To build the best ESB, you have to select the best open source components too!

If you've ever wondered how we build the worlds best Free and Open Source ESB, in terms of performance, features and ease of use; then read about it all on Thanking great Open Source projects and teams!

To build the best Open Source ESB, we *had* to select; and only depend on; the best Open Source components - and not the old cupboards or the stinking kitchen sinks, that some vendors cannot yet entangle themselves from!

Did I mentioned that our complete distribution is just ~35MB? Compare it to some others who call 135MB upto ~1G ESB's as light. I guess after putting lipstick on the Pig, they now like to call it "light-weight" like a damsel too ;) But sadly, some are still stuck in the days of hub-and-spoke messaging, and some made the whole world around them too SOAPy and slippery too soon as they didn't take enough REST at the right time.

Being the youngest ESB in town, and to Win! - we had no choice but to select the best technologies and open source projects that we would depend on and use.

Introducing true Zero-Copy proxying, with memory mapped files and Non-Blocking IO was killer in performance. Some even tried to copy the keywords without implementing the code, and even without understanding what Zero-Copy was :)

We also introduced the concept of using Java (classes or just fragments) or any JSR 223 scripting language (such as Javascript, Groovy, Ruby) for mediation. Although many vendors falsely claim that their ESB is "configuration" only - the configuration language they use is no different from a programming language. However, someone new must learn that "XML"ish language to "program business logic" - and no one is going to tell him the equivalent of a try {} catch {} finally block that can be reliably used in that context :)

Our approach was to let the user decide the way he is going to mediate, using a language and/or technologies he is already aware of. But unlike some ESBs where you write Java code, then use some script to compile, bundle and deploy - the UltraESB just requires you to write the lines - and the compilation is hidden! You can even debug your mediation from within your favourite IDE - either IDEA, Eclipse or Netbeans.

When we wanted to introduce clustering almost an year back, we looked at Apache ZooKeeper and knew at first sight that it was the right one for us - call it link at first sight ;) !

Selecting a UI framework was the most time consuming.. and we looked at GWT based, and other frameworks, based on the servlet model but did not find the right balance. We wanted to use the best of HTML5/CSS3 and be in control always.

The combination of HTML5/CSS3/JQuery/datatables - connected via JSON using Pure - to a Wink REST application secured by Shiro and implemented via JMX with direct calls to the remote ESB was one of the main introductions in this last release! But it was all worth it! Except for a few issues on the IE browser (which statistically a lot fewer people use now) everything has been smooth. But we will follow up a 1.4.1 release with these fixes for IE too.

UltraESB Code Quality Metrics - updated for v1.4.0

We've updated the code quality metrics for the free and open source UltraESB v1.4.0 release recently, and our glad to still have a 43.3% code coverage (46.1% of the lines and 36.2% of branches) and 15.5% of code comments (46% documented API)

The total lines of code increased to 33,563 spread over 69 packages and 426 classes, with v1.4.0 which introduced the web based administration console - UConsole, support for clustering, automated round-robin restarts, XACML and even faster XML performance with FastXML

By building each sample as both an end-user executable, and documented sample, as well as a JUnit test case has paid a lot. AFAIK we ship the most number of samples for any ESB with 48 sample configurations with over 100 sample proxy services!

UConsole - the node/cluster management console of the UltraESB

Sampath has published a detailed article on the architecture of the UConsole with a brief introduction to some of its functions. The UConsole is an ultra light-weight Jetty based web application that uses HTML5/CSS3 to render the pages. All front end pages and scripts thus can be safely cached by a browser, and all pages fetch data over a JSON API exposed by a REST application deployed using Apache Wink.

The UConsole is the graphical management console, to the re-vamped JMX based UltraESB management API using MXBeans. Thus any function that can be monitored or performed via the UConsole, can be performed via JMX too.

One neat aspect of the UConsole is the ability to connect to any node via its JMX URL. So when you connect to a cluster, you can connect to any single node of the cluster. Apache ZooKeeper ensures that you can perform any cluster wide function from any connected node. There is no "administration server", master or slaves, or any single point of failure!

We intend to develop a command line administration interface over the JSON/REST API in future, when it will add value to the product.

UltraESB v1.4.0 development builds are available for feedback!

Checkout the new blog [http://blog.adroitlogic.org] which describes some of the new features of the upcoming v1.4.0 - including clustering, caching, configuration changes at runtime, XACML and many more!

Mulesoft publishes new results against the ESB Performance Testing Framework

Mulesoft recently published new results on the performance of the Mule ESB v 3.1 over Mule 2.2.7 benchmarked in July 2008.

"Mule 3.1 performance is in average 10% better than its predecessor version 2.2.7, performing better when the number of concurrent consumers gets bigger and much better when dealing with XSLT transformations (around 15% better)."

A clear difference from the previous results from Mulesoft is that they are only testing concurrency levels of 20,40,80 & 160 this time, while dropping the 320, 640, 1280 and 2560 concurrent user test scenarios from this round! They have not published results against the 100K message sizes either - or for the WS-Security test case introduced by AdroitLogic last February [See http://esbperformance.org].

Its interesting to note both Mule and WSO2 now follows a pattern of performance testing only against previous versions of their own ESBs - when previously both published performance figures over each other; other open source ESBs and even commercial equivalents.

AdroitLogic first published the figures for its UltraESB in February 2010, and took a decision to compare its performance against competition - but not to release the names of the other products tested. Instead, it ships the complete ESB Performance Test Framework including scripts to run and convert results into CSVs, so that an end user can easily compare its performance against virtually any other comparative ESB (even commercial alternatives) on an exact same hardware and software configuration for a true apples-to-apples comparison.

Enterprise Service Bus [ESB] Comparison: Poll a directory for a file and upload it with an ...

Enterprise Service Bus [ESB] Comparison: Poll a directory for a file and upload it with a HTTP PUT: "Here is an example of a File polling service, which then sends the file to an authenticated HTTP service as a PUT request"