What is ESB?

Enterprise Service Bus (ESB) is a communication model that allows business applications to communicate with each other. An ESB framework involves a system of business rules and common standards, and a software mechanism that passes messages from source to destination.

An ESB generally exists as standalone middleware. It can deliver a Service-Oriented Architecture (SOA), typically for on-premise networks.

How does ESB Work?

ESB is one of the more established integration methods and has long been popular among organizations with heterogeneous, disparate applications on their network.

Before ESB, most integrations worked on a point-to-point basis, often relying on hand-coded integrations or manual file uploads. This approach was unreliable and created scalability issues, as each new discrete application required an n+1 number of integrations.

The Enterprise Service Bus model helped smooth the integration process by implementing the SOA principles. This strategy imagines a service-provider and service-consumer model, with providers and consumers connected by a bridge.

An ESB provides that bridge. Each application connects to the ESB via an adapter that handles tasks like message formatting and API calls. Doing this means that each application only has a single connection point to the adapter, which in turn connects to the ESB. If the enterprise wants to add a new application, they just need to add a new adapter. If they want to remove the application, they can delete the existing adapter.

The model typically works like this:

1. Message generation: An event occurs on the source system. This triggers an event that generates a message to the ESB.
2. Message formatting: All applications may have their own way of formatting and storing data, but the adapter formats these into a canonical messaging format. Usually, this format is XML, although some ESBs support flat files, JSON and other formats. Once prepared, the adapter passes a message to the ESB.
3. Message validation: The ESB structure will validate and categorize the request according to established business rules. This stage may also involve a basic level of data transformation if the destination requires it.
4. Conflict resolution: The ESB uses business rules to resolve any conflicts that may arise. For example, two systems may try to send simultaneous messages to the same system. In this instance, the ESB would sequence the messages according to the rules.
5. Message integration: The ESB passes the message to the destination system, and this system responds accordingly. For example, the system may add the incoming data to its database.
6. Confirmation and validation: Where required, the destination can send a response back to the source system to confirm that the communication was successful. This receipt will pass in the other direction back through the ESB.

The ESB middleware is generally stateless, which makes it unsuitable for large-scale data transformation. Instead, businesses would use an Extract, Transform, Load process, which provides an intermediary staging layer.

ESB vs. iPaaS

ESB is somewhat in the shadow of a newer approach, Integration Platform as a Service, or iPaaS.

iPaaS works in the same basic way as ESB. It sits as an integration layer between applications and allows them to pass data between each other. However, there are some key differences between the two.

On-premise vs. Cloud-based

ESB generally sits on an on-premise server and connects systems on the enterprise network. Some ESB solutions can integrate with cloud services, but the middleware itself often sits on a local machine. This can be faster and means that data never leaves the enterprise network. However, the organization must provide adequate server space for the ESB.

iPaaS is a cloud-based service that third parties host and offer to businesses. It integrates easily with other cloud services, but it can also act as a hybrid solution, connecting to both cloud and on-premise systems. Because it’s cloud-based, users have a great deal of flexibility when they need to scale up or down.

Legacy vs. Emerging

ESB has been around as a concept since the 90s, although the principle was first properly instantiated in 2002. The ESB model anticipates working with a number of disparate sources, each of which stores and transmits data in a different way. Organizations that rely on older systems may rely on their existing ESB.

iPaaS is a product of the cloud era and often works best with other cloud-based applications. iPaaS can integrate with older on-premise systems, but often this takes a greater deal of configuration work than integrating with newer services. Newer companies, as well as those going through digital transformation, may choose to adopt iPaaS for their infrastructure.

Adapters vs. Integration Libraries

ESB uses adapters to communicate between applications. Adapters often require someone to code and configure them, and that person must have an understanding of the data structure of the source application. Each application has its own way of communicating, which may include file uploads and SOAP or RESTful APIs.

iPaaS solutions generally offer a library of integrations for popular cloud-based applications. This allows users to perform low-code or even no-code integrations, which means that non-IT users can integrate systems on iPaaS. If an application is not supported in the library, the user may have to code one.

Pass-through vs. Intermediate Layer

ESB is stateless and not designed to hold large quantities of data at any time. Messages pass through the ESB from source to destination. While these messages may undergo some transformation, the ESB can’t perform any kind of large-scale transformation.

iPaaS usually can hold more data and perform more sophisticated operations on incoming data. This can be useful if the enterprise needs to cleanse, validate, or integrate data when it passes between one source or another.

However, when the operation involves a large movement of data or complicated integrations, most organizations will use an ETL (Extract, Transform, Load) process. ETL solutions like Xplenty hold incoming data in a staging database, where the ETL can perform extensive transformations. This is the best approach for activities involving large-scale data movement, such as populating a data warehouse.