This is a bit of panic in my professional world. “Do we need to be an ESB, should we deploy an ESB?” “We have an ESB, you don’t!” “We had an ESB 10 years ago – and so we are fully prepared to do this.” “We don’t have the resources to deploy an ESB and we will therefore wait and see.” The term was developed back when SOA was the rage and we all were trying to bust up our monolithic enterprise applications into distinct components “everyone could use” and in most cases it failed miserably. It was just too hard to bring it all together, even with an ESB. Today ESBs are pretty much various forms of message oriented middleware – focusing more on the communications between these applications than the business services themselves.
What is an ESB? Back in 2015 leading development platform vendors like TIBCO were delivering enterprise service bus solutions that focused on doing the work necessary to assemble cohesive environments that had many different technologies, databases, applications, etc. Mulesoft began to do the same with middleware and messaging technology designed to simplify the deployment of new technology as a service to multiple development teams and environments in the enterprise. What I’m learning is that while we in the business world are just now starting to talk about ESBs and microservices – these companies have pretty much moved on to the integration of everything from data to processes to platforms to business domains.
Back in 2016 TIBCO’s Kai Wähner talked bout the death of the ESB; but really he was defining the future of it. By 2017 we were describing operating environments as a PaaS (Platform as a Service) with companies like Amazon and Salesforce taking their advanced computing environments and timesharing that technology out to the rest of the world to build new applications. In 2019 it is pretty much mainstream to think about all of these applications running on someone else hardware, virtualized and accessible via the internet with advanced building blocks and tools to create applications rapidly and efficiently that work at scale! It is truly an amazing time to be selling technology. But it is also a very confusing time. As I shared in my previous post – most companies aren’t ready for this, don’t understand it and aren’t there yet. Especially my target market – the banking industry – they are just now beginning to adopt this technology.
So back to ESBs and what it means today. Today we talk about Microservices and APIs (an acronym for application program interface). I talk about Microservices in a previous post. A nice article about developing mobile APIs is on Upwork.com where Nicholas Wright defines an API as a technical development environment that enables access to another party’s application or platform. I like to simplify the distinction between what you run and develop in-house vs what you go and get elsewhere. I know it is an oversimplification but it is easy enough to describe APIs as an application running in the cloud that you access when you need it. Microservices are more like legos used to assemble business applications in the organization; reusable programs that address specific functions with the data, logic, algorithms and other components all encapsulated. Sure, Microservices running in the cloud can power APIs and APIs can therefore be considered a key component of microservices. But companies and software providers have been delivering APIs for many years – often with terrible documentation and with archaic messaging protocols and formats. Today’s APIs based on web services technology are much more true to form and closely related to microservices. They are truly based on reliable standard approach and therefore more easily consumed and collaborated with via a tool such as an ESB.
In 2016 – Wikipedia defined an ESB as follows:
En enterprise service bus (ESB) is a software architecture model used for designing and implementing communication between mutually interacting software applications in a service-oriented architecture (SOA). Its primary use is in enterprise application integration (EAI) of heterogeneous and complex landscapes.
If you want to know what it is defined as today follow this link
In 2016 it was used for:
Integration, orchestration, routing. (some kinds of) event processing / correlation business activity monitoring
Watch Kai Wähnet’s talk here Do Good Microservices Spell The Death of the Enterprise Service Bus?
Legacy Integration (e.g. with a powerful core processing connector)
API and REST integration and business services
Messaging (WebSockets. MQTT, AMQP, …) to the Internet of Things (IoT)
Today we have to first define how we build services, where we run them and how they are used before defining an ESB. Back to my earlier point – its really more about the messaging than the platform now. The reality is, at least in the big banks ESBs get lumped together with payment hubs and legacy messaging systems and many end up looking like those crazy Red Devil Busses in Panama – each a unique and souped up version of some pretty archaic technology.
Vendors like Mulesoft will want you to completely replace the way you define communication between systems; adopting a standard approach to building interfaces. This then defines the APIs and the programming tools – allowing you to adopt their approach, and the microservices and APIs in their marketplace. Owned by Salesforce.com now – its pretty much going to continue to become a marketplace and development platform – ultimately on their PaaS, in my opinion. I had some direct experience with the Salesforce.com development community during my time at Terafina, and prior to Mulesoft the key challenge for Salesforce professional services companies and developers was integration with legacy systems, getting access to data – to build more robust data driven applications. Companies like Terafina build these integration layers; but now Salesforce.com will eventually make it a part of their marketplace – eventually assembling core integration APIs into a subscription model.
Tibco has moved on to what they call an ESB bundle that includes legacy integration tools but also orchestration, messaging and monitoring tools. The TIBCO BusinessWorks integration platform focuses on data sources, it was a pioneer in high speed transaction processing – defining ESM as an extension of JSM (Java Message Service). The bottom line – investing in an ESB is not a magic bullet – for all of this to work you have to rethink your business applications, redefine your business domains as distinct functions and think differently about the way you assemble them. And there are so many tools available as companies move to cloud computing platforms, containerization, DevOps and Agile. I’ll do my best to post about those in the future as I find it all very fascinating.
Everyone I ask pretty much puts these and other companies that claim to provide ESBs really as integration middleware platforms. They don’t provide the domain expertise necessary to use them. Companies like IBM, NCR and the big bank core processors (Fiserv, FIS, Finastra, JHA) are starting to help banks do this – not trying to deliver an all purpose ESBs but rather to build microservices based solutions that are deigned specifically to work in the banking domain. Not to become the silver bullet for all things but rather to zero in on specific legacy applications and help banks to refactor those applications into a modern, flexible architecture that can take advantage of the latest platforms and tools.