Rise of Enterprise Integration: Our Business Landscape Calls for Architectural Rethink

A large multi-national enterprise typically comprises thousands of applications, with data inputs spanning hybrid/multi-cloud environments, IoT/mobile devices, and distributed operations. All this data, in turn, necessitates enterprise integration.

These data events, by their very nature, happen in real-time—a customer places an order online, a supplier updates available inventory, a passenger scans a boarding pass, a sensor detects a sudden temperature change. These events are not synchronous, and they all trigger follow-on actions that ripple throughout different departments and operations across an enterprise.

The vast volumes of information in play, all containing data critical to an enterprise’s day-to-day operations, are underscoring a dramatic shift in the way increasingly globalised business systems are integrated.

Old Enterprise Integration Ways Cannot Match the Needs of Today’s Real-Time Business World

Traditional methods such as iPaaS and ESB are still being used to “knit” together data across this complex web of disparate systems. However, they struggle to keep pace with the demands of today’s data-driven landscape. Synchronous, point-to-point solutions simply aren’t built to handle the ever-growing volumes of real-time data coursing through modern enterprises to an ever-increasing number of applications.

Traditional enterprise integration methods result in a complex web of connectivity that is difficult to understand, fragile, cannot deal with bursts and is generally not robust to failures and outages. This creates longer and longer application runtimes which, in turn, result in poor user experience.

To simplify enterprise integration, become more real-time, and more connected, organisations need to adopt an “event-driven” approach.

Event-Driven Data Needs Event-Driven Integration

Event-driven integration  offers a new and innovative approach to connecting systems through the instantaneous sharing of real-time events.

At its core, whenever an event occurs within a system, a message is published to a central hub called an event broker. Other systems subscribe to this broker, or a network of event brokers, called an event mesh, receiving messages in real-time and reacting accordingly. As we say in the real world—they are all on the same page! This on-demand, “data as a service” approach unlocks a level of flexibility and scalability far more dynamic, responsive, and real-time than traditional methods.

The Analyst Community Is Already There

Leading analysts now recognise event-driven integration as a key component to optimise real-time movement of business-critical data. Gartner identified this trend in the evolution of IT systems architecture, heralding an “event native” mindset that moved from looking at IT systems as being “data (centric) custodians” to one that looks at IT systems as comprising the “nervous system” of an enterprise. More to the point, viewing “data in motion” as opposed to “data at rest” as the real source of effective decision-making.

It should be noted that this event-driven enterprise integration approach works perfectly well with an organisation’s existing Integration Platform as a Service (iPaaS), essentially augmenting the iPaaS with an event-driven platform. In fact, with iPaaS and event-driven integration working in tandem, IT teams can migrate the right information flows incrementally to move towards becoming event-driven, allowing a phased implementation of key business processes over time.

This is reflected by IDC recently, noting in “IDC Market Glance – Connectivity Automation, 2Q24“: “Another trend that is helping fuel the growth of event-driven architecture (EDA) is the potential to utilise EDA in conjunction with iPaaS to split queuing, avoid bottlenecks, and manage workload and data traffic asynchronously. With an event broker layer, an organisation can also ensure that they have visibility into the state of queued messages even if there is a sending error.”  

Turning Integration Inside Out

An “event-driven” approach means rethinking and rearchitecting enterprise integration.

Enterprise integration of the past 20 years has been fairly centralised, with monolithic applications with many dependencies. Current integration approaches have placed integration components in the data path—that is, at the core. Integration, whether real-time or batch, requires connectivity and transformation. The challenge with centralised integration approaches is that they couple connectors, transformations, mappings, and potentially transactional context into one deployable runtime, through which all data must flow in a synchronous manner. As such, the integration component can become a bottleneck and suffer from many of the same problems as monolithic applications.

Event-driven enterprise integration turns this approach inside out.

It moves integrations and connectors to the edge, with decentralised and real-time data flow and events in the middle. The result is an application and integration architecture that is more agile, scalable, robust, and real time—much like event-driven microservices.

Today’s use cases demand an enterprise integration architecture that can handle both traffic bursts and slow/offline consumers without impairing performance; can scale to handle an increase in consumers, producers and in data volume; and can guarantee delivery of data even to temporarily unavailable consumers. It also demands an architecture that lets you easily plug in processing components and distribution mechanisms without impairing the overall design and even to easily adopt new technologies you had not even imagined before, such as Gen AI Agents and LLMs.

Going Inside Out Unlocks Transformative Business Insights

Arriving at an optimal event-driven enterprise integration implementation does not happen overnight. It is an evolutionary process that can be measured in four transformative milestones:

Breaking Down Data Silos

Traditional systems often create data silos, hindering accessibility and informed decision-making. Event-driven integration fosters information democratisation and availability, allowing businesses to readily access the data they need, when they need it.

Dealing with Unexpected Bursts in Traffic

The decoupling of components serves as a “shock absorber” that gracefully deals with unexpected bursts of traffic, such as sudden spikes in demand, leading to vastly increased order volumes. This decoupling also provides a more robust infrastructure that is tolerant of failures and outages and prevents cascading failures.

Nurturing Innovation at Warp Speed

Event-driven integration streamlines the process of integrating new applications and services, empowering businesses to innovate faster and stay ahead of the curve. If you used event-driven integration today, it would be faster and easier to give your Gen AI applications real-time enterprise context to work from because you would just add these new components to your existing event distribution architecture.

Keeping the User’s Finger on the Pulse

Enabling real-time data integration provides users with a more consistent and up-to-date view of information, leading to a smoother and more efficient experience.

Event-Driven Integration in the Real World: Organisations That Have ‘Thought Out’ Of The Traditional Integration Box

Word is getting out, as organisations across industries such as financial services, manufacturing, retail, and more are beginning to embrace event-driven uses cases.

Heineken, as part of its strategy to become the world’s most connected brewer, implemented an event-driven enterprise integration strategy where production line events trigger real-time inventory updates and automatic order fulfillment for distributors—on a scale across 350 global and local beer and cider brands it sells in 190-plus countries.

Leading German grocery chain, EDEKA, leverages an event-driven enterprise integration approach to modernise its supply chain and merchandise management systems, replacing synchronous batch updates between siloed systems with real-time data sharing. Powered by a continuous event-driven flow of information, EDEKA is now supporting a better shopping experience for customers.

In fact, a recent IDC Infobrief found that 82% of survey respondents have either begun or are planning to add between two to three event-driven use cases in the near future.

Rethinking Integration Architecture to Meet Today’s Needs

Increasing data volumes and connectivity levels amid shifting consumption models and customer expectations are changing the way large organisations need to architect the flow of critical information across their business. Traditional enterprise integration approaches are no longer able to support businesses that have to match customer, employee, and supplier requirements in real time.

Event-driven enterprise integration offers a path for multinational businesses to modernise their integration strategy and empower their business to be more adaptable, scalable, and robust—in a landscape that is only going to continue to become more digitised and real time.