Growth. This is the single-word answer to a question that many organizations consider self-evident. After all, data has taken a front seat in today’s businesses that it’s hard to find an individual who disagrees with the notion that the ability to leverage data is a key success factor for most businesses.
The amount of data we collectively generate 24/7/365 continues to grow, and with it our desire and actual ability to utilize data for various purposes. As the value potential of data grows, so does the business around making this happen. The big picture can be segmented using the idea of a “data supply chain” which essentially consists of three core areas:
- Data sources and the related storage (i.e. raw material)
- Data integration and data management (i.e. logistics and refinement)
- Analytics and algorithms (i.e. production and packaging).
While the “consumer facing” area of manipulating and visualizing data usually gets the most attention, the focus here is on the second and equally crucial element in delivering value from data: data integration and data management. So, let’s have a closer look at what we can expect to happen within the world of data integration over the next 5 years.
The market continues to grow while boundaries of the traditional integration domains dissolve
Integration has traditionally been separated into various sub-domains based on the different integration patterns, including but not limited to A2A, B2B, ETL, MFT, etc. As advancements in integration technology continue to make it not only possible but also practical to handle multiple integration use cases with a single solution (Liaison’s ALLOY Platform is a prime example of this integration technology evolution), focusing on a combination of all integration domains becomes increasingly relevant for understanding the big picture of where the integration market is heading.
Overall, this “hybrid integration market” is forecasted to grow from USD 17.14 Billion in 2017 to USD 33.60 Billion by 2022 (MarketsandMarkets, 2017) which amounts to a compound annual growth rate (CAGR) of 14.4%. For someone who has followed the development of the integration market over the years, this expected annual growth rate may not be terribly revolutionary. However, since this aggregate number masks the vast diversity among different integration technologies at their various stages of the lifecycle, what goes on behind this overall number just might be.
Digital transformation and cloud computing alter the market dynamics for traditional integration technologies
The demand for integration solutions is varied, with many dynamics to consider. The main drivers for this demand are the usual suspects, including: 1) accelerating demand for digital products and services as companies proceed with plans for digital transformation; 2) the continuing trend of moving applications from on-premises server rooms to the cloud to increase flexibility, scalability and efficiency; and 3) the ever-increasing amount and variety of data available from both internal and external sources.
The implications that these drivers have on the different types of integration technologies can be summarized as follows:
- Application Integration: Along with the actual applications, application integration middleware is also moving from on-premises to the cloud (i.e. traditional enterprise service buses and other middleware are being complemented and/or replaced by cloud-based solutions like Integration Platform as a Service, or iPaaS). The remaining on-premises applications continue to be served by traditional ESBs, which still represents the vast majority of the overall integration market size as long as an organization’s needs do not significantly change. However, any growth that on-premises integration technologies will experience is largely in replacing the established solutions with more cost-efficient open source on-premises integration tools, whereas the overall market is strongly moving towards the cloud.
- B2B Integration: In general, B2B integration is growing at a similar rate as the overall integration market and, in spite of the market being quite mature, its significance for businesses is increasing. The main driver behind this is the evolution of traditional supply chains towards increasingly automated, global supply networks. Due to the aforementioned market maturity, growth in B2B integration mainly comes from organizations replacing their well-established legacy B2B integration tools and platforms with modern solutions that are better suited to serve the evolving digital requirements and data analytics initiatives.
- Data Integration: Solutions for enterprise data integration need to support requirements for more varied data movement, but also governance and management of data as organizations face increasingly diverse requirements across business functions. The need for breaking down data silos and blending data from various sources to uncover business insights (while also ensuring compliance) challenges the traditional bulk/batch approach, which is being replaced by a more flexible approach that brings additional delivery styles including data virtualization, message-oriented data movement and data replication/synchronization to complement the more traditional styles like ETL (extract, transform, load). This enables meeting the real-time requirements when needed while utilizing batch processing where it’s most efficient to do so.
Integration complexity drives growth of professional integration services
As the complexity of integration requirements increases, so does the need for specialized skills. To a short extent, this development is counteracted by the fact that some of the simplest integration use cases can be handled by “citizen integrators”, rather than integration experts, with the help of self-service tools. However, professional services are expected to be one of the highest growing areas within the hybrid integration market within the next five years.
It will be very interesting to see how the forces of integration tool consolidation and increasing complexity interact to shape the daily integration operations in organizations. What everyone seems to be unanimous about is that data and data sources will continue to grow exponentially, pointing to more complex integrations. New skills are required to serve these evolving needs, which leads to redefining the core skill sets each IT organization deems mandatory to have in-house, and which ones to source from the market as and when needed.
As organizations get better at moving data around, managing it becomes a priority
With all the restructuring, modernization and other developments going on in the integration space, there is plenty of work for the IT department and their vendor partners to do to get integration operations streamlined and aligned with business needs and objectives. However, once you have the required access to data sorted out, the focus naturally shifts to questions like “can I trust this data” and “what more can I do with it.” Enter: data management.
Forward-thinking organizations take data management into consideration while developing their approach for integration. Some vendors, like Liaison Technologies, believe data management and integration are converging as disciplines and should be treated holistically. One interesting data point on the increasing significance of this topic is the projected growth of the master data management (MDM) market. Numbers and market definitions may vary between different research companies, but according to one forecast the MDM market will grow from the estimated USD 11.52 Billion in 2016 to 37.98 Billion by 2022 (Zion Market Research, 2017). This amounts to a CAGR of around 22%, and means that based on the two market reports referenced in this text, the MDM market will surpass the integration market in size by 2020.
An interesting question is how long will integration and data management continue to be treated as separate markets, and when will we start referring to them as the “data platform market.” Market definitions and segmentation aside, integration and data management are interrelated and equally crucial elements in extracting value from data that organizations cannot afford to ignore.
Solutions like Liaison’s ALLOY™ Platform are built for the future and their design reflects and evolves with these market developments as organizations continue to improve and optimize their ability to leverage their ever-growing data assets. To support these efforts, Liaison works with IT and business decision makers across industries and across the globe to help build an enterprise’s data strategy, define and develop the capabilities they need, and design and run the solutions that power their data operations. If you wish to find out more about this approach, you can contact Liaison’s data experts here.