This article is excerpted from the report:
“The World of Integration”
available for download here.
In Part One: Necessary Nuisance or Enabler? we discussed how enterprise software companies who once pitched that you would “never need anything else” have also had to admit that, yes, integration is critical to the burgeoning portfolios they sell.
Integration Vendors/Tools Segment
The largest segment in the market is that of neutral tools and suites that are designed to work with a myriad of application vendors (thus we use the term neutral). Often there are specialties by technology environment, such as middleware for device integration or integration for enterprise software; and by industry. In addition, and very important to this market is the growth of open source. If one considers the huge number of processes and end points that can be integrated to, open source can give a leg up on the daunting work of having to implement on a large scale and thus, can reduce the cost.
Offerings on the market have various names such as Enterprise Application Integration, Enterprise Service Bus, and Middleware. Lately, the term Data Integration is becoming part of the landscape as well. All of these have different components and functions. Some vendors are eschewing these limiting categories and now offer unified platforms: IPaaS, Integration Platforms as a Service. The platform approach provides a huge portfolio of services1 that can work with complementary technologies such as Master Data Management (MDM) and provide services such as data cleaning, standards enforcement, and the opportunity to have a data warehouse that sits on top to provide a broad opportunity for dashboards, monitoring, and analytics.
Within this sector we have B2B and A2A segments. To note, B2B frequently has not been considered part of the integration market per se, leaving EDI and other trading-partner management tools as its own segment. But that line has been blurred by the many mergers and acquisitions, partnerships, and embedded offerings of so many solutions companies. Buyers can now shop from a price list of these capabilities. Or a more intriguing approach is Unified Integration, which is a fully architected integration solution, placing B2B and A2A in one harmonious approach. This is emerging as a powerful segment as well. (See Figure 1)
Figure 1: Integration Modules and Capabilities
But wait, there’s more! APIs have become their own segment. (It used to be the domain of application vendors who would ‘publish’ the methods for developers to build integrations into their solutions). This is an interesting and growing area due to the explosion of end points as well as the need to build business process extensions within or between enterprises. We differentiate between APIs and Business Process Extension segments (which will be discussed more at length below).
Segments and Capabilities
Segments in Integration tools sets are:
Application Integration — an umbrella term used to describe a variety of capabilities from Enterprise Application integration (EAI), Middleware, ETL, and so on. The focus is applications, though, and not the movement of data outside the enterprise.
Enterprise Service Bus — intelligent message queues and routing of data.
Trading Partner Integration (TPI) — EDI and all the variants to manage B2B communications — should include web services. Web services are a key capability in a full TPI/B2B solution today. XML/HTTP, JSON (Java), SOAP (Objects) and REST (which maintains the web representation of a partner’s web page) are all techniques. HTTP and Java scripting allow the integration and exchange of information. Why would users also need REST? Think of a process like ecommerce where the partner, rather than the ecommerce seller, has the product. The product data, pricing, and so on from the supplier’s web page can be accessed and data processed between sites.
Unified Integration — here the solution is architected to accept both application and trading partner files. A rules engine is referenced to translate/transform the data and deliver it. For external transactions, the data file is moved to a schedule/outbox just as with EDI. Of course, this can also be a cloud-based application.
Business Process Orchestrations/Management or BPM — the visual design and automation of the workflow. Often, enterprise solutions have workflow engines with nice business process design GUIs as a strong feature within their solution. This is a best practice to provide to customers. However today, many ERP companies have a multi-package portfolio of solutions, so the BPM may not work with all products, or with all releases/versions. A neutral solution may be a viable choice in such an instance. Also for firms who have a rich portfolio of applications, BPM is hard to use across them all. Thus, BPM is a smaller segment within the integration market.2
BPM can also be part of a development platform suite. Suites have many more tools for developers architecting applications from scratch. That may be an option for some organizations who want the greatest extension of the neutral tools they purchase. But for this report we are more focused on applications and process integration vs. software development.3
Data Integration — the definition here has been morphing a bit in the market and there are conflicting definitions. By some, Data Integration is a superset of application integration which would include application integration and MDM (Master Data Management), data transformation and cleansing, API libraries, and possibly BPM.4
API — traditional, large software firms created application integration libraries and tools to provide third-party applications with the methods and rules to interact. Today there are thousands of APIs that are published for many integrations: for example, from a connection from FedEx and UPS to apps on mobile devices. Many of these are open source, so although critical, the revenue share in the market is smaller. However, these are often included in Application Integration suites.
Business Process Extensions (BPE) — often called APIs. However, ChainLink differentiates these from APIs since they are expressed in business language such as, “Print a barcode label; find inventor; execute drop ship; send service request; send ship notification; issue a material return authorization,” and so on.5 BPEs can use a variety of technical enablers beyond APIs, such as EDI and workflow to create/integrate the whole process.IoT Middleware — as we mentioned, there are specialized types of middleware. Device integration is not new, but IoT has given it a refresh.6
Microservers — an interesting and growing capability in the market is the use of a microserver. Think of a microserver as specific data sync for small packets or subsets of data, rather than having to move all the data into a software solution such as transportation or service management. Examples would be IoT data being collected from equipment monitoring or data coming from many devices on vehicles and other equipment,7 a DASH-like signal from your washing machine or coffeemaker, and so on.
Gateways — B2B gateways and the accompanying hardware server handle inbound and outbound messages, emails, or FTP file communications. They may also perform mapping, translation, and integration. This traffic handles community standards-based communications such as AS2, as well as common security standards such as SFTP and FTPS. Gateway software also translates community format standards such as such as EDI and SWIFT.
Real-time Data Streaming and Processing — these most often deal with location-aware data streams, social streams, IoT, and big data streams. The RDP can be simple: analytics process, filter, and move the data; or the platform may have additional technology such as complex event processing (CEP) analytics on top of the data streams.
Open Systems — one segment of the market is based on open standards and shareable object libraries to build and maintain applications. The prevailing attitude by some large solution providers is that open systems are just for small or mid-market. We challenge that perspective; it is a fact that larger enterprises are seeking open source due to the lower cost, ease of adoption, and benefits. Open communities are evolving, and more and more being offered all the time. In fact, the U.S. Department of Defense has stated as a key goal the adoption of open source foundations not just for business systems, but for more ‘unique’ applications.8 Their Open Systems Architecture program, OSA, is being actively pursued by all branches of the service. They include these standards for their suppliers to adhere to as well.
IPaaS — we put this last since, at its most realized, it is a platform that would encompass all of the above. Again there are conflicting definitions; some think of this as API/BPE for mid-business. However, what many large enterprise players are looking for is an accessible suite of integration tools in the cloud (private or public). The cost of integration is so huge that even the larger enterprises are looking for ways to offload the integration and software upgrades, as well as have access to the connectors and APIs to the burgeoning end points that are important to integrate to. Rather than having to go it alone, having a service continue to update these libraries has made IPaaS a growth market.
IPaaS is not just an offering by the neutrals, but can be incorporated, as we mentioned, in SaaS application solutions. It should be noted that not all SaaS solutions offer this. IPaaS or integration tools are also offered by the development tools vendors such as MuleSoft, Dell Boomi, Jitterbit, and Informatica; and are offered by supply chain vendors such as One Network and GT Nexus.
Overall growth for the integration market is around 8% year over year, and within that we see the IPaaS segment growing more than 11% for the next five years.9 This means, of course, that IPaaS will take an increasing share of the integration market over time.
Benefits and Trade-Offs
Large neutral integration technology companies, it should be noted, still have price cards they use to sell a variety of integration tools sets. They frequently offer a lot of in-house expertise in how to integrate to the many big commercial packaged software providers. However, as we all know, each company’s data model is different and it always seems to take a lot more time and resources (which equate to costs) to manage everything.
Tools players are most dominant in the integration world, providing ultimate flexibility which, of course, is the issue in integration. The potential downside is that IT departments and users are on their own to develop and manage applications. They have to create and model the business process from scratch and expend the time and resources required for completeness. Organizations today will still do this, but selectively.10 Anything a neutral provider can do to provide templates and object libraries, of course, does mitigate some of the ‘from scratch’ work.
The market has seen the rise of the unified platform as an approach to integration. The web has been a catalyst for these platforms, but platforms can also be on premise. The integration platform can provide a variety of methods, but the best provide rich B2B, A2A, and Business Process tools. The benefit here is the acquisition of one solution. We see the unified approach growing over time, especially as the EDI profession wanes.11 EDI is becoming a scarce expertise; hence, companies are turning more to managed services or the unified platform. The challenge is that A2A departments have been separate from the B2B teams, and purchasing has been done distinctly. A2A programmers just don’t have the skill for EDI and the world of trading partners and compliance. That will also have to change over time.
Application Software — as mentioned, integration can be built in. In-residence should be comprised of a huge portfolio of use cases, templates, and object libraries. If these applications are not built as open source, though, they may be challenged since they are limited to only the apps they support. By definition, integration needs to be broad. Therefore, purchasing an integration suite from the application vendors, though convenient, may limit the whole purpose of integration.
This is that old question: best of breed or ERP? ERP/application vendors have very complex and broad priorities so they may not always be on top of the next innovation. Of course, the ERP vendor surely knows best how to integrate to their product; whereas best of breed thinks only about innovating their solutions for integration. We advise buyers to go for integration packages that have broader versatility, whether purchased from an app vendor or not, to integrate multiple brand and non-branded software products.
We continue this discussion in Part Three: Innovators and Innovations.
_______________________________
1 At minimum, translate, clean data, and route transactions between systems. — Return to article text above
2 There are more names and nuances here such as ADE (application development environment); and iBPM, as in intelligent BPM. The micro-classifications go on and vendors and analysts try to make a name for themselves. Buyers should consider a broad range of these tools since these nuances simply reflect vendors’ additional capabilities and may not necessarily be worthy of a whole new market category; thus, chasing a micro-classification may limit your ability to export the best tools to meet your needs. — Return to article text above
3 Tools like Pegasystems, Appian, Red Hat, as examples. — Return to article text above
4 We think this term confuses, though, since DBMS products do integrate data. And Integration implies ETL (extract, transform, and load), anyway. We also have seen the term ‘data transformation solution.’ This also may confuse, since these systems are basically unified integration solutions. — Return to article text above
5 A great example here is Oz Development. Users fill out a form and check off the boxes of process extensions they want to use and the end point they need to integrate to. Behind the scenes is all that ugly integration code that users don’t have to deal with, e.g., a technical API library. Another great example for retail is Logicbroker that has applets such as inventory location between the ecommerce seller and supplier, drop ship, and so on. For manufacturing, an emerging player is The Trading Portal. Behind the scene this is often EDI and the end users never see that. — Return to article text above
6 RFID and IoT solution providers often provide platforms for development and integration solutions. These solutions focus on device layer integration, publish/subscribe, and data movement, and may have microservers to consolidate data. You can read about some of these here. — Return to article text above
7 For military there are applications programs like VICTORY (Vehicle Integration for C4ISR/EW Interoperability), GVA (Generic Vehicle Architecture), Future Airborne Capability Environment (FACE), and Better Buying Power (BBP) 3.0. Ultimately, a common desired outcome from these programs is for program managers to design interoperability into future C4ISR (Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance) systems. Ecommerce applications are Amazon’s DASH, or IoT applications from a myriad of consumer equipment companies. — Return to article text above
8 Ibid — Return to article text above
9 If we could include SAP’s HANA, ION, ICE and so on from the software applications players, this number would be even bigger. Today, though, a lot of the HANA world is custom. We assume over time, however, that SAP will have great core object libraries that will be available to and offered to an IPaaS HANA product for the market. Oracle, as we understand it, has an Oracle Application Integration Architecture (AIA) which they are packaging up as a kind of bundle which includes Fusion and what they call Process Integration Packs — prebuilt workflow for processes such as ordering, as well as API for SAP and so on. The challenge in determining the real market size is that firms like Oracle and SAP often give away these tools as part of a large software deal. In addition, many vendors don’t break out their revenue numbers. — Return to article text above
10 IT departments in large enterprises with lots of proprietary solutions (such as financial services) tend to go with these companies and you will note that is the biggest market for TIBCO, Informatica, IBM, etc. — Return to article text above
11 Read: The World Without Us — EDI Professionals — Return to article text above
To view other articles from this issue of the brief, click here.