MFT, EDI and other B2B Alphabet Soups

The History

In our recent research, we noted that B2B technologies are of high interest and are on purchasing lists for 2012. In a recent webinar I did with MSDynamicsWorld, about 60% of the attendees declared that they were going to purchase an EDI solution this year. Amazingly, after a more than thirty-year run, it seems that Electronic Data Interchange (EDI) is alive and thrives!

It often helps to have some perspective on technology over time. Each innovation brings both excitement and frustrations. Early adopters champion the benefits and work out the kinks so that the rest of us can enjoy sustaining benefits and interoperability. (For a good write-up on EDI methods, projects, etc., you can read An Introduction to EDI from the DiCentral web site.)

In trying to understand why something works the way it does today, we often lose sight of (or never knew) the constraints under which these technologies were developed. In the recent webinar on adoption of B2B communications, I discussed the past, present, and future of some of the communications technologies, especially as far as EDI/ B2B goes.

Figure 1 provides a chart of the forty year span as we moved from the world of paper-based systems to the automated and integrated world of today. (Without revealing the age of some of the team members here, we can remember how excited they were to get their modems.) Modems allowed us to communicate system-to-system over telephone lines (vs. shipping tapes and card decks) and created a revolution in B2B communications. The famous American Hospital Supply ASAP case study was a novel breakthrough. In the 1960s,¹ the supplier, AHS, put modems in customers’ offices to enable customers to order direct.²

The Need for Standards

But along with all those 010110 bits and bytes, we needed standards to allow data to be systematically read between systems and between companies: thus, the birth of ANSI, UCC and the UN’s EDIFACT. I won’t bore you with all the details, but many industries were struggling to communicate. Transportation, government, and financial services were the most notable leaders to organize and get serious about standards. In fact, without standards we could not have global commerce. Today there are dozens of standards groups for different industries.³ (For a list of EDI standards groups, click here.)

One important point in the EDI story, though, is how we commercialized it. Communications frequently got interrupted (talk about a ‘bad cell phone zone’). Com lines were crashing all the time and systems were often ‘down,’ stymying successful transmissions. Static on the line often introduced errors that made reading and interpreting the data strings difficult. In response, a novel idea was developed — stream your characters to a service bureau that could store your transmission and then make the multiple attempts to ‘knock on the door’ of the recipients. Thus the creation of Value Added Networks (VANs). (VAN providers, please forgive this rendition of history.) Charges were determined based on transmission size or by number of characters, criteria still used by the industry today.

There is one other element of note regarding FTP, or File Transfer Protocol. Also developed a very long time ago, these nifty pieces of code were written, or derived from code written, by highly creative programmers (a renegade lot who often refused to dress fashionably) who loved creative, complex code. In fact, most of the FTP approaches were developed before the industry got serious about structured programming techniques. One of the problems with FTP was that in the millions of lines of code within an enterprise, nobody knew what they were, to whom they were going, and if they were secure.

Fast forward fifteen years to the internet and the collaboration movement. Could it be possible to share data in real-time? This was the challenge posed in the late nineties with the adoption of internet-based information sharing. Many industries are still struggling to achieve this real-time visibility.

The internet also introduced a different and much more affordable approach to file and data sharing. So after decades of EDI VANS and EDI software knocking on mail boxes of customers and trading partners, new protocols and approaches have been developed. XML, AS2⁴ (HTTPS) for example, represent just such a change, creating the secure internet communication protocol now mandated by retail mega companies, for example, Amazon, and Walmart. (You can see a great chart of protocols and where they are used here at the Cleo Communications web site.)

Today’s world is all about speed, standards, and data monitoring. Security has become a big factor due to concerns about intellectual property violations, privacy and hacking issues, and assurance that the right systems have received the right data as expected (non-repudiation). Hence, we have a promising and growing market from technologies such as Managed File Transfer (MFT) and RFID (Radio Frequency Identification). They provide methods for secure and rapid communication and interoperability between trading partners, systems, and people.

A great case study of just such a migration from FTP to MFT is the Michigan Department of Information Technology project done with Ipswitch.They had a “terabyte of data or more every month moving among 13 state agencies and 5,000 external partners, each with its own formats and processing requirements for handling data and transactions.”⁵ This situation is not unusual for big organizations.

Many organizations have very little control over document transfers — and that is costly and dangerous. It is not that employees are purposefully defrauding their employers by sending confidential documents by email; rather they are seeking convenience (as well as getting a confirmation from a human, that the document was received). But we all know that this is not a good management strategy for those institutions with restricted distribution policies for corporate information. Thus, an MFT approach today can work in the ‘P2P’ realm of business as well as replace the old fashion VANs for B2B transmissions.

Data and Communications Divide

Another important element of this historical discussion concerns what I’ll call the EDI+ industry (EDI and MFT) and how it has evolved the segmenting of data and standards from the communications platform. In the 1980s and through most of the ‘90s, EDI data was inseparable from the platform technology. If a company implemented EDI, the data standards and communication vehicle were part of the EDI system — they were inseparable.But that all changed with the internet, and later, RFID.The reality is that EDI is a data standard — and communication vehicles vary widely. Data can be transported by VAN, directly over the internet, MFT, by cellular network or even paper. Whether a bar-coded paper label or an encoded RFID tag, the communication platform is appropriate for the job at hand. For example, offline environments might require paper or other onboard intelligent devices; whereas online systems need access from the office or warehouse.

Conclusion — We Have Choices

An important element today is the ability for even small business to participate in large trading networks with large customers. EDI projects used to be big and expensive, requiring a knowledgeable, specialized team. But today we have portals and MFT and EDI cloud solutions that can provide solutions at a modest cost.⁶ This gives us an opportunity for rich communications and a more accurate picture of our business and trade.

Amazingly, adoption of EDI is not yet 100% and still is growing. We will discuss the reasons why in upcoming articles. As we continue this series we will provide a description of the market landscape and explore your options for securing and managing your data for large and small companies.

_________________________

References:

You can replay the EDI, MFT and into the Clouds webinar here.

Here is other recent writing on EDI, MFT and Compliance

More on Cloud

LinkedIn groups for more on EDI and communicate with the community of EDI users:

EDI

EDI Professionals Worldwide

________________________________________________________

To view other articles from this issue of the brief, click here.