EDI API - The Definitive Guide (2021)

This is a complete guide to how EDI and API co-exist in 2021.

So if you want to learn how to use EDI as an API, you’ll enjoy the tips in this new guide.

Let’s dive right in.

• Chapter 1: EDI and API Fundamentals
• Chapter 2: EDI vs. API
• Chapter 3: Transport, security & reliability
• Chapter 4: Speed & size
• Chapter 5: Cost
• Chapter 6: Why is EDI so complex?
• Chapter 7: EDI Message Standards
• Chapter 8: EDI and API, together

Chapter 1: EDI and API Fundamentals

EDI stands for Electronic Data Interchange and in its broadest sense is referred to as the standard for B2B communication. EDI is used interchangeably as the established set of practices for businesses to communicate with each other. It’s not a particular technology or architectural principle, and the same applies to API.

Wikipedia: What is EDI

EDI covers all aspects of how business data is exchanged between companies (or trading partners in EDI terminology) such as transport, security, custom configuration and agreements, data encryption, data formats, reliability, traceability and the ability to transport aggregated, potentially large volumes of data in a single transaction.

The primary industry drivers for EDI have been Retail, Supply Chain, Manufacturing, Logistics, and later eCommerce and Healthcare. Every industry has benefited from EDI, and most are still heavily reliant on it. After all, it was created as the biggest logistics and supply operation in the post WW2 period.

API stands for Application Programming Interface and is a generalized term to express the mechanisms of how software applications can communicate with each other.

What exactly IS an API?

In very much the same way as EDI, API covers the different aspects of machine-to-machine communication such as transport, security, data formats, reliability, versioning, traceability, etc. Modern web APIs provide multiple options for stream-based data transportation in near real-time as well as large volumes; just think of Netflix, YouTube, or Snapchat, which all employ a multitude of APIs for their core and critical operations.

It is then safe to say that in principle, EDI refers to the standards governing the data exchange between businesses, and API refers to the standards governing the data exchange between software applications.

Even with a naked eye, one can see that the two are somewhat overlapping in that they standardize the exchange between different actors in a predictable, reliable and easy to implement way, however, from two different perspectives, EDI from the business side and API from the technology side.

Chapter 2: EDI vs. API

Perhaps you have read some of those articles on the Internet comparing EDI to API, suggesting that the two are mutually exclusive and can’t be perceived in accord with each other, thus prompting for a decision to choose either one but not both. This is usually a sign of misconception and is used to favor one approach over the other, depending on the author’s preference rather than any technological evidence.

As far as the translation and validation of EDI documents are concerned, EDI and API can live happily together, and anyone exploring their options in this day and age has the unique opportunity to get the best of both worlds. No more EDI vs. API type of ill-conceived and misleading articles, because an API for EDI already exists, is already used by businesses and is undoubtedly part of the future of EDI.

The preamble for all “EDI is not compatible with API” or “EDI vs. API” articles on the Internet is in regards to the EDI Message Standards. We’ll get back to this topic but first, let’s explore some of the other similarities between EDI and API.

Chapter 3: Transport, security & reliability

How is business data exchanged between trading partners, the EDI way?

VANs

In the olden days, one would use paper documents and regular mail. Then came the Internet and the modems. And then the so-called VANs (or Value Added Networks, without the “value” some might say) offered an electronic distribution network such that any business can safely trade with any other business with an active subscription to the VAN or the VAN’s greater network, over the Internet, regardless of their data format.

I bet it sounded like the gates to Heaven. And it probably was. The whole of the corporate world signed up for it to avoid missing out on lucrative opportunities.

VAN services are trans industry and act as clearinghouses for trading transactions, thus presenting technical, procedural, and systems engineering challenges. Most importantly, they require cooperation between companies, however, mainly on the VAN’s terms.

AS2

VANs are a separate topic themselves, I just wanted to pave the way for introducing AS2 — the mainstream communication protocol for EDI. Allegedly, AS2 was born as an alternative to VANs and was championed by Walmart. Amazon, Kroger, Lowe’s, and others soon followed suit. With AS2 and the Internet, the point-to-point EDI was born. Businesses had the alternative to either choose trading via VAN(s) or establish direct connections with each other. Some had a hybrid of both.

EDI

EDI documents are usually transported via AS2, AS3, SFTP (or FTPS, FTP), and even manually by email. AS2 provides security based on digital certificates, encryption, and reliability based on MDNs, or protocol acknowledgments.

AS2 is usually applied as a higher-level protocol on top of the underlying HTTP and MIME. In effect, it uses a similar transport protocol as most web APIs do (HTTP & HTTPS).

API

Modern web APIs are built with security and reliability in mind. The concrete design is up to the particular product and implementation that utilizes the API, however, concerns such as security, reliability, and encryption are considered the baseline for every API implementation. HTTPS provides secure transport and API Management solutions provide language and platform-agnostic off-the-shelf solutions for API setup.

The options here are limitless and nearly every API hosting provider such as Amazon, Microsoft, or Google offers mature API management and security solutions that are actively developed and are at the forefront of technology. At EdiNation we use Microsoft Azure API Management. Even the conservative world of banking had cast its eyes towards the opportunities that the modern age presents.

Open Banking

Understanding What it Takes to Secure Your API

APIs provide fine-grained security options, based on Open Authentication (OAuth), Transport Layer Security (TLS), Enterprise user registry authentication, LDAP, quota policies, etc. On top of that most API Management offerings come with built-in audit functionality for a quick audit trail of user logins, updates, usage analytics, exception alerting, and third-party integrations.

Conclusion

Both EDI and API offer mature tooling around transport, security, and reliability, however, EDI is pretty much tied to AS2, whereas APIs can choose from the latest and most advanced solutions, and are highly scalable and configurable.

This is becoming a particularly popular topic in the so-called “Digital Transformation Age,” advocating the clear advantage of investing in replacing legacy software systems with their modern equivalent, to be better prepared for the future. Hence, transitioning EDI transportation from AS2 to whatever APIs have to offer is becoming a hot topic. On a high level, this might sound expensive and overkill, but in many situations, the underlying infrastructure, hosting APIs and AS2 servers, is one and the same thing, so replacing one with the other can be considered more of an upgrade rather than a complete revamp.

Chapter 4: Speed & size

The beauty of the EDI format was its ability to compress data in such a way, that it is optimal in size and at the same time retains a certain level of readability by a human. Converting the nested structure of an EDI transaction to a linear set of letters and symbols and providing the specification to decode it back at the destination is how EDI works. This very functionality allows for:

1. Smaller size, the markup renders the optimal number of letters and symbols to be able to translate the data, the compression is better than JSON, YAML, and XML (all of which need a key for every value). EDI only needs the value. For example, my first name, Kamen, represented in all 4 would look like:

• JSON {firstName:”Kamen”} /19 symbols
• YAML firstName:Kamen /15 symbols
• XML <firstName>Kamen</firstName> /28 symbols
• EDI *Kamen /6 symbols

EDI is 2.5 times smaller than the second-placed YAML.

2. Security when data is encrypted and signed using AS2

3. Reliability via AS2’s MDNs

4. Accuracy because the data must adhere to its specification

Why did size matter? Well, remember that EDI originated back when every byte mattered and you could get a 20MB Commodore PC with no screen for just $1500! Also, remember that EDI started being transported via paper documents and faxes, and had to be in a state of “readability” so that every partner who owned the specification, also in paper, could easily translate it.

But there was also another reason, financial. Some VANs charged by the byte and/or by the number of transactions. So the cheapest data transfer option was to batch EDI transactions in a large file. If it were XML, it would have been several times more expensive. So size mattered. A lot.

Modern APIs allow for massive data upload and download, using streaming and ramping up cloud technology. Think of YouTube, DropBox, or Netflix; they do data transfers and video streaming on a massive scale.

How Netflix Works

In effect, size doesn’t matter in the information age. At least not as much as it did back in the day. Modern APIs utilize the latest and the brightest technology in terms of speed and bandwidth in the Internet era. From a financial perspective though, VANs still charge by the byte, not quite the same as the broadband providers.

Chapter 5: Cost

This brings us to the next topic. How do EDI and API fare in cost?

Let’s compare the two options for EDI communication, direct connection, or point-to-point connection when companies connect directly with each other, and intermediary connection, when companies connect through a value network/VAN.

1. Point-to-point EDI (also known as do-it-yourself EDI) example, sending a purchase order from company ABC to company XYZ:

The following factors affect the overall cost:

• EDI Software
• AS2 Software
• Maintaining EDI conversion and maps between ABC and XYZ
• Maintaining EDI specifications for purchase order
• Dedicated software and EDI resources

Now multiply this by the number of connections and factor in the additional effort for maintaining connection overlappings and isolation. And this is for each of the partners.

The obvious benefits of a P2P setup are:

• You have full control over the data and process
• Everything is configurable and managed by you
• Easier integration with the rest of the in-house applications
• Predictable cost of ownership

The obvious drawback here is the cost of EDI and AS2 Software (which often is sold as one package). Popular products are Microsoft BizTalk, MuleSoft, IBM Sterling Integrator, etc. The flexibility to having full control is somewhat undermined by the price of EDI software, usually in the range of tens of thousands of USD per year.

2. Managed EDI (via VANs or value networks)

The options here are ubiquitous. You have old-school VANs, modern web portals/mailboxes, multi-connector B2B networks (Ariba, Tradeshift), and new kids on the block (Orderful). Despite their technological and implementation differences, the common property is that they all handle the “EDI stuff” for you.

For example, if you decided to sell your items on Amazon and become a vendor, you need to trade via EDI with Amazon. An immediate option is to hire a middleman to do just that for you because you’ve heard all the horror stories about EDI and prefer to pay handsomely just to be left alone.

Here it is, I said it. New businesses are afraid of EDI and their perception of EDI is that it is 1. Complex and 2. Expensive. And they are right!

An example of sending the same purchase order from company ABC to company XYZ via VAN:

There are only two factors that affect the overall cost:

• How greedy is your management provider
• How much would it cost you to maintain an often turbulent relationship with your management provider

We’ve seen a fair number of companies over the years hoping to switch EDI providers or completely step away and implement in-house systems. Frankly, it’s also been hard for the EDI providers themselves as technology has moved on, a lot since the 90s, and the competition brings in all kinds of shiny new tools and methodologies, which they have to somehow patch into their legacy monoliths.

Also, remember the price, but most importantly the “hidden” price contributing to the total cost of ownership (TCO). This is what I meant by the second factor. It’s quite often that in order to not incur an additional cost, customers avoid making any changes to the established channel with their provider, and perform all adjustments/amendments to the always evolving EDI processing, themselves, outside of that channel.

Companies prefer to invest extra time and resources in a variety of tooling, under their control and outside their provider’s reach. The reason for this is 1. Cost and 2. Flexibility. Providers are notorious for overcharging and protracting the execution of any such tweak in time.

The obvious benefits of managed EDI are:

• Less involvement in EDI/agreements/mappings, etc. past the initial setup
• Not operating/maintaining separate infrastructure for EDI
• No need for EDI Software
• EDI is practically a black box and you exchange business information with all the companies across your network using your own data formats

The logical drawbacks are:

• Too expensive, however, getting more expensive by the term
• Lack of flexibility to change the channel once up and running, which more or less renders the “managed” label obsolete
• You can never tap in programmatically into the EDI data in case you wanted to

Okay, that’s great but what about APIs? How do they fare in cost? Well, this wouldn’t be a valid comparison, much like apples and oranges, however, the following would be accurate statements:

• APIs are widespread and the cost to host a web API is comparable to what you’d pay to your cloud hosting provider. The software to connect to APIs or build APIs is free and almost every programming language supports API.
• API resources are widely available. The latest StackOverflow developer survey shows that web technologies are the most popular among developers worldwide. You can find a web developer far easier and far cheaper than an EDI one.
• Data transport via APIs is secure, faster, and allows for massive data sets.
• You can easily integrate new APIs into your existing architecture and even reuse software components.
• APIs align well with all the latest practices in communication, software architecture, and the Web in general.
• The API specifications are free and collaborative. OpenAPI and Swagger are de-facto the go-to options when it comes to documenting APIs and are maintained by the likes of Microsoft, Oracle, RedHat, IBM, etc.
• API tooling is available as open-source, API adoption is fast due to the popularity of the web technologies and the developer communities.

I won’t go any further. You get the idea. APIs are prevalent, inexpensive, at the forefront of technology, and will play a vital role in the future web infrastructure.

Conclusion: EDI is expensive, API is cheap. Well, that’s it. “Digital transformation” remember? Only if it were somehow possible to combine the two and get a better, cost-effective alternative, with minimal up-front cost and huge potential for whatever it is to come.

Chapter 6: Why is EDI so complex?

So far I’ve covered some of the aspects that EDI vs. API style of comparison would normally find relatable. And I’ve concluded that from a technical, practical and financial point of view (that is on the part of the customer and not the managed provider), there are mainly positives in using API over EDI when it comes to security, reliability, speed, size, and cost. Or at least there are no downsides.

The biggest driver to NOT transform working EDI applications in any way has always been complexity. EDI is complex. Why change something that works? And why change something that is so complex that it might cause more grief than measurable gain?

I remember working on a project a long time ago, the goal of which was to create a better software engine that calculates the amount of money each GP practice in England needs to be paid. The reason was that the existing engine had a mind of its own, was hard to maintain and extend, and was producing suspicious results. In fact, no one in England knew why GP doctors were paid what they were paid. Fact.

It was so complex and with so many inputs/rules (like that if a patient was a smoker, the surgery was paid more) that 3 months into the project, and it was abandoned. It was impractical and costly to walk the walk. Better the devil you know.

The complexity of EDI stems from the same thing that was brought in to reduce that very same complexity. I know, it sounds absurd but isn’t it often like this in life?

Enter EDI Message Standards.

Chapter 7: EDI Message Standards

EDI’s main selling point is maintaining a rigid format for every business document. An abstraction of a document, let’s say a purchase order, or an invoice, that allows for all permutations and combination of data it can possibly ever need. A set structure with extension points in the form of repetitions, groups, optional elements, and EDI codes.

This format is governed by the EDI specification. EDI maintains a clear separation, the specification and the actual EDI message, are distinct entities. To be able to both code and decode an EDI message, you need the specification.

An example specification for a purchase order in EDIFACT:

An example EDI file for a purchase order in EDIFACT:

EDI message formats were meant to be the single representation of business documents. The lingua-franca of B2B. When company ABC wants to trade with company XYZ and exchange purchase orders, invoices, and dispatch advices, neither of the partners should care about data formats. It’s imperative that if both companies “know” EDI and exchange data in EDI format, there will be no issues whatsoever because they both communicate in the same language.

The reality, however, took a bit of a downturn. The 4 main contributors for this were/are:

• Collaboration, or the lack of it. EDI standardization is governed by a third party. ASC in America, UNECE in Europe, etc. They do release new versions of all business documents bi-annually, however, the pace of business is faster. To keep up companies have taken the liberty to negotiate amendments to the standards amongst themselves, bypassing the governing organizations, and hoping that they will be allowed to retrospectively sneak in those amendments into the official standards later.
• Power struggle. It’s widely known that VICS, UCS, and ASC had a breakout and whole industries took their own way in governing and maintaining EDI standards. Let’s face it, trusting a third party to dictate the shape of business formats is idealistic with a flavor of monopoly. This caused a lot of friction and businesses wanted not only control over their data but also the formats that mold this data.
• Natural. EDI has been here for a while. Multiply 2 versions every year, for every business document for the last 40 years. Now multiply this by the number of trading partners that connected with each other over the years. That’s like 70% of all B2B communication worldwide. It’s the nervous system of global trade. Managed EDI was created to solve this complexity which EDI was meant to solve in the first place but suffered from the same 2 issues as the ones above, collaboration failure and power struggle.
• Poor tooling. Standardizing business transactions is one thing. Enforcing this standard is another. A correct association would be to having separate institutions for lawmaking and law enforcement as we do in every democratic country. EDI is the law, but there is no law enforcement. This means that complying with EDI is voluntary, error-prone, and subject to interpretation. The core issue is the lack of machine-readable meta-format to represent EDI documents. Just look at the example above, it’s made for humans, not for machines.

From a purely technical perspective, this boils down to data formats and mapping. EDI lacks a formal machine-readable format. The EDI government bodies know how a purchase order should look like, the businesses exchanging EDI messages know how a purchase order should look like, however, the machines and software that actually transports and processes these purchase orders doesn’t know what a purchase order is. So it moves it around blindly, as a shapeless blob of data that could be anything.

The closest to a universal EDI format was SEF (Standard Exchange Format) devised by Foresight Corporation, now owned by tech giant Tibco. If you google it, it’s going to come back with a handful of actionable results. It’s powerful but a thing of the past. And still, it’s only a meta-format, lacking proper and affordable tooling. We, at EdiNation, offer our own SEF converter to OpenAPI for X12 and EDIFACT.

Let’s leave EDI aside for a moment and turn our attention to non-EDI communication, such as B2C, or the rise of Web and APIs. It surely must have similar problems, albeit not so heavily regulated?

The wonderful world of the modern Web presents us with the best, and often free, tools for manipulating data of any kind. OpenAPI specification, which started from Swagger, JSON schema, SOAP, and WSDL are the main standards for representing and validating documents in JSON or XML. The community and tooling around these technologies are abundant and promptly updated.

API specifications are the ultimate response to the issues EDI has which we covered earlier, such as document versioning, mapping, collaboration, and standards for describing modern APIs. What is more important is that all of the API specifications were developed to be both machine-readable and human-readable. They can be seamlessly automated and integrated for a fraction of the time and cost you’d need for any EDI implementation.

Conclusion: APIs have excellent, popular, updated, and often free tools. APIs have machine-readable and human-readable standards/specifications, such as OpenAPI. EDI has a limited and very expensive tooling. Most importantly, there isn’t a broadly adopted standard for representing and validating EDI documents.

Chapter 8: EDI and API, together

I think it’s glaringly obvious. The solution to all of EDI’s shortcomings. To bring EDI to the modern age it needs a practical solution to:

1. Have a new, free and machine-readable standard for representing and validating EDI messages, with adequate and free tools
2. Align well with APIs
3. Allow for easy migration of all existing specifications to this new standard

As I mentioned at the start of this guide, there are no technical impediments in combining EDI and API, on the contrary, there are only positives.

At EdiNation, we attacked this problem frontally and created:

1. A new standard for representing and validating EDI documents based on OpenAPI,
2. An EDI translator and validator API
3. A library of interactive, machine-readable and human-readable EDI specifications for all of the popular EDI standards like X12, HIPAA, EDIFACT, IATA, EANCOM, VDA, IAIABC, etc.
4. A tool to convert old SEF and EdiFabric specifications to the new OpenAPI standard

You are probably wondering what is the catch? How come no one thought of this before? Is it a scam? Well, it’s all available for you to evaluate, simply sign up and give it a go. It looks simple and rudimentary, but don’t take this the wrong way, we’ve invested thousands of hours in researching and developing a workable solution that runs EDI on API. The SEF importer alone took us two months to develop and test.

This brings me to another episode in my career, back in 2011, when I was tasked to implement the data source that would collect data for all visitors, guards, personnel, and general staff within the London 2012 Olympic venues. It was a vital piece of data that other applications would use to display and render on screens across control rooms and produce reports from. I can’t disclose any technical details but can safely say that the solution I came up with back then was powerful but at the same time so simple that some people didn’t believe it did what I claimed it did. It was antithetical to how people would usually approach such a problem. But it worked. London Olympics 2012 was labeled as one of the best-organized sports events in history. It’s complex to make things simple.

This guide was supposed to provide a comparison between EDI and API and offer a solution for them to co-exist. I’m not trying to lean one way or another, just present the facts and what is possible and available, in my own view.

What we created with EdiNation empowers EDI to:

Technical

• Have a human-readable and machine-readable standard for representing EDI documents, based on OpenAPI. Here is how the format of an X12 837 P medical claim looks like.
• Describing EDI formats with OpenAPI in YAML or JSON means that interpreting and consuming EDI data would be native for most programming languages.
• Translation and validation between EDI messages and their OpenAPI representations are seamless. Consumers can choose whether to exchange EDI data as EDI messages or directly as “JSON”.
• Automatic import of SEF specifications for X12 and EDIFACT.
• Resolve the versioning and partner-specific dialects hell by maintaining distinct data models and benefiting from the nature of JSON serialization.

Strategical

• Lower the bar for adopting EDI or replacing existing EDI systems.
• Reduce the go-to-market time for EDI implementations.
• Reduce the cost of developing, owning, and maintaining EDI solutions in-house.
• Reuse existing code and integrations and apply them to EDI. You don’t need separate software or hardware dedicated to EDI only.
• Collaborate on data formats quicker and more accurately by exchanging machine-readable EDI message formats.
• Hop on the modern infrastructure for APIs and the Web, and control your data and processes.
• Onboard new trading partners in hours not weeks.
• The OpenAPI EDI standard is free, anyone can create tooling for it, we do not hold any IP rights over it and happily share it with the world.

The apparent drawback is that this is new and requires looking at EDI from a different angle. Oh, and it’s only applicable to point-to-point EDI integrations, right? Not really!

What we build in EdiNation is not a full-fledged EDI product. It’s a standard and the tooling around it to make it usable. All bundled up in a web API. You can think of it as the crafting table in Minecraft, but for EDI.

We’ve assisted companies (as both EdiFabric and EdiNation) to build:

• VANs
• Logistics platforms
• ERPs
• EDI servers
• Medical claims platforms
• Supply chain software add-ons for EDI
• Amazon vendor integration
• Airline ticketing and availability systems
• Insurance software
• Government institutions and educational software
• many more…

We’ve witnessed the wide adoption of the principles outlined above, because businesses evolve and adapt, and favor realistic solutions to harness the benefits of both EDI and API in contrast to having to choose one or the other and get lost in silos.

What’s even better is that you might already be halfway there. If you represent a company that exchanges EDI files with trading partners, I won’t be surprised if you already use EdiFabric or EdiNation internally, without knowing it. Ask around the developer teams, we have thousands of implementations worldwide. Or perhaps your trading partners do? If so, consider surfacing that implementation to the outer layer of your connection points and expose it to the world. You’ll be astonished by the result.

Originally published at https://docs.edination.com on June 17, 2020.