Back in 2015, I was leading an engineering team developing a digital textbook rental platform. At the time, microservices were relatively new, but we decided that the flexibility and scalability made them a good choice for our platform.
Back then, adopting loosely coupled microservices was really challenging. The tooling wasn’t as good as it is today, there weren’t nearly as many people using microservices at startups, and many of the terms we use now hadn’t been coined yet.
Things have come a long way in the past six years, and one of the most exciting developments is the rise of third-party API-based platforms and service meshes. Now, instead of building all your microservices from the ground up, you have the option to integrate with pre-built services to add backend functionality to your application in an instant.
In this post, I’ll introduce the idea of a service mesh as a means of binding your internal and third-party services together. I’ll use a microservices-based e-commerce architecture for my examples, but many of the concepts introduced in this post will help you create better integrations between any application and the third-party services it relies on.
A service mesh describes a set of infrastructure components that allow microservices to communicate with one another. In contrast to an API gateway that sits in front of your public-facing HTTP layer, a service mesh connects the internal elements of your application and dictates how they can interact with one another.
A common use case for a service mesh is connecting containers in a Kubernetes deployment. Because containers are isolated from one another, the service mesh provides network rules that dictate which containers can pass data to others and how. Engineers often use sidecar deployments to accomplish this, but implementation details can vary.
While typically used to marshal data between pieces of your core application, a service mesh is also useful for standardizing connections to integral third-party services.
“With the service mesh pattern, we are outsourcing the network management of any inbound or outbound request made by any service (not just the ones that we build but also third-party ones that we deploy) to an out-of-process application.”
For example, let’s say you’re building an e-commerce application. You might start off with a Cart Service, Order Service, and a Product Information Management (PIM) Service:
As your application and product catalog scales, you might realize that it’s a better investment of your engineers’ time to focus on the user experience than building a PIM from scratch. So, you decide to integrate with a third-party PIM like fabric.
Next, you realize that you need to deliver multichannel notifications to your users, so you integrate with another third-party tool like Courier or Nexmo. You can use your service mesh to proxy calls to these third-party services and control the flow of data to your existing internal services.
The service mesh pattern ensures that communication between your internal and external services happens seamlessly, and allows you to more easily swap between in-house and third-party solutions as your application evolves. Because the service mesh acts as an interface between your services, you shouldn’t have to refactor all your services every time you update one.
While the most universal purpose of a service mesh is to connect services across a network, it isn’t limited to passively transporting data. Because your service mesh touches all requests between services, it sits in a perfect spot to handle several key application functions:
While the above functions apply generally to any service mesh, there are some specific considerations you should make if you’re building a service mesh for an e-commerce application. First, let’s take a look at how a service mesh might connect the various microservices an e-commerce application is likely to use:
Your e-commerce service mesh will be responsible for making sure all your various APIs are able to connect and stay active. For example, the service mesh will handle encryption between your Auth Service and your Order Service to keep user data safe. It will ensure that your PIM scales up or incorporates load balancing to handle failures. And it will log errors when an item without a price (from the Promotions Service) is added to an order (via the Orders Service).
To start building a service mesh for your e-commerce application:
Whether you use an off-the-shelf service mesh or you build your own microservice integration layer, there are a few things that are critical to consider. In this section, I’ll review some of the most pertinent challenges.
Every step in a networked request adds latency to the total time it takes to receive data from your application. Many of the biggest challenges in using microservices are related to the speed of network requests, and because speed is so important to e-commerce applications, your service mesh needs to be prepared for this.
Unfortunately, it’s not easy to control network requests, especially when you rely on third-party solutions.
“It’s important for developers to understand the limitations of a third-party service before relying on them at scale. Can they keep up with your expected demand while maintaining the performance you require? Is their stated SLA compatible with yours?”
There are a few ways you can mitigate speed issues using a service mesh, though:
Improved security almost always comes at the cost of speed, but depending on the data moving across your service, it might be worth the tradeoff. Giving your service mesh responsibility for authenticating and authorizing requests between your services allows you to centralize your security rules and makes keeping them up to date much easier.
“The softest target in most organizations is the app layer, and attackers know this. Microservices thus both make this problem harder and easier for the defenders. Harder, because as apps are decomposed, the amount of network interdependencies grows and most dev teams themselves don’t fully understand the connectivity mesh of their apps.”
Security between microservices is part of the challenge, but the other part is data privacy across services. For example, some countries impose data storage laws that don’t allow identifying information to leave their country. If you’re operating a distributed e-commerce application with customers around the world, it’s important that your service mesh can handle data localization requirements.
A significant obstacle we faced when implementing microservices in 2015 was tracing and understanding errors, and we weren’t alone.
“One of the biggest challenges that we hear from users is how difficult it is to capture metrics and data from their microservices… This is why most organizations in the ‘evaluation’ stages of adopting microservices are unsure when they will be able to deploy production applications with this type of architecture.”
Often, users complained about cryptic error messages, and our engineers struggled to figure out where in the request’s lifecycle our application failed. By implementing observability, logging, and error handling at the service mesh layer, you can have a much clearer picture of what’s going on in your microservices.
This is something you’ll also have to consider when integrating with third-party APIs. If they provide clear HTTP error codes and messages, your service mesh can interpret these failures and pass errors up the chain appropriately. If not, you’ll have a really hard time understanding what went wrong.
Whether you’re building microservices or integrating data from third-party e-commerce APIs, you’ll need to think about how data moves across the network. Too many teams treat this as an afterthought by building independent services and allowing them to communicate directly, but by adding a service mesh, you can abstract these connections away. This allows you to build a more performant, secure, and scalable distributed application.
Building a service mesh yourself is a big undertaking, so if you’re using container-based microservices check out Consul, Istio, and Kuma. Finally, if you’re looking to integrate a powerful headless e-commerce platform into your microservices-based architecture, take a look at Fabric. Our team can help you build a robust backend for your online store or mobile application without spending months of engineering time.
Tech advocate and writer @ fabric. Previously CTO @ Graide Network.