Integration challenges in microservices architecture with gRPC & REST

Posted on February 11, 2022 by Rakesh Girija Ramesan Nair and Gourishankar ValsalakumariNeelakantapillai

CNCF projects highlighted in this post

Guest post by Rakesh Girija Ramesan Nair, Senior Technology Architect with Infosys Limited, and Gourishankar ValsalakumariNeelakantapillai, Technology Lead with Infosys Limited

Abstract: This blog is envisaged to explain the integration challenges that technologies like gRPC and REST bring into E2E microservices architecture. It summarizes the problems evident currently while implementing microservices which needs to primarily 

1. communicate internally between services and
2. externally to user or 3^rd party systems

It also provides distinct insights on the absence of a standardized solution across the technology stack from any of the available open-source frameworks.  

Introduction

Adoption of microservices architecture is on the rise and is being widely accepted for the flexibility including maintainability and scalability that it brings to the table. With containerization, microservices architecture becomes even more powerful allowing users to create applications focusing on their capabilities rather than solving the dependencies. Cloud native application development is powered by microservices architecture using containers.

Distributed systems are complex to design and becomes even more complicated with the varying nature of business requirements For achieving E2E business capability multiple microservices need to be interconnected or invoked. Choice of integration technologies become critical, the common approach being followed currently is for any inter-service communication leverage gRPC (Google Remote Procedure Call) and for any client facing services leverage REST (Representational State Transfer) API.

gRPC – follows RPC API implementation leveraging HTTP 2.0 protocol and protocol buffers for message exchange.

REST – architecture follows HTTP protocol and data format used for messaging is JSON or XML

Problem statement

The challenge in designing and developing a capability which need to be consumed internally by other services and exposed to 3^rd party systems or users 

Let us consider a sample scenario, of an order management system consisting of an order manager and product inventory microservice. 

• The Product Inventory services holds all the product details & its relationships including the various categories. REST API is required to expose the product details & its relationships to external systems and user interfaces.
• The Order Manager service interfaces with another digital channel which acts as the front-end system for customer ordering. This internally invokes the Product Inventory service to validate the product inventory details.

In the current scheme of things there are multiple approaches possible where we could address such a requirement, few such options are detailed below: –

Option 1:

Following an approach where any inter-service communication leverage gRPC and for any client-facing services leverage REST.

• Expose the Product Inventory service via gRPC for inter-service communication 

We have used Protobuf definition for the contract and used java to generate server-side implementation.

• Additional coding like creating a rest controller and translation of response is required to expose the same as REST API for 3^rd party systems to consume.

Additional coding complexity and dependency management for handling both gRPC and REST.

Follow microservices aggregator pattern,

Option 2:

• Create an aggregator service which would expose the REST API capabilities by aggregating the responses from different services or have the wrapper REST API service implemented. This would also have the gRPC client implementation that is required to communicate with other internal services for aggregating the response. Translation code to create API response from protocol buffers will be included here. 

gRPC and protocol buffer forces developers to strictly adhere to a contract ensuring that messages are secure and not lost between communications. While the contract first nature and co-developed approach of defining RPC is good between related services, the aggregator service brings an overhead.

Inference

Architects put a lot of thought into designing distributed systems. Having an efficient integration pattern defined is key to the success of the solution.  

Below is a summarized view of the various integration options and the challenges: –

• Expose data as REST (JSON based) internally and externally: -This approach which is most popular, unfortunately does not cater for all requirements. This is not ideal for data intensive inter service communications due to the limitations of JSON payloads and HTTP protocol.
• Expose gRPC internally and externally: – Data exchange happens in binary format and is not humanly readable. gRPC relies on HTTP2.0 which has limited support from modern browsers.
• Create REST and gRPC :- 
  • As explained in the options earlier additional coding and integration overhead.
  • Lack of mature gRPC implementation across technologies like java python or node from any of the widely adopted open-source frameworks.

It is important to think through and consider these various integration patterns while we consider designing our next microservices based solutions.