Load Balancing gRPC on Kubernetes using Istio

About gRPC

What is gRPC?

gRPC is a modern open source high performance Remote Procedure Call (RPC) framework, efficiently connecting connecting services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication

Why gRPC?

  1. Performance
  • HTTP/2 protocol is compact and efficient both in sending and receiving
  • gRPC messages are serialized using a format called Protobuf, which is up to 6 times faster than JSON
  1. Streaming (read more)
  2. Deadline/timeouts and cancellation (read more)

gRPC Load Balancing on Kubernetes/Openshift

The Problem

gRPC leverages the HTTP/2 protocol which sends multiple packets over a single connection, because of the multiplexing feature. In comparison, HTTP creates and closes connections per request. The issue is, native Kubernetes Services only split requests by connection. Due to this, gRPC services hosted as K8s deployments ends up reusing existing connections and sending new requests to the same k8s Pod.

The Solution

There are two options to load balance gRPC requests on Kubernetes

  • Using a Headless service
  • Using a Proxy (Istio!)

Balancing gRPC Traffic using Istio

Pre-requisites

With Istio installed on the cluster, the next step is to inject proxy containers into the application pods. The simplest way to do this is to enable auto injection to your namespace, as follows:

$ kubectl label namespace <your namespace> istio-injection=enabled

This ensures that any deployment that is in the namespace, is injected with proxy sidecars.

Now, you will observe that the gRPC load is now being balanced across the pods.

Istio Traffic Management

With this default set up of Istio with K8s, Istio now distributes traffic across each service’s pool of pods using a round-robin model. However, this basic service discovery and load balancing can be easily be improved, to have a better control over traffic flow, giving power for A/B testing, the freedom to choose your load balancing strategy and set inbound rules for your microservices.

This can be done using Istio’s specific K8s custom resource definitions (CRDs). These are:

Virtual services & Destination rules are the key building blocks for traffic management using Istio.

Virtual Services

A virtual service lets you configure how requests are routed to a service within an Istio service mesh, building on the basic connectivity and discovery provided by Istio and your platform.

(Example)

Destination Rules

While the virtual service let’s you configure how traffic is routed, A destination rule dictates what happens to the traffic for that destination. It allows you to set your preferred load balancing strategy. The different options are:

  • Random: Requests are forwarded at random to pods in the destination deployment.
  • Weighted: Requests are forwarded to pods according to a specific percentage.
  • Least requests: Requests are forwarded to instances with the least number of requests.

And that’s it! We have successfully load balanced gRPC traffic on K8s!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store