Load Balancing gRPC on Kubernetes using Istio
About gRPC
What is gRPC?
gRPC is a modern open source high performance Remote Procedure Call (RPC) framework, efficiently connecting connecting services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication
Why gRPC?
- Performance
- HTTP/2 protocol is compact and efficient both in sending and receiving
- gRPC messages are serialized using a format called Protobuf, which is up to 6 times faster than JSON
gRPC Load Balancing on Kubernetes/Openshift
The Problem
gRPC leverages the HTTP/2 protocol which sends multiple packets over a single connection, because of the multiplexing feature. In comparison, HTTP creates and closes connections per request. The issue is, native Kubernetes Services only split requests by connection. Due to this, gRPC services hosted as K8s deployments ends up reusing existing connections and sending new requests to the same k8s Pod.
The Solution
There are two options to load balance gRPC requests on Kubernetes
- Using a Headless service
- Using a Proxy (Istio!)
Balancing gRPC Traffic using Istio
Pre-requisites
- Istio
- gRPC services
- K8s Deployment and Service configurations
With Istio installed on the cluster, the next step is to inject proxy containers into the application pods. The simplest way to do this is to enable auto injection to your namespace, as follows:
$ kubectl label namespace <your namespace> istio-injection=enabled
This ensures that any deployment that is in the namespace, is injected with proxy sidecars.
Now, you will observe that the gRPC load is now being balanced across the pods.
Istio Traffic Management
With this default set up of Istio with K8s, Istio now distributes traffic across each service’s pool of pods using a round-robin model. However, this basic service discovery and load balancing can be easily be improved, to have a better control over traffic flow, giving power for A/B testing, the freedom to choose your load balancing strategy and set inbound rules for your microservices.
This can be done using Istio’s specific K8s custom resource definitions (CRDs). These are:
Virtual services & Destination rules are the key building blocks for traffic management using Istio.
Virtual Services
A virtual service lets you configure how requests are routed to a service within an Istio service mesh, building on the basic connectivity and discovery provided by Istio and your platform.
(Example)
Destination Rules
While the virtual service let’s you configure how traffic is routed, A destination rule dictates what happens to the traffic for that destination. It allows you to set your preferred load balancing strategy. The different options are:
- Random: Requests are forwarded at random to pods in the destination deployment.
- Weighted: Requests are forwarded to pods according to a specific percentage.
- Least requests: Requests are forwarded to instances with the least number of requests.
And that’s it! We have successfully load balanced gRPC traffic on K8s!
