Kubernetes - Communication Between Services

Prerequisites

  • Docker
  • Minikube

Description

In this post I show a simple example of how services communicate with each other inside the kubernetes cluster. First I’ll explain the scenario, then build the application.

Scenario

In the previous post Kubernetes - Exposing Two Services Using Ingress we had two services and exposed them outside the cluster using the ingress controller. The user could send a request to the service of choice depending on their workflow. However say they only have one workflow - they want to find the named entities from images of text. Using the previous architecture, they would have to make two requests.

First they would have to send the image to the ocr-service and get the text, then they would have to send that text to the ner-service to extract the named entities. To make the process more efficient for the use case, we will introduce a third service following the orchestrator design pattern. The user will get the named entities that are present in the text of the image in only one request.

The orchestrator-service will be responsible for accepting the image, sending it to the ocr-service, receiving the extracted text, then sending that text to the ner-service. Finally it will respond with the named entities to the original request.

Communication between services

This scenario requires the orchestrator-service to communicate with the ner-service and ocr-service inside the cluster. If you run these three services on a local machine you would point the orchestrator to some URL like http://localhost:80/api/endpoint. However, inside the cluster there is no common IP that all pods share such as ‘localhost’

Each pod that is created is assigned its own unique IP. If you delete a pod and Kubernetes recreates it, the pod that takes its place will have a new IP. Therefore there is no single IP we can use to comunicate with.

DNS

Domain Name System (DNS) is a system for associating IP addresses with easy to remember names. Kubernetes automatically configures an internal DNS service that provides a method for service discovery. This service is called CoreDNS. It listens for service and endpoint events which are triggered when you create, update or delete Kubernetes services and their associated pods. When those events occur CoreDNS updates its DNS records.

Every service defined in the cluster (including the DNS server itself) is assigned a DNS name. You use these names to target services from within the cluster instead of using IP address. The names are of the form

servicename.namespace.svc.cluster.local

Services can be resolved by the service name alone from within the same namespace. Pods can access services in other namespaces by specifying the namespace.

In this example we will deploy everything within the same (default) namespace, so we will target a URL that looks like

http://servicename:80

As long as you keep the service names consistent between deployments, your services will always be able to locate each other even if the underlying pod IP addresses are constantly changing.

In this example, the orchestrator microservice will need to communicate with other services inside the cluster. It does so by targeting the service names as defined in their kubernetes specification files. Note how each address has been set to the name of the service.

Build Docker Images

First make sure to point your shell to the minikube docker daemon. To get the command for your shell run minikube docker-env.

For example on Windows we use the following command

For more information please refer to the section Use local images by re-using the Docker daemon

Finally, make note of the minikube IP address by running this command.

minikube ip

ner-service

From the ner-service folder run docker build . -t ner

ocr-service

From the ocr-service folder run docker build . -t ocr

orchestrator-service

From the orchestrator-service folder run docker build . -t orchestrator

Create Kubernetes Deployments

In the root of the folder for the sample code (where all the yaml files are located) apply all three yaml deployments for the three services.

kubectl apply -f ner-deployment.yaml
kubectl apply -f ocr-deployment.yaml
kubectl apply -f orchestrator-deplyment.yaml

Create NodePort Service

In this example I apply the nodeport-service to expose the orchestrator outside the cluster. Note this is just for testing that the orchestrator is able to communicate with the two services and returns the expected result. 

kubectl apply -f nodeport-service.yaml

Test

Run kubegtl get services, you should see three services of type CLusterIP and one NodePort service.

Then using the following command create a POST request with the body containing the test image to the endpoint api/NER on the minikube IP, port 30001 as follows

curl --verbose --request POST --header "Content-Type:multipart/form-data" --form "file=@test-image.jpg"  192.168.64.4:30001/api/NER`

You should see the following output

When finished remove the nodeport service by running

kubectl delete -f nodeport-service.yaml

Sample Code

References