Tutorial: Configure, Deploy an Edge Application on Cloud Native Edge Infrastructure

This is the last installment of the 4-part series (Part 1) (Part 2) (Part 3) where we configure and deploy an artificial intelligence workload running on an “edge” Internet of Things (AIoT) workload on cloud native edge infrastructure. The application does predictive maintenance of turbines. For the background and explanation of the use case, refer to the previous part of the tutorial.

👁 Image

Start by cloning the GitHub repository that contains the code, configuration, and Kubernetes deployment manifests. Feel free to explore the source code, configuration, and manifests available in the repository. You can build container images for each of the services to store them in a private registry.

As we start deploying each microservice, I will discuss the design decisions and configuration choices.  If you have a K3s cluster configured with Project Calico and Portworx, you can deploy the entire application without building the images.

kubectl apply -f deploy/

If you want to deploy one artifact at a time, start with the namespace. The workload runs in a dedicated namespace, aiot.

kubectl apply -f deploy/aiot-ns.yaml

Deploying the Mosquitto MQTT Broker

The Mosquitto pod acting as the MQTT broker relies on a persistent volume claim to persist logs and data. This PVC uses dynamic provisioning based on a Portworx storage class which is optimized for databases. The same storage class is used for the InfluxDB time-series database.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
 name: db-sc
provisioner: pxd.portworx.com
parameters:
 io_profile: "db_remote"
 repl: "3"

Deploy the storage class followed by the Mosquitto service.

kubectl apply -f deploy/db-sc.yaml

kubectl apply -f deploy/mosquitto.yaml

Deploying Fan Simulators

The fan simulators are configured as pods with environment variables pointing to the MQTT broker and the topic associated with the telemetry. The environment variable FAULT will decide if the simulator publishes anomalous data. The DEVICE_ID variable assigns an arbitrary value to the device identifier.

 initContainers:
 - name: wait-for-mosquitto
 image: janakiramm/wait
 args: ["--timeout=60", "mosquitto:1883"]
 containers:
 - name: fan-1
 image: janakiramm/fan
 imagePullPolicy: Always
 env:
 - name: MQTT_HOST
 value: "mosquitto"
 - name: MQTT_TOPIC
 value: "fan/messages"
 - name: FAULT
 value: "0"
 - name: DEVICE_ID
 value: "1"

The InitContainer will wait for 60 seconds for the Mosquitto broker to become available before timing out. This will ensure that the pod doesn’t experience CrashLoopBackOff while waiting for the dependent service.

Deploy two fans one of which is configured to ingest anomalous data.

kubectl apply -f deploy/fan-1.yaml

kubectl apply -f deploy/fan-2.yaml

Now, we have two simulators publishing telemetry to an MQTT topic.

Deploying AI Inference and Prediction Service

The inference pod downloads the latest version of the model and exposes that as a REST endpoint. An InitContainer checks for the presence of the model in a well-known directory and pulls the compressed TensorFlow model only if it’s missing. This approach avoids downloading and copying the model each time the deployment is scaled.

As the inference service scales, the model needs to be available to multiple pods through a shared filesystem. To enable this, we define a different Portworx storage class with the SharedV4 flag to create a globally shared namespace volume that can be used by multiple pods.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
 name: infer-sc
provisioner: pxd.portworx.com
parameters:
 repl: "1"
 sharedv4: "true"

kubectl apply -f deploy/infer-sc.yaml

After creating the storage class, let’s create the inference service.

kubectl apply -f deploy/infer.yaml

With the inference service in place, we can deploy the predictor microservice that acts as the intermediary between the MQTT broker and the AI model.

The values related to the MQTT broker and the endpoint of the inference service are passed as environment variables.

 containers:
 - name: predict
 image: janakiramm/predict
 env:
 - name: MQTT_HOST
 value: "mosquitto"
 - name: MQTT_DEV_TOPIC
 value: "fan/messages"
 - name: MQTT_PREDICT_TOPIC
 value: "fan/anomaly"
 - name: SCORING_URL
 value: "http://infer:5000/score"

kubectl apply -f deploy/predict.yaml

Since the inference microservice is using shared volume, we can easily scale the number of replicas.

kubectl scale deploy/infer --replicas=3

Deploying Telegraf and InfluxDB

InfluxDB is used as a time-series database to store the telemetry data coming from the fan simulators and the prediction service. It is configured as a stateful set backed by the PVCs created from the same Portworx storage class used by Mosquitto.

kubectl apply -f deploy/influxdb.yaml

Telegraf connects InfluxDB with Mosquitto through a configuration file which is created as a Kubernetes config map.

kubectl apply -f deploy/telegraf.yaml

Check the logs of the Telegraf pod to confirm the flow of messages from Mosquitto to InfluxDB.

TELEGRAF_POD=$(kubectl get pods -n aiot -l app=telegraf -o jsonpath='{.items[0].metadata.name}')

kubectl logs -f $TELEGRAF_POD

👁 Image

Deploying Grafana and Configuring the Dashboard

A config map associated with Grafana pod configures InfluxDB as the datasource. This bundling helps us quickly import an existing dashboard.

apiVersion: v1
kind: ConfigMap
metadata:
 name: grafana-datasources
 namespace: aiot
data:
 influxdb.yaml: |-
 {
 "apiVersion": 1,
 "datasources": [
 {
 "access":"proxy",
 "editable": true,
 "name": "influxdb",
 "orgId": 1,
 "type": "influxdb",
 "database": "fan",
 "url": "http://influxdb:8086",
 "version": 1
 }
 ]
 }

Deploy Grafana with the below command:

kubectl apply -f deploy/grafana.yaml

After accessing the Grafana dashboard, login with admin/admin and change the password. Go to the manage dashboards section and click on the import button.

👁 Image

Copy the content of fan.json from the dashboard directory, paste it into the textbox, and click the load button.

👁 Image

With the configuration in place, you would be able to access the below dashboard:

👁 Image

Configuring Network Policies

Since the K3s cluster is configured with Calico-based CNI, we can secure the network. A sample policy to prevent access to the AI inference is included in the deploy directory.

kubectl apply -f deploy/netpol.yaml

Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.

Feature image by Dmitrii Bardadim from Pixabay.