58 Dispatches · 5 Desks · 68 Topics · 3 Series

Unofficial Azure Club

“IaaS, PaaS, Cloud Native, Kubernetes, Everything is possible in this website :)”

Cloud Native C17

Building a WASM Serverless Solution with KEDA HTTP Add-on and Slight Containerd Shim

By Yingting Huang 6 min read

Introduction

In this article, we will explore how to use KEDA, the KEDA HTTP add-on, and Slight Containerd Shim to build a WASM serverless solution. The solution provides scale-to/from-zero support, reduced cold-start time, and cloud integration capabilities such as Azure Blob, App Configuration, Service Bus, and more. The diagram below shows how it works.

WASM serverless architecture
WASM serverless architecture
  • Incoming HTTP requests are routed through Nginx Ingress Controller to the KEDA HTTP interceptor. The interceptor keeps track of the number of pending HTTP requests: requests that it has forwarded but the app has not returned yet.
  • The KEDA HTTP add-on operator runs inside the Kubernetes cluster, watches for HTTPScaledObject resources, and creates a ScaledObject for the Deployment specified in the HTTPScaledObject resource.
  • ScaledObject points to the interceptor as the KEDA external scaler and uses gRPC to get the size of the pending queue. Based on this queue size, it reports scaling metrics to KEDA. As the queue size increases, the scaler instructs KEDA to scale up as appropriate.
  • KEDA manages 0<->1 scaling and leverages HPA for 1 <-> n scaling.
  • Kubernetes watches for WASM Pods being created, notices that their runtime is wasmtime-slight-v1, and leverages the Slight containerd shim to start the SpiderLightning host runtime for the WASM application.
  • The WASM application receives the HTTP request and replies back.

Before jumping into the implementation, let’s take a closer look at the components used in the solution.

WASM

WebAssembly (WASM) is an open standard that defines a portable binary-code format for executable programs. It is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications.

WASM allows serverless applications to be faster and more efficient. WASM enables the deployment of scripts and programs that can run on the serverless environment with fewer resources than traditional scripting languages. This makes serverless applications more cost-effective, as well as faster and more reliable. Additionally, using WASM to develop serverless applications allows for easier portability and integration with different platforms.

The Open Container Initiative (OCI) provides support for packaging a WASM application into a container image. Generally speaking, a minimal WASM container image is typically a few hundred kilobytes in size. For example, a sample WASM container image used in this solution is only 64.7KB.

IMAGE                                                               TAG                     IMAGE ID            SIZE
…
ghcr.io/huangyingting/rust-slight                                   main                    8ce9ccccf9eea       64.7kB
…

KEDA and KEDA HTTP Add-on

KEDA is a Kubernetes-based Event Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed.

KEDA provides a reliable and well-tested solution for scaling your workloads based on external events. However, KEDA doesn’t provide an HTTP-based scaler. The KEDA HTTP Add-on allows Kubernetes users to automatically scale their HTTP servers up and down (including to/from zero) based on incoming HTTP traffic. The diagram below is copied from the KEDA HTTP Add-on design page and shows how it works.

KEDA HTTP add-on architecture
KEDA HTTP add-on architecture

SpiderLightning and Slight Containerd Shim

SpiderLightning (Slight) is an open source WASM host (based on WasmTime) for building and running fast, secure, and composable cloud microservices with WebAssembly. It defines a set of WebAssembly Interface Types (i.e., WIT) files that abstract distributed application capabilities, such as state management, pub/sub, event-driven programming, and more.

Slight Containerd Shim is a containerd shim powered by the SpiderLightning engine. It allows you to run WASM applications developed with SpiderLightning SDKs (C and Rust are currently supported).

Implementation

The solution implementation requires deploying several components, including:

  • Slight Containerd Shim
  • Nginx Ingress Controller
  • KEDA and KEDA HTTP add-on
  • Redis - used by demo application to illustrate SpiderLightning capabilities
  • Sample application and manifests

The detailed steps are below.

Install Slight Containerd Shim

Refer to “Deis Labs Containerd Wasm Shims” section from my previous article Run WASM applications from Kubernetes

Deploy Nginx Ingress Controller

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx

Deploy KEDA and KEDA HTTP add-on

Deploy KEDA

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
kubectl create namespace keda
helm install keda kedacore/keda --namespace keda

Deploy KEDA HTTP add-on

helm install http-add-on kedacore/keda-add-ons-http --namespace keda

Deploy Redis

 helm repo add bitnami https://charts.bitnami.com/bitnami
 helm repo update
 helm install redis bitnami/redis

Sample Application and Manifests

A sample WASM application is created to read/write/delete Redis cache. The source code is located in this repo.

To deploy this application, follow these steps:

  • Generate a YAML file with the content below. Replace REPLACE_IT_WITH_REDIS_ADDRESS with the Redis address. If you followed the Redis deployment step above, the Redis address will generally be redis://redis-master.redis:6379. Replace REPLACE_IT_WITH_FQDN with an FQDN name for external access, for example, rust-slight.yourdomain.com.
apiVersion: v1
kind: Namespace
metadata:
  name: rust-slight
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rust-slight
  namespace: rust-slight
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rust-slight
  template:
    metadata:
      labels:
        app: rust-slight
    spec:
      runtimeClassName: wasmtime-slight
      containers:
        - name: rust-slight
          image: ghcr.io/huangyingting/rust-slight:main
          imagePullPolicy: IfNotPresent
          command: ["/"]
          env:
          - name: REDIS_ADDRESS
            value: REPLACE_IT_WITH_REDIS_ADDRESS
---
apiVersion: v1
kind: Service
metadata:
  name: rust-slight
  namespace: rust-slight
spec:
  type: ClusterIP
  ports:
    - protocol: TCP
      port: 3000
      targetPort: 3000
  selector:
    app: rust-slight
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rust-slight
  namespace: keda
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: REPLACE_IT_WITH_FQDN
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: keda-add-ons-http-interceptor-proxy
            port:
              number: 8080
---
kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: rust-slight
    namespace: rust-slight
spec:
    host: REPLACE_IT_WITH_FQDN
    targetPendingRequests: 200
    scaleTargetRef:
        deployment: rust-slight
        service: rust-slight
        port: 3000
    replicas:
        min: 0
        max: 10
  • Run kubectl apply -f manifest.yaml to deploy the application
  • Notice the application is scaled to 0 as currently there is no HTTP request
kubectl get deploy -n rust-slight 
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
rust-slight   0/0     0            0           34s
  • Send an HTTP request to see what happens
curl http://rust-slight.yourdomain.com/read
  • After 1 or 2 seconds, our WASM application echoes HTTP request back
user-agent: curl/7.81.0
accept: */*
x-forwarded-for: X.X.X.X, 192.168.2.19
x-forwarded-host: rust-slight.yourdomain.com
x-forwarded-port: 80
x-forwarded-proto: http
x-forwarded-scheme: http
x-real-ip: X.X.X.X
x-request-id: 6ccba5facab304b26cbb129b92fca9f6
x-scheme: http
accept-encoding: gzip
  • Check deployment again, it is scaled out to 1
kubectl get deploy -n rust-slight 
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
rust-slight   1/1     1            1           2s
  • Application itself supports redis read, write and delete, here are some sample commands that you can play with
# Write
curl -X PUT -d '{"key": "version", "value": "1.0.0"}' http://rust-slight.yourdomain.com/create
# Update
curl -X POST -d '{"key": "version", "value": "1.0.1"}' http://rust-slight.yourdomain.com/update
# Delete
curl -X DELETE -d '{"key": "version"}' http://rust-slight.yourdomain.com/delete
# Read
curl http://rust-slight.yourdomain.com/read

Related dispatches