Multi-user MQTT platform

Mosquitto is usually the first candidate to come to mind when looking for an MQTT broker. However, by default, Mosquitto manages users using a password file. This makes it difficult to easily add or remove users, especially when the broker is deployed in Kubernetes.

I recently stumbled upon the mosquitto-go-auth plugin, which allows for the authentication to be performed by an external service, by connecting to the latter via, for example, HTTP requests.

This is ideal as it allows to outsource the authentication to my user management and authentication service. As the requests sent by the plugin are not exactly in the format expected by the user management service, I created a simple gateway service that proxies the requests and converts them accordingly.

Thanks to this architecture, user accounts can easily be created and the credentials of those accounts can be used to authenticate connecting MQTT clients. On top of that, the MQTT authentication gateway is configured to only allow users to publish and subscribe to topics that begin with their username, thus preventing the access to someone else's data.

Moreover, with an appropriate use of configmaps, Mosquitto can be deployed entirely in Kubernetes while provided an SSL secured MQTT broker.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mosquitto-data
  namespace: iot
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mosquitto
  namespace: iot
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mosquitto
  template:
    metadata:
      labels:
        app: mosquitto
    spec:
      containers:
      - name: mosquitto
        image: iegomez/mosquitto-go-auth
        ports:
        - containerPort: 8883
        - containerPort: 9001
        volumeMounts:
        - mountPath: "/etc/mosquitto/"
          name: config
        - mountPath: "/mosquitto/certs/"
          name: certs
        - mountPath: "/mosquitto/data/"
          name: data
      volumes:
      - name: config
        configMap:
          name: mosquitto-config
      - name: certs
        secret:
          secretName: mosquitto-certs
      - name: data
        persistentVolumeClaim:
          claimName: mosquitto-data
---
apiVersion: v1
kind: Service
metadata:
  name: mosquitto-nodeport
  namespace: iot
spec:
  type: NodePort
  selector:
    app: mosquitto
  ports:
  - port: 8883
    nodePort: 30883
---
apiVersion: v1
kind: Service
metadata:
  name: mosquitto-clusterip
  namespace: iot
spec:
  type: ClusterIP
  selector:
    app: mosquitto
  ports:
  - port: 9001
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mosquitto-config
  namespace: iot
data:
  mosquitto.conf: |

    persistence true
    persistence_location /mosquitto/data/

    log_dest stdout

    # Raw MQTT with TLS (MQTTS)
    listener 8883
    protocol mqtt

    cafile /mosquitto/certs/ca.crt
    keyfile /mosquitto/certs/tls.key
    certfile /mosquitto/certs/tls.crt

    allow_anonymous false

    auth_plugin /mosquitto/go-auth.so
    auth_opt_log_dest stdout
    auth_opt_log_level debug

    auth_opt_backends http

    auth_opt_http_host mosquitto-auth-gateway
    auth_opt_http_port 80
    auth_opt_http_getuser_uri /getuser
    auth_opt_http_aclcheck_uri /aclcheck
    auth_opt_http_superuser_uri /superuser

    # MQTT over Websocket using TLS (WSS)
    # Not too sure why has to be defined after and options dont need to be repeated
    listener 9001
    protocol websockets


---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: mosquitto
  namespace: iot
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - mqtt.example.com
    secretName: mosquitto-certs
  rules:
  - host: mqtt.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: mosquitto-clusterip
          servicePort: 9001
---

As such, this architecture achieves a cloud native multi-user MQTT broker.

The source code for the authentication gateway is available on GitLab.