TLS

NGSI-LD asks for HTTPS, with TLS 1.2 supported. Andromeda can terminate TLS itself, or run on plaintext behind a reverse proxy that does. Both work. What matters is that no client can reach a plaintext port directly.

TLS ends at	Who holds the certificate	Choose it when
The broker	The broker, which reloads it on rotation	Clients reach the broker directly
A proxy	The proxy	A proxy already checks callers, as Security recommends

Terminating TLS in the broker

[server]
transport = "tls"

[tls]
certificate = "/etc/andromeda/tls/tls.crt"
private_key = "/etc/andromeda/tls/tls.key"

The certificate file holds the chain, leaf first, and the key can be PKCS#8, SEC1 or PKCS#1. Every listener that terminates TLS presents this one identity, which means the API, HTTP/3 when it is on, and the management listener with management.transport = "tls". HTTP/3 shares the API's host and port, and its Alt-Svc header tells clients the two are the same server, so one certificate is the only consistent choice.

andromeda configuration validate refuses the configuration when a listener terminates TLS and either path is missing. The files themselves are read when the listeners start. A file that can't be read, holds no certificate or key, or doesn't match the other one stops the broker with a message that names it, such as no private key found in /etc/andromeda/tls/tls.key.

With the default tls.minimum_protocol_version = "tls1.2", a connection can use TLS 1.2 or 1.3, and TLS 1.0 and 1.1 are never offered. "tls1.3" drops 1.2, the version the specification asks every implementation to support, so choose it only when you know every client speaks 1.3. HTTP/3 always uses TLS 1.3.

The TCP listener offers HTTP/2 and HTTP/1.1 through ALPN, in that order, and the client chooses. A client gets tls.handshake_timeout, 10 seconds, to finish its handshake, so one that connects and stalls holds up nobody else.

Rotating the certificate

There's nothing to restart. The broker watches the directories that hold the certificate, the key and the client CA bundle, not just the files. That catches every usual way of replacing them, whether a symbolic link swapped to a new directory, certbot moving a new file into place, or a plain copy over the old one.

After a change, the broker waits until the files have been quiet for tls.reload_settle_after, 2 seconds, and then loads them. A rotation that writes the certificate and the key a moment apart is therefore read once, with both in place. If the writes are further apart, the first load finds a mismatched pair and is rejected, and the second succeeds.

New handshakes on every TLS listener get the new certificate. Connections already open keep their session, and pick up the new certificate when they reconnect. A load that fails, such as a half-written file or a key that no longer matches its certificate, changes nothing, and the broker keeps presenting the previous certificate.

Client certificates

tls.client_authentication.mode decides whether clients must show a certificate. A certificate a client does show is checked against the CA bundle in tls.client_authentication.ca.

Mode	A client without a certificate	A certificate from an unknown issuer
"disabled", the default	Served	Not asked for
"optional"	Served	Fails the handshake
"required"	Refused	Fails the handshake

The same rule applies on every TLS listener, HTTP/3 included, and the CA bundle is reloaded along with the rest. With "disabled", authentication happens somewhere else, if you need it at all.

The management listener

On a port of its own, the management listener stays on plaintext unless you set management.transport = "tls", because usually only your health checks and Prometheus reach it. With "tls", it presents the same certificate and applies the same client certificate rule. In "shared-port" mode, it shares the API's listener and transport.

Terminating TLS at a proxy

Keep the broker on plaintext, which is the default, and put a reverse proxy in front, such as nginx, Envoy, HAProxy, Traefik or a cloud load balancer. The proxy holds the certificate and forwards to the broker over a network you control.

Never let a client reach the plaintext port

Bind the broker to a private interface or a container network, and publish only the proxy. A reachable plaintext port defeats the point of TLS. With forwarded fields believed from any peer, it also lets any client choose the scheme and host of the URLs the broker stores.

The proxy must offer TLS 1.2, should offer TLS 1.3 too, and must not offer 1.0 or 1.1. In nginx, that is ssl_protocols TLSv1.2 TLSv1.3;.

The URLs the broker hands out

Some of the URLs the broker builds are absolute and outlive the request. The URL of every @context it stores is kept, and handed to later readers and to every notification of the subscription that created it. Behind a TLS proxy, the request that reaches the broker is plaintext, and a proxy that rewrites Host hands it an internal name. Without help, those URLs start with http:// and may name a host no client can reach. There are two ways to fix that.

• Set server.public_url

  Give the public address, path prefix included, such as https://broker.example.org or https://example.org/context-broker. Every absolute URL is then built on it, whatever the request says. This is the simpler choice, and the only one that works when the proxy serves the broker under a path. Subscriptions forwarded to context sources also give it as the address their notifications come back to.

• Believe the proxy's forwarded fields

  Set http.forwarded.trust, as the next section describes. The scheme then comes from the proto parameter of Forwarded, or from X-Forwarded-Proto when Forwarded names no protocol. The host and port come from the host parameter of Forwarded, or from X-Forwarded-Host when Forwarded names no host, and from the request's own Host when neither does. Choose this for a broker reached under several names, since each request then gets URLs on the name its client used.

server.public_url wins when both are set. Location headers and pagination links are relative paths, so their scheme and host are right behind any proxy without either setting.

Trusting forwarded fields safely

Forwarded, X-Forwarded-Proto and X-Forwarded-Host are ordinary request fields, and anyone who can reach the broker can send them. Whoever the broker believes chooses the scheme and host of the URLs it stores and hands to later readers, which is why the broker ignores them by default. http.forwarded.trust says whose fields it believes.

• "none", the default

  Nobody's.

• "proxies"

  Only those of a peer inside one of the addresses or subnets http.forwarded.proxies lists, such as ["10.0.0.0/8", "fd00::/8"]. A client that reaches the broker directly is ignored even when it sends the fields. An IPv4 peer reaching a dual-stack listener is matched as its IPv4 address.

• "any"

  Everyone's. Use it only when the network lets nothing but the proxy reach the broker, for example a container network where the proxy's address changes.

Whichever you choose, each believed proxy must replace the fields rather than append to them. The broker reads the leftmost element, which is the one a client sent if the proxy appended. Proxies differ in what they do by default.

• nginx

  proxy_set_header with a value built from $scheme and $http_host replaces the field. A value built from $http_forwarded or $http_x_forwarded_host carries the client's value along, and turns the replacement back into an append.

• Caddy

  Replaces X-Forwarded-Proto and X-Forwarded-Host on its own, but passes on a Forwarded the client sent unchanged, and the broker reads Forwarded first. Replace it with header_up Forwarded "proto={scheme};host=\"{hostport}\"".

• Traefik

  Also replaces X-Forwarded-Proto and X-Forwarded-Host and passes Forwarded on. It can't build a value from the request, so remove Forwarded with a headers middleware whose customRequestHeaders sets it to an empty value.

The broker reads only http or https as the scheme, and only a host with an optional port as the host. It never reads X-Forwarded-Port, because the port belongs in the forwarded host. A field it can't read is ignored, and the request carries on with what it addressed.

When the proxy forwards no host, it must pass the client's Host through with its port. In nginx, use $http_host rather than $host, since $host drops the port, and a proxy on port 8443 would then have the broker store URLs that point at 443. Envoy passes the client's :authority through unless a route sets host_rewrite_literal or auto_host_rewrite. Behind a proxy that must rewrite Host and can forward nothing, set server.public_url.

Believing a forwarded host rests on the same trust as believing the forwarded scheme. A proxy the broker believes already chooses Host itself, so believing its forwarded host gives it nothing it lacked. It does let a proxy that rewrites Host, or one that serves several public names, reach correct URLs without a fixed server.public_url.

Serving the broker under a path

The broker serves its API under /ngsi-ld/v1 at the root of its own port, and has no setting for a path in front of that. A proxy that serves it under a prefix, such as https://example.org/context-broker, strips the prefix before it forwards a request. A proxy that keeps the prefix reaches no route, and every request gets 404.

Then set server.public_url to the prefixed address. Forwarded fields carry no prefix, so with them in its place every URL the broker builds leaves the prefix out, and no subscription is forwarded. With server.public_url set, Location headers and the links that page through a collection start with the prefix, such as /context-broker/ngsi-ld/v1/entities/urn:ngsi-ld:Building:1 or /context-broker/ngsi-ld/v1/entities?type=Building&limit=2&offset=2. Stored @context URLs, and the endpoint of the subscriptions the broker forwards to context sources, start with the whole address.

When you strip a prefix, forward the rest of the path exactly as the client sent it. An entity id can hold a /, which the client sends as %2F, and a proxy that decodes the path first splits the id in two. These ways of stripping keep %2F as it was sent, behind nginx, Caddy and Traefik alike.

• nginx

  Take the path from $request_uri, for example with a map that drops the prefix, and pass it in proxy_pass http://broker$path. A rewrite, or a proxy_pass with a path of its own, decodes it first.

• Caddy

  uri strip_prefix /context-broker inside a handle /context-broker/ngsi-ld/* block.

• Traefik

  A stripPrefix middleware with prefixes: ["/context-broker"] on a router whose rule is PathPrefix(`/context-broker/ngsi-ld/`).

With any other proxy, check that an id holding %2F reaches the broker as it was sent. The deployment planner writes the whole setup for nginx, Caddy and Traefik when you answer that the proxy serves the broker under a path.

An nginx example

server {
    listen 8080;
    return 301 https://$host:8443$request_uri;
}

server {
    listen 8443 ssl;
    http2 on;

    ssl_certificate     /etc/nginx/certs/broker.crt;
    ssl_certificate_key /etc/nginx/certs/broker.key;
    ssl_protocols TLSv1.2 TLSv1.3;

    client_max_body_size 64m;

    location /ngsi-ld/ {
        proxy_pass http://broker:1056;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_set_header Host $http_host;
        proxy_set_header Forwarded "proto=$scheme;host=\"$http_host\"";
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $http_host;

        proxy_connect_timeout 5s;
        proxy_send_timeout 35s;
        proxy_read_timeout 35s;
    }
}

Run the broker beside it with ANDROMEDA__HTTP__FORWARDED__TRUST=proxies and ANDROMEDA__HTTP__FORWARDED__PROXIES set to the proxy's address, and with no published port of its own.

• client_max_body_size matches the broker's default request_limits.max_body of 64 MiB.
• The timeouts follow its default request_limits.timeout of 30 seconds, as the next section explains.
• The first server block does the redirect to HTTPS, which the broker doesn't.
• Only /ngsi-ld/ is forwarded, so /livez, /readyz and /metrics stay off the public path when they share the API port.

The deployment planner writes the same for nginx, Caddy and Traefik from your answers.

Timeouts and limits at the proxy

Set the proxy's limits from the broker's own, so that the broker acts first and the client gets the broker's answer, with problem details, rather than the proxy's.

• Wait request_limits.timeout plus 5 seconds for the broker's answer

  35 seconds with the default. Past request_limits.timeout, the broker answers 500 itself. A proxy that waits exactly as long races it and sometimes answers 504 first, while 10 ms longer already let the broker's answer through every time behind nginx, Caddy and Traefik, and the 5 seconds leave room for a broker under load. The setting is proxy_read_timeout in nginx, response_header_timeout in Caddy, and responseHeaderTimeout of a servers transport in Traefik.

• Give a request body as long to arrive

  A slow body counts against request_limits.timeout too, and the broker answers it with 408. In nginx, that is proxy_send_timeout, which bounds sending the request to the broker. In Traefik, it is the entry point's readTimeout, which bounds reading the request from the client and is 60 seconds unless set.

• Set no shorter limit on writing the answer back

  A Traefik entry point with a writeTimeout below the broker's time drops the broker's answer, so keep it at 0s.

• Give up connecting to the broker after a few seconds

  5 in the planner's files. A running broker accepts at once, and a stopped one is refused at once, so this matters only when the broker's host doesn't answer at all.

• Close idle connections to the broker after any time you like

  60 seconds in the planner's nginx file. The broker sets no idle timeout of its own, so the proxy always closes an idle connection first, and never reuses one the broker is closing.

• Accept bodies up to request_limits.max_body, each with a Content-Length

  One byte more then gets the proxy's own 413. The broker answers a body without a length with 411, as NGSI-LD asks, so the proxy has to buffer a chunked upload. nginx does by default, and Traefik does with its buffering middleware, which also sets the limit. Caddy passes a chunked body on as it came.

Stopping the broker needs nothing extra at the proxy. The broker refuses new connections at once, which the proxy answers with 502, and lets the requests in flight finish within server.graceful_drain_timeout, which equals request_limits.timeout by default. A request still running when that drain ends is cut off, and the proxy answers it with 502 as well.

Strict-Transport-Security

Strict-Transport-Security is off by default. A browser that receives it uses HTTPS for the host until max-age runs out, so turn it on only when the host will stay on HTTPS that long.

[http.security_headers.strict_transport_security]
mode = "enabled"
max_age = "180days"
include_subdomains = "disabled"

With this, every API response to a request that arrived over the TLS listener carries Strict-Transport-Security: max-age=15552000. Set include_subdomains = "enabled" to add includeSubDomains, which makes browsers use HTTPS for every subdomain of the host too. The header is left out when http.security_headers.mode is "disabled", and a response on the plaintext listener never carries it, since RFC 6797 section 7.2 has browsers ignore it over plain HTTP.

Behind a proxy that terminates TLS, the broker listens in plaintext and sends no copy, so the proxy supplies the header. If the proxy adds its own while the broker also sends one over TLS, the response carries two, and a browser follows only the first. Have the proxy replace the broker's header rather than add a second one.

• nginx

  proxy_hide_header Strict-Transport-Security beside add_header Strict-Transport-Security "max-age=15552000" always, both at the level of the server block. An add_header inside a location stops the ones above it from applying there.

• Caddy

  header >Strict-Transport-Security "max-age=15552000" is applied after the broker's answer and replaces its copy. A header without the > adds a second one.

• Traefik

  stsSeconds in a headers middleware replaces it.

What the broker doesn't do

• Serve TLS and plaintext on one port, or redirect plaintext requests to HTTPS

  A TLS port fails a plaintext client's handshake. Behind a proxy, the redirect is the proxy's job.

• Issue or renew certificates, or staple OCSP responses

  A certificate manager writes the files, and the broker picks them up.

• Choose ciphers

  It has no cipher settings beyond the minimum protocol version.

Checking it works

For a quick test of TLS in the broker, make a self-signed certificate and start the broker with it.

openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout broker.key -out broker.crt \
  -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1" \
  -addext "basicConstraints=critical,CA:FALSE"

ANDROMEDA__SERVER__TRANSPORT=tls \
ANDROMEDA__TLS__CERTIFICATE=$PWD/broker.crt \
ANDROMEDA__TLS__PRIVATE_KEY=$PWD/broker.key \
andromeda

Then check that HTTPS answers, and that a TLS 1.2 client gets through.

curl --cacert broker.crt https://localhost:1056/ngsi-ld/v1/info/sourceIdentity
openssl s_client -connect 127.0.0.1:1056 -tls1_2 </dev/null

Replace the two files with a new pair and run openssl s_client -connect 127.0.0.1:1056 </dev/null | openssl x509 -noout -serial again. It shows the new serial without a restart.

Behind a proxy, check the URL the broker stores. Create a @context through the proxy and list it.

curl -X POST https://broker.example.org:8443/ngsi-ld/v1/jsonldContexts \
  -H 'Content-Type: application/json' \
  -d '{
    "@context": {
        "example": "https://example.org/example"
    }
}'
curl 'https://broker.example.org:8443/ngsi-ld/v1/jsonldContexts?details=true'

The URL of the listed context must start with https://broker.example.org:8443/.

• It starts with http://

  The broker has neither server.public_url nor forwarded fields it believes. With http.forwarded.trust = "proxies", check that http.forwarded.proxies holds the address the proxy connects from.

• The port is missing

  The proxy sends $host instead of $http_host.

What to watch

• tls_material_reloaded at info

  A rotated certificate is now in use.

• tls_reload_rejected at error

  A rotation was refused, and the previous certificate is still in use. The line says why.

• tls_handshake_failed at debug

  A connection failed its handshake, as a port scan, a health check that doesn't speak TLS or a plaintext client does. A busy deployment sees many, which is why they are only at debug.

If it goes wrong

• Clients get stored @context URLs that start with http://

  The broker is behind a TLS proxy and doesn't know it. Set server.public_url, or believe the forwarded fields of a proxy that replaces them, with http.forwarded.trust.

• A rotated certificate isn't being served

  Look for tls_reload_rejected. A key that doesn't match its certificate, or a file still being written, keeps the old certificate in place.

• The broker doesn't start after you set server.transport = "tls"

  Both tls.certificate and tls.private_key must be set, readable and matching. andromeda configuration validate catches a missing path, and a bad file shows at startup.

• A health probe fails against the TLS port

  A probe that only opens a TCP connection never finishes a handshake. Probe over HTTPS, or probe the plaintext management port.

• Every request through a proxy that serves the broker under a path gets 404

  The proxy keeps the prefix. Strip it before forwarding, and set server.public_url to the prefixed address.

Where Andromeda interprets the specification

The specification says "The API shall support HTTP over TLS", that TLS 1.2 as defined by RFC 5246 "shall be supported", and that "HTTP without TLS is not recommended". It also says "The apiRoot includes the scheme ("http" or "https"), host and optional port, and an optional prefix string", which is what server.public_url sets. It has the broker store the URL it serves a @context at. Forwarded is defined by RFC 7239, which has each proxy append its element after those already present (section 4). RFC 8996 deprecates TLS 1.0 and 1.1. RFC 6797 section 8.1 has a browser ignore Strict-Transport-Security received over plain HTTP, and follow only the first of several.