Federated
A federation joins domains that already run brokers of their own. A city might have a parks department, a transport company and a water utility, each with its own NGSI-LD infrastructure and its own team. The federation broker registers each domain as a context source, and applications that want the whole city ask only the federation broker.
The parts are those of a distributed deployment, with two differences. Each source is a complete broker with its own database, and a registration describes a domain coarsely, by the types it holds and the area it covers, rather than entity by entity.
- ApplicationsOne point of access
- answers, notifications
- Federation brokerRegisters the domains, holds no domain data
- answers to what the federation forwards
- Parks departmentAndromedaDomain brokerPostgreSQLIts own databaseTransport companyAndromedaDomain brokerPostgreSQLIts own databaseWater utilityAny NGSI-LD brokerAnother product
When it fits
- Each domain is run by a different team, with its own schedule, upgrades and data model.
- Applications should have one address, and the domains should keep owning their data.
- Domains come and go without anything else changing. Joining adds a registration, and leaving removes it.
A domain needs nothing beyond the ordinary NGSI-LD API, so it can run another broker product. The federation broker keeps none of the domains' data, only the registrations that describe them.
What runs
- The federation broker with a PostgreSQL server. It holds the registrations and whatever else you choose to keep there, and none of the domains' entities.
- A broker and a database for every domain, at the domain's own address. Each domain's operators keep running theirs the way they do now.
A forwarded request carries the alias of every broker it has passed in its Via header. A broker that finds its own alias there treats the request as a loop and doesn't send it on, so two brokers with the same federation.identity.source_alias block each other's requests. Left out, the alias is the host of server.public_url, which means brokers that share a host name have to set it.
What to set
On the federation broker:
On each domain broker, only its own federation.identity values have to differ. A domain that has context sources of its own is the case hierarchical describes.
The Federation guide shows how to write the registrations, and how location and scope keep a request away from domains that can't answer it.
Tenants
A tenant has registrations of its own, so register the domains in every tenant of the federation broker that should see them. The federation broker's tenants and a domain's tenants are separate and don't have to share names. A registration can add headers to every request it forwards through contextSourceInfo, and that is how it names the tenant to use at the domain. Tenants describes how a tenant is chosen.
Example deployment
Three brokers, each with a database of its own, a federation broker and two domains. One .env holds the three passwords. In a real federation the domains run on their own hosts, and the domain addresses below would be theirs.
Start it
cat > .env <<EOF
POSTGRES_PASSWORD_FEDERATION=$(openssl rand -hex 24)
POSTGRES_PASSWORD_PARKS=$(openssl rand -hex 24)
POSTGRES_PASSWORD_TRANSPORT=$(openssl rand -hex 24)
EOF
name: andromeda
x-postgresql: &postgresql
image: ghcr.io/mskvarc/postgresql:18.6-timescaledb2.31.0-postgis3.6.4-tsl
restart: unless-stopped
mem_limit: 2g
cpus: 1
x-andromeda: &andromeda
image: ghcr.io/mskvarc/andromeda:1.0.0
restart: unless-stopped
services:
postgresql:
<<: *postgresql
environment:
POSTGRES_USER: andromeda
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD_FEDERATION:?set it in .env}
POSTGRES_DB: andromeda
volumes:
- postgresql-data:/var/lib/postgresql
andromeda:
<<: *andromeda
depends_on:
postgresql:
condition: service_healthy
environment:
ANDROMEDA__DATABASE__CONNECTION: postgres://andromeda:${POSTGRES_PASSWORD_FEDERATION}@postgresql:5432/andromeda
ANDROMEDA__SERVER__PUBLIC_URL: http://andromeda:1056/
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ALIAS: federation
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ID: urn:ngsi-ld:ContextSourceIdentity:federation
ANDROMEDA__FEDERATION__COOLDOWN__MODE: exponential
ports:
- "1056:1056"
postgresql-parks:
<<: *postgresql
environment:
POSTGRES_USER: andromeda
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD_PARKS:?set it in .env}
POSTGRES_DB: andromeda
volumes:
- postgresql-parks-data:/var/lib/postgresql
andromeda-parks:
<<: *andromeda
depends_on:
postgresql-parks:
condition: service_healthy
environment:
ANDROMEDA__DATABASE__CONNECTION: postgres://andromeda:${POSTGRES_PASSWORD_PARKS}@postgresql-parks:5432/andromeda
ANDROMEDA__SERVER__PUBLIC_URL: http://andromeda-parks:1056/
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ALIAS: parks
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ID: urn:ngsi-ld:ContextSourceIdentity:parks
postgresql-transport:
<<: *postgresql
environment:
POSTGRES_USER: andromeda
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD_TRANSPORT:?set it in .env}
POSTGRES_DB: andromeda
volumes:
- postgresql-transport-data:/var/lib/postgresql
andromeda-transport:
<<: *andromeda
depends_on:
postgresql-transport:
condition: service_healthy
environment:
ANDROMEDA__DATABASE__CONNECTION: postgres://andromeda:${POSTGRES_PASSWORD_TRANSPORT}@postgresql-transport:5432/andromeda
ANDROMEDA__SERVER__PUBLIC_URL: http://andromeda-transport:1056/
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ALIAS: transport
ANDROMEDA__FEDERATION__IDENTITY__SOURCE_ID: urn:ngsi-ld:ContextSourceIdentity:transport
volumes:
postgresql-data:
postgresql-parks-data:
postgresql-transport-data:
Only the federation broker publishes a port. It reaches the domain brokers over the Compose network, and a domain's own clients would keep reaching it the way they do now. Start everything with docker compose up -d.
Register the domains
Register the two domains at the federation broker, each by the types it holds.
curl -i http://localhost:1056/ngsi-ld/v1/csourceRegistrations \
-H 'Content-Type: application/json' \
-d '{
"id": "urn:ngsi-ld:ContextSourceRegistration:parks",
"type": "ContextSourceRegistration",
"information": [
{
"entities": [
{
"type": "Park"
},
{
"type": "Tree"
}
]
}
],
"endpoint": "http://andromeda-parks:1056/"
}'
curl -i http://localhost:1056/ngsi-ld/v1/csourceRegistrations \
-H 'Content-Type: application/json' \
-d '{
"id": "urn:ngsi-ld:ContextSourceRegistration:transport",
"type": "ContextSourceRegistration",
"information": [
{
"entities": [
{
"type": "Bus"
},
{
"type": "BusStop"
}
]
}
],
"endpoint": "http://andromeda-transport:1056/"
}'
Query across the domains
Write a Tree to andromeda-parks and a Bus to andromeda-transport, the way the domains' own producers would. The domain brokers publish no port, so these calls run from a throwaway container on the Compose network, which name: andromeda calls andromeda_default.
docker run --rm --network andromeda_default curlimages/curl \
-i http://andromeda-parks:1056/ngsi-ld/v1/entities \
-H 'Content-Type: application/json' \
-d '{
"id": "urn:ngsi-ld:Tree:1",
"type": "Tree"
}'
docker run --rm --network andromeda_default curlimages/curl \
-i http://andromeda-transport:1056/ngsi-ld/v1/entities \
-H 'Content-Type: application/json' \
-d '{
"id": "urn:ngsi-ld:Bus:1",
"type": "Bus"
}'
One query at the federation broker now returns both, each from the domain that holds it.
curl -s 'http://localhost:1056/ngsi-ld/v1/entities?type=Tree,Bus' \
-H 'Accept: application/ld+json'