Distributed

Andromeda can answer for data it doesn't hold. You register another NGSI-LD system as a context source, saying which entities and attributes it has. Andromeda then forwards matching requests to it and merges the answers with its own data. A client sees one entity even when its attributes live in several systems.

The registration decides almost everything, such as which requests reach the source, how its data relates to yours, and what happens when it fails. This guide shows how to register a source, what changes once you have, and how to tell when a source is misbehaving. When the sources are whole systems run by other teams, Federation adds what changes at that scale.

Registering a source

http

POST /ngsi-ld/v1/csourceRegistrations
Content-Type: application/json

{
    "id": "urn:ngsi-ld:ContextSourceRegistration:fleet",
    "type": "ContextSourceRegistration",
    "information": [
        {
            "entities": [
                {
                    "type": "Vehicle"
                }
            ],
            "propertyNames": [
                "speed"
            ]
        }
    ],
    "endpoint": "https://fleet.example.org/"
}

curl

curl -i http://localhost:1056/ngsi-ld/v1/csourceRegistrations \
  -H 'Content-Type: application/json' \
  -d '{
    "id": "urn:ngsi-ld:ContextSourceRegistration:fleet",
    "type": "ContextSourceRegistration",
    "information": [
        {
            "entities": [
                {
                    "type": "Vehicle"
                }
            ],
            "propertyNames": [
                "speed"
            ]
        }
    ],
    "endpoint": "https://fleet.example.org/"
}'

rust

let registration = CSourceRegistration::builder("https://fleet.example.org/")
    .id(iri!("urn:ngsi-ld:ContextSourceRegistration:fleet"))
    .information([RegistrationInfo::builder()
        .entities(["Vehicle"])
        .property_names(["speed"])
        .build()?])
    .build()?;

let outcome = client.registrations().register(&registration).send().await?;

python

response = requests.post(
    f"{BASE}/csourceRegistrations",
    json={
        "id": "urn:ngsi-ld:ContextSourceRegistration:fleet",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Vehicle",
                    },
                ],
                "propertyNames": [
                    "speed",
                ],
            },
        ],
        "endpoint": "https://fleet.example.org/",
    },
)
assert response.status_code == 201

HTTP/1.1 201 Created
Location: /ngsi-ld/v1/csourceRegistrations/urn:ngsi-ld:ContextSourceRegistration:fleet

From now on, a query for vehicles, or for speed, also goes to https://fleet.example.org/. endpoint is the source's API root, without /ngsi-ld/v1.

information lists what the source holds. Each entry names entity types, optionally narrowed by id or idPattern, and the attributes in propertyNames and relationshipNames. You can also bound a source by area with location, by Scope with scope, and by time with observationInterval. A source is asked only about what its registration covers, and only for the attributes it covers. A source that holds every Vehicle but registers only speed is never asked for its brand.

The broker that holds the registration does the forwarding. The source needs nothing beyond the ordinary NGSI-LD API, and doesn't need to know the broker exists.

Choosing a mode

The mode decides how the source's data relates to yours. It is set per registration, so one broker can mix modes across sources.

• inclusive, the default. The source adds to what the broker holds, and answers from the broker and from the source are merged.
• exclusive. The source alone holds the registered attributes of the entities it names. The broker doesn't store them, and sends reads and writes for them to the source.
• redirect. Requests for what the registration covers go to the source, and the broker answers nothing of its own for them.
• auxiliary. The source only fills gaps. It adds an attribute nobody else supplied, and never overrides anything. An entity nobody else holds comes back with everything the source sent. It takes only reads.

An exclusive registration names its entities by id, and lists at least one attribute.

A registration that overlaps another proxied registration, or context data the broker already holds, is refused with 409 Conflict. The overlap check applies to exclusive and redirect registrations, and to the proxied footprint of inclusive ones alike.

Choosing the operations

"operations": [
    "federationOps",
    "updateOps"
]

operations lists the requests the source accepts, by operation name or by group. The default group, federationOps, covers reads, type and attribute discovery, subscriptions and EntityMaps. It does not cover writes. A write reaches a source only when its registration lists a group or operation that includes it, such as updateOps. A source is never sent a request its registration doesn't list.

Six operations that the groups name are never forwarded, whichever groups a registration advertises. These are reading, updating and deleting an EntityMap, reading the context source identity, and reading or listing subscriptions. The broker answers each from its own state, so a source's copies of an EntityMap, a subscription or an identity never appear in the answer. The broker places and removes its copies of subscriptions at sources through the create, update and delete operations, which are forwarded.

An auxiliary registration defaults to retrieveOps instead. It must stay within the consumption operations, and federationOps would take it outside them.

Querying with sources

An ordinary query reaches every source whose registration matches it.

http

GET /ngsi-ld/v1/entities?type=Vehicle&attrs=speed

curl

curl -i 'http://localhost:1056/ngsi-ld/v1/entities?type=Vehicle&attrs=speed'

rust

let speed: AttributeName = "speed".parse()?;

let page = client
    .entities()
    .query()
    .entity_type("Vehicle")
    .attrs(vec![speed])
    .send()
    .await?;

python

response = requests.get(
    f"{BASE}/entities",
    params={
        "type": "Vehicle",
        "attrs": "speed",
    },
)
assert response.status_code == 200

The broker asks each matching source for what it covers. It checks every entity that comes back as it would check a local write, and merges the answers attribute by attribute. An entity a source returns that a local write would refuse is left out, and the answer says so in an NGSILD-Warning header. Distributed operations explains the merge and the warnings.

To leave the sources out, send local=true.

http

GET /ngsi-ld/v1/entities?type=Vehicle&local=true

curl

curl -i 'http://localhost:1056/ngsi-ld/v1/entities?type=Vehicle&local=true'

rust

let page = client
    .entities()
    .query()
    .entity_type("Vehicle")
    .locality(Locality::Local)
    .send()
    .await?;

python

response = requests.get(
    f"{BASE}/entities",
    params={
        "type": "Vehicle",
        "local": "true",
    },
)
assert response.status_code == 200

To pick sources by what their registrations say, use csf, a filter over registration members written in the q syntax. Sorting works only when no registration takes part, so add local=true when you need a sorted answer.

When a source fails

A failed request to one source doesn't fail the whole answer. The rest comes back, with a warning naming the failed source.

A source whose registration sets a management.cooldown is held back for that long after a failure, so it doesn't slow every request down. While it is held back, you get a 199 warning. The broker can hold back every failing source with a growing cooldown when federation.cooldown.mode is "exponential".

When an operation goes to a single source, that source's failure decides the status, 504 for a timeout, 502 for any other error, and 404 when the source doesn't have the entity.

The registration keeps count. Read it back to see how the source is doing.

http

GET /ngsi-ld/v1/csourceRegistrations/urn:ngsi-ld:ContextSourceRegistration:fleet

curl

curl -i http://localhost:1056/ngsi-ld/v1/csourceRegistrations/urn:ngsi-ld:ContextSourceRegistration:fleet

rust

let registration = client.registrations().retrieve(&fleet).send().await?;

python

response = requests.get(FLEET)
assert response.status_code == 200

• status is ok or failed.
• timesSent and timesFailed give how many requests went to the source, and how many failed.
• lastSuccess and lastFailure give when the source last answered, and when it last failed.

Distributed operations covers the cooldown and its settings.

Finding and watching registrations

GET /ngsi-ld/v1/csourceRegistrations?type=Vehicle lists the registrations for a type. Like an entity query, it needs a type, attributes, q or a geoquery.

To hear when registrations change, subscribe with POST /ngsi-ld/v1/csourceSubscriptions. The body is a subscription, and each notification describes the registrations that were created, changed or removed. Subscriptions and notifications covers the notification format.

Removing a source

http

DELETE /ngsi-ld/v1/csourceRegistrations/urn:ngsi-ld:ContextSourceRegistration:fleet

curl

curl -i -X DELETE http://localhost:1056/ngsi-ld/v1/csourceRegistrations/urn:ngsi-ld:ContextSourceRegistration:fleet

rust

let outcome = client.registrations().delete(&fleet).send().await?;

python

response = requests.delete(FLEET)
assert response.status_code == 204

The answer is 204, and requests stop going to the source. Data the source holds stays there. The broker never copied it, so removing a registration deletes nothing at the source.

Security

A registration makes the broker send requests to the address it names, with any headers contextSourceInfo asks for. Anyone who can register a source can therefore make the broker call any address it can reach, unless federation.forwarding.private_addresses is "refused", which rules out loopback, private and reserved addresses. federation.forwarding.allowed_addresses exempts the subnets your sources sit in, and federation.forwarding.denied_addresses rules out addresses and subnets of your choosing. The broker never follows a redirect from a source. Security explains what to restrict.