Skip to main content

jsonld

jsonld is a JSON-LD 1.1 processor. It processes contexts, expands, compacts and flattens documents, and turns them into RDF. It is a fork of json-ld by Timothée Haudebourg, with the upstream history kept, and it builds on iri-rs for IRIs and jstrict for JSON. The jsonld crate brings together jsonld-core, jsonld-syntax, jsonld-context-processing, jsonld-expansion, jsonld-compaction and jsonld-serialization, and jsonld-cli runs the algorithms from the command line.

The same contexts on every request​

A converter runs JSON-LD once per document. A server that speaks JSON-LD runs it on every request, and nearly every request names the same few @contexts. Processing a context is the costly step, so jsonld does it as seldom as it can.

  • Expanding a document processes each distinct @context in it once, however many nodes repeat it. A ProcessingCache passed to process_with_cache carries processed contexts from one document to the next, and it checks a hit against the inputs the entry was built from, not only against their hash.
  • A context that references nothing remote and imports nothing is processed synchronously, without the futures the algorithm needs when it may call a loader.
  • The inverse context and the list of terms that can serve as prefixes are built the first time compaction asks for them, and kept with the context until one of its definitions changes. The compact IRIs chosen against a context are memoised on it, up to a fixed number.
  • A term of up to 23 bytes, which covers nearly every term of a real context, is stored inline, so creating, cloning or dropping it allocates nothing. A longer term is reference-counted. Nothing is interned for the life of the process, so the memory spent on terms never outgrows the contexts a program keeps.

Memoisation costs memory and only pays off when the same contexts and terms come back. A batch job that converts unrelated documents gains little from it and may come out slower. The repository's benchmarks are a development tool and no comparison with upstream is published, so measure your own documents if speed decides the choice.

Two settings matter when documents come from people you don't trust.

  • Hash maps use foldhash by default, which is fast but not resistant to hash flooding. The ahash feature swaps in ahash.
  • A ProcessingCache never evicts. A stream of distinct contexts through one grows it until it is cleared, so clear it or keep processed contexts in a bounded cache of your own.

JSON in and out​

Documents are jstrict values, whose objects keep their entries in the order written, a repeated key included. JSON-LD 1.1 section 9 requires the keys of an object to be unique, and a processor can only enforce that if the parser shows it a repeated key. A context that defines a term twice is refused with duplicate key, and a node object that names a property twice keeps the values of both.

The serde-json and sonic-rs features convert at the boundary. sonic_rs::Object keeps order and repeated keys, so reading from it loses nothing, and writing to it leaves a repeated key once, with its last value. serde_json::Map holds one value per key and sorts its keys unless the preserve_order feature of serde_json is on.

IRIs are compared as strings, as JSON-LD and RDF require, and iri-rs explains why that is the right rung.

The sample needs iri-rs with its static feature and jsonld with serde-json.

use iri_rs::{IriBuf, iri};
use jsonld::{JsonLdProcessor, NoLoader, RemoteDocument, syntax::{Parse, Value}};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let document = RemoteDocument::new(
        Some(IriBuf::from(iri!("https://example.com/buildings"))),
        Some("application/ld+json".parse()?),
        Value::parse_str(
            r#"{
                "@context": {"name": "https://schema.org/name"},
                "@id": "urn:ngsi-ld:Building:1",
                "name": "Town hall"
            }"#,
        )?
        .0,
    );

    // The context is inline, so it is processed synchronously and the loader is never called.
    let expanded = document.expand(&NoLoader).await?;
    println!("{}", expanded.into_serde_json());
    // [{"@id":"urn:ngsi-ld:Building:1","https://schema.org/name":[{"@value":"Town hall"}]}]
    Ok(())
}

Conformance​

The repository runs the W3C JSON-LD 1.1 API test suite for the four of its seven manifests whose algorithms jsonld implements, namely expand, compact, flatten and toRdf. Every test that runs passes. The few that are skipped each link to the discussion behind the skip, and a test written for JSON-LD 1.0 runs in 1.0 mode rather than being skipped. Negative tests, the ones that expect an error, check that processing fails, but not yet that it fails with the error code the manifest names.

The other three manifests cover turning RDF into JSON-LD, extracting JSON-LD from HTML and HTTP content negotiation, which jsonld does not implement.

When upstream fits better​

  • Your code already works with iref, json-syntax or rdf-types values. Upstream json-ld builds on them, and this fork would need a conversion wherever the two meet.
  • Your workload is nothing like a server's, such as a batch of one-off documents. Measure both before you choose.

Where Andromeda uses it​

Andromeda processes the @context of a request once into an active context, and keeps it in a cache of its own that is bounded in size and age. It reuses a kept context only while every document it was built from still resolves to the same one. Its typed expansion and compaction then ask jsonld about one term at a time, through the library's own IRI expansion and term selection, and hand anything they don't model to the full algorithms, which start from the same active context. Differential tests hold the compaction walk to the processor's output. A name in q, attrs or orderBy is expanded the same way, one term at a time. What the broker does with your names shows what a client sees of all this.

The core @context releases Andromeda accepts are embedded byte for byte, so processing them never touches the network (Core @context releases). Other contexts reach the processor through the broker's own loader, within the limits of JSON-LD configuration (Fetching remote @context documents). Most NGSI-LD terms the code names are constants that vocab! generates from the core @context at compile time.

The JSON tree of jsonld may appear only in the JSON-LD engine and in the type that holds JSON literals, and a check enforces that (Three forms of JSON). Standards lists the JSON and JSON-LD specifications Andromeda follows.

Last updated on