Skip to main content

Target languages in q

Filter LanguageProperties by full language tags and ranges in q. The standard grammar only takes color[en]. With this extension, q=marque[en-GB]=="Vauxhall Viva" works as Annex C prints it, and at the highest level q=name[en]=="Lift" also matches a name stored under en-GB. The default is the standard behaviour.

At a glance​

The three levels​

  • The standard grammar (default, mode = "disabled")

    A target language has to look like an attribute name, so [en] works and [en-GB] is refused.

  • language-tags (mode = "enabled", the default level)

    Also accepts every RFC 5646 language tag, such as [en-GB] or [zh-Hant-TW]. This only adds: requests the standard grammar accepts behave exactly as before.

  • language-ranges (mode = "enabled", level = "language-ranges")

    Also accepts RFC 4647 extended ranges such as [de-*-DE], and reads every target language as a range. This changes what existing filters select: [en] now matches en-GB too.

Turning it on​

[extensions.q_target_languages]
mode = "enabled"
level = "language-tags"

Be careful with language-ranges. Moving to it, or away from it, changes the meaning of every stored subscription whose q names a target language.

Using it​

At language-tags:

GET /ngsi-ld/v1/entities?type=Car&q=marque[en-GB]=="Vauxhall Viva"

At language-ranges, q=name[en]=="Lift" matches an entity whose name holds {"en-GB": "Lift"}, and q=name[de-*-DE]=="Aufzug" matches {"de-Latn-DE": "Aufzug"}.

How keys are matched​

At every level, a target language matches a key without regard to case: [en-gb] selects the key written en-GB.

Under the standard grammar and at language-tags, the match is tag equality, so [en] doesn't select en-GB.

At language-ranges, every target language is an extended language range, matched by extended filtering. The first subtags have to match, or the range's first subtag is *. Every later subtag of the range that isn't * has to turn up further along the tag. Subtags can be skipped on the way, but never a singleton such as the x that starts a private-use part. Some examples:

  • [de-*-DE] matches de-DE, de-Latn-DE and de-DE-x-goethe, but not de-Deva or de-x-DE.
  • [*-GB] matches every tag whose region is GB.
  • [en] matches en and every tag below it.

[*] on its own is the standard "every language" marker at every level, and matches the strings of every key.

How operators decide​

Every operator is decided string by string over the selected keys:

  • ==, a value range, the ordering operators and ~= hold when one selected string satisfies them.
  • !=, !~= and an excluded value range hold when the languageMap exists and no selected string satisfies them.

Rare cases​

Sorting, stored subscriptions and csf
  • orderBy reads a target language by tag equality at every level, and refuses a range with BadRequestData. Ordering needs one value, and there's no rule for picking one among several matching strings.
  • A stored subscription is evaluated under the broker's current level. A subscription whose q names [en] starts selecting en-GB when the broker moves to language-ranges, and stops when it moves back. A stored full tag or range keeps working after the level is lowered, because it was accepted at a level that supported it. A range has no tag-equality reading, so it keeps filtering as a range.
  • A one-member group spelled like an attribute name (color[en]) keeps both of its standard readings at every level.
  • Context Source Properties are never LanguageProperties, so a one-member csf bracket only ever names a member of a compound value. A target language that no member name spells, such as [en-GB], selects nothing.

What gets refused​

  • 400 BadRequestData for a full tag below language-tags, or a range with * below language-ranges

    Raise the level, or use a target language the current level accepts. This applies to Query Entities, Query Temporal Evolution, subscription create and update (q and csf), orderBy, and every other operation that compiles a q.

  • 400 BadRequestData for a bracket member that is neither a name, a tag nor a range

    Fix the target language.

  • 400 BadRequestData naming a registration

    A context source the query reaches doesn't support the target language you used. See below.

Context sources​

Before it runs, a query or temporal query asks the context sources it reaches what they support, by reading their identity. It does this when its q names a target language beyond the standard grammar, or when the broker is at language-ranges and the q names any target language. Each source is asked at most once every five minutes. The sources are those whose registrations take part in the query. A source's level is read from the tokens it advertises, and a source without an identity counts as reading the standard grammar.

  • If the q names a target language a reached source's level doesn't accept, the query is refused with BadRequestData naming the registration. That source would refuse a full tag outright, and a range has no tag-equality reading, so nothing it could answer would be what you asked for.
  • Otherwise the query runs at the lowest level among this broker and every source it reaches. The forwarded q is unchanged.
  • If that drops a language-ranges broker to tag equality, the response carries NGSILD-Warning: 199 - "language range filtering not applied: registration <id> does not advertise languageRangeFiltering". Each source still answers under its own reading. A source matching by tag equality returns tag-equal matches, while a source at language-ranges still filters by range, so a mixed response holds the union of both. No forwarded parameter can tell a filtering source to match by tag equality.
  • A forwarded subscription copy carries the q unchanged, and each source notifies under its own reading. A copy held by a tag-equality source notifies only tag-equal matches. The subscriber isn't warned about that partial answer, because notifications carry no NGSILD-Warning.
  • Creating an EntityMap from a query, Purge Entities and snapshot capture read only this broker's entities, under its own level.

Why it is an extension​

The specification's q grammar doesn't allow a - in the group that names the target language. A conformant broker reads color[en] and refuses marque[en-GB] with BadRequestData. The informative examples in the specification do print marque[en-GB], but the normative grammar doesn't produce it. extensions.mode = "conformance" forces the standard grammar.

language-tags only turns refused requests into answered ones. language-ranges goes further and changes a filter the specification already defines. The specification makes the target value of a LanguageProperty the value under the key of the languageMap that matches the target language, and makes the keys language tags, which are compared without regard to case. Under that reading [en] selects the key en and nothing else. A broker at language-ranges answers a conformant request differently.

Last updated on