Skip to main content

Query language

The q parameter keeps the entities whose attribute values pass a test, such as q=speed>50;brandName!="Mercedes". Strings go in double quotes. A bare word that isn't a URI or a time value is refused rather than guessed at, so a typo can't quietly select nothing.

Writing a query​

A query is one or more terms. A term on its own tests that the attribute exists. With an operator, it compares the attribute's value.

GET /ngsi-ld/v1/entities?type=Vehicle&q=speed>50;brandName!="Mercedes"
HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "id": "urn:ngsi-ld:Vehicle:A",
        "type": "Vehicle",
        "speed": {
            "type": "Property",
            "value": 80
        },
        "brandName": {
            "type": "Property",
            "value": "Volvo"
        }
    }
]

The examples on this page are written unencoded so they stay readable. In a real URL, percent-encode the expression, including its quotes, spaces, | and ;.

OperatorMeaningValue
nonethe attribute existsnone
==equal, or one ofa value, a list or a range
!=not equal, none ofa value, a list or a range
> >= < <=ordered comparisona number, string or time value
~=matches a patterna regular expression
!~=doesn't match a patterna regular expression

; joins terms that must all hold, and | joins terms of which one must hold. a;b|c reads as (a;b)|c. Parentheses override that, and they can nest, as in q=((speed>50|rpm>3000);brandName=="Mercedes"). Spaces, tabs and line breaks between the parts of an expression are ignored.

Reaching into an attribute​

An attribute path can go three ways.

  • temperature.observedAt reaches a sub-attribute, here the observedAt of temperature. Each name after a dot is a sub-attribute of the one before it.
  • address[city] reaches the member city inside a structured value. sensor.rawdata[airquality.particulate] goes through a sub-attribute and then two levels into its value.
  • owner{age} crosses a Relationship and tests the entity it points at. See Linked entities.

On a LanguageProperty, the brackets name a language instead, as in name[en]. See Language strings.

Each name in a path is a term of your @context, a prefixed name such as ex:speed, or an absolute IRI. An IRI holding a dot goes in double quotes, because a bare dot separates sub-attributes, as in q="https://example.org/speed">50. The quotes change nothing else, so "speed">50 is speed>50.

Values​

How a value is written decides how it is compared, so text that looks the same can mean two things.

Numbers​

JSON numbers, compared by their exact decimal value. 1.0 equals 1.00, and two numbers that differ in the eighteenth decimal are told apart. A number past the range in Numbers is refused.

Strings​

Double-quoted JSON strings, with the JSON escapes, such as \" for a quote, \n for a line break, or \u and four hex digits for any character. A character outside the Basic Multilingual Plane takes a surrogate pair.

== and != compare character by character. >, < and ranges sort by the Unicode Collation Algorithm, under the CLDR root order. So Bravo sorts between alpha and charlie, and not before every lower-case word.

true and false​

Only after == and !=. These values have no order, so > on them is refused.

Date-times, dates and times​

Written bare, such as 2024-01-01T00:00:00Z, 2024-01-01 or 12:00:00Z. A bare one compares as a time with values declared as that type. A quoted one is a plain string, so the same text can match a time in one place and a string in another. Time values in q has the details, and the format they follow is under Date-times.

URIs​

Bare or quoted, and only after == and !=. q=locatedIn==urn:ngsi-ld:City:Pisa and q=locatedIn=="urn:ngsi-ld:City:Pisa" select the same entities. A single URI is also compared in its normalised spelling (see Relationship targets).

Ranges​

temperature==10..20 holds for every value from 10 to 20, both included. temperature!=10..20 holds for every other value, including a value of another type. A range takes numbers, strings or time values, and only goes after == and !=.

Lists​

color=="black","red" holds when the value is one of them, and color!="black","red" holds when it is none of them. Lists only go after == and !=. A list of URIs can be written bare, just as a single URI can, as in locatedIn==urn:ngsi-ld:City:Pisa,urn:ngsi-ld:City:Lucca.

Bare words are refused​

A bare word such as status==active is refused with 400 BadRequestData, whatever the attribute. Quote it, as in status=="active". The same goes for a word inside a list, a range, or any term of a longer expression. One bare word refuses the whole query.

GET /ngsi-ld/v1/entities?type=Vehicle&q=status==active
HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "type": "https://uri.etsi.org/ngsi-ld/errors/BadRequestData",
    "status": 400
}

Only type and status are shown here, and the real body carries the rest of the problem document. The fix is q=status=="active".

What a term compares with​

What a term tests depends on the kind of attribute it lands on.

Property​

Its value. When the value is an array, == tests whether the array holds the operand, and != tests whether it doesn't. >, <, ranges and patterns don't match an array. An empty array still exists, so q=tags selects it, but no == does.

Structured value​

A structured value isn't compared as a whole. Address one of its members instead, as in address[city]=="Berlin". The member names are terms of your @context, like attribute names.

A member holding an array is compared the way an array value is. With "rooms": [4, 5], address[rooms]==4 matches, and neither address[rooms]!=4 nor address[rooms]>3 does. A path can't step through an array, so with "floors": [{"level": 1}, {"level": 2}] the member address[floors.level] doesn't exist and no term on it matches. The bracketed path is read as an ECMAScript member expression, and an array has no member named level.

JsonProperty​

Its json is raw JSON, and its keys are never terms. Name the attribute in jsonKeys, and the members in brackets are read as written, as in q=parkingTickets[value]=="Overstay 60 minutes"&jsonKeys=parkingTickets. Every operator works on a member reached this way, and a dotted path goes deeper. Arrays inside it follow the same rules as in a structured value.

LanguageProperty​

Its strings. See Language strings.

ListProperty​

Its valueList, as one ordered list. == tests whether it holds the operand, and != tests whether it doesn't. >, <, ranges and patterns never match a list.

Relationship​

Its object, or every object when it holds several. Only == and != apply. Every other operator, a range included, matches no Relationship.

ListRelationship​

Its objectList. Only == and != apply, as membership.

VocabProperty​

Its vocab, expanded to a full IRI. Compare it with the IRI, as in q=gender=="http://example.org/Male", or quote the term and name the attribute in expandValues, as in q=gender=="Male"&expandValues=gender.

GeoProperty​

A geometry isn't a value q compares. Filter on it with a geoquery (see Geometry and geoqueries).

Several values and several instances​

A term over an attribute with several values passes when any one of them passes. The same holds across datasetId instances, and for sub-attributes too. A term like temperature.accuracy<0.2 selects an entity whose temperature has an accuracy of 0.1 under one datasetId and 0.5 under another, because one instance passes. An instance past its expiresAt takes no part.

Sub-attributes also reach the members every attribute can carry. temperature.observedAt>2024-06-01T00:00:00Z, temperature.unitCode=="CEL" and temperature.expiresAt>2150-01-01T00:00:00Z work like any other sub-attribute. A path names its sub-attribute under its own attribute, so b.x never matches a value held at a.x.

Mismatched types​

When the operand and the value are of different types, the term is unequal. == and every ordering fail, and != holds. So speed!="50" selects a speed of 50 that is stored as a number, even though speed=="50" doesn't. Over a valueList of numbers, ==2, !="2" and !=true select the entity, while =="2", !=2 and ==true don't.

A value's type is its declared datatype when it has one, with the valueType coming before the literal's own @type. Without a declaration it is the value's JSON type, whatever the value spells (see Declared numbers and Booleans). Each value of an array, a valueList included, gets its type the same way, so ["4", "7"] beside a valueType of xsd:integer holds the number 4.

The type of a quoted operand is String. With "zip": 3310 in address, address[zip]=="3310" and address[zip]<"4" select nothing, and address[zip]!="3310" selects the entity. With "zip": "3310" it's the other way round for ==3310 and !=3310. A member of a structured value follows the same rule, declared only by its own @type. A member of a JsonProperty is raw JSON and has only its JSON type, and so does a Context Source Property in csf.

Language strings​

A LanguageProperty holds one string, or several, per language. A term without a language reads all of them, and passes when any one string does. q=name=="Lift" selects {"en": "Lift", "de": "Aufzug"}, and name[*] means the same.

With a language, the term reads only the strings under that key, as in q=name[en]=="Lift". The key matches by case-insensitive tag equality, so [en] reads a key written EN, but not en-GB. A bracket holding a full tag such as [en-GB] is refused by default. The Target languages in q extension accepts full tags and ranges.

!= and !~= hold when no string matches. That includes an entity whose map has no string in the named language, so q=name[en]!="Door" selects {"fr": "Porte"}, and name[en]!~="^D" selects it too. The negated operators still need the attribute itself, so an entity without name is never selected.

The other operators need a string to compare. name[en]=="Door" and name[en]>"C" never select {"fr": "Porte"}, and neither does the existence test name[en]. That asymmetry is easy to miss, because a negated term reads "no English string matches", and a positive one reads "an English string matches".

The lang parameter chooses which string comes back. It doesn't change what q reads.

Patterns​

name~="^Merc" holds when a string value matches the regular expression, and name!~="^Merc" holds when the attribute holds a string that doesn't. A pattern only applies to strings. A number, a boolean, a Relationship object or an array matches neither operator, so an attribute holding only a number is never selected by !~=. The one exception is a language map without the named language (see Language strings).

The pattern is an IEEE 1003.2 extended regular expression with a few common extensions. (?i) folds case across Unicode, so (?i)^čebela$ selects Čebela too, and \w and \d cover non-ASCII letters and digits. . also matches a line break.

Write the pattern in quotes. A pattern made only of letters, digits, _, -, :, / and # can also go bare, as in name~=Merc, but any other character ends a bare pattern or refuses it. Unlike a string value, a pattern keeps its backslashes as written, so "^example\.com$" matches a literal full stop. The constructs Andromeda refuses are listed under Query parameters.

Inside a bracket expression, an equivalence class such as [[=e=]] and a collating symbol such as [[.e.]] stand for the one character they name, so name~="M[[=e=]]rc" matches Mercedes. Patterns compare under the C collation, where [[=e=]] holds e alone and not é. Any other [ inside a bracket expression is an ordinary character, so [[abc]] matches a] and [].

expandValues and jsonKeys​

Both parameters take a comma-separated list of attribute names.

expandValues expands the quoted strings compared with those attributes through your @context, the way a VocabProperty's vocab is expanded when it is written. With "Male": "http://example.org/Male" in the context, q=gender=="Male"&expandValues=gender compares with http://example.org/Male. Only quoted strings and URIs are expanded. A bare word is still refused, even for an attribute expandValues names.

jsonKeys marks attributes whose value is raw JSON, so the member names in their brackets are read as written instead of expanded. A Context Source Property in csf is always read that way.

A subscription keeps its q, expandValues and jsonKeys, and gives them back as you wrote them.

Linked entities​

A term can look into the entity a Relationship points at. q=isParkedIn{category}=="commercial" selects the cars parked in a commercial building. The comparison goes after the closing brace, never inside it. Hops nest, as in owner{employer{city}}=="Berlin", and a hop without a comparison tests that the linked entity has the attribute.

A hop follows every object of the Relationship, in every datasetId instance, and selects the entity when any linked entity passes. It needs join, and it can't go deeper than joinLevel (see Linked entities).

A list of entity types before a colon limits the linked entities a hop considers. isParkedIn{Building,Garage:category}=="commercial" ignores a car parked in a Warehouse, whatever its category. The list needs at least two types. With one, isParkedIn{Building:category} reads as the prefixed attribute name Building:category.

Temporal queries​

On the Temporal API, q decides which entities are answered. A term passes when an attribute instance inside the time window passes it, or when the instance still in effect at the window's start passes it. A deleted instance holds no value and passes nothing. Every datasetId counts, whatever the datasetId parameter asks for.

q doesn't remove instances from the answer. An entity it selects comes back with every instance its other parameters ask for.

Where else q appears​

Subscriptions​

A subscription's q is checked when the subscription is written, and it decides which changed entities are notified. It accepts every expression an entity query accepts, with the same meaning. q=n alone is an existence test, so the subscription is notified of an entity that has n and never of one without it.

Context source registrations​

Discovery and context source registration subscriptions take a q over entities and a csf over the registration's Context Source Properties, in the same grammar. The csf targets are the members a registrant writes (registrationName, description, mode, management, operations, expiresAt and the rest of the registration's defined members) and any property the registrant adds. mode reads as inclusive when the registration declares none, and expiresAt is a DateTime, so csf=expiresAt>2030-01-01T00:00:00Z compares instants.

A member that is an array, such as operations, scope or datasetId, is compared by membership. csf=operations=="queryEntity" selects a registration whose operations lists queryEntity among others, and csf=operations!="queryEntity" one whose operations doesn't list it.

id, type and the delivery statistics the broker keeps (status, timesSent, timesFailed, lastSuccess, lastFailure) are not targets. csf=timesSent matches no registration, and csf=id gets 400 BadRequestData, because the core @context maps id to a JSON-LD keyword rather than to an attribute name.

Purge Snapshots​

DELETE /ngsi-ld/v1/snapshots takes a q over a Snapshot's own id, type, snapshotLifetime, snapshotPriority and endpoint. The whole grammar applies, with every operator, ranges, lists, patterns, ; binding tighter than |, and parentheses. snapshotPriority is a Number and the rest are Strings, so q=snapshotPriority>8 compares numbers.

snapshotLifetime compares as the duration text the Snapshot reads back with, which is the configured default when it was created without one. So q=snapshotLifetime=="PT2H" selects a Snapshot given two hours, but q=snapshotLifetime>"PT1H" compares text, not durations. Snapshot lifetimes are described in Snapshots.

A value no Snapshot can hold is compared, not refused, so q=snapshotPriority==11 selects nothing and gets 404 ResourceNotFound. A Snapshot without an endpoint matches no term on it, != included. A term on any other name, an output-only member such as expiresAt included, gets 400 BadRequestData.

Forwarded queries​

A query forwarded to a context source carries its q, spelled in the @context used for that source (see Distributed operations).

What gets refused​

  • 400 BadRequestData for an expression the grammar doesn't allow, such as speed> with nothing after the operator, an unclosed hop rel{, a comparison inside a hop's braces, or an empty q. Andromeda doesn't run the part it could read.
  • 400 BadRequestData for a bare word. Quote it, because expandValues doesn't make a bare word acceptable.
  • 400 BadRequestData for true, false or a URI after >, >=, < or <=. Use == or != instead.
  • 400 BadRequestData for a malformed value, such as a number past the supported range, a time value with a numeric offset or a seventh fractional digit, a string with an escape JSON doesn't define, or a raw control character.
  • 400 BadRequestData for a pattern outside the supported dialect (see Patterns).
  • 400 BadRequestData for a path more than 32 levels deep. Sub-attributes and hops count together against the same bound.
  • 400 BadRequestData for a hop without join, or one deeper than joinLevel. Add join, or raise joinLevel.
  • 400 BadRequestData for a bracket that names neither a member nor a language, such as label[en_GB-x]. A full tag like [en-GB] needs the Target languages in q extension.
  • 403 TooComplexQuery for a filter making more than 1,000 comparisons. Every Query Term is one comparison and a value list is one per value, so a==1,2,3;b>4 makes four. Split the query, or raise request_limits.max_q_comparisons.
  • 403 TooComplexQuery for a filter nesting more than 16 levels. Each level of ; and | combinations counts, so a;b nests one level, and a;(b|c) and a|b;c two. Parentheses around a single term add none, but parentheses written more than 64 deep are refused whatever the limit. The limit is request_limits.max_q_nesting, at most 64.

Both limits apply to entity and temporal queries, deletes by query, and subscriptions, and a query over them is refused before any context source is asked. A subscription stored before a limit was lowered keeps notifying.

Where Andromeda interprets the specification​

The query language is defined by the specification, meaning its grammar, the target value of each attribute family, and what each operator means. A query that is not "syntactically valid (as per the referred clauses 4.9 and 4.10)" raises BadRequestData, and Andromeda gives every q it can't read that answer, on every operation. A temporal q is checked against "all the Attribute instances resulting from the initial filtering performed by the temporal query".

A quoted value is a JSON string, so its quotes are part of it. Date-times, dates and times are bare alternatives, as the specification's eighth example writes them. A bare word comes from no production except URI, true and false. Example 12 writes q=gender==Male&expandValues=gender, but the grammar has no place for Male, and Andromeda follows the grammar.

String equality follows the JSON specification, and ordering follows the Unicode Collation Algorithm, which the query language recommends. The CLDR root order is the default for strings. The specification also says "URI comparison should be performed so that the number of false negatives is minimized". That is why a URI operand is compared in its normalised spelling as well.

A few of the specification's examples and members reach past its grammar, and Andromeda follows them.

  • The grammar has no whitespace, but the examples put spaces in lists and after operators (color!= "black", "red"), so spaces, tabs and line breaks between the parts of an expression are skipped. Entity type selections are read the same way.
  • An attribute name is a letter followed by letters, digits and _, and a path is "always a composition of short hand names". Attribute names also have a prefixed form, and a broker reads a filter against expanded IRIs, so a path name can be a prefixed name or an absolute IRI, quoted when it holds a dot.
  • A value list leaves out URI, but the prose compares a Relationship with "a list of URIs", so a list takes bare URIs as a single operand does.
  • A regular expression is written without quotes in the grammar, yet a pattern holds characters the query uses as syntax, so a pattern goes in quotes, and a bare one made of name characters is read too.

The specification calls a query too complex to be resolved TooComplexQuery, which is how Andromeda answers a q past its comparison or nesting limit. Counting the instance in effect when a temporal window opens follows the specification's validity rules and its worked example.

The grammar's QueryTermAssoc doesn't nest, but its sixth example does, so Andromeda accepts nested parentheses. The grammar puts the entity type list of a hop before a further hop and needs at least two types in it. Example 14 writes sensor{Device:humidity} with one type, which the grammar reads as a prefixed attribute name, and so does Andromeda. The specification leaves open whether the list must exclude other types, and Andromeda reads it as a restriction.

!= over a language map is defined as "No matching value is found". An entity without a string in the named language therefore passes it, and Andromeda reads !~= the same way. jsonKeys and the eleventh example are why members of a raw JSON value aren't expanded, and a subscription has its own expandValues and jsonKeys. RegExp requires IEEE 1003.2. Andromeda admits that dialect as the regex crate reads it, and refuses the few constructs where the two readings differ (see Query parameters).

Last updated on