Name caches

Every tenant stores the full IRIs of its attribute names and entity types once. They live in the dictionary tables attribute_names and entity_type_names, and everywhere else the broker uses a small integer key. The dictionaries only grow. A name gets a row the first time it is written, and keeps it. Andromeda never refuses a write for bringing one more name, because the specification lets an entity carry any type or attribute IRI.

To avoid reading those tables on every request, each broker process keeps two caches per tenant, one for each dictionary, with a fixed memory budget. A name's key never changes once it is written, so an evicted name can't come back wrong. A miss reads the dictionary table, with one indexed SELECT for all the names a request is missing.

How the caches work

Each entry is charged $c = \ell + 160$ bytes, where $\ell$ is the length of its IRI in bytes. A 45-character IRI costs $45 + 160 = 205$ bytes, and the budget is an upper bound on what the cache really uses. It is a ceiling, not an allocation, since a cache holds only the names the tenant has used since it was loaded.

When a cache is full, a newcomer only gets in if it is used more often than the entry it would push out. A stream of names written once can't push out the names every request needs. A name that isn't used for database.name_cache.idle_eviction_after leaves the cache by itself.

When a tenant is loaded, database.name_cache.loading decides whether each dictionary is read into its cache straight away, as far as the budget allows, or filled from use alone.

How much memory

The most the name caches can hold in one process is:

$$M_{\max} = N \cdot 2 \cdot S$$

where:

• $M_{\max}$ is the memory the name caches can hold in one broker process.
• $N$ is the number of tenants loaded, at most tenancy.residency.max_resident.
• $2$ is the two caches each tenant keeps, one per dictionary.
• $S$ is database.name_cache.max_size_per_dictionary.

With the defaults, $N = 64$ and $S = 16\ \text{MiB}$, so $M_{\max} = 64 \cdot 2 \cdot 16\ \text{MiB} = 2\ \text{GiB}$. When a tenant is unloaded, its caches go with it.

One tenant can hold up to 32 MiB of names with the defaults, which is about 80,000 names of 45 characters per dictionary. A tenant whose data models use a few hundred names holds about 100 KB.

Sizing the caches

Most deployments should keep the defaults. A data model with hundreds to a few thousand names fits with room to spare, and nothing is ever evicted.

If you run many tenants, or tenants you can't trust to keep their vocabulary small, lower max_size_per_dictionary to 1 to 4 MiB to cap the worst case. Normal tenants still fit. A tenant inventing junk names only pushes out its own rarely used ones, and pays reads for them.

If a tenant has a very large set of names in active use, such as generated attribute names, raise max_size_per_dictionary once the names used within idle_eviction_after exceed about 80,000 per dictionary. The alternative is to accept the extra reads.

If a tenant has a stable vocabulary that is used in rare bursts, such as a nightly batch, set idle_eviction_after to "unlimited". The names then survive the quiet hours, and only the budget evicts.

To see how many names a tenant has, run SELECT count(*) FROM attribute_names and SELECT count(*) FROM entity_type_names in its database. That gives an upper bound on what each cache needs.

What to watch

The name dictionary metrics show lookups, evictions and the entries held. A miss rate that stays high means the cache is too small for the names in use.

If it goes wrong

• Memory grows with the number of tenants

  Lower database.name_cache.max_size_per_dictionary or tenancy.residency.max_resident.

• A tenant's requests read the dictionary tables all the time

  Its names don't fit the budget. Raise max_size_per_dictionary, or check whether the tenant generates attribute names.