Skip to content

General Rules

This page describes the conventions that apply to every canonical document. The artefact pages describe only the fields specific to each artefact, and link back here for everything else.

Document structure

A canonical document is a single YAML mapping. It has two parts:

  1. The header, which is the same for every artefact type. It identifies the format and the structure, and holds the properties common to all maintainable artefacts (names, annotations, validity and so on).
  2. The body, which holds the content specific to the artefact type, such as the items of a codelist or the components of a data structure. The body of a stub is empty, or holds only the few properties a stub keeps.
format: "fusion-yaml(1.0.0)"              # header
ref: "Codelist=SDMX:CL_FREQ(2.1)"         # header
name: "Code list for Frequency (FREQ)"    # header
items:                                    # body
- A: "Annual"
- M: "Monthly"

There are no YAML document start (---) or end markers, comments, anchors or tags.

Document header

The header fields are written in the order below. Only format and ref are always present. The other fields are written only when the structure has a value for them.

Field Value Written when
format "fusion-yaml(1.0.0)", the format name and version. This is the version of the canonical format, not of SDMX: a document of any SDMX version uses the same format version always, first field
ref the structure class and its short URN, Class=AGENCY:ID(VERSION), for example "Codelist=SDMX:CL_FREQ(2.1)". Custom structure classes include their package, custom.mypackage.MyType=AGENCY:ID(1.0) always, second field
url the structure URL, where the full structure can be obtained the structure has a structure URL
serviceUrl the service URL the structure has a service URL
externalReference true the structure is an external reference, and is not a stub (a stub is always an external reference)
final true the structure is final
validity the validity period, see Validity the structure has a valid from or valid to date
uri a URI string the structure has a URI
name, name-xx the names, see Localised text always at least one name
description, description-xx the descriptions, see Localised text the structure has descriptions
annotations an array of annotations, see Annotations the structure has annotations
links an array of links, see Links the structure has links

All other fields belong to the body.

format: "fusion-yaml(1.0.0)"
ref: "Codelist=ACY:CL_FREQ(1.0)"
final: true
validity: "2020-01-01T00:00:00/2030-12-31T00:00:00"
uri: "https://acy.org/codelist/CL_FREQ"
name: "Frequency"
name-fr: "Fréquence"
description: "The frequency of the observations"
annotations:
- id: "ANN"
  title: "Example annotation"
items:
- A: "Annual"

Scalars and quoting

  • Strings are always double-quoted: name: "Annual". Keys are not quoted.
  • Numbers are written unquoted, for example minOccurs: 1 and maxLength: 50. A value that the information model holds as text is written as a string even when it looks like a number, for example a validation rule result, or a custom structure instance value.
  • Booleans are written as true or false. Most boolean flags are written only when they are true, and their absence means false (final, partial, multilingual, sequence, …). Where both values carry meaning (for example startInclusive in a time range), the field is always written.
  • Dates and periods are written as strings, see Validity.
  • Absent values. A field with no value is left out. null and empty strings, arrays and objects are never written.

Simple and complex values

This is the most important rule of the format. A value is written in its simplest form, and only becomes an object when it carries more information than the simple form can hold.

The clearest example is a code in a codelist. A code that has only an identifier and an English name is written as a single ID: "name" pair:

items:
- A: "Annual"

When the code has anything else, for example a description, a parent, a name in another language or an annotation, the same code becomes an object keyed by its identifier, holding a field for each property:

items:
- A:
    name: "Annual"
    description: "Once a year"

The same principle is used throughout the format:

Simple form Complex form Becomes complex when
an item - A: "Annual" - A: {name, description, …} the item has more than an English name, see Items
a component - "FREQ" - FREQ: {codelist, …} the component has a representation, an id that differs from its concept, annotations, … see Components
a single value measures: "OBS_VALUE" an array measures: [ … ] there is more than one value, see Value or array
key values REF_AREA: "UK" REF_AREA: {values, cascade, …} the selection uses cascading or a time range, see Key values

A reader must therefore accept both forms wherever the complex form is possible. Each artefact page states when each of its values changes from simple to complex.

Value or array

A field that can hold several values is written as a single scalar when it has exactly one value, and as an array when it has more than one. Nothing is written when it has no values.

measures: "OBS_VALUE"      # one value
measures:                  # several values
- "OBS_VALUE"
- "BIRTHS"

Arrays are written in block style, with the - at the same indentation as the parent key.

Every multi-valued field of simple values follows this rule, including contact details, series keys, concept roles and mapping targets.

Ordering

Fields are always written in the order given by the writer for the artefact, which is the order of the field reference tables in this documentation. Array entries keep the order they have in the structure (items, dimensions, steps, …). Collections that SDMX treats as unordered are sorted, so that their order cannot vary: localised text (including annotation text) is sorted by locale, key values are sorted alphabetically, concept scheme aliases are sorted by URN, and maps keyed by an id (for example the fixed values of a structure map) are sorted by key. Collections whose order is part of the structure are written in the order the structure holds them; the artefact pages note where this applies.

Localised text

Names, descriptions and other localised text are written as one field per locale. English (en) is the default locale and is written with the plain field name. Every other locale is written with the locale appended after a hyphen.

name: "Frequency"
name-de: "Frequenz"
name-fr: "Fréquence"
description: "The frequency of the observations"
description-fr: "La fréquence des observations"

The locales are sorted, with English written first as the plain field. All names are written before any description. The same rule applies to every localised field, including annotation text (text, text-fr) and link titles (title, title-fr).

Every locale is always written. sdmx-core can restrict the localised text a structure returns to the locales of the current request (a thread-local locale filter in TextTypeUtil). The canonical writer switches this filter off while it writes, so the document, and therefore the checksum, does not depend on the caller's locale.

Note

Custom structure instances are the one exception. A localised property value is always written with its locale, including English (definition-en), because an instance can hold a property value with no locale at all, and the suffix is what tells the two apart. See Custom Structure Instance.

Annotations

Annotations are written as an array under annotations. Each annotation is an object containing only the fields that have a value, in this order:

Field Value
id the annotation id
title the annotation title
type the annotation type
text, text-xx the annotation text, see Localised text
uri the annotation URL
value the annotation value (SDMX 3.0)
annotations:
- id: "annotationId"
  title: "Annotation Title"
  type: "NOTE"
  text: "An English note"
  text-fr: "Une note en français"
  value: "Annotation Value"

Annotations can appear on the structure (in the header) and on any identifiable inside it (items, components, groups and so on), where they are written after the object's own fields and before its links.

Links (SDMX 3.0) are written as an array under links. Each link contains only the fields that have a value, in this order:

Field Value
rel the link relationship
url the link target (href)
uri the link URI
urn the URN of the linked structure, as a canonical value, for example "Codelist=SDMX:CL_FREQ(2.1)"
type the link type
title, title-xx the link titles, see Localised text
links:
- rel: "custom"
  url: "https://server.org/sdmx/v2/structure/userdefinedoperatorscheme/SDMX/OPS/1.0/SUM"
  urn: "UserDefinedOperator=SDMX:OPS(1.0).SUM"
  type: "sdmx_artefact"

Links are always the last field of the object they belong to.

References

References to other structures are written in one of two forms.

Keyed by class. When the field name itself says what is referenced, the key is the lower-case structure class and the value is the short URN, AGENCY:ID(VERSION), followed by the identifiable path where the target is not a maintainable.

codelist: "SDMX:CL_FREQ(2.1)"
valuelist: "ACY:VL_UNIT(1.0)"
conceptscheme: "SDMX:CROSS_DOMAIN_CONCEPTS(2.0)"

This is how a component's enumeration is written: the key (codelist, valuelist, geographiccodelist, geogridcodelist, conceptscheme, …) tells the reader the type of the enumeration. Custom structure classes include their package, for example custom.mypackage.mytype.

Labelled list of one type. When a field holds references that are all of one type, and its name says which (for example dataStructures, rulesetSchemes), the values are short URNs, using value or array.

rulesetSchemes:
- "ACY:RS1(1.0)"
- "ACY:RS2(1.0)"

Canonical value. When the field name does not say what type of structure is referenced (for example attachment, target, conceptRole, or a map keyed by a dimension id), the value carries the class: Class=AGENCY:ID(VERSION), or Class=AGENCY:ID(VERSION).ITEM.PATH for an identifiable.

attachment: "DataStructure=ACY:EXR(1.0)"
conceptRole: "Concept=SDMX:CROSS_DOMAIN_CONCEPTS(2.0).FREQ"
urn: "UserDefinedOperator=SDMX:OPS(1.0).SUM"

Identifiable objects

Objects that have an identifier inside a structure (items, components, groups, steps, rules, …) are written as an array of single-key objects. The key is the identifier, and the value is an object holding the remaining properties:

groups:
- SIBLING:
    dimensions:
    - "FREQ"
    - "REF_AREA"

This keeps the identifier visually first and keeps the order of the array. Within the object, the common properties are written in a fixed position:

  • names and descriptions first (for nameable objects),
  • then the uri,
  • then the fields specific to the object,
  • then annotations,
  • then links.

Where an object has nothing to write other than its identifier, the simple form is used instead.

Items

The items of an item scheme (codes, concepts, categories, agencies, …) are written under items as identifiable objects, in the order they have in the scheme.

An item is written in the simple form - ID: "English name" when it:

  • has exactly one name, and that name is English, and
  • has no descriptions, annotations, links or URI, and
  • has no properties specific to its type (for example a code's parent, a concept's core representation, a hierarchical item's children, a validity period).

Otherwise it is written in the object form, keyed by its identifier:

items:
- _T:
    name: "Total - All activities"
    description: "This aggregate includes all activities"
- _X: "Unspecified"
- M:
    name: "Male"
    parent: "T"

Some item types are always written in the object form, because they always have properties of their own (for example the items of VTL schemes). Each artefact page states its own rule.

In hierarchical item schemes (category schemes, reporting taxonomies, …), the child items are written under an items field of their parent, with the same rules:

items:
- ECO_STAT:
    name: "Economic statistics"
    items:
    - ECO_ACCOUNTS: "Economic accounts"
- MACROECO_STAT: "Macroeconomic statistics"

An item scheme that is partial writes partial: true as the first body field.

Components

The components of data and metadata structures (dimensions, attributes, measures, metadata attributes) are written as identifiable objects, but they are keyed by their concept identity, not by their own identifier, because in SDMX a component's identifier is the identifier of its concept unless it is overridden.

Concept scheme. The concept schemes used by the components are written once, before the components:

  • If all components use concepts from a single concept scheme, it is written as conceptscheme, and components are keyed by the concept id alone.

    conceptscheme: "TEST:CS(1.0)"
    dimensions:
    - "FREQ"
    - "REF_AREA"
    
  • If the components use several concept schemes, each scheme is given an alias under conceptschemes, and the component key is ALIAS.CONCEPT_ID. The alias is the concept scheme id; when two schemes share an id a number is appended (CS, CS1, CS2). The aliases are sorted by concept scheme URN.

    conceptschemes:
      CS: "ECB:CS(1.0)"
      CS1: "SDMX:CS(1.0)"
      CONCEPTS: "TEST:CONCEPTS(1.0)"
      CS2: "TEST:CS(1.0)"
    dimensions:
    - "CS2.FREQ"
    - "CS1.INDICATOR"
    - CONCEPTS.T:
        id: "TIME_PERIOD"
    

Component object. A component is written as an object containing, in order:

  • id, only when the component id differs from the concept id (as with the time dimension above),
  • uri,
  • the representation (not written for the time dimension, which has a fixed representation),
  • the fields specific to the component type,
  • annotations and links.

Simple form. When none of these is present (the id equals the concept id and there is no representation, URI, annotation, link or type-specific field), the component is written as the plain string key: - "FREQ". Data structure attributes and metadata attribute usages are always written as objects: an attribute always has a relationship, and a usage with no relationship is an empty object (see Data Structure Definition).

Representation and text format

A representation (of a component, or the core representation of a concept) is written as fields directly in the owning object, not as a nested object:

Field Value
codelist, valuelist, … the enumeration, as a reference keyed by class
text format fields see below
minOccurs number
maxOccurs number, or "unbounded"

The text format fields are written only when they have a value, in this order: textType, maxLength, minLength, maxValue, minValue, startValue, endValue, startTime, endTime, interval, timeInterval, pattern, multilingual (only when true), decimals, sequence (only when true). The text type uses its SDMX name (String, AlphaNumeric, ObservationalTimePeriod, …). All the numeric facets are written as numbers.

- SOURCE:
    textType: "AlphaNumeric"
    maxLength: 5
    minOccurs: 1
    maxOccurs: 2

Validity and time ranges

A validity period (of a structure, or of an item) is written as a single ISO 8601 interval string, start/end. Either side may be empty when the period is open, and the precision of each date is kept.

validity: "2020-01-01T00:00:00/2030-12-31T00:00:00"
validity: "2009/2009"

A time range (used in constraints) is written as a timeRange object:

Field Value Written when
range true the time range is a range between a start and an end date. Otherwise only one of start or end is present
start the start date, in SDMX format there is a start date
startInclusive true or false there is a start date (always written with it)
end the end date, in SDMX format there is an end date
endInclusive true or false there is an end date (always written with it)
validFrom date there is a valid from date
validTo date there is a valid to date
timeRange:
  range: true
  start: "2002-02-02"
  startInclusive: false
  end: "2003-03-03"
  endInclusive: true

Key values

Constraints select values for a component (a dimension, an attribute, a metadata attribute) using key values. They are written as a field keyed by the component id.

Simple form. When the selection is a plain list of values, the values are written directly, using value or array. The values are sorted.

includedCube:
  FREQ: "A"
  REF_AREA:
  - "FR"
  - "UK"

A value that has its own validity period has the period appended after a |:

  REF_AREA:
  - "DE|/1997"
  - "FR|1995/"
  - "UK|1990/2000"

Complex form. When the selection uses cascading, prefix removal or a time range, it is written as an object. Each value appears in exactly one of the value lists:

Field Value Written when
removePrefix true the prefix is removed (SDMX 3.0 removePrefix)
values the values selected without cascading there are any
cascade the values selected with all of their children, including the value itself there are any
cascadeExcRoot the values whose children are selected, excluding the value itself there are any
timeRange a time range the selection has a time range
excludedCube:
  REF_AREA:
    values:
    - "SA"
    - "UK"
    - "US"
    cascade: "EUR"
    cascadeExcRoot: "ASIA"

Stubs

A stub is a structure that holds only its identity and the structure URL from which the full structure can be obtained. In the canonical format a stub is written with its full header (including url) and no body:

format: "fusion-yaml(1.0.0)"
ref: "Codelist=ESTAT:CL_ACTIVITY(1.0.2)"
url: "https://sdmx.stub.com"
name: "TEST"

There is no explicit stub flag. A document is a stub when it has a structure URL (url) and no content other than the properties a stub keeps. This follows the definition of MaintainableBean.isStub(), so the structure that is read back reports the same stub status as the structure that was written. A full structure that has a structure URL but no content (for example a codelist with no codes) is, by the same definition, a stub. A service URL (serviceUrl) does not make a document a stub.

A few types keep some body properties on a stub, because the stub is not meaningful without them, for example the grid definition of a geo grid codelist, or the attachment of a data constraint. These are written in the stub body, and are listed on the artefact page.

Omitted values and defaults

To keep the document short and to give each structure exactly one form, values are left out when they can be derived. A reader restores them as follows:

Omitted Restored as
a locale suffix on name, description, title, text English (en)
a boolean flag (final, partial, externalReference, multilingual, sequence, …) false
a component id the concept id
the representation of the TIME_PERIOD dimension the fixed time dimension representation
an item written in the simple form an item with only an English name
any optional field not set

Type-specific defaults (for example required on a data structure attribute or measure) are described on the artefact pages.