General Rules¶
This page describes the conventions that apply to every canonical document. The artefact pages describe only the fields specific to each artefact, and link back here for everything else.
Document structure¶
A canonical document is a single YAML mapping. It has two parts:
- The header, which is the same for every artefact type. It identifies the format and the structure, and holds the properties common to all maintainable artefacts (names, annotations, validity and so on).
- The body, which holds the content specific to the artefact type, such as the items of a codelist or the components of a data structure. The body of a stub is empty, or holds only the few properties a stub keeps.
format: "fusion-yaml(1.0.0)" # header
ref: "Codelist=SDMX:CL_FREQ(2.1)" # header
name: "Code list for Frequency (FREQ)" # header
items: # body
- A: "Annual"
- M: "Monthly"
There are no YAML document start (---) or end markers, comments, anchors or tags.
Document header¶
The header fields are written in the order below. Only format and ref are always present. The other fields are
written only when the structure has a value for them.
| Field | Value | Written when |
|---|---|---|
format |
"fusion-yaml(1.0.0)", the format name and version. This is the version of the canonical format, not of SDMX: a document of any SDMX version uses the same format version |
always, first field |
ref |
the structure class and its short URN, Class=AGENCY:ID(VERSION), for example "Codelist=SDMX:CL_FREQ(2.1)". Custom structure classes include their package, custom.mypackage.MyType=AGENCY:ID(1.0) |
always, second field |
url |
the structure URL, where the full structure can be obtained | the structure has a structure URL |
serviceUrl |
the service URL | the structure has a service URL |
externalReference |
true |
the structure is an external reference, and is not a stub (a stub is always an external reference) |
final |
true |
the structure is final |
validity |
the validity period, see Validity | the structure has a valid from or valid to date |
uri |
a URI string | the structure has a URI |
name, name-xx |
the names, see Localised text | always at least one name |
description, description-xx |
the descriptions, see Localised text | the structure has descriptions |
annotations |
an array of annotations, see Annotations | the structure has annotations |
links |
an array of links, see Links | the structure has links |
All other fields belong to the body.
format: "fusion-yaml(1.0.0)"
ref: "Codelist=ACY:CL_FREQ(1.0)"
final: true
validity: "2020-01-01T00:00:00/2030-12-31T00:00:00"
uri: "https://acy.org/codelist/CL_FREQ"
name: "Frequency"
name-fr: "Fréquence"
description: "The frequency of the observations"
annotations:
- id: "ANN"
title: "Example annotation"
items:
- A: "Annual"
Scalars and quoting¶
- Strings are always double-quoted:
name: "Annual". Keys are not quoted. - Numbers are written unquoted, for example
minOccurs: 1andmaxLength: 50. A value that the information model holds as text is written as a string even when it looks like a number, for example a validation rule result, or a custom structure instance value. - Booleans are written as
trueorfalse. Most boolean flags are written only when they aretrue, and their absence meansfalse(final,partial,multilingual,sequence, …). Where both values carry meaning (for examplestartInclusivein a time range), the field is always written. - Dates and periods are written as strings, see Validity.
- Absent values. A field with no value is left out.
nulland empty strings, arrays and objects are never written.
Simple and complex values¶
This is the most important rule of the format. A value is written in its simplest form, and only becomes an object when it carries more information than the simple form can hold.
The clearest example is a code in a codelist. A code that has only an identifier and an English name is written as a
single ID: "name" pair:
When the code has anything else, for example a description, a parent, a name in another language or an annotation, the same code becomes an object keyed by its identifier, holding a field for each property:
The same principle is used throughout the format:
| Simple form | Complex form | Becomes complex when |
|---|---|---|
an item - A: "Annual" |
- A: {name, description, …} |
the item has more than an English name, see Items |
a component - "FREQ" |
- FREQ: {codelist, …} |
the component has a representation, an id that differs from its concept, annotations, … see Components |
a single value measures: "OBS_VALUE" |
an array measures: [ … ] |
there is more than one value, see Value or array |
key values REF_AREA: "UK" |
REF_AREA: {values, cascade, …} |
the selection uses cascading or a time range, see Key values |
A reader must therefore accept both forms wherever the complex form is possible. Each artefact page states when each of its values changes from simple to complex.
Value or array¶
A field that can hold several values is written as a single scalar when it has exactly one value, and as an array when it has more than one. Nothing is written when it has no values.
Arrays are written in block style, with the - at the same indentation as the parent key.
Every multi-valued field of simple values follows this rule, including contact details, series keys, concept roles and mapping targets.
Ordering¶
Fields are always written in the order given by the writer for the artefact, which is the order of the field reference tables in this documentation. Array entries keep the order they have in the structure (items, dimensions, steps, …). Collections that SDMX treats as unordered are sorted, so that their order cannot vary: localised text (including annotation text) is sorted by locale, key values are sorted alphabetically, concept scheme aliases are sorted by URN, and maps keyed by an id (for example the fixed values of a structure map) are sorted by key. Collections whose order is part of the structure are written in the order the structure holds them; the artefact pages note where this applies.
Localised text¶
Names, descriptions and other localised text are written as one field per locale. English (en) is the default
locale and is written with the plain field name. Every other locale is written with the locale appended after a hyphen.
name: "Frequency"
name-de: "Frequenz"
name-fr: "Fréquence"
description: "The frequency of the observations"
description-fr: "La fréquence des observations"
The locales are sorted, with English written first as the plain field. All names are written before any description.
The same rule applies to every localised field, including annotation text (text, text-fr) and link titles
(title, title-fr).
Every locale is always written. sdmx-core can restrict the localised text a structure returns to the locales of the
current request (a thread-local locale filter in TextTypeUtil). The canonical writer switches this filter off while it
writes, so the document, and therefore the checksum, does not depend on the caller's locale.
Note
Custom structure instances are the one exception. A localised property value is always written with its locale,
including English (definition-en), because an instance can hold a property value with no locale at all, and the
suffix is what tells the two apart. See Custom Structure Instance.
Annotations¶
Annotations are written as an array under annotations. Each annotation is an object containing only the fields that
have a value, in this order:
| Field | Value |
|---|---|
id |
the annotation id |
title |
the annotation title |
type |
the annotation type |
text, text-xx |
the annotation text, see Localised text |
uri |
the annotation URL |
value |
the annotation value (SDMX 3.0) |
annotations:
- id: "annotationId"
title: "Annotation Title"
type: "NOTE"
text: "An English note"
text-fr: "Une note en français"
value: "Annotation Value"
Annotations can appear on the structure (in the header) and on any identifiable inside it (items, components, groups and so on), where they are written after the object's own fields and before its links.
Links¶
Links (SDMX 3.0) are written as an array under links. Each link contains only the fields that have a value, in this
order:
| Field | Value |
|---|---|
rel |
the link relationship |
url |
the link target (href) |
uri |
the link URI |
urn |
the URN of the linked structure, as a canonical value, for example "Codelist=SDMX:CL_FREQ(2.1)" |
type |
the link type |
title, title-xx |
the link titles, see Localised text |
links:
- rel: "custom"
url: "https://server.org/sdmx/v2/structure/userdefinedoperatorscheme/SDMX/OPS/1.0/SUM"
urn: "UserDefinedOperator=SDMX:OPS(1.0).SUM"
type: "sdmx_artefact"
Links are always the last field of the object they belong to.
References¶
References to other structures are written in one of two forms.
Keyed by class. When the field name itself says what is referenced, the key is the lower-case structure class and
the value is the short URN, AGENCY:ID(VERSION), followed by the identifiable path where the target is not a
maintainable.
codelist: "SDMX:CL_FREQ(2.1)"
valuelist: "ACY:VL_UNIT(1.0)"
conceptscheme: "SDMX:CROSS_DOMAIN_CONCEPTS(2.0)"
This is how a component's enumeration is written: the key (codelist, valuelist, geographiccodelist,
geogridcodelist, conceptscheme, …) tells the reader the type of the enumeration. Custom structure classes include
their package, for example custom.mypackage.mytype.
Labelled list of one type. When a field holds references that are all of one type, and its name says which
(for example dataStructures, rulesetSchemes), the values are short URNs, using
value or array.
Canonical value. When the field name does not say what type of structure is referenced (for example
attachment, target, conceptRole, or a map keyed by a dimension id), the value carries the class:
Class=AGENCY:ID(VERSION), or Class=AGENCY:ID(VERSION).ITEM.PATH for an identifiable.
attachment: "DataStructure=ACY:EXR(1.0)"
conceptRole: "Concept=SDMX:CROSS_DOMAIN_CONCEPTS(2.0).FREQ"
urn: "UserDefinedOperator=SDMX:OPS(1.0).SUM"
Identifiable objects¶
Objects that have an identifier inside a structure (items, components, groups, steps, rules, …) are written as an array of single-key objects. The key is the identifier, and the value is an object holding the remaining properties:
This keeps the identifier visually first and keeps the order of the array. Within the object, the common properties are written in a fixed position:
- names and descriptions first (for nameable objects),
- then the
uri, - then the fields specific to the object,
- then
annotations, - then
links.
Where an object has nothing to write other than its identifier, the simple form is used instead.
Items¶
The items of an item scheme (codes, concepts, categories, agencies, …) are written under items as
identifiable objects, in the order they have in the scheme.
An item is written in the simple form - ID: "English name" when it:
- has exactly one name, and that name is English, and
- has no descriptions, annotations, links or URI, and
- has no properties specific to its type (for example a code's parent, a concept's core representation, a hierarchical item's children, a validity period).
Otherwise it is written in the object form, keyed by its identifier:
items:
- _T:
name: "Total - All activities"
description: "This aggregate includes all activities"
- _X: "Unspecified"
- M:
name: "Male"
parent: "T"
Some item types are always written in the object form, because they always have properties of their own (for example the items of VTL schemes). Each artefact page states its own rule.
In hierarchical item schemes (category schemes, reporting taxonomies, …), the child items are written under an items
field of their parent, with the same rules:
items:
- ECO_STAT:
name: "Economic statistics"
items:
- ECO_ACCOUNTS: "Economic accounts"
- MACROECO_STAT: "Macroeconomic statistics"
An item scheme that is partial writes partial: true as the first body field.
Components¶
The components of data and metadata structures (dimensions, attributes, measures, metadata attributes) are written as identifiable objects, but they are keyed by their concept identity, not by their own identifier, because in SDMX a component's identifier is the identifier of its concept unless it is overridden.
Concept scheme. The concept schemes used by the components are written once, before the components:
-
If all components use concepts from a single concept scheme, it is written as
conceptscheme, and components are keyed by the concept id alone. -
If the components use several concept schemes, each scheme is given an alias under
conceptschemes, and the component key isALIAS.CONCEPT_ID. The alias is the concept scheme id; when two schemes share an id a number is appended (CS,CS1,CS2). The aliases are sorted by concept scheme URN.
Component object. A component is written as an object containing, in order:
id, only when the component id differs from the concept id (as with the time dimension above),uri,- the representation (not written for the time dimension, which has a fixed representation),
- the fields specific to the component type,
annotationsandlinks.
Simple form. When none of these is present (the id equals the concept id and there is no representation, URI,
annotation, link or type-specific field), the component is written as the plain string key: - "FREQ". Data
structure attributes and metadata attribute usages are always written as objects: an attribute always has a
relationship, and a usage with no relationship is an empty object (see
Data Structure Definition).
Representation and text format¶
A representation (of a component, or the core representation of a concept) is written as fields directly in the owning object, not as a nested object:
| Field | Value |
|---|---|
codelist, valuelist, … |
the enumeration, as a reference keyed by class |
| text format fields | see below |
minOccurs |
number |
maxOccurs |
number, or "unbounded" |
The text format fields are written only when they have a value, in this order: textType, maxLength, minLength,
maxValue, minValue, startValue, endValue, startTime, endTime, interval, timeInterval, pattern,
multilingual (only when true), decimals, sequence (only when true). The text type uses its SDMX name (String,
AlphaNumeric, ObservationalTimePeriod, …). All the numeric facets are written as numbers.
Validity and time ranges¶
A validity period (of a structure, or of an item) is written as a single ISO 8601 interval string, start/end. Either
side may be empty when the period is open, and the precision of each date is kept.
A time range (used in constraints) is written as a timeRange object:
| Field | Value | Written when |
|---|---|---|
range |
true |
the time range is a range between a start and an end date. Otherwise only one of start or end is present |
start |
the start date, in SDMX format | there is a start date |
startInclusive |
true or false |
there is a start date (always written with it) |
end |
the end date, in SDMX format | there is an end date |
endInclusive |
true or false |
there is an end date (always written with it) |
validFrom |
date | there is a valid from date |
validTo |
date | there is a valid to date |
timeRange:
range: true
start: "2002-02-02"
startInclusive: false
end: "2003-03-03"
endInclusive: true
Key values¶
Constraints select values for a component (a dimension, an attribute, a metadata attribute) using key values. They are written as a field keyed by the component id.
Simple form. When the selection is a plain list of values, the values are written directly, using value or array. The values are sorted.
A value that has its own validity period has the period appended after a |:
Complex form. When the selection uses cascading, prefix removal or a time range, it is written as an object. Each value appears in exactly one of the value lists:
| Field | Value | Written when |
|---|---|---|
removePrefix |
true |
the prefix is removed (SDMX 3.0 removePrefix) |
values |
the values selected without cascading | there are any |
cascade |
the values selected with all of their children, including the value itself | there are any |
cascadeExcRoot |
the values whose children are selected, excluding the value itself | there are any |
timeRange |
a time range | the selection has a time range |
Stubs¶
A stub is a structure that holds only its identity and the structure URL from which the full structure can be obtained.
In the canonical format a stub is written with its full header (including url) and no body:
format: "fusion-yaml(1.0.0)"
ref: "Codelist=ESTAT:CL_ACTIVITY(1.0.2)"
url: "https://sdmx.stub.com"
name: "TEST"
There is no explicit stub flag. A document is a stub when it has a structure URL (url) and no content other than
the properties a stub keeps. This follows the definition of MaintainableBean.isStub(), so the structure that is
read back reports the same stub status as the structure that was written. A full structure that has a structure URL
but no content (for example a codelist with no codes) is, by the same definition, a stub. A service URL (serviceUrl)
does not make a document a stub.
A few types keep some body properties on a stub, because the stub is not meaningful without them, for example the grid definition of a geo grid codelist, or the attachment of a data constraint. These are written in the stub body, and are listed on the artefact page.
Omitted values and defaults¶
To keep the document short and to give each structure exactly one form, values are left out when they can be derived. A reader restores them as follows:
| Omitted | Restored as |
|---|---|
a locale suffix on name, description, title, text |
English (en) |
a boolean flag (final, partial, externalReference, multilingual, sequence, …) |
false |
a component id |
the concept id |
the representation of the TIME_PERIOD dimension |
the fixed time dimension representation |
| an item written in the simple form | an item with only an English name |
| any optional field | not set |
Type-specific defaults (for example required on a data structure attribute or measure) are described on the artefact
pages.