Skip to content

Data Constraint

A Data Constraint limits the data that is allowed for a structure, or describes the data that actually exists. It is attached to a Data Structure Definition, Dataflow, Provision Agreement or data source. It can define cube regions (allowed values for each component) and key sets (whole series keys). The document starts with ref: "DataConstraint=AGENCY:ID(1.0)".

The writer is ContentConstraintCanonicalWriter.

Simple form

An allowed-content constraint attached to one structure, with a cube region in which each value is a string or an array of strings:

format: "fusion-yaml(1.0.0)"
ref: "DataConstraint=ACY:CONS2(1.0)"
name: "CONS2 Name"
attachment: "DataStructure=ACY:EXR(1.0)"
includedCube:
  REF_AREA:
  - "ASIA"
  - "EUR"
excludedCube:
  INDICATOR: "BIR%"
Source: dataconstraint/testMultipleCubes.yaml

The document starts with the common header. Then come these body fields:

  • attachment: what the constraint is attached to. Each entry is a canonical value (class, =, short URN), because the target can be several types. There can be more than one target. Following the value-or-array rule, a single target is written as a string. Multiple targets are sorted by URN.
  • includedCube / excludedCube: the dimension values of the included and excluded cube regions. Each entry is a key value keyed by the dimension id. Values are sorted, and may contain the % wildcard.

Complex form

Role

The constraint role is written only for actual constraints, as role: "Actual". Allowed is the default and is never written.

format: "fusion-yaml(1.0.0)"
ref: "DataConstraint=ACY:CONS2(1.0)"
name: "CONS2 Name"
role: "Actual"
attachment: "DataStructure=ACY:EXR(1.0)"
includedCube:
  REF_AREA:
  - "ASIA"
  - "EUR"
excludedCube:
  INDICATOR: "BIR%"
Source: dataconstraint/testActualConstraint.yaml

Cube regions: dimension and attribute values

A cube region can constrain both dimensions and attributes, and a dimension and an attribute may share an id. So the two are written under separate fields. Dimension values go under includedCube / excludedCube, and attribute values under includedCubeAttributes / excludedCubeAttributes:

includedCube:
  TIME_FORMAT:
  - "P1M"
  - "P1Y"
includedCubeAttributes:
  TIME_FORMAT: "P1D"
Source: dataconstraint/testCubeDimensionAndAttributeSameId.yaml

Key values: simple vs object

A key value is written in its simple form, ID: value(s), unless it has a removed prefix, cascading values or a time range. If it has any of these, it becomes an object (key values).

Validity on individual values does not by itself make a key value an object. Each value with a validity period is written as VALUE|PERIOD:

attachment: "ProvisionAgreement=ACY:EXR(1.0)"
includedCube:
  REF_AREA:
  - "DE|/1997"
  - "FR|1995/"
  - "UK|1990/2000"
Source: dataconstraint/testCubeConstraintWithValidity.yaml

When values cascade to their child codes, the object form splits them by type:

excludedCube:
  REF_AREA:
    values:
    - "SA"
    - "UK"
    - "US"
    cascade: "EUR"
    cascadeExcRoot: "ASIA"
Source: dataconstraint/testCascadeWithNonCascade.yaml

  • values holds the values that do not cascade.
  • cascade holds the values that cascade and include the value itself.
  • cascadeExcRoot holds the values that cascade to their children only, excluding the value itself.

Each list is sorted and follows the value-or-array rule.

A removed prefix is written as removePrefix: true, which also forces the object form:

includedCube:
  REF_AREA:
    removePrefix: true
    values:
    - "ASIA"
    - "EUR"
Source: dataconstraint/testCubesRemovePrefix.yaml

A key value with a time range (typically TIME_PERIOD) is written as an object with a timeRange object. The format is described in validity.

Key sets (series)

Included and excluded series keys are written together under one series object. The component ids are written once, as headers. Each key is then written as a single string of values separated by :, in header order:

series:
  dimensions:
  - "FREQ"
  - "REF_AREA"
  - "INDICATOR"
  - "INDUSTRY"
  attributes:
  - "COLLECTION"
  - "UNIT"
  included: "A:UK:ABC::A:0"
  excluded: "M::ABC:REV1_%:B:1"
Source: dataconstraint/testSeriesConstraintWithAttrValues.yaml

The rules for building a key string:

  • dimensions lists every dimension id used by any included or excluded key, in the order first seen. attributes does the same for attribute (component) values, and is omitted when no key has attribute values.
  • In each key, the dimension values come first, then the attribute values if there are attribute headers. A component the key does not constrain is left empty, so A:UK:ABC: has no INDUSTRY value.
  • Multiple values for one component are joined with +, as in A:UK+FR+DE:ABC.
  • A value whose prefix is removed starts with ~, as in A:UK:~ABC.
  • A key with a validity period ends with |PERIOD, as in A:DE:ABC|2008/ or Q:UK:ABC|/2010.
  • dimensions, attributes, included and excluded each follow the value-or-array rule: a single entry is written as a string. included and excluded keep the order of the constraint's keys; they are not sorted.

series:
  dimensions:
  - "FREQ"
  - "REF_AREA"
  - "INDICATOR"
  included:
  - "A:UK:ABC"
  - "A:FR:ABC"
  - "A:DE:ABC|2008/"
  - "Q:UK:ABC|/2010"
  - "Q:DE:ABC|2008/2009"
Source: dataconstraint/testSeriesConstraintWithValidity.yaml (trimmed)

Attachment to data sources or a data set

A constraint can also be attached to data sources. Each one is written as an object in attachmentDataSources:

format: "fusion-yaml(1.0.0)"
ref: "DataConstraint=ACY:CONS11(1.0)"
name: "CONS11 Name"
role: "Actual"
attachmentDataSources:
- url: "https://acy.org/data/source"
  type: "Simple"
Source: dataconstraint/testDataSourceAttachment.yaml

A data source object holds url, then type (REST, WebService or Simple), then wsdlUrl and wadlUrl. A data source with none of the three type flags set has no type.

An attachment to a data set or metadata set would be written as attachmentDataSet or attachmentMetadataSet, an object holding provider (the canonical value of the data provider) and setId. These fields are reserved: the information model has no implementation of a data set or metadata set reference, so no constraint can hold one and neither field is written in practice.

Reference period, release calendar and metadata target region

  • referencePeriod: an object with start and end, each in SDMX date format and each written only when set.
  • releaseCalendar: an object with periodicity, offset and tolerance, each written only when set. For example:
releaseCalendar:
  periodicity: "P1M1D"
  offset: "P5D"
  tolerance: "P1D"
  • metadataTargetRegion (SDMX 2.1 content constraints): an object with these fields:
    • include: a boolean, always written, even when false.
    • report and metadataTarget.
    • keyValues: an array. Each element is an object holding one key value, plus objectReferences (canonical value or values) when there are any.
    • attributes: a map of key values.

Field reference

Field Value Written when
role "Actual" the constraint defines actual data present
attachment canonical value(s), sorted by URN, value or array the attachment references structures
attachmentDataSources array of data source objects the attachment has data sources
attachmentDataSet object: provider, setId the attachment is a data set reference
attachmentMetadataSet object: provider, setId the attachment is a metadata set reference
series object: dimensions, attributes, included, excluded there are included or excluded series keys
includedCube map of dimension id to key value the included cube region has dimension values
includedCubeAttributes map of attribute id to key value the included cube region has attribute values
excludedCube map of dimension id to key value the excluded cube region has dimension values
excludedCubeAttributes map of attribute id to key value the excluded cube region has attribute values
referencePeriod object: start, end the constraint has a reference period
releaseCalendar object: periodicity, offset, tolerance the constraint has a release calendar
metadataTargetRegion object (see above) the constraint has a metadata target region

Data source object (attachmentDataSources[]):

Field Value Written when
url data URL set
type REST, WebService or Simple one of the type flags is set (checked in that order)
wsdlUrl URL set
wadlUrl URL set

series object:

Field Value Written when
dimensions dimension id(s), value or array always
attributes attribute id(s), value or array some key has attribute values
included key string(s), value or array there are included keys
excluded key string(s), value or array there are excluded keys

Key value object (inside a cube region or a metadata target region):

Field Value Written when
removePrefix true the prefix is removed
values non-cascading value(s), sorted there are any
cascade cascading value(s), including the root there are any
cascadeExcRoot cascading value(s), excluding the root there are any
timeRange time range object the key value has a time range

Stub

A stub data constraint keeps its attachment. After the header, the stub writes the attachment fields (attachment, attachmentDataSources, attachmentDataSet / attachmentMetadataSet) when the stub has them:

format: "fusion-yaml(1.0.0)"
ref: "DataConstraint=ACY:CONS3(1.0)"
url: "https://sdmx.stub.com"
name: "CONS3 Name"
attachment: "DataStructure=ACY:EXR(1.0)"
Source: dataconstraint/testStub.yaml

Only a non-complete stub of an allowed constraint has an attachment. A complete stub and a stub of an actual constraint have none, so only the header is written. role is never written in a stub.

Reading notes

  • If role is absent, the constraint is read as Allowed. role: "Allowed" is also accepted.
  • includedCube and includedCubeAttributes are read into the same cube region, and so are the two excluded fields.
  • When a series key is read, an empty value means the component is unconstrained, a leading ~ sets remove prefix, and | introduces the validity period (from/to, either side optional). The number of :-separated values must equal the number of dimensions plus attributes.
  • attachmentDataSet and attachmentMetadataSet cannot be read. The reader rejects them, because there is no implementation of a data or metadata set reference.