Layered Config Tree

A configuration structure that supports cascading layers.

Layered Config Tree allows base configurations to be overridden by multiple layers with cascading priorities. The configuration values are presented as attributes of the configuration object and are the value of the keys in the outermost layer of configuration where they appear.

For example:

>>> config = LayeredConfigTree(layers=['inner_layer', 'middle_layer', 'outer_layer', 'user_overrides'])
>>> config.update({'section_a': {'item1': 'value1', 'item2': 'value2'}, 'section_b': {'item1': 'value3'}}, layer='inner_layer')
>>> config.update({'section_a': {'item1': 'value4'}, 'section_b': {'item1': 'value5'}}, layer='middle_layer')
>>> config.update({'section_b': {'item1': 'value6'}}, layer='outer_layer')
>>> config.section_a.item1
'value4'
>>> config.section_a.item2
'value2'
>>> config.section_b.item1
'value6'

class layered_config_tree.main.ConfigNode(layers, name)[source]

A priority based configuration value.

A ConfigNode represents a single configuration value with priority-based layers. The intent is to allow a value to be set from sources with different priorities and to record what the value was set to and from where.

For example, a simulation may need certain values to always exist, and so it will set them up at a “base” layer. Components in the simulation may have a different set of priorities and so override the “base” value at a “component” level. Finally a user may want to override the simulation and component defaults with values at the command line or interactively, and so those values will be set in a final “user” layer.

A ConfigNode may only have a value set at each layer once. Attempts to set a value at the same layer multiple times will result in a DuplicatedConfigurationError.

The ConfigNode will record all values set and the source they are set from. This sort of provenance with configuration data greatly eases debugging and analysis of simulation code.

This class should not be instantiated directly. All interaction should take place by manipulating a LayeredConfigTree object.

Parameters:

layers (list[str])
name (str)

property name: str: The name of this configuration value.

property accessed: bool: Whether or not this node has been accessed.

property metadata: list[dict[str, Any | str | None]]: All values and associated metadata for this node.

freeze()[source]

Convert the ConfigNode to read-only.

This can be used to create a contract around when the configuration is modifiable.

Return type:: None

get_value(layer=None)[source]

Return the value at the specified layer.

If no layer is specified, the outermost (highest priority) layer at which a value has been set will be used.

Return type:

Any

Parameters:

layer (str | None) – Name of the layer to retrieve the value from.

Raises:

ConfigurationKeyError – If no value has been set at any layer (i.e. the ConfigNode is empty).
MissingLayerError – If values exist but not at the requested layer.

update(value, layer, source)[source]

Set a value for a layer with optional metadata about source.

Return type:

None

Parameters:

value (Any) – Data to store in the node.
layer (str | None) – Name of the layer to use. If no layer is provided, the value will be set in the outermost (highest priority) layer.
source (str | None) – Metadata indicating the source of this value.

Raises:

ConfigurationError – If the node is frozen.
ConfigurationKeyError – If the provided layer does not exist.
DuplicatedConfigurationError – If a value has already been set at the provided layer or a value is already in the outermost layer and no layer has been provided.

class layered_config_tree.main.ConfigIterator(config_tree)[source]

An iterator over the keys of a LayeredConfigTree.

Parameters:: config_tree (LayeredConfigTree)

class layered_config_tree.main.LayeredConfigTree(data=None, layers=[], name='')[source]

A container for configuration information.

Each configuration value is exposed as an attribute the value of which is determined by the outermost layer which has the key defined.

Parameters:

data (InputData | None)
layers (list[str])
name (str)

freeze()[source]

Convert the LayeredConfigTree to read only.

This is useful for loading and then freezing configurations that should not be modified at runtime.

Return type:: None

items()[source]

Return an iterable of all (child_name, child) pairs.

Return type:: Iterable[tuple[str, LayeredConfigTree | ConfigNode]]

keys()[source]

Return an Iterable of all child names.

Return type:: Iterable[str]

values()[source]

Return an Iterable of all children.

Return type:: Iterable[LayeredConfigTree | ConfigNode]

unused_keys()[source]

List all values in the LayeredConfigTree that haven’t been accessed.

Return type:: list[str]

to_dict()[source]

Convert the LayeredConfigTree to a nested dictionary.

All metadata is lost in this conversion.

Return type:: dict[str, Any]

get(keys, default_value=None, layer=None)[source]

Return the value at the key or key path in the outermost layer.

Return type:

Any

Parameters:

keys (str | list[str]) – The string or ordered list of strings to look for in the tree starting from the outermost layer.
default_value (Any) – The value to return if and only if the final key in the key path does not exist.
layer (str | None) – The name of the layer to retrieve the value from.

Notes

The default_value will only be used if final key in the key path does not exist but the rest of the key path does.

Returns:

The value at the key or nested keys and at the requested layer (the outer, by default). default_value (None, by default) is returned if the full key path except for the final key exists at an explicitly-requested layer.

Raises:

TypeError – If the keys parameter is not a string or a list of strings.

Parameters:

keys (str | list[str])
default_value (Any)
layer (str | None)

Return type:

Any

get_tree(keys)[source]

Return the LayeredConfigTree at the key or key path from the outermost layer.

Return type:

LayeredConfigTree

Parameters:

keys (str | list[str]) – The key or key path to look up from the outermost layer.

Returns:

The LayeredConfigTree located at the key or key path provided starting from the outermost layer.

Raises:

TypeError – If the keys parameter is not a string or list of strings.
ConfigurationKeyError – If any of the keys in the key path do not exist in the tree.
ConfigurationError – If the data at the final key in the key path is not a LayeredConfigTree.

update(data, layer=None, source=None)[source]

Add additional data into the LayeredConfigTree.

Return type:

None

Parameters:

data (dict[str, Any] | str | Path | LayeredConfigTree | None) –
The data used to update the LayeredConfigTree.
- dict : Flat or nested dictionaries may be provided. Keys of dictionaries at all levels must be strings.
- str : Strings provided can be yaml formatted strings, which will be parsed into a dictionary using standard yaml parsing. Alternatively, a path to a yaml file may be provided and the file will be read in and parsed.
- pathlib.Path : A path object to a yaml file will be interpreted the same as a string representation.
- LayeredConfigTree : Another LayeredConfigTree can be used. All source information will be ignored and the provided layer and source will be used to set the metadata.
layer (str | None) – The name of the layer to store the value in. If no layer is provided, the value will be set in the outermost (highest priority) layer.
source (str | None) – The source to attribute the value to.

Raises:

ConfigurationError – If the LayeredConfigTree is frozen or attempting to assign an invalid value.
ConfigurationKeyError – If the provided layer does not exist.
DuplicatedConfigurationError – If a value has already been set at the provided layer or a value is already in the outermost layer and no layer has been provided.

metadata(name)[source]

Return all values and associated metadata for the named child.

Return type:: list[dict[str, Any]]
Parameters:: name (str) – The name of the child to retrieve metadata for.
Returns:: A list of dictionaries, each containing ‘layer’, ‘source’, and ‘value’ keys for every layer at which the child has a value set.
Raises:: ConfigurationKeyError – If no child with the given name exists.