ml_collections.config_flags package¶

Config flags module.

DEFINE_config_dict¶

config_flags.DEFINE_config_dict(config: ml_collections.config_dict.config_dict.ConfigDict, help_string: str = 'ConfigDict instance.', flag_values: absl.flags._flagvalues.FlagValues = <absl.flags._flagvalues.FlagValues object>, lock_config: bool = True, sys_argv: Optional[List[str]] = None, **kwargs) → absl.flags._flagvalues.FlagHolder¶

Defines flag for inline ConfigDict’s compatible with absl flags.

Similar to DEFINE_config_file except the flag’s value should be a ConfigDict instead of a path to a file containing a ConfigDict. After the flag is parsed, FLAGS.name will contain a reference to the ConfigDict, optionally with some values overridden.

Typical usage example:

script.py:

from ml_collections import config_dict
from ml_collections import config_flags


config = config_dict.ConfigDict({
    'field1': 1,
    'field2': 'tom',
    'nested': {
        'field': 2.23,
    }
})


_CONFIG = config_flags.DEFINE_config_dict('my_config', config)
...

print(_CONFIG.value)

The following command:

python script.py -- --my_config.field1 8
                    --my_config.nested.field=2.1

will print:

field1: 8
field2: tom
nested: {field: 2.1}

Parameters:

name – Flag name.
config – ConfigDict object.
help_string – Help string to display when –helpfull is called. (default: “ConfigDict instance.”)
flag_values – FlagValues instance used for parsing. (default: absl.flags.FLAGS)
lock_config – If set to True, loaded config will be locked through calling .lock() method on its instance (if it exists). (default: True)
sys_argv – If set, interprets this as the full list of args used in parsing. This is used to identify which overrides to define as flags. If not specified, uses the system sys.argv to figure it out.
**kwargs – Optional keyword arguments passed to Flag constructor.

Returns:

a handle to defined flag.

DEFINE_config_file¶

config_flags.DEFINE_config_file(default: Optional[str] = None, help_string: str = 'path to config file.', flag_values: absl.flags._flagvalues.FlagValues = <absl.flags._flagvalues.FlagValues object>, lock_config: bool = True, sys_argv: Optional[List[str]] = None, **kwargs) → absl.flags._flagvalues.FlagHolder¶

Defines flag for ConfigDict files compatible with absl flags.

The flag’s value should be a path to a valid python file which contains a function called get_config() that returns a python object specifying a configuration. After the flag is parsed, FLAGS.name will contain a reference to this object, optionally with some values overridden.

During flags parsing, every flag of form –name.([a-zA-Z0-9]+.?)+=value and -name.([a-zA-Z0-9]+.?)+ value will be treated as an override of a specific field in the config object returned by this flag. Field is essentially a dot delimited path inside the object where each path element has to be either an attribute or a key existing in the config object. For example –my_config.field1.field2=val means “assign value val to the attribute (or key) field2 inside value of the attribute (or key) field1 inside the value of my_config object”. If there are both attribute and key-based access with the same name, attribute is preferred.

Typical usage example:

script.py:

from ml_collections import config_flags

_CONFIG = config_flags.DEFINE_config_file('my_config')

print(_CONFIG.value)

config.py:

def get_config():
  return {
      'field1': 1,
      'field2': 'tom',
      'nested': {
          'field': 2.23,
      },
  }

The following command:

python script.py -- --my_config=config.py
                    --my_config.field1 8
                    --my_config.nested.field=2.1

will print:

{'field1': 8, 'field2': 'tom', 'nested': {'field': 2.1}}

It is possible to parameterise the get_config function, allowing it to return a differently structured result for different occasions. This is particularly useful when setting up hyperparameter sweeps across various network architectures.

parameterised_config.py:

def get_config(config_string):
  possible_configs = {
      'mlp': {
          'constructor': 'snt.nets.MLP',
          'config': {
              'output_sizes': (128, 128, 1),
          }
      },
      'lstm': {
          'constructor': 'snt.LSTM',
          'config': {
              'hidden_size': 128,
              'forget_bias': 1.0,
          }
      }
  }
  return possible_configs[config_string]

If a colon is present in the command line override for the config file, everything to the right of the colon is passed into the get_config function. The following command lines will both function correctly:

python script.py -- --my_config=parameterised_config.py:mlp
                    --my_config.config.output_sizes="(256,256,1)"


python script.py -- --my_config=parameterised_config.py:lstm
                    --my_config.config.hidden_size=256

The following will produce an error, as the hidden_size flag does not exist when the “mlp” config_string is provided:

python script.py -- --my_config=parameterised_config.py:mlp
                    --my_config.config.hidden_size=256

Parameters:

name – Flag name, optionally including extra config after a colon.
default – Default value of the flag (default: None).
help_string – Help string to display when –helpfull is called. (default: “path to config file.”)
flag_values – FlagValues instance used for parsing. (default: absl.flags.FLAGS)
lock_config – If set to True, loaded config will be locked through calling .lock() method on its instance (if it exists). (default: True)
sys_argv – If set, interprets this as the full list of args used in parsing. This is used to identify which overrides to define as flags. If not specified, uses the system sys.argv to figure it out.
**kwargs – Optional keyword arguments passed to Flag constructor.

Returns:

a handle to defined flag.

Additional Methods¶

`ml_collections.config_flags.config_flags.is_config_flag`(flag)	Returns True iff flag is an instance of _ConfigFlag.
`ml_collections.config_flags.config_flags.GetValue`(…)	Gets value of a single field.
`ml_collections.config_flags.config_flags.GetType`(…)	Gets type of field in config described by a config_path.
`ml_collections.config_flags.config_flags.GetTypes`
`ml_collections.config_flags.config_flags.SetValue`(…)	Sets value of field described by config_path.