Getting the state of an experiment bench#
To replicate an experiment, it is useful to compare the state of the original and the current experiment bench, ensuring they are as close as possible. Phileas has different tools that allow to do this.
Example bench#
Let us work with the following simple experiment bench:
1bench = """
2simulated-motors:
3 loader: phileas-mock_motors-phileas
4
5simulated-oscilloscope:
6 loader: phileas-mock_oscilloscope-phileas
7 probe: electric-field-probe
8 motors: simulated-motors
9"""
10
11experiment = """
12oscilloscope:
13 interface: oscilloscope
14 amplitude: !range
15 start: 0.1
16 end: 10
17 steps: 5
18
19iteration: !sequence [1, 2]
20"""
21
22factory = ExperimentFactory(bench, experiment)
23factory.initiate_connections()
Verifying the effectiveness of an applied configuration#
After having configured an instrument, it is possible that its effective
configuration differs from the required one. You can use
get_effective_instrument_configuration()
and
get_effective_experiment_configuration()
to know the exact state of the instrument.
In our case, the amplitude of the oscilloscope can only be a power of 10. Configuring it, and getting its effective configuration, can be done with:
1for conf in factory.experiment_config:
2 factory.configure_experiment(conf)
3 effective_conf = factory.get_effective_instrument_configuration("oscilloscope")
4
5 required_amp = conf["oscilloscope"]["amplitude"]
6 effective_amp = effective_conf["amplitude"]
7 print(f"Configuration: {conf}")
8 print(f"Amplitude, required: {required_amp}\tEffective: {effective_amp}\n")
Configuration: {'iteration': 1, 'oscilloscope': {'amplitude': 0.1}}
Amplitude, required: 0.1 Effective: 0.1
Configuration: {'iteration': 1, 'oscilloscope': {'amplitude': 2.575}}
Amplitude, required: 2.575 Effective: 1
Configuration: {'iteration': 1, 'oscilloscope': {'amplitude': 5.05}}
Amplitude, required: 5.05 Effective: 10
Configuration: {'iteration': 1, 'oscilloscope': {'amplitude': 7.525}}
Amplitude, required: 7.525 Effective: 10
Configuration: {'iteration': 1, 'oscilloscope': {'amplitude': 10.0}}
Amplitude, required: 10.0 Effective: 10
Configuration: {'iteration': 2, 'oscilloscope': {'amplitude': 0.1}}
Amplitude, required: 0.1 Effective: 0.1
Configuration: {'iteration': 2, 'oscilloscope': {'amplitude': 2.575}}
Amplitude, required: 2.575 Effective: 1
Configuration: {'iteration': 2, 'oscilloscope': {'amplitude': 5.05}}
Amplitude, required: 5.05 Effective: 10
Configuration: {'iteration': 2, 'oscilloscope': {'amplitude': 7.525}}
Amplitude, required: 7.525 Effective: 10
Configuration: {'iteration': 2, 'oscilloscope': {'amplitude': 10.0}}
Amplitude, required: 10.0 Effective: 10
Let us now consider the case where we set the lazy and snake parameters of the configuration:
1exp_conf = factory.experiment_config.with_params(lazy=True, snake=True)
2for conf in exp_conf:
3 print(conf)
{'iteration': 1, 'oscilloscope': {'amplitude': 0.1}}
{'oscilloscope': {'amplitude': 2.575}}
{'oscilloscope': {'amplitude': 5.05}}
{'oscilloscope': {'amplitude': 7.525}}
{'oscilloscope': {'amplitude': 10.0}}
{'iteration': 2}
{'oscilloscope': {'amplitude': 7.525}}
{'oscilloscope': {'amplitude': 5.05}}
{'oscilloscope': {'amplitude': 2.575}}
{'oscilloscope': {'amplitude': 0.1}}
The amplitude of the oscilloscope is not always configured. Thus, it is not
efficient to retrieve its configuration at each step of the loop. In this case,
we can give
get_effective_experiment_configuration()
an empty configuration that it then fills, ignoring missing fields:
1for conf in exp_conf:
2 factory.configure_experiment(conf)
3 effective_conf = factory.get_effective_experiment_configuration(conf)
4
5 print(f"Required: {conf}\nEffective: {effective_conf}\n")
Required: {'iteration': 1, 'oscilloscope': {'amplitude': 0.1}}
Effective: {'oscilloscope': {'amplitude': 0.1}}
Required: {'oscilloscope': {'amplitude': 2.575}}
Effective: {'oscilloscope': {'amplitude': 1}}
Required: {'oscilloscope': {'amplitude': 5.05}}
Effective: {'oscilloscope': {'amplitude': 10}}
Required: {'oscilloscope': {'amplitude': 7.525}}
Effective: {'oscilloscope': {'amplitude': 10}}
Required: {'oscilloscope': {'amplitude': 10.0}}
Effective: {'oscilloscope': {'amplitude': 10}}
Required: {'iteration': 2}
Effective: {}
Required: {'oscilloscope': {'amplitude': 7.525}}
Effective: {'oscilloscope': {'amplitude': 10}}
Required: {'oscilloscope': {'amplitude': 5.05}}
Effective: {'oscilloscope': {'amplitude': 10}}
Required: {'oscilloscope': {'amplitude': 2.575}}
Effective: {'oscilloscope': {'amplitude': 1}}
Required: {'oscilloscope': {'amplitude': 0.1}}
Effective: {'oscilloscope': {'amplitude': 0.1}}
Note
Not all loaders implement
get_effective_configuration(). They are
ignored by the methods which retrieve the effective configuration of an
instrument or an entire bench.
Dumping the complete state of an experiment bench#
Retrieving the configuration of an instrument is not sufficient to guarantee
that its state is the same as in another experiment. The experiment factory
provides the
dump_bench_state() method, whose
purpose is to build the most complete representation of the state of an
experiment bench.
1pprint(factory.dump_bench_state())
BenchState(version='0.5',
instruments={'simulated-motors': InstrumentState(loader='phileas-mock_motors-phileas',
interfaces=['2d-motors'],
configuration={},
id='mock-motors-driver:1',
state={'x': 0.0,
'y': 0.0}),
'simulated-oscilloscope': InstrumentState(loader='phileas-mock_oscilloscope-phileas',
interfaces=['oscilloscope'],
configuration={'motors': 'simulated-motors',
'probe': 'electric-field-probe'},
id='mock-oscilloscope-driver:1',
state={'amplitude': 0.1,
'bit_width': 8,
'fw_version': '12.3',
'probe': 'ElectricFieldProbe'})})
You can use the state as is. Alternatively, you can export it to YAML, allowing to save it or compare it to another saved state:
1saved_state = factory.dump_bench_state().to_yaml()
2print(saved_state)
version: '0.5'
instruments:
simulated-motors:
loader: phileas-mock_motors-phileas
interfaces:
- 2d-motors
configuration: {}
id: mock-motors-driver:1
state:
x: 0.0
y: 0.0
simulated-oscilloscope:
loader: phileas-mock_oscilloscope-phileas
interfaces:
- oscilloscope
configuration:
probe: electric-field-probe
motors: simulated-motors
id: mock-oscilloscope-driver:1
state:
probe: ElectricFieldProbe
amplitude: 0.1
bit_width: 8
fw_version: '12.3'
to_yaml() also takes an optional argument,
which is the path of a file where the state can be stored.
factory.dump_bench_state().to_yaml(Path("state.yaml"))
If you have a bench and an experiment configuration file, and a script which registers the required loaders, you can use the CLI to dump the state of the bench:
$ python -m phileas --bench bench.yaml --experiment experiment.yaml --script script.py
version: '0.5'
instruments:
simulated-motors:
loader: phileas-mock_motors-phileas
interfaces:
- 2d-motors
configuration: {}
id: mock-motors-driver:1
state:
simulated-oscilloscope:
loader: phileas-mock_oscilloscope-phileas
interfaces:
- oscilloscope
configuration:
probe: electric-field-probe
motors: simulated-motors
id: mock-oscilloscope-driver:1
state:
Note
You can use exported states to compare them, with diff for example.
Once you have stored a state, you can use it later to restore the bench:
1state = BenchState.from_yaml(saved_state)
2factory.restore_bench_state(state)
---------------------------------------------------------------------------
UnionMatchError Traceback (most recent call last)
Cell In[8], line 1
----> 1 state = BenchState.from_yaml(saved_state)
2 factory.restore_bench_state(state)
File ~/checkouts/readthedocs.org/user_builds/phileas/checkouts/latest/phileas/factory.py:432, in BenchState.from_yaml(cls, source)
429 raw = _yaml_load_parser.load(f)
431 config = dacite.Config(forward_references={"_NoDefault": None})
--> 432 return dacite.from_dict(data_class=BenchState, data=raw, config=config)
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:69, in from_dict(data_class, data, config)
67 if key in data:
68 try:
---> 69 value = _build_value(type_=field_type, data=data[key], config=config)
70 except DaciteFieldError as error:
71 error.update_path(field.name)
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:105, in _build_value(type_, data, config)
103 data = _build_value_for_union(union=type_, data=data, config=config)
104 elif is_generic_collection(type_):
--> 105 data = _build_value_for_collection(collection=type_, data=data, config=config)
106 elif cache(is_dataclass)(orig(type_)) and isinstance(data, Mapping):
107 data = from_dict(data_class=type_, data=data, config=config)
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:150, in _build_value_for_collection(collection, data, config)
148 if isinstance(data, Mapping) and is_subclass(collection, Mapping):
149 item_type = extract_generic(collection, defaults=(Any, Any))[1]
--> 150 return data_type((key, _build_value(type_=item_type, data=value, config=config)) for key, value in data.items())
151 elif isinstance(data, tuple) and is_subclass(collection, tuple):
152 if not data:
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:150, in <genexpr>(.0)
148 if isinstance(data, Mapping) and is_subclass(collection, Mapping):
149 item_type = extract_generic(collection, defaults=(Any, Any))[1]
--> 150 return data_type((key, _build_value(type_=item_type, data=value, config=config)) for key, value in data.items())
151 elif isinstance(data, tuple) and is_subclass(collection, tuple):
152 if not data:
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:107, in _build_value(type_, data, config)
105 data = _build_value_for_collection(collection=type_, data=data, config=config)
106 elif cache(is_dataclass)(orig(type_)) and isinstance(data, Mapping):
--> 107 data = from_dict(data_class=type_, data=data, config=config)
108 for cast_type in config.cast:
109 if is_subclass(type_, cast_type):
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:69, in from_dict(data_class, data, config)
67 if key in data:
68 try:
---> 69 value = _build_value(type_=field_type, data=data[key], config=config)
70 except DaciteFieldError as error:
71 error.update_path(field.name)
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:103, in _build_value(type_, data, config)
101 return data
102 if is_union(type_):
--> 103 data = _build_value_for_union(union=type_, data=data, config=config)
104 elif is_generic_collection(type_):
105 data = _build_value_for_collection(collection=type_, data=data, config=config)
File ~/checkouts/readthedocs.org/user_builds/phileas/envs/latest/lib/python3.13/site-packages/dacite/core.py:143, in _build_value_for_union(union, data, config)
141 if not config.check_types:
142 return data
--> 143 raise UnionMatchError(field_type=union, value=data)
UnionMatchError: can not match type "dict" to any type of "instruments.state" union: typing.Union[NoneType, bool, str, int, float, dict[bool | str | int | float, ForwardRef('DataTree')], list[ForwardRef('DataTree')]]
restore_bench_state() has an
optional argument strict which, when set, prevents restoring states whose
version, instrument names, loaders, interfaces and ids do not exactly match.
1state_with_invalid_id = BenchState.from_yaml("""
2version: '0.4'
3instruments:
4 simulated-motors:
5 loader: phileas-mock_motors-phileas
6 interfaces:
7 - 2d-motors
8 configuration: {}
9 id: mock-motors-driver:2
10 state:
11 x: 0.0
12 y: 0.0
13""")
14factory.restore_bench_state(state_with_invalid_id)
Replication of the software environment of an experiment#
Given properly implemented loaders, with the same bench and experiment configuration files, Phileas can guarantee the proper replication of an experiment. However, the user is in charge of the replication of the software environment of the experiment.
For this, different tools exist, such as conda or poetry. It is recommended
that you use them, and store their lock files alongside state.yaml.