Simple example#

This pages uses a simple experiment to illustrate some of the basic features of Phileas. It shows how to measure the values of a field at different spatial positions.

Experimental setup and drivers#

This experiment is based on two instruments:

  • a probe is used to measure the amplitude of the electric field. It is connected to an oscilloscope.

  • A motorized stage is used to position the probe.

We want to plot different field maps, so for this we need to automate the positioning of the probe and the measurement of the field. Two drivers are used, allowing to control

  • the maximum amplitude of the oscilloscope. It must be high enough to avoid saturation, but low enough to prevent quantization noise. Different amplitudes are tried to solve this issue.

  • The motorized stage has thgrees of freedom, denoted \(X\), \(Y\) and \(Z\), which also computer-controlled.

This example uses simulated instruments, availabe in the phileas.mock_instruments module. They can be used as follows:

1motors = mock_instruments.Motors()
2probe = mock_instruments.ElectricFieldProbe(motors)
3oscilloscope = mock_instruments.Oscilloscope(probe)
4oscilloscope.amplitude = 1
5
6print(f"Field value with {motors}: {oscilloscope.get_measurement()}")
7motors.set_position(x=1, y=1.5)
8print(f"Field value with {motors}: {oscilloscope.get_measurement()}")
INFO:phileas.phileas.mock_instruments:[Motors] Connection initiated.
INFO:phileas.phileas.mock_instruments:[Oscilloscope] Connection initiated.
Field value with Motors(x=0.0, y=0.0, id='mock-motors-driver:1'): 0.5
Field value with Motors(x=1, y=1.5, id='mock-motors-driver:1'): 0.0234375
1probe.get_amplitude()
np.float64(0.024688715247306627)

If we break this down,

  1. the drivers are instanciated,

  2. the instruments are configured,

  3. and finally, measurements are gathered.

We will now see how to use Phileas to automate these measurements for more positions.

Describing the experimental bench#

The first step to drive the instruments is to describe what what instruments are available on the bench, and how to connect to them. This gives Phileas all the information required to instanciate the drivers. This is done by the bench configuration file, which is a simple YAML file. In our case, as the instruments are simulated, we can define it like this:

1bench = """
2simulated-motors:
3    loader: phileas-mock_motors-phileas
4
5simulated-oscilloscope:
6    loader: phileas-mock_oscilloscope-phileas
7    probe: electric-field-probe
8    motors: simulated-motors
9"""

This bench configuration states that two instruments, simulated-motors and simulated-oscilloscope are available. Then, it tells the simulated oscilloscope what probe is used, and how it is driven. Notice the loader attribute : it is a special field, which is required by Phileas for instantiating the drivers. It is explained in the following section.

See also

Bench configuration file further describes the syntax of this file.

Warning

Here, the content of the bench configuration file is stored in a Python str, as it integrates easily with notebooks. Usually, it is preferred to store it in a file, conventionally called bench.yaml.

Describing the instrument parameters#

The instruments available on the bench can now be configured. This is the purpose of the experiment configuration file. It is also a YAML file, which describes

  • what instruments are required by the experiment,

  • how to configure them,

  • and is usually used to document the experiment itself, working as a lab notebook.

Let us define it as

 1experiment = """
 2description: Measure the electric field for a single probe position.
 3
 4motors:
 5    interface: 2d-motors
 6    x: 1
 7    y: 1.5
 8
 9oscilloscope:
10    interface: oscilloscope
11    amplitude: 1
12"""

It contains two entries motors and oscilloscope, which are identified as instrument entries by their interface key. It is a requirement, which states that instrument motors must be a 3d-motors. Phileas uses this internally to match the required instrument with the available ones. Besides the interface reserved keyword, each instrument entry describe its required configuration.

Finally, this file can also contain arbitrary data, which is convenient to document the experimental process. Here, the description entry simply states the purpose of the experiment, but it could be much more detailed.

See also

Experiment configuration file further describes the syntax of this file.

Warning

Here, the content of the experiment configuration file is stored in a Python str, as it integrates easily with notebooks. Usually, it is preferred to store it in a file, conventionally called experiment.yaml.

Letting Phileas handle instruments with loaders#

The bench configuration file describes the available instruments of a bench, whereas the experiment configuration file requires some of them. Internally, Phileas relies on loaders to match the requirements to the available instruments.

A loader is a simple Python class, which

  • is identified by a name, which is used in the bench configuration file. In this example, loaders phileas-mock_motors-phileas and phileas-mock_oscilloscope-phileas are used.

  • Additionally, it has can support multiple interfaces, which are requested by the experiment configuration file. In this example, interfaces 2d-motors and oscilloscope are used.

Here, loaders MotorsLoader and OscilloscopeLoaders are internally used. Phileas then uses them to

  • instanciate the corresponding drivers

  • and automatically configure the instruments, which is covered later in this example.

See also

Implementing loaders covers the details necessary for designing your own loaders.

Putting it all together in a script#

For now, we have declared the instruments available on our simulated bench, and expressed what instruments are actually required by the experiment, and how to configure them. We can then actually use Phileas to handle the instruments initialization, which is done by an ExperimentFactory:

1factory = phileas.ExperimentFactory(bench, experiment)
INFO:phileas:Bench configuration supplied as a data tree.
INFO:phileas:Bench instrument simulated-motors assigned to loader <class 'phileas.mock_instruments.MotorsLoader'>.
INFO:phileas:Bench instrument simulated-oscilloscope assigned to loader <class 'phileas.mock_instruments.OscilloscopeLoader'>.
INFO:phileas:Experiment configuration supplied as an iteration tree.
INFO:phileas:Matching experiment instrument motors with bench instrument simulated-motors.
INFO:phileas:Matching experiment instrument oscilloscope with bench instrument simulated-oscilloscope.

Warning

Here, bench and experiment are str. If you store the configurations in dedicated files, simply supply them as pathlib.Path objects, such as

factory = phileas.ExperimentFactory(Path("bench.yaml"), Path("experiment.yaml"))

Phileas logs some of its internal operations. Here, we use them to monitor them. Not how motors - the name of the instrument in the experiment configuration - is matched with simulated-motors - the name of the instrument on the bench. Similarly, oscilloscope is matched with simulated-oscilloscope.

Note

This separation in two levels allows to ease the replication of an experiment. The useful information of an experiment, and what it exactly does, is in the experiment configuration file, which should be portable. On the other hand, the bench configuration file contains all the non-portable information which is specific to a given bench. To replicate an experiment, only the bench configuration file has to be adapted.

Using the bench configuration file, and the loaders it requests, the drivers can then be instantiated. They are then accessible in factory.experiment_instruments.

1factory.initiate_connections()
2motors: mock_instruments.Motors = factory.experiment_instruments["motors"]
3oscilloscope: mock_instruments.Oscilloscope = factory.experiment_instruments["oscilloscope"]
INFO:phileas:Initiating connections to the used instruments.
INFO:phileas.simulated-motors:Initiating connection.
INFO:phileas.phileas.mock_instruments:[Motors] Connection initiated.
INFO:phileas.simulated-oscilloscope:Initiating connection.
INFO:phileas.phileas.mock_instruments:[Oscilloscope] Connection initiated.
INFO:phileas:Connections initiation is done.

Note

Notice the oscilloscope: mock_instruments.Oscilloscope type hint. It is not required, yet it allows to document the type of oscilloscope, which couldn’t be statically determined otherwise. It is recommended to add type hints if you want to benefit from the guarantees of a static type checker like mypy or pyright.

Finally, the instruments can be configured, and the measurements gathered:

1factory.configure_experiment()
2print(f"Field value with {motors}: {oscilloscope.get_measurement()}")
INFO:phileas.simulated-motors:Position set to {'x': 1, 'y': 1.5}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
Field value with Motors(x=1, y=1.5, id='mock-motors-driver:1'): 0.02734375

As the logs indicate, the motors and oscilloscope are properly configured to the values specified in the experiment config file. This results in the same measurement as in the initial example, with manual instruments configuration.

Mapping the electric field value#

Let us now go further, and record the field values for multiple probe positions. Let us say that we want to regularly map the region with \(0 \ge X, Y \ge 1\), with \(3\) different position along each axis. This can be done directly in the experiment configuration file:

 1experiment = """
 2description: Measure the electric field in the region 0 <= X, Y <= 1.
 3
 4motors:
 5    interface: 2d-motors
 6    x: !range
 7        start: 0
 8        end: 1
 9        steps: 3
10    y: !range
11        start: 0
12        end: 1
13        steps: 3
14
15oscilloscope:
16    interface: oscilloscope
17    amplitude: 1
18"""

Note how the x andy parameters of motors have been replaced by a !range object. This is an example of a custom YAML type, that is used by Phileas to represent multiple configurations with a single configuration file.

Let us introduce two types. First, a data tree represent the data that can be stored in a usual YAML file. In other words, it is built with list and dict objects, eventually storing numeric, boolean, string or null values. A data tree thus represents a single configuration to be applied to an instrument, or even the configuration of a whole experimental bench.

Now, Phileas introduces another similar structure : the iteration tree. Put simply, an iteration tree represents a collection of data trees, with the iteration order over it. Experiment configuration files are actually parsed to iteration trees, so that they can represent all the configurations of a (potentially complex) experiment.

The experiment iteration tree of a factory is available in its experiment_config attribute, which can be iterated over like a usual Python collection. This allows us to slightly modify the code of the last section to apply multiple configurations to the instruments.

1factory = phileas.ExperimentFactory(bench, experiment)
2factory.initiate_connections()
3motors: mock_instruments.Motors = factory.experiment_instruments["motors"]
4oscilloscope: mock_instruments.Oscilloscope = factory.experiment_instruments["oscilloscope"]
5
6for config in factory.experiment_config:
7    factory.configure_experiment(config)
INFO:phileas:Bench configuration supplied as a data tree.
INFO:phileas:Bench instrument simulated-motors assigned to loader <class 'phileas.mock_instruments.MotorsLoader'>.
INFO:phileas:Bench instrument simulated-oscilloscope assigned to loader <class 'phileas.mock_instruments.OscilloscopeLoader'>.
INFO:phileas:Experiment configuration supplied as an iteration tree.
INFO:phileas:Matching experiment instrument motors with bench instrument simulated-motors.
INFO:phileas:Matching experiment instrument oscilloscope with bench instrument simulated-oscilloscope.
INFO:phileas:Initiating connections to the used instruments.
INFO:phileas.simulated-motors:Initiating connection.
INFO:phileas.phileas.mock_instruments:[Motors] Connection initiated.
INFO:phileas.simulated-oscilloscope:Initiating connection.
INFO:phileas.phileas.mock_instruments:[Oscilloscope] Connection initiated.
INFO:phileas:Connections initiation is done.
INFO:phileas.simulated-motors:Position set to {'x': 0.0, 'y': 0.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 0.0, 'y': 0.5}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 0.0, 'y': 1.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 0.5, 'y': 0.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 0.5, 'y': 0.5}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 0.5, 'y': 1.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 1.0, 'y': 0.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 1.0, 'y': 0.5}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'x': 1.0, 'y': 1.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.

Yet, we notice that the oscilloscope amplitude is repeatedly configured, without changing. Similarly, the x position only changes every 3 iterations. The experiment configuration can be modified to perform lazy iteration: only modified values are returned at each iteration.

1lazy_configuration = (factory.experiment_config
2    .with_params(["motors"], lazy=True)
3    .with_params(["oscilloscope"], lazy=True)
4)
5
6for config in lazy_configuration:
7    factory.configure_experiment(config)
INFO:phileas.simulated-motors:Position set to {'x': 0.0, 'y': 0.0}.
INFO:phileas.simulated-oscilloscope:Amplitude set to 1.
INFO:phileas.simulated-motors:Position set to {'y': 0.5}.
INFO:phileas.simulated-motors:Position set to {'y': 1.0}.
INFO:phileas.simulated-motors:Position set to {'x': 0.5, 'y': 0.0}.
INFO:phileas.simulated-motors:Position set to {'y': 0.5}.
INFO:phileas.simulated-motors:Position set to {'y': 1.0}.
INFO:phileas.simulated-motors:Position set to {'x': 1.0, 'y': 0.0}.
INFO:phileas.simulated-motors:Position set to {'y': 0.5}.
INFO:phileas.simulated-motors:Position set to {'y': 1.0}.

with_params() modifies a parameter of an entry of an iteration tree (here, one of its nodes), and returns it.

See also

Iteration trees extensively covers the capabilities offered by iteration trees.

Configuring logging#

Up to now, we have used the logs of Phileas in order to peek at its internal operations. Let us clean the outputs by keeping only warnings.

1phileas.logger.setLevel(logging.WARNING)

Storing results with xarray#

Let us now see how to store the results of the measurements, so that we can proceed to the data analysis stage of the experiment.

Different Python data containers can be used, yet our example is perfect for xarray ones. You can think of them as multi-dimensional arrays, where the dimensions are annotated to make the container meaningfull.

See also

This page does not present xarray. See this page for terminology on its containers.

Phileas allows to simply prepare an xarray container for data storage.

 1coords, dims_name, dims_shape = iteration_tree_to_xarray_parameters(
 2    factory.experiment_config
 3)
 4results = xr.Dataset(
 5    data_vars=dict(
 6        electric_field=(dims_name, np.full(dims_shape, np.nan)),
 7    ),
 8    coords=coords
 9)
10results
<xarray.Dataset> Size: 352B
Dimensions:                 (motors.x: 3, motors.y: 3)
Coordinates:
    description             <U56 224B 'Measure the electric field in the regi...
  * motors.x                (motors.x) float64 24B 0.0 0.5 1.0
  * motors.y                (motors.y) float64 24B 0.0 0.5 1.0
    oscilloscope.amplitude  int64 8B 1
Data variables:
    electric_field          (motors.x, motors.y) float64 72B nan nan ... nan nan

The results dataset contains a single variable, which is the measurement of the electric field. See how most of the information of the experiment configuration file is kept in the dataset, including both varying parameters such as motors.x and fixed values such as description.

The acquisition loop can then be modified to incorporate measurements storage.

1for config in factory.experiment_config:
2    flat_config = {
3        coord: v for coord, v in flatten_datatree(config).items() if coord in dims_name
4    }
5    factory.configure_experiment(config)
6    results.electric_field.loc[flat_config] = oscilloscope.get_measurement()

The flatten_datatree() utility is used to converted the hierarchical structure of a data tree to a flat structure which can be used for indexing the xarray storage.

1flat_config
{'motors.x': 1.0, 'motors.y': 1.0}

Once the current experiment index is built, the results can be modified using their loc modifier. Finally, results contains the values of the electric field at the different requested positions.

1results.electric_field
<xarray.DataArray 'electric_field' (motors.x: 3, motors.y: 3)> Size: 72B
array([[0.5      , 0.5      , 0.5      ],
       [0.5      , 0.5      , 0.5      ],
       [0.0546875, 0.4765625, 0.1875   ]])
Coordinates:
    description             <U56 224B 'Measure the electric field in the regi...
  * motors.x                (motors.x) float64 24B 0.0 0.5 1.0
  * motors.y                (motors.y) float64 24B 0.0 0.5 1.0
    oscilloscope.amplitude  int64 8B 1

See also

Storing results covers additional ways of storing results.

Getting further in the parameters generation#

We have seen how to acquire simple datasets, but the \(3 \times 3\) grid used for measurements up to now might not have a resolution suitable for data analysis. Let us increase the number of measurements points, which will allow us to plot the value of the electric field in space.

 1configs = (factory.experiment_config
 2    .with_params(["motors", "x"], start=0, end=1, steps=20)
 3    .with_params(["motors", "y"], start=0, end=1, steps=20)
 4
 5)
 6coords, dims_name, dims_shape = iteration_tree_to_xarray_parameters(configs)
 7results = xr.Dataset(
 8    data_vars=dict(
 9        electric_field=(dims_name, np.full(dims_shape, np.nan)),
10    ),
11    coords=coords
12)
13results
14
15for config in configs:
16    flat_config = {
17        coord: v for coord, v in flatten_datatree(config).items() if coord in dims_name
18    }
19    factory.configure_experiment(config)
20    results.electric_field.loc[flat_config] = oscilloscope.get_measurement()
21
22results
<xarray.Dataset> Size: 4kB
Dimensions:                 (motors.x: 20, motors.y: 20)
Coordinates:
    description             <U56 224B 'Measure the electric field in the regi...
  * motors.x                (motors.x) float64 160B 0.0 0.05263 ... 0.9474 1.0
  * motors.y                (motors.y) float64 160B 0.0 0.05263 ... 0.9474 1.0
    oscilloscope.amplitude  int64 8B 1
Data variables:
    electric_field          (motors.x, motors.y) float64 3kB 0.5 0.5 ... 0.1914

Notice how the size of the dimensions has increased to 20, as requested. However, the acquisition code remained the same. We can now plot the measurements:

1results.electric_field.plot(x="motors.x", y="motors.y")
<matplotlib.collections.QuadMesh at 0x7dd65605dfd0>
../_images/2d592507f48025a4c29c61a4a39240d4c424c9290b9292edb740cd102b4b2cca.png

This figure exhibits some saturation: most of the measured values are fixed at \(0.5\). Let us have a look at the required configurations, using to_pseudo_data_tree(), which converts an iteration tree in a more readable format.

See also

This user guide section presents pseudo data trees more extensively.

1import pprint
2pprint.pprint(configs.to_pseudo_data_tree())
{'description': 'Measure the electric field in the region 0 <= X, Y <= 1.',
 'motors': {'x': LinearRange(start=0,
                             end=1,
                             default_value=_NoDefault(),
                             steps=20),
            'y': LinearRange(start=0,
                             end=1,
                             default_value=_NoDefault(),
                             steps=20)},
 'oscilloscope': {'amplitude': 1}}

The amplitude of the oscilloscope is configured to 1, and it does not have an offset, so this means that the maximum value it can measure is 0.5. This explains why we obtained saturated measurements. Let us vary the amplitude of the oscilloscope in order to find a suitable value. Here, we use replace_node() to replace the type of a node in the iteration tree. This allows us to change the fixed amplitude value of the oscilloscope with a geometrically distributed sequence.

 1configs = (configs
 2    .replace_node(["oscilloscope", "amplitude"], GeometricRange, start=1, end=10000, steps=3)
 3)
 4coords, dims_name, dims_shape = iteration_tree_to_xarray_parameters(configs)
 5results = xr.Dataset(
 6    data_vars=dict(
 7        electric_field=(dims_name, np.full(dims_shape, np.nan)),
 8    ),
 9    coords=coords
10)
11
12for config in configs:
13    flat_config = {
14        coord: v for coord, v in flatten_datatree(config).items() if coord in dims_name
15    }
16    factory.configure_experiment(config)
17    results.electric_field.loc[flat_config] = oscilloscope.get_measurement()
18
19results
<xarray.Dataset> Size: 10kB
Dimensions:                 (motors.x: 20, motors.y: 20,
                             oscilloscope.amplitude: 3)
Coordinates:
    description             <U56 224B 'Measure the electric field in the regi...
  * motors.x                (motors.x) float64 160B 0.0 0.05263 ... 0.9474 1.0
  * motors.y                (motors.y) float64 160B 0.0 0.05263 ... 0.9474 1.0
  * oscilloscope.amplitude  (oscilloscope.amplitude) float64 24B 1.0 100.0 1e+04
Data variables:
    electric_field          (motors.x, motors.y, oscilloscope.amplitude) float64 10kB ...

The resulting data can finally be plotted to check what has just been acquired:

Hide code cell source

 1# Prepare an image grid
 2fig = plt.figure(figsize=(18, 7))
 3grid = ImageGrid(
 4    fig,
 5    111,
 6    (1, len(results["oscilloscope.amplitude"])),
 7    axes_pad=0.2,
 8    cbar_location="bottom",
 9    cbar_mode="each",
10    cbar_pad=None,
11    aspect=False,
12    cbar_size="3%",
13)
14
15for (lab, data), ax, cax in zip(
16    results.electric_field.groupby("oscilloscope.amplitude"), grid, grid.cbar_axes
17):
18    contours = ax.contour(data.squeeze(), norm=LogNorm(), colors="w")
19    ax.clabel(contours, inline=1, fontsize=10)
20    im = ax.imshow(data.squeeze(), interpolation="nearest", norm=LogNorm())
21    cax.colorbar(im)
22    ax.set_xticks([])
23    ax.set_yticks([])
24    ax.set_title(f"Amplitude {data['oscilloscope.amplitude'].item():.0f}")
../_images/6c1be5f9c57633fe6ad8f6af60992604b22ff7b51873669b2535a26d4e1f49fc.png

Using these measurements, the experimenter can now properly configure the amplitude of his oscilloscope, in order to measure the phenomenon of interest.

Conclusion and getting further#

This example illustrates how Phileas work, and how you can prepare a simple experiment with it. Notably, it splits an experiment in three parts:

  • the bench configuration file describes the instruments that are available on an experimental setup, and is not portable;

  • the experiment configuration file describes what an experiment does, and is usually what the experimenter works with;

  • the acquisition script handles everything else, and notably the measurements and the storage of their results.

The experiment configuration file and the acquisition script are meant to be portable. This architecture reduces the efforts required to replicate an experiment and share it with others.

Now that you have seen at a glance how to use Phileas, you can dive into more advanced examples.

You can go to the user guide to have detailed information about specific features of Phileas.