Reading Experimental Data

In this section, we will be reading and exploring data generated by parameter scan or data acquisition. Check this tutorial about how to Reading experimental data in HDF5 formats.

The experimental data reading API in LISO is very similar to the EXtra-data API, which is extensively used by beamline scientists and users at European XFEL. Indeed, the design of data reading API in LISO is inspired by EXtra-data and aims to stay as close as possible to it. It is foreseen in the near future, LISO will be able to read a virtual dataset (VDS) consisting of both data from the accelerator and the user sides.

However, there are fundamental differences between DOOCS and Karabo. Every signal (data) known to DOOCS is called a channel. A channel is identified by its unique address consisting of 4 components of the form of Facility/Device/Location/Property. While in Karabo, the data are categorized into control data identified by Device/Property and instrument data identified by Device/OutputChannel. The control data is also called “slow data” due to historical reasons, while the instrument data is also known as “fast data” or “pipeline data”.

In Karabo, the bottleneck of the data pipeline is the data from bunch-resolved (pulse-resolved) area detectors like AGIPD, LPD and DSSC, which can produce up to thousands of megapixel images per second. To the best of my knowledge, there is no such kind of detectors on the accelerator side.