data_event_handlers_files#

Handling of file-based data events.

This module contains Data Event Handler classes that manipulate file-based events.

class OmBaseFileDataEventHandlerMixin(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

designated_collector_rank()#

Return type:: Literal[‘first’, ‘last’]

skip_rank_finalization()#

Return type:: bool

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)#

Initializes Pilatus single-frame file event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Pilatus file-based event handling on the collecting node, so this function actually does nothing.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

None

class PilatusFilesEventHandler(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)#

Parameters:

node_rank (int)
node_pool_size (int)

Return type:

None

event_generator(*, node_rank, node_pool_size)#

Retrieves Pilatus single-frame file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to the content of an individual single-frame CBF file. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node will process fewer events than the others.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

Generator[dict[str, Any], None, None]

extract_data(*, event)#

Extracts data from a Pilatus single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

event (dict[str, Any]) – A dictionary storing the event data.

Returns:

A dictionary storing the extracted data.

Each dictionary key identifies a Data Source in the event for which data has been retrieved.

The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

OmDataExtractionError – Raised when data cannot be extracted from the event.

Return type:

dict[str, Any]

initialize_event_data_retrieval()#

Initializes data data retrieval from Pilatus single-frame files.

This function initializes the retrieval of single standalone data events from Pilatus single-frame data files.

Please see the documentation of the base Protocol class for additional information about this method.

Return type:: None

retrieve_event_data(event_id)#

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Pilatus single-frame file event identifier corresponds to the relative or absolute path to a file containing the event data.

Parameters:: event_id (str) – A string that uniquely identifies a data event.
Returns:: All data related to the requested event.
Return type:: dict[str, Any]

class Jungfrau1MFilesDataEventHandler(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)#

Initializes Pilatus single-frame file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

None

event_generator(*, node_rank, node_pool_size)#

Retrieves Jungfrau 1M file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single single detector data frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

Generator[dict[str, Any], None, None]

extract_data(*, event)#

Extracts data from a Jungfrau 1M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

event (dict[str, Any]) – A dictionary storing the event data.

Returns:

A dictionary storing the extracted data.

Each dictionary key identifies a Data Source in the event for which data has been retrieved.

The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

OmDataExtractionError – Raised when data cannot be extracted from the event.

Return type:

dict[str, Any]

initialize_event_data_retrieval()#

Initializes event data retrieval from Jungfrau 1M HDF5 files.

This function initializes the retrieval of single standalone data events from Jungfrau 1M HDF5 files.

Please see the documentation of the base Protocol class for additional information about this method.

Return type:: None

retrieve_event_data(event_id)#

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Jungfrau 1M unique event identifier is a string consisting of the absolute or relative path to the master HDF5 file attached to the event, and the index of the event within the file, separated by the ‘//’ symbol.

Parameters:: event_id (str) – A string that uniquely identifies a data event.
Returns:: All data related to the requested event.
Return type:: dict[str, Any]

class EigerFilesDataEventHandler(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)#

Initializes Pilatus single-frame file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

None

event_generator(*, node_rank, node_pool_size)#

Retrieves Eiger file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single detector data frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last node processes fewer events than the others.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

Generator[dict[str, Any], None, None]

extract_data(*, event)#

Extracts data from an Eiger file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

event (dict[str, Any]) – A dictionary storing the event data.

Returns:

A dictionary storing the extracted data.

Each dictionary key identifies a Data Source in the event for which data has been retrieved.

The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

OmDataExtractionError – Raised when data cannot be extracted from the event.

Return type:

dict[str, Any]

initialize_event_data_retrieval()#

Initializes event data retrieval from Eiger files.

This function initializes the retrieval of single standalone data events from Eiger files.

Please see the documentation of the base Protocol class for additional information about this method.

Return type:: None

retrieve_event_data(event_id)#

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. An Eiger event identifier is a string consisting of the absolute or relative path to the HDF5 data file attached to the event, and the index of the event within the file, separated by ‘//’ symbol.

Parameters:: event_id (str) – A string that uniquely identifies a data event.
Returns:: All data related to the requested event.
Return type:: dict[str, Any]

class RayonixMccdFilesEventHandler(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)#

Initializes Pilatus single-frame file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

None

event_generator(*, node_rank, node_pool_size)#

Retrieves Rayonix MX340-HS single-frame file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to the content of an individual single-frame mccd file. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

Generator[dict[str, Any], None, None]

extract_data(*, event)#

Extracts data from a Rayonix MX340-HS single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

event (dict[str, Any]) – A dictionary storing the event data.

Returns:

A dictionary storing the extracted data.

Each dictionary key identifies a Data Source in the event for which data has been retrieved.

The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

OmDataExtractionError – Raised when data cannot be extracted from the event.

Return type:

dict[str, Any]

initialize_event_data_retrieval()#

Initializes event data retrievals from Rayonix MX340-HS single-frame files.

This function initializes the retrieval of single standalone data events from Rayonix MX340-HS single-frame files.

Please see the documentation of the base Protocol class for additional information about this method.

Return type:: None

retrieve_event_data(event_id)#

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Rayonix MX340-HS event identifier is the full absolute or relative path to the mccd file associated with the event.

Parameters:: event_id (str) – A string that uniquely identifies a data event.
Returns:: All data related to the requested detector event.
Return type:: dict[str, Any]

class Lambda1M5FilesDataEventHandler(*, source, parameters)#

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.
The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

source (str) – A string describing the data event source.
data_sources –
A dictionary containing a set of Data Source class instances.
- Each dictionary key must define the name of a data source.
- The corresponding dictionary value must store the instance of the [Data Source class][om.protocols.data_retrieval_layer.OmDataSourceProtocol] # noqa: E501 that describes the source.
monitor_parameters – An object storing OM’s configuration parameters.
parameters (DataRetrievalLayerParameters)

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)#

Initializes Pilatus single-frame file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

None

event_generator(*, node_rank, node_pool_size)#

Retrieves Lambda 1.5M file events.

This method overrides the corresponding method of the base class: please also refer to the documentation of that class for more information.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single detector frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

node_rank (int) – The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.
node_pool_size (int) – The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

Return type:

Generator[dict[str, Any], None, None]

extract_data(*, event)#

Extracts data from a Lambda 1.5M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

event (dict[str, Any]) – A dictionary storing the event data.

Returns:

A dictionary storing the extracted data.

Each dictionary key identifies a Data Source in the event for which data has been retrieved.

The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

OmDataExtractionError – Raised when data cannot be extracted from the event.

Return type:

dict[str, Any]

initialize_event_data_retrieval()#

Initializes event data retrievals from Lambda 1.5M HDF5 files.

This function initializes the retrieval of single standalone events from Lambda 1.5M HDF5 files.

Please see the documentation of the base Protocol class for additional information about this method.

Return type:: None

retrieve_event_data(event_id)#

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Lambda 1.5M unique event identifier is a string consisting of two parts. The first part is the absolute or relative path to the HDF5 file storing the first panel of the detector data frame attached to the event (“m01.nxs”), while the second part is the index of the event within the file. The two parts are separated by the ‘//’ symbol.

Parameters:: event_id (str) – A string that uniquely identifies a data event.
Returns:: All data related to the requested event.
Return type:: dict[str, Any]