cfelpyutils package¶

Submodules¶

cfelpyutils.cfel_crystfel module¶

Utilities for interoperability with the CrystFEL software package.

This module contains reimplementation of Crystfel functions and utilities.

cfelpyutils.cfel_crystfel.load_crystfel_geometry(filename)¶

Loads a CrystFEL geometry file into a dictionary.

Reimplements the get_detector_geometry_2 function from CrystFEL amost verbatim. Returns a dictionary with the geometry information. Entries in the geometry file appears as keys in the returned dictionary. For a full documentation on the CrystFEL geometry format, see:

tfel/manual-crystfel_geometry.html

Parameters:	filename (str) – filename of the geometry file
Returns:	dictionary with the geometry loaded from the file
Return type:	detector (dict)

cfelpyutils.cfel_cxi module¶

Utilities for writing multi-event files in the CXIDB format.

This module contains utilities to write files that adhere to the CXIDB file format:

http://www.cxidb.org/cxi.html .

class cfelpyutils.cfel_cxi.CXIWriter(filename, max_num_slices=5000)¶

Bases: object

Writing of multi-event CXIDB files.

Implements a simple low-level CXIDB file format writer for multi event files. it allows the user to write data “stacks” in the CXIDB files, making sure that the entries in all stacks are synchronized.

A CXI Writer instance manages one file. A user can add a stack to a CXI Writer instance with the add_stack_to_writer function, which also writes the first entry in the stack. The user can then add to the writer all the stacks that he wants in the file. Once all stacks are added, the user initializes them with the initialize_stacks function. After initialization, no more stacks can be added. Instead, entries can be appended to the existing stacks, using the append_data_to_stack function.

A “slice” (a set of synced entries in all the stacks in the file) can be written to the a file only after an entry has been appended to all stacks in the file. Conversely, after an entry has been appended to a stack, the user cannot append another entry before a slice is written. This ensures synchronization of the data in all the stacks.

A file can be closed at any time. In any case, the writer will not allow a file to contain more than the number_of_entries specified during instantiation.

Simple non-stack entries can be written to the file at any time, before or after stack initialization (provided of course that the file is open). Entries and stacks will general never be overwritten unless the overwrite parameter is set to True.

Example of usage of the stack API:

c1 = 0 c2 = 0

f1 = CXIWriter(‘test1.h5’, ) f2 = CXIWriter(‘test2.h5’, )

f1.add_stack_to_writer(‘detector1’, ‘/entry_1/detector_1/data’, numpy.random.rand(2, 2),: ‘frame:y:x’)
f2.add_stack_to_writer(‘detector2’, ‘/entry_1/detector_1/data’, numpy.random.rand(3, 2),: ‘frame:y:x’, compression=False, chunk_size=(1,3,2))

f1.add_stack_to_writer(‘counter1’, ‘/entry_1/detector_1/count’, c1) f2.add_stack_to_writer(‘counter2’, ‘/entry_1/detector_1/count’, c2)

f1.write_simple_entry(‘/entry_1/detector_1/name’, ‘FrontCSPAD’) f2.write_simple_entry(‘/entry_1/detector_1/name’, ‘BackCSPAD’)

f1.initialize_stacks() f2.initialize_stacks()

a = numpy.random.rand(2, 2) b = numpy.random.rand(3, 2)

c1 += 1 c2 += 2

f1.append_data_to_stack(‘detector1’, a) f2.append_data_to_stack(‘detector2’, b)

f1.append_data_to_stack(‘counter1’, c1) f2.append_data_to_stack(‘counter2’, c2)

f1.write_stack_slice_and_increment() f2.write_stack_slice_and_increment()

f1.close_file() f2.close_file()

add_stack_to_writer(name, path, initial_data, axes=None, compression=True, chunk_size=None, overwrite=True)¶

Adds a new stack to the file.

Adds a new stack to the CXI Writer instance. The user must provide a name for the stack, that will identify the stack in all subsequents operations. The user must also provide the data that will be written as the initial entry in the stack (initial_data). This initial entry is used to set the size and type of data that the stack will hold and these parameters are in turn be used to validate all data that is subsequently appended to the stack.

Parameters:

name (str) – stack name.
path (str) – path in the hdf5 file where the stack will be written.
(Union[numpy.ndarray, str, int, float] (initial_data) – initial entry in the stack. It gets written to the
as slice 0. Its characteristics are used to validate all data subsequently appended to the stack. (stack) –
axes (str) – the ‘axes’ attribute for the stack, as defined by the CXIDB file format.
compression (Union[None, bool,str]) – compression parameter for the stack. This parameters works in the same
as the normal compression parameter from h5py. The default value of this parameter is True. (way) –
chunk_size (Union[None, tuple]) – HDF5 chuck size for the stack. If this parameter is set to None, the
writer will compute a chuck size automatically (CXI) –
use the provided tuple to set the chunk size. (will) –
overwrite (bool) – if set to True, a stack already existing at the same location will be overwritten. If set
False, an attempt to overwrite a stack will raise an error. (to) –

append_data_to_stack(name, data)¶

Appends data to a stack.

Appends data to a stack, validating the data to make sure that the data type and size match the previous entries in the stack. Only one entry can be appended to each stack before writing a slice across all stacks with the write_slice_and_increment.

Parameters:	name (str) – stack name, defining the stack to which the data will be appended. (Union (data) – numpy.ndarray, str, int, float): data to write. The data will be validated against the type size of previous entries in the stack. (and) –

close_file()¶

Closes the file.

Closes the file for writing, ending all writing operations.

get_file_handle()¶

Access to the naked h5py file handle.

This function allows access to the a naked h5py handle for the file managed by the CXI Writer. This allowa operations on the file that are not covered by CXI Writer API. Use it at your own risk.

Returns:	an h5py file handle to the file managed by the writer.
Return type:	fh (h5py.File)

initialize_stacks()¶

Initializes the stacks.

Initializes the stacks in the CXI Writer instance. This fixes the number and type of stacks in the file. No stacks can be added to the CXI Writer after initialization.

write_simple_entry(path, data, overwrite=False)¶

Writes a simple, non-stack entry in the file.

Writes a simple, non-stack entry in the file, at the specified path. A simple entry can be written at all times, before or after the stack initialization.

Parameters:	path (str) – path in the hdf5 file where the entry will be written. (Union (data) – numpy.ndarray, str, int, float): data to write overwrite (bool) – if set to True, an entry already existing at the same location will be overwritten. If set False, an attempt to overwrite an entry will raise an error. (to) –

write_stack_slice_and_increment()¶

Writes a slice across all stacks and resets the writer for the next slice.

Writes a slice across all stacks in the file. It checks that an entry has been appended to each stack, and writes all the entries on top of the relevant stacks in one go. If an entry is missing in a stack, the function will raise an error. After writing the slice, the function resets the writer to allow again appending data to the stacks.

cfelpyutils.cfel_fabio module¶

Utilities based on the fabio python module.

This module contains utilities based on the fabio python module. files.

cfelpyutils.cfel_fabio.read_cbf_from_stream(stream)¶

Reads a cbfimage object out of a data string buffer.

Read a data string buffer received as a payload from the PETRAIII P11 sender, and creates a cbfimage object from it (See the documentation of the fabio python module).

Parameters:	stream (str) – a data string buffer received from the PETRAIII P11 sender.
Returns:	a cbfimage object containing the data extracted from the string buffer.
Return type:	cbf_obj (fabio.cbfimage)

cfelpyutils.cfel_geom module¶

Utilities for CrystFEL-style geometry files.

This module contains utilities for the processing of CrystFEL-style geometry files.

cfelpyutils.cfel_geom.apply_geometry_from_file(data_as_slab, geometry_filename)¶

Parses a geometry file and applies the geometry to data.

Parses a geometry file and applies the geometry to detector data in ‘slab’ format. Turns a 2d array of pixel values into an array containing a representation of the physical layout of the detector, keeping the origin of the reference system at the beam interaction point.

Parameters:	data_as_slab (numpy.ndarray) – the pixel values to which geometry is to be applied. geometry_filename (str) – geometry filename.
Returns:	Array containing a representation of the physical layout of the detector, with the origin of the reference system at the beam interaction point.
Return type:	im_out (numpy.ndarray data_as_slab.dtype)

cfelpyutils.cfel_geom.apply_geometry_from_pixel_maps(data_as_slab, yx, im_out=None)¶

Applies geometry in pixel map format to data.

Applies geometry, in the form of pixel maps, to detector data in ‘slab’ format. Turns a 2d array of pixel values into an array containing a representation of the physical layout of the detector, keeping the origin of the reference system at the beam interaction point.

Parameters:	data_as_slab (numpy.ndarray) – the pixel values to which geometry is to be applied. yx (tuple) – the yx pixel maps describing the geometry of the detector; each map is a numpy.ndarray. im_out (Optional[numpy.ndarray]) – array to hold the output; if not provided, one will be generated automatically. –
Returns:	Array containing a representation of the physical layout of the detector, with the origin of the reference system at the beam interaction point.
Return type:	im_out (numpy.ndarray data_as_slab.dtype)

cfelpyutils.cfel_geom.pixel_maps_for_image_view(geometry_filename)¶

Parses a geometry file and creates pixel maps for pyqtgraph visualization.

Parse the geometry file and creates pixel maps for an array in ‘slab’ format containing pixel values. The pixel maps can be used to create a representation of the physical layout of the detector in a pyqtgraph ImageView widget (i.e. they apply the detector geometry setting the origin of the reference system is in the top left corner of the output array).

Parameters: geometry_filename (str) – geometry filename.

Returns:

pixel maps

slab_shape tuple (int, int): shape of the original geometry uncorrected array (the pixel values in “slab” format).

img_shape tuple (int, int): shape of the array needed to contain the representation of the physical layout of the detector.

Return type: (y, x) (numpy.ndarray int, numpy.ndarray int)

cfelpyutils.cfel_geom.pixel_maps_from_geometry_file(fnam)¶

Parses a geometry file and creates pixel maps.

Extracts pixel maps from a CrystFEL-style geometry file. The pixel maps can be used to create a representation of the physical layout of the detector, keeping the origin of the reference system at the beam interaction point.

Parameters:	fnam (str) – geometry filename.
Returns:	slab-like pixel maps with respectively x, y coordinates of the pixel and distance of the pixel from the center of the reference system.
Return type:	x,y,r (numpy.ndarray float, numpy.ndarray float, numpy.ndarray float)

cfelpyutils.cfel_hdf5 module¶

Utilities for HDF5 files.

This module contains utilities for the processing of HDF5. This module builds on what the h5py module already provides.

cfelpyutils.cfel_hdf5.load_nparray_from_hdf5_file(data_filename, data_group)¶

Loads a numpy.ndarray from an HDF5 file.

Parameters:	data_filename (str) – filename of the file to read. data_group (str) – internal HDF5 path of the data block to read.
Returns:	numpy array with the data read from the file.
Return type:	nparray (numpy.ndarray)

cfelpyutils.cfel_optarg module¶

Utilities for parsing command line options and configuration files.

This module contains utilities for parsing of command line options and configuration files.

cfelpyutils.cfel_optarg.parse_parameters(config)¶

Sets correct types for parameter dictionaries.

Reads a parameter dictionary returned by the ConfigParser python module, and assigns correct types to parameters, without changing the structure of the dictionary.

The parser tries to interpret each entry in the dictionary according to the following rules:

If the entry starts and ends with a single quote or double quote, it is interpreted as a string.
If the entry starts and ends with a square bracket, it is interpreted as a list.
If the entry starts and ends with a brace, it is interpreted as a dictionary.
If the entry is the word None, without quotes, then the entry is interpreted as NoneType.
If the entry is the word False, without quotes, then the entry is interpreted as a boolean False.
If the entry is the word True, without quotes, then the entry is interpreted as a boolean True.
If none of the previous options match the content of the entry, the parser tries to interpret the entry in order as:
- An integer number.
- A float number.
- A string.
The first choice that succeeds determines the entry type.

Parameters:	config (class RawConfigParser) – ConfigParser instance.
Returns:	dictionary with the same structure as the input dictionary, but with correct types assigned to each entry.
Return type:	monitor_params (dict)

cfelpyutils.cfel_psana module¶

Utilities based on the psana python module.

This module provides utilities that build on the functionality provided by the psana python module.

cfelpyutils.cfel_psana.dirname_from_source_runs(source)¶

Returns a directory name based on a psana source string.

Takes a psana source string (e.g exp=CXI/cxix....) and returns a string that can be used as a subdirectory name or a prefix for files and directories.

Parameters:	source (str) – a psana source string (e.g. exp=CXI/cxi....).
Returns:	a string that can be used as a filename or a prefix .
Return type:	dirname (str)

cfelpyutils.cfel_psana.psana_event_inspection(source)¶

Prints the structure of psana events.

Takes a psana source string (e.g. exp=CXI/cxix....) and inspects the structure of the first event in the data, printing information about the the content of the event.

Parameters:	source (str) – a psana source string (e.g. exp=CXI/cxix....).

cfelpyutils.cfel_psana.psana_obj_from_string(name)¶

Converts a string into a psana object type.

Takes a string and returns the python object type described by the string.

Parameters:	name (str) – a string describing a python type.
Returns:	the python type described by the string.
Return type:	mod (type)

Module contents¶

Utilities for CFEL software projects.

This module provides utilities (functions, classes, etc.) used by many CFEL projects.

It contains the following submodules (non of which are automatically imported):

cfelfabio: utilities based on the fabio python module (a module that allows python to work with several file format used in x-ray imaging).

cfelgeom: utilities for processing CrystFEL-style geometry files.

cfelhdf5: utilities for processing HDF5 files (based on the h5py python module).

cfeloptarg: utilities to parse command line arguments and parameter files.

cfelpsana: utilities that expand the functionality of the psana python module.

Table Of Contents

This Page