HDF5 and NEXUS

EIGER data is stored in HDF5 (Hierarchical Data Format) files. HDF is designed to store and organize large amounts of numerical data.  It is ideally suited for the demands of high performance detectors, large research facilities and users who need to handle vast amounts of data.

EIGER detectors acquire up to 180,000 frames and about 360 gigabyte of raw data per minute. Any file format or strategy that aims at storing only one image per file is therefore not suitable for EIGER. One of the main benefits of HDF5 is providing a transparent and efficient way of storing and handling a large number of images in a small number of files. Furthermore, HDF5 offers versatile means of storing experiment metadata in the same file structure as the detector data. Many analytical packages such as MATLAB, IDL, and R support HDF5. These and numerous other advantages make HDF5 the preferred file format for many synchrotrons and large research facilities.
 

The NeXus data format uses HDF5 as a container and provides design principles, definitions, and data structures for storing of scientific data. All EIGER detector data and metadata is stored in NeXus classes in the HDF5 files written by the detector software. Beamline scientists and synchrotron users benefit from an expandable set of NeXus application definitions. Using NeXus definitions such as NXmx or NXtomo, all metadata describing a macromolecular crystallography or tomography experiment can be stored in the same files structure as the detector data.