pyEELSMODEL.io_tools package

Submodules

pyEELSMODEL.io_tools.dm_ncempy module

THIS CODE IS FROM NCEMPY (https://openncem.readthedocs.io/en/latest/#) It is GPLv3 and MIT license

A module to load data and meta data from DM3 and DM4 files into python as written by Gatan’s Digital Micrograph program. .. note:

General users:
    Use the simplified dm.dmReader() function to load the data and meta
    data as a python dictionary.
Advanced users and developers:
    Access the file internals through the dm.fileDM() class.
On Memory mode:
  The fileDM support and "on memory" mode that pre-loads the file data in
  memory and read operations during header parsing are performed against
  memory. This can significantly improve performance when the file resides in
  a parallel file system (PFS) because latency of seek operations PFSs is very
  high.

pyEELSMODEL.io_tools.dm_ncempy.dmReader(filename, dSetNum=0, verbose=False, on_memory=True)

A simple function to parse the file and read the requested dataset. Most users will want to use this function to simplify reading data directly into memory and to retriece the spatial axes (i.e. energy axis). :param filename: The filename to open and read into memory :type filename: str :param dSetNum:

The number of the data set to read. Almost always should be = 0.
Default = 0

Parameters:

verbose (bool) – Allow extra printing to see file internals. Default = False
on_memory (bool, default True) – Whether to use the on_memory option of fileDM. Usually provides much faster data reading.

Notes

Use the coords key for spatial axes (i.e. energy loss for spectra).

Returns:

A dictionary of keys where the data is in the ‘data’ key. Other metadata is contained in other named keys such as ‘pixelSize’. ‘coords’ contains the coordinate axes with the proper origin and

scale

applied (i.e. the energy loss axis for EELS data)

Return type:

dict

Example

Load data from a single image dm3 file and display it >> from ncempy.io import dm >> im0 = dm.dmReader(‘filename.dm3’) >> plt.imshow(im0[‘data’]) #show the single image from the data file

class pyEELSMODEL.io_tools.dm_ncempy.fileDM(filename, verbose=False, on_memory=True)

Bases: object

Opens the file and reads in the header. Data is loaded using the getDataset method. .. attribute:: xSize, ySize, zSize

The shape of the data for each data set in the file. Each value is the same as the shape attribute for an ndarray.

type:

list

zSize2

The shape of the 4th dimension for a 4D file (i.e. 4D-STEM data)

Type:: list

dataSize

The total number of bytes in each dataset. Similar to numpy’s nbytes attribute for an ndarray.

Type:: list

dataOffset

The starting byte number for each dataset. This can provide fast access directly to the data if needed by seeking to this byte number.

Type:: list

dataShape

The total number of dimensions in eahc dataset. Similar to numpy’s ndims attribute for an ndarray.

Type:: list

file_name

The name of the file

Type:: str

file_path

A pathlib.Path object for the open file

Type:: pathlib.Path

fid

The file handle.

Type:: file

numObjects

The number of datasets in the file. Often (but not always) the file contains a thumbnail and the raw data. The thumbnail will always be the first object. See thumbnail attribute.

Type:: list

thumbnail

Tells whether the first object or dataset in the file is a thumbnail. If true then this object is always skipped in methods and it is assumed that the user wants to skip this. Can retrive this thumbnail using a built-in method if desired.

Type:: bool

scale

The real size of the pixel. Real and reciprical space are supported.

Type:: list

scaleUnit

The unit name as a string for each dimension of each dataset.

Type:: list

origin

The origin for the real or reciprocal space scaling for each dimension. Be careful, this value is actually meant to be scaled by the scale before being used. See ncempy.io.dmReader() for proper handling of this especially for spectroscopy data.

Type:: list

allTags

Contains all tags in the DM file as key value pairs.

Type:: dictionary

Examples

Read data from a file containing a single image into memory >> from ncempy.io import dm >> with dm.fileDM(‘filename.dm4’) as dmFile1: >> dataSet = dmFile1.getDataset(0) Example of reading a full multi-image DM3 file into memory: >> with dm.fileDM(‘imageSeries.dm3’)as dmFile2: >> series = dmFile2.getDataset(0)

allTags

dataOffset

dataShape

dataSize

dataType

fid

fileSize

file_name

file_path

fromfile(*args, **kwargs)

Reads data from a file or memory map. Calls np.fromfile and np.frombuffer depending on the on_memory mode of the fileDM. .. note:

This is essentially a passthrough function to Numpy's frombuffer
and fromfile depending on the class variable on_memory.

Parameters:

*args – dtype and count are required
**kwargs

Returns:

Data read from the file as a 1d ndarray.

Return type:

ndarray

getDataset(index)

Retrieve a dataset from the DM file. .. rubric:: Notes

Most DM3 and DM4 files contain a small “thumbnail” as the first dataset written as RGB data. This function ignores that dataset if it exists. To retrieve the thumbnail use the getThumbnail() function. The pixelOrigin returned is not actually the start of the coordinates. The start of the energy axis for EELS (for example) will be pixelSize * pixelOrigin. dmReader() returns the correct coordinates. The correct origin is: pixelSize * pixelOrigin and be careful about the sign as it seems some datasets might use -pixelOrigin in the previous equation.

Parameters:

index (int) –

The number of the data set to retrieve ignoring the thumbnail.: If a thumbnail exists then index = 0

actually corresponds to the second data set in a DM file.

Returns:

A dictionary of the data and meta data. The data is associated with the ‘data’ key in the dictionary.

Return type:

dict

getMemmap(index)

Return a numpy memmap object (read-only) for the dataset requested.: This is very useful

for very large datasets to avoid loading the entire data set into memory. No meta data is returned. :param index: The number of the dataset in the DM file. :type index: int

Returns:: A read-only numpy memmap object with access to the data. The file will continue to be open as long as the memmap is open. Delete the memmap to close the file.
Return type:: numpy.memmap

getSlice(index, sliceZ, sliceZ2=0)

Retrieve a slice of a dataset from the DM file. The data set will have a shape according to 3D = [sliceZ,Y,X] or 4D: [sliceZ2,sliceZ,Y,X] Note: Most DM3 and DM4 files contain a small “thumbnail” as the first dataset written as RGB data. This function ignores that dataset if it exists. To retrieve the thumbnail use the getThumbnail() function. Warning: DM4 files with 4D data sets are written as [X,Y,Z1,Z2]. This code currently gets the [X,Y] slice. Getting the [Z1,Z2] slice is not yet implemented. Use the getMemmap() function to retrieve arbitrary slices of large data sets. :param index: The number of the dataset in the DM file. :type index: int :param sliceZ: The slice to get along the first dimension (C-ordering)

for 3D datasets or 4D datasets.

Parameters:: sliceZ2 (int) – For 4D dataset
Returns:: A dictionary containing meta data and the data.
Return type:: dict

getThumbnail()

Read the thumbnail saved as the first dataset in the DM file as an: RGB array.

This is not fully tested. Be careful using this. :returns: Numpy array of size [3,Y,X] which is an RGB thumbnail. :rtype: ndarray

numObjects

on_memory

origin

parseHeader(): Parse the header by reading the root tag group. This ensures the file pointer is in the correct place.

scale

scaleOrigin

scaleUnit

seek(fid, offset, from_what=0)

Positions the reading head for fid. fid can be a file or memory map. Follows the same convention as file.seek :param fid: File or memory map. :type fid: file id :param offset: Number of bytes to move the head forward (positive value)

or backwards (negative value).

Parameters:: from_what (int) – Reference point to use in the head movement. 0: for beginning of the file (default behavior), 1: from the current head position, and 2: from the end of the file.

tell(): Return the current position in the file. Switches mode based on on_memory mode. :returns: pos – The current position in the file. :rtype: int

thumbnail

verbose

writeTags(new_folder_path_for_tags=None)

Write out all tags as human readable text to a text file in the same directory (or a user definable directory) and with a the

same name as the DM file.

Parameters:: new_folder_path_for_tags (str or pathlib.Path, Optional) – Allow user to define a different path than the directory of the current file.

xSize

ySize

zSize

zSize2

pyEELSMODEL.io_tools.hdf5_io module

pyEELSMODEL.io_tools.hdf5_io.load_elements_and_edges(filename)

pyEELSMODEL.io_tools.hdf5_io.load_h5py(filename)

pyEELSMODEL.io_tools.hdf5_io.load_hspy(filename)

pyEELSMODEL.io_tools package

Submodules

pyEELSMODEL.io_tools.dm_ncempy module

pyEELSMODEL.io_tools.hdf5_io module

Module contents