JSONLabNotebook: A persistent store in JSON format

Note

This style of notebook is fine for storing small datasets, and those that need to be accessed in a very portable manner, but is very wasteful for large datasets, for which an HDF5LabNotebook is almost certainly a better choice.

class epyc.JSONLabNotebook(name: str, create: bool = False, description: str = None)

A lab notebook that persists intself to a JSON file. This is the most basic kind of persistent notebook, readable by virtually any tooling.

Using JSON presents some disadvantages, as not all types can be represented. Specifically, exceptions from the metadata of failed experiments (with Experiment.EXCEPTION) will be saved as strings. We also need to convert datetime objects to ISO-format strings when saving.

Parameters:
  • name – JSON file to persist the notebook to
  • create – if True, erase existing file (defaults to False)
  • description – free text description of the notebook

Persistence

JSON notebooks are persistent, with the data being saved into a file identified by the notebook’s name. Committing the notebook forces a save.

JSONLabNotebook.isPersistent() → bool

Return True to indicate the notebook is persisted to a JSON file.

Returns:True
JSONLabNotebook.commit()

Persist to disc.

JSON file access

If you want to make sure that the file is closed and commited after use you can use code such as:

with JSONLabNotebook(name='test.json', create=True).open() as nb:
    nb.addResult(rc1)
    nb.addResult(rc2)

After this the notebook’s underlying file will be closed, with the new results having been saved.

Structure of the JSON file

The version 1 file format is flat and stores all results in a single block. This has been replaced by the version 2 format that has a structure the follows the structure of the result sets in the notebook.

Important

epyc can still read version 1 JSON notebooks, but will only save in the version 2 format.

The top level JSON object consists of elements holding the notebook title and some housekeeping attributes. There is also a nested dict holding result sets, keyed by their tag.

Each result set object contains elements for its description and any attributes. There are also two further nested JSON objects, one holding results and one holding pending results. Each result is simply a results dict rendered in JSON; each pending result is a job identifier mapped to the parameters controlling the pending result.