Data objects

DataManager

This dataclass is used to modify metadata stored in Datasource objects and the metadata store. DataManager instances are created by the data_manager function.

class openghg.dataobjects.DataManager(metadata, store)

__init__(metadata, store)

__str__()

Return str(self).

Return type:

str

delete_datasource(uuid)

Delete Datasource(s) in the object store. At the moment we only support deleting the complete Datasource.

NOTE: Make sure you really want to delete the Datasource(s)

Parameters:

uuid (list | str) – UUID(s) of objects to delete

Return type:

None

Returns:

None

refresh()

Force refresh the internal metadata store with data from the object store.

Return type:

None

Returns:

None

restore(uuid, version='latest')

Restore a backed-up version of a Datasource’s metadata.

Parameters:

• uuid (str) – UUID of Datasource to retrieve

• version (str | int) – Version of metadata to restore

Return type:

None

Returns:

None

update_attributes(uuid, version='latest', data_vars=None, update_global=True, to_update=None, to_delete=None)

Update the attributes of the stored Dataset.

This takes UUIDs of Datasources (and optionally a version tag) and updates the associated attributes: - to update attributes pass in a dictionary of key/value pairs to update. - to delete attributes pass in a list of keys to delete.

Parameters:

• uuid (list | str) – UUID(s) of Datasources to be updated.

• version (str | list[str]) – optional version string

• data_vars (str | list[str] | None) – optional list of data vars to update; if None, then only global attributes will be updated.

• update_global (bool) – if True, update global attributes.

• to_update (dict | None) – Dictionary of metadata to add/update. New key/value pairs will be added.

• updated. (If the key already exists in the metadata the value will be)

• to_delete (str | list | None) – Key(s) to delete from the metadata

Return type:

None

Returns:

None

update_metadata(uuid, to_update=None, to_delete=None)

Update the metadata associated with data.

This takes UUIDs of Datasources and updates the associated metadata. To update metadata pass in a dictionary of key/value pairs to update. To delete metadata pass in a list of keys to delete.

Parameters:

• uuid (list | str) – UUID(s) of Datasources to be updated.

• to_update (dict | None) – Dictionary of metadata to add/update. New key/value pairs will be added.

• updated. (If the key already exists in the metadata the value will be)

• to_delete (str | list | None) – Key(s) to delete from the metadata

Return type:

None

Returns:

None

view_backup(uuid=None, version=None)

View backed-up metadata for all Datasources or a single Datasource if a UUID is passed in.

Parameters:

uuid (str | None) – UUID of Datasource

Returns:

Dictionary of versioned metadata

Return type:

dict

SearchResults

This dataclass is returned by the OpenGHG search functions and allows easy retrieval and querying of metadata retrieved by the search function.

class openghg.dataobjects.SearchResults(metadata=None, start_result=None, start_date=None, end_date=None)

This class is used to return data from the search function. It has member functions to retrieve data from the object store.

Parameters:

• keys – Dictionary of keys keyed by Datasource UUID

• metadata (dict | None) – Dictionary of metadata keyed by Datasource UUID

• start_result (str | None) –

  ?

__init__(metadata=None, start_result=None, start_date=None, end_date=None)

__repr__()

Return repr(self).

Return type:

str

__str__()

Return str(self).

Return type:

str

static df_to_table_console_output(df)

Process the DataFrame and display it as a formatted table in the console.

Parameters:

df (DataFrame) – The DataFrame to be processed and displayed.

Return type:

None

Returns:

None

retrieve(dataframe=None, version='latest', sort=True, **kwargs)

Retrieve data from object store using a filtered pandas DataFrame

Parameters:

• dataframe (DataFrame | None) – pandas DataFrame

• version (str) – Version of data requested from Datasource. Default = “latest”.

• sort (bool) – Sort data by time in retrieved Dataset

• **kwargs (Any) – Metadata values to search for

Returns:

ObsData object(s)

Return type:

ObsData / List[ObsData]

retrieve_all(version='latest', sort=True)

Retrieves all data found during the search

Parameters:

• version (str) – Version of data requested from Datasource. Default = “latest”.

• sort (bool) – Sort by time. Note that this may be very memory hungry for large Datasets.

Returns:

ObsData object(s)

Return type:

ObsData / List[ObsData]

uuids()

Return the UUIDs of the found data

Returns:

List of UUIDs

Return type:

list

ObsData

This dataclass is returned by data retrieval functions such as get_obs_surface and the SearchResults retrieve function.

class openghg.dataobjects.ObsData(metadata, data=None, uuid=None, version=None, start_date=None, end_date=None, sort=True, elevate_inlet=False, attrs_to_check=None)

This class is used to return observations data. It be created with a preloaded xarray Dataset or with a UUID and version number to retrieve data from Datasource zarr store.

__eq__(other)

Return self==value.

Return type:

bool

__getitem__(key)

Returns the data attribute (xarray Dataset) when the site name is specified. Included as a compatability layer for legacy format as a dictionary containing a Dataset for each site code.

key (str): Site code

Return type:

Any

__hash__ = None

__iter__()

Returns site code as the key for the dictionary as would be expected.

Return type:

Iterator

__len__()

Returns number of key values (fixed at 1 at present)

Return type:

int

plot_timeseries(title=None, xlabel=None, ylabel=None, units=None, logo=True)

Plot a timeseries

Return type:

Figure

FluxData

This dataclass is used to return observations data from the get_flux function

class openghg.dataobjects.FluxData(metadata, data=None, uuid=None, version=None, start_date=None, end_date=None, sort=True, elevate_inlet=False, attrs_to_check=None)

This class is used to return flux/emissions data from the get_flux function

Parameters:

• data (Dataset | None) – xarray Dataframe

• metadata (dict) – Dictionary of metadata including model run parameters

__str__()

Return str(self).

Return type:

str

ObsColumnData

This dataclass is used to return observations data from the get_obs_column function

class openghg.dataobjects.ObsColumnData(metadata, data=None, uuid=None, version=None, start_date=None, end_date=None, sort=True, elevate_inlet=False, attrs_to_check=None)

This class is used to return observations data from the get_obs_column function

Parameters:

• data (Dataset | None) – xarray Dataset

• metadata (dict) – Dictionary of metadata including model run parameters

__str__()

Return str(self).

Return type:

str

plot_timeseries(title=None, xlabel=None, ylabel=None, units=None, logo=True)

Plot a timeseries

Return type:

Figure

FootprintData

This dataclass is used to return observations data from the get_footprint function

class openghg.dataobjects.FootprintData(metadata, data=None, uuid=None, version=None, start_date=None, end_date=None, sort=True, elevate_inlet=False, attrs_to_check=None)

This class is used to return observations data from the get_footprint function

Parameters:

• data (Dataset | None) – xarray Dataset

• metadata (dict) – Dictionary of metadata including model run parameters

__str__()

Return str(self).

Return type:

str

BoundaryConditionsData

This dataclass is used to return observations data from the get_bc function

class openghg.dataobjects.BoundaryConditionsData(metadata, data=None, uuid=None, version=None, start_date=None, end_date=None, sort=True, elevate_inlet=False, attrs_to_check=None)

This class is used to return boundary conditions data from the get_bc function

Parameters:

• data (Dataset | None) – xarray Dataframe

• metadata (dict) – Dictionary of metadata including model run parameters

__str__()

Return str(self).

Return type:

str