Data objects#

DataManager#

This dataclass is used to modify metadata stored in Datasource objects and the metadata store. DataManager instances are created by the data_manager function.

class openghg.dataobjects.DataManager(metadata, store)[source]#

__init__(metadata, store)[source]#

__str__()[source]#

Return str(self).

Return type:: str

delete_datasource(uuid)[source]#

Delete a Datasource in the object store. At the moment we only support deleting the complete Datasource.

NOTE: Make sure you really want to delete the Datasource(s)

Parameters:: uuid (Union[List, str]) – UUID(s) of objects to delete
Return type:: None
Returns:: None

refresh()[source]#

Force refresh the internal metadata store with data from the object store.

Return type:: None
Returns:: None

restore(uuid, version='latest')[source]#

Restore a backed-up version of a Datasource’s metadata.

Parameters:

uuid (str) – UUID of Datasource to retrieve
version (Union[str, int]) – Version of metadata to restore

Return type:

None

Returns:

None

update_metadata(uuid, to_update=None, to_delete=None)[source]#

Update the metadata associated with data. This takes UUIDs of Datasources and updates the associated metadata. If you want to delete some metadata

Parameters:

uuid (Union[List, str]) – UUID(s) of Datasources to be updated.
to_update (Optional[Dict]) – Dictionary of metadata to add/update. New key/value pairs will be added.
updated. (If the key already exists in the metadata the value will be) –
to_delete (Union[str, List, None]) – Key(s) to delete from the metadata

Return type:

None

Returns:

None

view_backup(uuid=None, version=None)[source]#

View backed-up metadata for all Datasources or a single Datasource if a UUID is passed in.

Parameters:: uuid (Optional[str]) – UUID of Datasource
Returns:: Dictionary of versioned metadata
Return type:: dict

SearchResults#

This dataclass is returned by the OpenGHG search functions and allows easy retrieval and querying of metadata retrieved by the search function.

class openghg.dataobjects.SearchResults(keys=None, metadata=None, start_result=None)[source]#

This class is used to return data from the search function. It has member functions to retrieve data from the object store.

Parameters:

keys (Optional[Dict]) – Dictionary of keys keyed by Datasource UUID
metadata (Optional[Dict]) – Dictionary of metadata keyed by Datasource UUID
start_result (Optional[str]) –
?

__init__(keys=None, metadata=None, start_result=None)[source]#

__repr__()[source]#

Return repr(self).

Return type:: str

__str__()[source]#

Return str(self).

Return type:: str

static df_to_table_console_output(df)[source]#

Process the DataFrame and display it as a formatted table in the console.

Parameters:: df (DataFrame) – The DataFrame to be processed and displayed.
Return type:: None
Returns:: None

classmethod from_json(data)[source]#

Create a SearchResults object from a dictionary

Parameters:: data (Union[bytes, str]) – Serialised object
Returns:: SearchResults object
Return type:: SearchResults

retrieve(dataframe=None, sort=True, elevate_inlet=False, **kwargs)[source]#

Retrieve data from object store using a filtered pandas DataFrame

Parameters:

dataframe (Optional[DataFrame]) – pandas DataFrame
sort (bool) – Sort data by date in retrieved Dataset
elevate_inlet (bool) – Elevate inlet to a variable within the Dataset, useful
data. (for ranked) –

Returns:

ObsData object(s)

Return type:

ObsData / List[ObsData]

retrieve_all(sort=False, elevate_inlet=False)[source]#

Retrieves all data found during the search

Parameters:

sort (bool) – Sort by time. Note that this may be very memory hungry for large Datasets.
elevate_inlet (bool) – Elevate inlet to a variable within the Dataset, useful
data. (for ranked) –

Returns:

ObsData object(s)

Return type:

ObsData / List[ObsData]

to_data()[source]#

Convert this object to a dictionary for JSON serialisation

Returns:: Dictionary of data
Return type:: dict

to_json()[source]#

Serialises the object to JSON

Returns:: JSON str
Return type:: str

uuids()[source]#

Return the UUIDs of the found data

Returns:: List of UUIDs
Return type:: list

ObsData#

This dataclass is returned by data retrieval functions such as get_obs_surface and the SearchResults retrieve function.

class openghg.dataobjects.ObsData(data, metadata)[source]#

This class is used to return observations data from the get_observations function

Parameters:

data (Dataset) – Dictionary of xarray Dataframes
metadata (Dict) – Dictionary of metadata

__delattr__(name)#: Implement delattr(self, name).

__eq__(other)[source]#

Return self==value.

Return type:: bool

__getitem__(key)[source]#

Returns the data attribute (xarray Dataset) when the site name is specified. Included as a compatability layer for legacy format as a dictionary containing a Dataset for each site code.

key (str): Site code

Return type:: Any

__hash__()#: Return hash(self).

__init__(data, metadata)#

__iter__()[source]#

Returns site code as the key for the dictionary as would be expected.

Return type:: Iterator

__len__()[source]#

Returns number of key values (fixed at 1 at present)

Return type:: int

__repr__()#: Return repr(self).

__setattr__(name, value)#: Implement setattr(self, name, value).

plot_timeseries(title=None, xlabel=None, ylabel=None, units=None, logo=True)[source]#

Plot a timeseries

Return type:: Figure

to_data()[source]#

Creates a dictionary package of this ObsData’s metadata and data.

Returns:: Dictionary of metadata and bytes
Return type:: dict

FluxData#

This dataclass is used to return observations data from the get_flux function

class openghg.dataobjects.FluxData(data, metadata)[source]#

This class is used to return flux/emissions data from the get_flux function

Parameters:

data (Dataset) – xarray Dataframe
metadata (Dict) – Dictionary of metadata including model run parameters

__delattr__(name)#: Implement delattr(self, name).

__eq__(other)#: Return self==value.

__hash__()#: Return hash(self).

__init__(data, metadata)#

__repr__()#: Return repr(self).

__setattr__(name, value)#: Implement setattr(self, name, value).

__str__()[source]#

Return str(self).

Return type:: str

ObsColumnData#

This dataclass is used to return observations data from the get_obs_column function

class openghg.dataobjects.ObsColumnData(data, metadata)[source]#

This class is used to return observations data from the get_obs_column function

Parameters:

data (Dataset) – xarray Dataset
metadata (Dict) – Dictionary of metadata including model run parameters

__delattr__(name)#: Implement delattr(self, name).

__eq__(other)#: Return self==value.

__hash__()#: Return hash(self).

__init__(data, metadata)#

__repr__()#: Return repr(self).

__setattr__(name, value)#: Implement setattr(self, name, value).

__str__()[source]#

Return str(self).

Return type:: str

FootprintData#

This dataclass is used to return observations data from the get_footprint function

class openghg.dataobjects.FootprintData(data, metadata)[source]#

This class is used to return observations data from the get_footprint function

Parameters:

data (Dataset) – xarray Dataset
metadata (Dict) – Dictionary of metadata including model run parameters

__delattr__(name)#: Implement delattr(self, name).

__eq__(other)#: Return self==value.

__hash__()#: Return hash(self).

__init__(data, metadata)#

__repr__()#: Return repr(self).

__setattr__(name, value)#: Implement setattr(self, name, value).

__str__()[source]#

Return str(self).

Return type:: str

BoundaryConditionsData#

This dataclass is used to return observations data from the get_bc function

class openghg.dataobjects.BoundaryConditionsData(data, metadata)[source]#

This class is used to return boundary conditions data from the get_bc function

Parameters:

data (Dataset) – xarray Dataframe
metadata (Dict) – Dictionary of metadata including model run parameters

__delattr__(name)#: Implement delattr(self, name).

__eq__(other)#: Return self==value.

__hash__()#: Return hash(self).

__init__(data, metadata)#

__repr__()#: Return repr(self).

__setattr__(name, value)#: Implement setattr(self, name, value).

__str__()[source]#

Return str(self).

Return type:: str