Documentation

This section of the documentation gives an overview of the public facing functions used in the Jupyter notebooks available at the OpenGHG hub. For developers documentation of the internal workings of the library are available in the developer API section.

modules

These modules are used to process observation data. Classes in this module should not be used directly as they are used by functions in the Client or LocalClient modules when either uploading data to the OpenGHG cloud platform or processing data for storage in a local object store.

ObsSurface: Process surface observation data

client

Classes within the client module are used to interact with the cloud based OpenGHG system.

Process: Upload and process observation data files
Search: Search for data in the object store
RankSources: Rank data sources by date range
JobRunner: Run jobs on a local or cloud HPC cluster

jobs

Classes within this module are used for running simulation jobs on high performace computing (HPC) clusters either locally or within the cloud based on a cluster as a service (CaaS) offering (see CitC).

SSHConnect: Connect via SSH to a HPC cluster
JobDrive: Create a cloud storage drive for use by the HPC job

localclient

For use with a local version of the object store. These functions make it easy to take advantage of the processing and export capabilities of OpenGHG on your local filesystem. Use of this module results in the creation of a local object store in a location controlled by the OPENGHG_PATH environment variable.

get_obs_surface(): Search for surface observations in the object store using a site, species and date range.
process_files(): Process files for storage in the object store
RankSources: Rank data sources by date range
Search: Search for data within the object store (soon to be deprecated in favour of get_obs

objectstore

Many of the functions in this submodule are only for internal use and will be renamed.

get_bucket(): Get a bucket (data container) for storing of data within the object store

processing

This submodule contains functions that are widely used in the processing functions found in modules.

assign_attributes(): Assign attributes to a dictionary of observation data in NetCDF format using get_attributes
get_attributes(): Write attributes to an in-memory NetCDF file to ensure it is CF-compliant
get_ceda_file(): Create a file that contains the correct attributes for uploading to the CEDA archive
recombine_datasets(): Recombine separate NetCDF files into a single file sorted by date
search(): The function that is used by openghg.client.Search and openghg.localclient.Search to search the object store
assign_data(): Assings data to exisiting Datasources or creates new Datasources