nsrdb.preprocessing.nasa_data_model.NasaDataModel

class NasaDataModel(input_data, output_pattern)[source]

Class to handle conversion of nasa data to standard uwisc style format for NSRDB pipeline

Methods

`derive_solar_angles`(ds)	Derive solar angles if not already present in the dataset.
`derive_stdevs`(ds)	Derive standard deviations used as training features.
`drop_vars`(ds)	Drop variables that are not part of the UWISC output schema.
`fill_missing_vars`(ds)	Fill any missing variables with NaN arrays.
`get_primary_input_file`(input_files)	Get the primary input file used for naming outputs.
`get_solar_azimuth`(ds)	Derive the solar azimuth angle for the dataset.
`get_solar_zenith`(ds)	Derive the solar zenith angle for the dataset.
`open_dataset`(input_file)	Get xarray dataset for raw input file.
`parse_timestamp`(input_file)	Parse a timestamp tuple from an input file path.
`parse_timestamp_string`(input_file)	Parse the output timestamp string from an input file path.
`process_dataset`(ds)	Run the shared UWISC preprocessing pipeline on a dataset.
`remap_cloud_phase`(ds)	Map source cloud phase flags to UWISC cloud types.
`remap_dims`(ds)	Rename dims and coords to standards and build 2D lat/lon grids.
`rename_vars`(ds)	Rename variables to uwisc conventions.
`run`(input_file, output_pattern)	Run conversion routine and write converted dataset.
`transform_raw_data`(ds)	Apply any subclass-specific preprocessing to the raw dataset.
`write_output`(ds, output_file)	Write converted dataset to the final output file.

Attributes

`CLOUD_TYPE_MAP`
`CLOUD_TYPE_SOURCE_VAR`
`NAME_MAP`
`TIMESTAMP_PATTERN`
`ds`	Get xarray dataset for raw input data.
`input_files`	Get the normalized list of input files.
`output_file`	Get output file name for the configured output pattern.
`primary_input_file`	Get the primary input file used for naming outputs.
`time_index`	Get a single-step time index for the primary input file.
`timestamp`	Get the parsed timestamp tuple for the primary input file.
`timestamp_string`	Get the parsed output timestamp string for the primary input.

classmethod open_dataset(input_file)[source]: Get xarray dataset for raw input file.

classmethod run(input_file, output_pattern)[source]: Run conversion routine and write converted dataset.

derive_solar_angles(ds): Derive solar angles if not already present in the dataset.

classmethod derive_stdevs(ds): Derive standard deviations used as training features.

classmethod drop_vars(ds): Drop variables that are not part of the UWISC output schema.

classmethod fill_missing_vars(ds): Fill any missing variables with NaN arrays.

classmethod get_primary_input_file(input_files): Get the primary input file used for naming outputs.

property output_file: Get output file name for the configured output pattern.

classmethod parse_timestamp(input_file): Parse a timestamp tuple from an input file path.

classmethod parse_timestamp_string(input_file): Parse the output timestamp string from an input file path.

property primary_input_file: Get the primary input file used for naming outputs.

process_dataset(ds): Run the shared UWISC preprocessing pipeline on a dataset.

classmethod remap_cloud_phase(ds): Map source cloud phase flags to UWISC cloud types.

remap_dims(ds): Rename dims and coords to standards and build 2D lat/lon grids.

property time_index: Get a single-step time index for the primary input file.

property timestamp: Get the parsed timestamp tuple for the primary input file.

property timestamp_string: Get the parsed output timestamp string for the primary input.

classmethod transform_raw_data(ds): Apply any subclass-specific preprocessing to the raw dataset.

classmethod write_output(ds, output_file): Write converted dataset to the final output file.