nsrdb.preprocessing.base_data_model.BaseUwiscDataModel

class BaseUwiscDataModel(input_data, output_pattern)

Bases: object

Shared preprocessing pipeline for UWISC-format conversion.

Methods

derive_solar_angles(ds)	Derive solar angles if not already present in the dataset.
derive_stdevs(ds)	Derive standard deviations used as training features.
drop_vars(ds)	Drop variables that are not part of the UWISC output schema.
fill_missing_vars(ds)	Fill any missing variables with NaN arrays.
get_primary_input_file(input_files)	Get the primary input file used for naming outputs.
get_solar_azimuth(ds)	Derive the solar azimuth angle for the dataset.
get_solar_zenith(ds)	Derive the solar zenith angle for the dataset.
open_dataset(input_data)	Open the raw input data as an xarray dataset.
parse_timestamp(input_file)	Parse a timestamp tuple from an input file path.
parse_timestamp_string(input_file)	Parse the output timestamp string from an input file path.
process_dataset(ds)	Run the shared UWISC preprocessing pipeline on a dataset.
remap_cloud_phase(ds)	Map source cloud phase flags to UWISC cloud types.
remap_dims(ds)	Rename dims and coords to standards and build 2D lat/lon grids.
rename_vars(ds)	Rename variables to uwisc conventions.
run(input_data, output_pattern)	Run the conversion routine and write the converted dataset.
transform_raw_data(ds)	Apply any subclass-specific preprocessing to the raw dataset.
write_output(ds, output_file)	Write converted dataset to the final output file.

Attributes

CLOUD_TYPE_MAP
CLOUD_TYPE_SOURCE_VAR
NAME_MAP
TIMESTAMP_PATTERN
ds	Get xarray dataset for raw input data.
input_files	Get the normalized list of input files.
output_file	Get output file name for the configured output pattern.
primary_input_file	Get the primary input file used for naming outputs.
time_index	Get a single-step time index for the primary input file.
timestamp	Get the parsed timestamp tuple for the primary input file.
timestamp_string	Get the parsed output timestamp string for the primary input.

property input_files

Get the normalized list of input files.

classmethod get_primary_input_file(input_files)

Get the primary input file used for naming outputs.

property primary_input_file

Get the primary input file used for naming outputs.

classmethod parse_timestamp(input_file)

Parse a timestamp tuple from an input file path.

classmethod parse_timestamp_string(input_file)

Parse the output timestamp string from an input file path.

property timestamp

Get the parsed timestamp tuple for the primary input file.

property timestamp_string

Get the parsed output timestamp string for the primary input.

property time_index

Get a single-step time index for the primary input file.

property output_file

Get output file name for the configured output pattern.

classmethod open_dataset(input_data)

Open the raw input data as an xarray dataset.

property ds

Get xarray dataset for raw input data.

classmethod transform_raw_data(ds)

Apply any subclass-specific preprocessing to the raw dataset.

get_solar_zenith(ds)

Derive the solar zenith angle for the dataset.

get_solar_azimuth(ds)

Derive the solar azimuth angle for the dataset.

classmethod rename_vars(ds)

Rename variables to uwisc conventions.

classmethod drop_vars(ds)

Drop variables that are not part of the UWISC output schema.

remap_dims(ds)

Rename dims and coords to standards and build 2D lat/lon grids.

classmethod fill_missing_vars(ds)

Fill any missing variables with NaN arrays.

derive_solar_angles(ds)

Derive solar angles if not already present in the dataset.

classmethod remap_cloud_phase(ds)

Map source cloud phase flags to UWISC cloud types.

classmethod derive_stdevs(ds)

Derive standard deviations used as training features.

process_dataset(ds)

Run the shared UWISC preprocessing pipeline on a dataset.

classmethod write_output(ds, output_file)

Write converted dataset to the final output file.

classmethod run(input_data, output_pattern)

Run the conversion routine and write the converted dataset.