mlclouds.data_handlers.ValidationData

class ValidationData(val_files, val_sites='all', features=['solar_zenith_angle', 'cloud_type', 'refl_0_65um_nom', 'refl_0_65um_nom_stddev_3x3', 'refl_3_75um_nom', 'temp_3_75um_nom', 'temp_11_0um_nom', 'temp_11_0um_nom_stddev_3x3', 'cloud_probability', 'cloud_fraction', 'air_temperature', 'dew_point', 'relative_humidity', 'total_precipitable_water', 'surface_albedo'], y_labels=['cld_opd_dcomp', 'cld_reff_dcomp'], all_sky_vars=('alpha', 'aod', 'asymmetry', 'cloud_type', 'cld_opd_dcomp', 'cld_reff_dcomp', 'ozone', 'solar_zenith_angle', 'ssa', 'surface_albedo', 'surface_pressure', 'total_precipitable_water'), one_hot_cats=None, predict_clearsky=True, test_set_mask=None)[source]

Bases: object

Load and prep validation data

Parameters:

  • val_files (str | list of str) – List of files to use for validation

  • val_sites (‘all’ | list of int) – Surfrad gids to use for validation. Use all if ‘all’

  • features (list of str) – Names of model input fields

  • y_labels (list of str) – Names of model output fields

  • all_sky_vars (list of str) – Names of fields used for the allsky algorithm

  • one_hot_cats (dict | None) – Categories for one hot encoding. Keys are column names, values are lists of category values. See phygnn.utlities.pre_processing.

  • predict_clearsky (bool) – Let phygnn predict properties for clear and cloudy time steps if true, else, only predict properties for cloudy time steps.

  • test_set_mask (None | numpy.ndarray of bool) – Set of full data set in val_files to use. If None, use full dataset.

Methods