mlclouds.validator.Validator

class Validator(model, val_sites='all', val_files=None, config={'epochs_a': 100, 'epochs_b': 90, 'features': ['solar_zenith_angle', 'cloud_type', 'refl_0_65um_nom', 'refl_0_65um_nom_stddev_3x3', 'refl_3_75um_nom', 'temp_3_75um_nom', 'temp_11_0um_nom', 'temp_11_0um_nom_stddev_3x3', 'cloud_probability', 'cloud_fraction', 'air_temperature', 'dew_point', 'relative_humidity', 'total_precipitable_water', 'surface_albedo'], 'hidden_layers': [{'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}], 'learning_rate': 0.0005, 'loss_weights_a': [1, 0], 'loss_weights_b': [0.5, 0.5], 'metric': 'relative_mae', 'n_batch': 64, 'one_hot_categories': {'flag': ['clear', 'ice_cloud', 'water_cloud', 'bad_cloud']}, 'p_fun': 'p_fun_all_sky', 'p_kwargs': {'loss_terms': ['mae_ghi']}, 'phygnn_seed': 0, 'surfrad_window_minutes': 15, 'training_prep_kwargs': {'add_cloud_flag': True, 'filter_clear': False, 'filter_daylight': True, 'filter_sky_class': False, 'nan_option': 'interp', 'sza_lim': 89}, 'y_labels': ['cld_opd_dcomp', 'cld_reff_dcomp']}, val_data=None, update_clear=False, update_cloudy=False, test_set_mask=None, save_timeseries=False)[source]

Bases: object

Run PhygnnModel predictions, run allsky using predicted cloud properties and compare to NSRDB baseline irradiance.

Parameters:

  • model (Phygnn instance) – Trained or loaded Phygnn model

  • val_sites (‘all’ | list of int) – Surfrad gids to use for validation. Use all if ‘all’

  • val_files (str | list of str | None) – File or list of file to use for validation from config file. Filenames must include the four-digit year and east/west text to indicate satellite. Must be None if val_data is set.

  • config (dict) – Phygnn configuration dict

  • val_data (None | ValidationData instance) – Use preloaded validation data or load from val_files if None. Must be none if val_files is set.

  • update_clear (bool) – If true, update cloud type for clear time steps with phygnn predictions

  • update_cloudy (bool) – If true, update cloud type for cloudy time steps with phygnn predictions

  • test_set_mask (None | numpy.ndarray of bool) – Set of full data set in val_files to use. If None, use full dataset.

  • save_timeseries (bool) – Save time series data to disk

Methods