mlclouds.autoxval.XVal
- class XVal(config={'epochs_a': 100, 'epochs_b': 90, 'features': ['solar_zenith_angle', 'cloud_type', 'refl_0_65um_nom', 'refl_0_65um_nom_stddev_3x3', 'refl_3_75um_nom', 'temp_3_75um_nom', 'temp_11_0um_nom', 'temp_11_0um_nom_stddev_3x3', 'cloud_probability', 'cloud_fraction', 'air_temperature', 'dew_point', 'relative_humidity', 'total_precipitable_water', 'surface_albedo'], 'hidden_layers': [{'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}, {'activation': 'relu', 'dropout': 0.1, 'units': 256}], 'learning_rate': 0.0005, 'loss_weights_a': [1, 0], 'loss_weights_b': [0.5, 0.5], 'metric': 'relative_mae', 'n_batch': 64, 'one_hot_categories': {'flag': ['clear', 'ice_cloud', 'water_cloud', 'bad_cloud']}, 'p_fun': 'p_fun_all_sky', 'p_kwargs': {'loss_terms': ['mae_ghi']}, 'phygnn_seed': 0, 'surfrad_window_minutes': 15, 'training_prep_kwargs': {'add_cloud_flag': True, 'filter_clear': False, 'filter_daylight': True, 'filter_sky_class': False, 'nan_option': 'interp', 'sza_lim': 89}, 'y_labels': ['cld_opd_dcomp', 'cld_reff_dcomp']})[source]
Bases:
objectTrain a PHYGNN using one or more satellite datasets then validate against the NSRDB baseline data using another satellite dataset to predict cloud parameters. The sites used for training may also be controlled.
- Parameters:
config (dict) – Dict of configuration options. See CONFIG for example.
Methods
load_model(fname)Load existing model from disk
plot(gid)Show statistics bar charts
save_model(fname)Save model to disk
save_stats(fname)Save statistics and model config to file
train([train_sites, train_files])Train PHYGNN model
validate([val_files, val_data, ...])Predict values using PHYGNN model and validation against baseline NSRDB data.
- train(train_sites=(0, 1, 2, 3, 5, 6), train_files='/projects/pxs/mlclouds/training_data/{year}_{area}_v322/mlclouds_surfrad_{area}_{year}.h5')[source]
Train PHYGNN model
- Parameters:
train_sites (list of int) – Sites to use for training
train_files (list | str) – File or list of files to use for training. Filenames must include the four-digit year and satellite domain (east|west).
- validate(val_files=None, val_data=None, update_clear=False, update_cloudy=False, save_timeseries=False)[source]
Predict values using PHYGNN model and validation against baseline NSRDB data.
- val_files: str | list of str | None
File or list of file to use for validation from config file. Filenames must include the four-digit year and east/west text to indicate satellite. Must be None if val_data is set.
- val_data: None | ValidationData instance
Use preloaded validation data or load from val_files if None. Must be none if val_files is set.
- update_clear: bool
If true, update cloud type for clear time steps with phygnn predictions
- update_cloudy: bool
If true, update cloud type for cloudy time steps with phygnn predictions
- save_timeseries: bool
Save time series data to disk
- load_model(fname)[source]
Load existing model from disk
- Parameters:
fname (str) – File name and path of pickle file
- save_model(fname)[source]
Save model to disk
- Parameters:
fname (str) – File name and path for pickle file