revrt.utilities.handlers.IncrementalWriter#

class IncrementalWriter(out_fp)[source]#

Bases: object

Stream results to disk by appending result in chunks to a file

File can be of type CSV or GeoPackage. A new file is created if one does not exist.

Parameters:: out_fp (path-like) – Path to output file.

Methods

`preprocess_chunk`(chunk)	Preprocess chunk before saving
`save`(result)	Write result chunk to file

preprocess_chunk(chunk)[source]#

Preprocess chunk before saving

By default this method just passes the data through (underlying assumption is that the chunk data is already in a DataFrame). If the underlying data is not in a DataFrame, users can subclass and overwrite this method to convert the data into a DataFrame for writing.

Parameters:: chunk (pandas.DataFrame or geopandas.GeoDataFrame) – A chunk of data that will eventually be written to file.
Returns:: pandas.DataFrame or geopandas.GeoDataFrame – A dataframe holding the chunk of data to be written to file.

save(result)[source]#

Write result chunk to file

Parameters:: result (pandas.DataFrame or geopandas.GeoDataFrame) – A chunk of data that will eventually be written to file.