compass.pipeline.collection.persistence.persist_documents#

async persist_documents(jurisdiction, collected_docs, *, relative_to=None)[source]#

Persist deduplicated documents for one jurisdiction

Parameters:

jurisdiction (compass.utilities.jurisdictions.Jurisdiction) – Jurisdiction whose deduplicated documents will be persisted and serialized into collection metadata.
collected_docs (compass.pipeline.collection.dedupe.DocumentDeDuplicator) – Deduplicated document collection containing {"doc", "from_steps"} entries for each persisted document.
relative_to (path-like, optional) – Base path used to store source_fp and parsed_fp as relative paths when possible. By default, None.

Returns:

dict – Serialized collection metadata for the jurisdiction, including jurisdiction identifiers and the persisted document records.