C ABI
infrastore-ffi compiles a cdylib (libinfrastore_ffi.{dylib,so,dll}) exposing a C-compatible
API over Store. The header crates/infrastore-ffi/include/infrastore.h is generated by cbindgen —
do not hand-edit it. The Julia binding is the primary consumer.
Conventions
- Return codes. Every function returns
int32_t.INFRASTORE_OK(0) is success; any other value is an error. Retrieve the human-readable detail withinfrastore_last_error_message. - Opaque handles.
InfraStoreandInfraStoreKeyare incomplete struct types; you only ever hold pointers. Free them withinfrastore_store_freeandinfrastore_key_free. - Out-parameters. Results are written through caller-provided pointers (
**out,*out_len, …). - Caller-owned buffers. Returned arrays come back through an out-pointer + length and must be
freed:
f64percentile buffers (double **) withinfrastore_buffer_free_f64, raw element-byte buffers (uint8_t **) withinfrastore_buffer_free_u8, timestamp and shape buffers (int64_t **) withinfrastore_buffer_free_i64, and theu64dims buffer frominfrastore_store_get_forecast(uint64_t **) withinfrastore_buffer_free_u64. - Typed arrays. Add functions take the element
dtypecode (0=f64, 1=f32, 2=i64, 3=i32, 4=u64, 5=bool),ndimsplus adims_ptrshape array ([length, k1, …]), and the raw little-endiandata_ptrofdata_byte_lenbytes. Reads return the dtype code and a raw byte buffer for the caller to decode;infrastore_store_get_singlefollows the same dtype-generic convention, returning the dtype, shape, and raw element bytes. owner_categoryis anint32_t(0 = Component,1 = SupplementalAttribute) passed to the add functions. The owner identity is the pair(owner_id, owner_category)— a component and a supplemental attribute may share a numericowner_idand stay distinct — and the category is recorded with the association at add time.extis an optional opaque, package-owned payload (typically JSON) passed verbatim to the add functions and stored uninterpreted.- Strings are null-terminated UTF-8. Optional string arguments (
ext,features_json,units) acceptNULL. - Features are passed as a JSON object string whose values are int / float / bool / string.
- Timestamps are
int64_tUnix milliseconds. Resolutions/horizons/intervals are ISO-8601 duration strings (e.g."PT1H","P1M","P1Y"); aNULL(or empty) string means unset. On output they are ownedchar *strings — free each withinfrastore_string_free.
Status Codes
| Macro | Value | Meaning |
|---|---|---|
INFRASTORE_OK | 0 | Success |
INFRASTORE_ERR_NULL_POINTER | 1 | A required pointer was NULL |
INFRASTORE_ERR_INVALID_UTF8 | 2 | A string argument was not UTF-8 |
INFRASTORE_ERR_INVALID_PARAMETER | 3 | A bad argument value |
INFRASTORE_ERR_NOT_FOUND | 4 | No matching series / array |
INFRASTORE_ERR_DUPLICATE | 5 | Key already exists |
INFRASTORE_ERR_INTEGRITY | 6 | On-disk inconsistency |
INFRASTORE_ERR_READ_ONLY | 7 | Write on a read-only store |
INFRASTORE_ERR_IO | 8 | I/O failure |
INFRASTORE_ERR_INCOMPATIBLE_FORMAT | 9 | The store on disk was written in a different, incompatible on-disk format than this build reads. There is no in-place upgrade. |
INFRASTORE_ERR_DUPLICATE_ASSOCIATION | 10 | An attachment or parent/child edge with the same identity already exists. Distinct from INFRASTORE_ERR_DUPLICATE, which is about time-series identity. |
INFRASTORE_ERR_INTERNAL | 99 | Unexpected internal error |
Lifecycle
int32_t infrastore_store_create(const char *path, bool in_memory, struct InfraStore **out);
/* compression_kind: 0 = none, 1 = DEFLATE (deflate_level 0-9 + shuffle). */
int32_t infrastore_store_create_with_compression(const char *path, bool in_memory, uint8_t compression_kind,
uint8_t deflate_level, bool shuffle, struct InfraStore **out);
int32_t infrastore_store_open(const char *path, bool read_only, struct InfraStore **out);
void infrastore_store_free(struct InfraStore *handle);
void infrastore_key_free(struct InfraStoreKey *key);
/* Frees any owned `char *` this library returns (resolutions, horizons, intervals, …). */
void infrastore_string_free(char *s);
void infrastore_buffer_free_f64(double *ptr, uint64_t len);
void infrastore_buffer_free_u8(uint8_t *ptr, uint64_t len);
void infrastore_buffer_free_i64(int64_t *ptr, uint64_t len);
void infrastore_buffer_free_u64(uint64_t *ptr, uint64_t len);
SingleTimeSeries
int32_t infrastore_store_add_single(struct InfraStore *handle,
int64_t owner_id, const char *owner_type,
int32_t owner_category, /* 0=Component, 1=SupplementalAttribute */
const char *name,
int64_t initial_ts_unix_ms, const char *resolution, /* ISO-8601 */
int32_t dtype, uint64_t ndims, const uint64_t *dims_ptr,
const uint8_t *data_ptr, uint64_t data_byte_len,
const char *ext, /* optional */
const char *features_json, /* optional */
const char *units, /* optional */
struct InfraStoreKey **out_key); /* owned; infrastore_key_free */
int32_t infrastore_store_get_single(const struct InfraStore *handle, const struct InfraStoreKey *key,
int64_t *out_initial_ts_unix_ms,
char **out_resolution, /* ISO-8601; infrastore_string_free */
int32_t *out_dtype,
int64_t **out_shape, uint64_t *out_shape_len, /* infrastore_buffer_free_i64 */
uint8_t **out_data, uint64_t *out_data_byte_len, /* infrastore_buffer_free_u8 */
char **out_ext); /* optional (NULL skips); owned, infrastore_string_free */
int32_t infrastore_store_remove(struct InfraStore *handle, const struct InfraStoreKey *key);
/* All-or-nothing batched remove: on any error (including one missing key)
nothing is removed. *out_removed receives the count on success. */
int32_t infrastore_store_remove_bulk(struct InfraStore *handle,
const struct InfraStoreKey *const *keys, uint64_t len,
uint64_t *out_removed);
int32_t infrastore_store_has(const struct InfraStore *handle, const struct InfraStoreKey *key, bool *out_present);
/* Key identity comparison and hashing (consistent with each other; the hash is
stable only within one process). */
int32_t infrastore_key_eq(const struct InfraStoreKey *a, const struct InfraStoreKey *b, bool *out_eq);
int32_t infrastore_key_identity_hash(const struct InfraStoreKey *key, uint64_t *out_hash);
NonSequentialTimeSeries
infrastore_store_add_non_sequential takes an explicit int64_t Unix-millisecond timestamp array
alongside the typed data buffer. infrastore_store_get_non_sequential returns owned timestamp,
shape, and raw-byte buffers (free with infrastore_buffer_free_i64, infrastore_buffer_free_i64,
and infrastore_buffer_free_u8) plus the dtype code. The shape is the full
[length, *element_shape] array shape (the first dim is time, so callers can recover an
N-dimensional per-step element shape). out_ext is the optional opaque package-owned payload copied
into a caller-allocated buffer of ext_cap bytes, with the full length reported in out_ext_len —
probe with a NULL/zero-capacity buffer first, then call again with a buffer of that length (as
infrastore_store_get_single documents for its shape and ext outputs).
int32_t infrastore_store_add_non_sequential(struct InfraStore *handle,
int64_t owner_id, const char *owner_type,
int32_t owner_category, const char *name,
const int64_t *timestamps_unix_ms, uint64_t timestamps_len,
int32_t dtype, uint64_t ndims, const uint64_t *dims_ptr,
const uint8_t *data_ptr, uint64_t data_byte_len,
const char *ext, const char *features_json,
const char *units,
struct InfraStoreKey **out_key);
int32_t infrastore_store_get_non_sequential(const struct InfraStore *handle, const struct InfraStoreKey *key,
int64_t **out_timestamps, uint64_t *out_timestamps_len,
int32_t *out_dtype,
int64_t **out_shape, uint64_t *out_shape_len, /* infrastore_buffer_free_i64 */
uint8_t **out_data, uint64_t *out_data_byte_len, /* infrastore_buffer_free_u8 */
char *out_ext, uint64_t ext_cap,
uint64_t *out_ext_len);
Attribute-Based Access
Resolve a series by its attributes instead of a key handle. A NULL/empty resolution means unset.
int32_t infrastore_store_has_by_attrs(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category, const char *name,
const char *resolution, const char *features_json, bool *out_present);
/* Name-less existence query: true iff owner_id has any series, optionally
filtered to a single ts_type (use_type selects whether ts_type applies). */
int32_t infrastore_store_has_for_owner(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category,
int32_t ts_type, bool use_type, bool *out_present);
int32_t infrastore_store_remove_by_attrs(struct InfraStore *handle,
int64_t owner_id, int32_t owner_category, const char *name,
const char *resolution, const char *features_json);
int32_t infrastore_store_get_array_by_hash(const struct InfraStore *handle, const uint8_t *data_hash,
int32_t *out_dtype,
uint8_t **out_data, uint64_t *out_byte_len); /* infrastore_buffer_free_u8 */
/* Count SingleTimeSeries (*out_sts) and DeterministicSingleTimeSeries (*out_dst)
associations that reference the 32-byte content hash data_hash, across all
owners — one catalog query to decide whether removing a SingleTimeSeries would
orphan a DST that shares its backing array. */
int32_t infrastore_store_count_array_references(const struct InfraStore *handle, const uint8_t *data_hash,
uint64_t *out_sts, uint64_t *out_dst);
/* Build a key handle from attributes for any ts_type, so an attribute-addressed
caller can reuse the key-based readers (infrastore_store_get_single,
infrastore_store_get_non_sequential, infrastore_store_get_forecast_by_key). A NULL resolution
or interval means unset. The returned key is owned; free it with infrastore_key_free. */
int32_t infrastore_make_key_from_attrs(int64_t owner_id, int32_t owner_category, const char *name,
int32_t ts_type,
const char *resolution, const char *interval, /* ISO-8601; NULL = unset */
const char *features_json,
struct InfraStoreKey **out_key);
/* List every key for owner_id (one per association, including derived
DeterministicSingleTimeSeries rows). Ownership is two-tiered: free each InfraStoreKey
with infrastore_key_free, then free the array with infrastore_keys_buffer_free. An owner with
no series yields *out_keys = NULL and *out_len = 0. */
int32_t infrastore_store_get_time_series_keys(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category,
struct InfraStoreKey ***out_keys, uint64_t *out_len);
void infrastore_keys_buffer_free(struct InfraStoreKey **ptr, uint64_t len);
/* Inspect an opaque key: type code, resolution (an owned ISO-8601 string, NULL
when unset — free with infrastore_string_free), owner id, owner category (0 = Component,
1 = SupplementalAttribute), name, and features (a JSON object string, "{}" when
empty — the shape the attribute-addressed entry points accept). The name and
features strings use probe-then-fetch — pass NULL buffers / 0 caps to read the
required lengths, then call again with len+1-byte buffers. */
int32_t infrastore_key_attributes(const struct InfraStoreKey *key,
int32_t *out_type, char **out_resolution, /* ISO-8601; infrastore_string_free */
int64_t *out_owner_id, int32_t *out_owner_category,
char *name_buf, uint64_t name_cap, uint64_t *out_name_len,
char *features_buf, uint64_t features_cap, uint64_t *out_features_len);
/* The whole metadata record for a key, as a JSON object with the shape of one
infrastore_store_list_time_series element: owner_id, owner_type, owner_category,
time_series_type, name, data_hash (64-char hex), initial_timestamp_ms, resolution,
horizon, interval, count, length, percentiles, dtype, element_shape, features,
units, ext — fields that do not apply to the key's type are null. One export
covers every time series type, static and forecast alike. Probe-then-fetch:
call with buf = NULL, cap = 0 to learn *out_len, then again with an
out_len+1-byte buffer. INFRASTORE_ERR_NOT_FOUND if the key names nothing stored. */
int32_t infrastore_store_get_metadata_by_key(const struct InfraStore *handle,
const struct InfraStoreKey *key,
char *buf, uint64_t cap, uint64_t *out_len);
/* Resolve attributes plus a requested type to the key of the one matching stored
series. requested_type is any stored type code (0..5) or
INFRASTORE_TYPE_ABSTRACT_DETERMINISTIC (100), which matches a stored Deterministic or
DeterministicSingleTimeSeries and yields the concrete one. Unlike
infrastore_make_key_from_attrs, which builds an identity without consulting the
catalog, this validates: INFRASTORE_ERR_NOT_FOUND on a miss and
INFRASTORE_ERR_INVALID_PARAMETER when several series match (narrow with a concrete
type, resolution, and/or interval). The name is historical — it is not
forecast-specific. Free the key with infrastore_key_free. */
int32_t infrastore_store_resolve_forecast_key(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category,
const char *name,
const char *resolution, const char *interval,
const char *features_json,
int32_t requested_type,
struct InfraStoreKey **out_key);
infrastore_store_get_metadata_by_key + infrastore_store_get_array_by_hash is the read path used
by bindings that maintain their own key objects (such as an InfrastructureSystems.jl-side store).
infrastore_make_key_from_attrs bridges the two addressing styles: it materializes a
InfraStoreKey from attributes that the key-based read functions accept directly, which is how a
binding reaches the metadata of an attribute-addressed series.
infrastore_store_get_time_series_keys enumerates an owner's keys (the only way to obtain a key for
a transform-derived DeterministicSingleTimeSeries), and infrastore_key_attributes reads back an
opaque key's type, name, features, and addressing so the caller can pick the matching key-based
reader.
Forecasts
The forecast types are created and read through the C ABI. ts_type is the TimeSeriesType
discriminant — 0 = SingleTimeSeries, 1 = NonSequentialTimeSeries, 2 = Deterministic,
3 = DeterministicSingleTimeSeries, 4 = Probabilistic, 5 = Scenarios — plus one request-only
sentinel, INFRASTORE_TYPE_ABSTRACT_DETERMINISTIC (100), described below. Forecast values are
dtype-generic raw little-endian byte buffers with explicit dimensions — the same dtype, ndims,
dims_ptr, data_ptr, data_byte_len convention as the static add functions (see the
data model for the conventional shapes); the store records
the windowing parameters in metadata and does not interpret the layout. A
DeterministicSingleTimeSeries (3) is read like any other forecast but cannot be written through
infrastore_store_add_forecast — it is derived via infrastore_store_transform_single_time_series.
The C ABI keeps these per-type add functions as low-level transport (the higher-level bindings layer
the generic add_time_series over them). infrastore_store_add_forecast accepts only ts_type
2 = Deterministic or 5 = Scenarios; infrastore_store_add_probabilistic adds the percentile
vector for Probabilistic. DeterministicSingleTimeSeries (3) is not addable through
infrastore_store_add_forecast — it errors and directs you to
infrastore_store_transform_single_time_series, which derives a DeterministicSingleTimeSeries
from every stored SingleTimeSeries (sharing the backing array) and writes the number transformed
to *out_count:
int32_t infrastore_store_add_forecast(struct InfraStore *handle,
int64_t owner_id, const char *owner_type, int32_t owner_category,
const char *name, int32_t ts_type,
int64_t initial_ts_unix_ms,
const char *resolution, const char *horizon, const char *interval, /* ISO-8601 */
uint64_t count,
int32_t dtype, uint64_t ndims, const uint64_t *dims_ptr,
const uint8_t *data_ptr, uint64_t data_byte_len,
const char *ext, /* optional */
const char *features_json, const char *units,
struct InfraStoreKey **out_key);
int32_t infrastore_store_add_probabilistic(struct InfraStore *handle,
int64_t owner_id, const char *owner_type,
int32_t owner_category, const char *name,
int64_t initial_ts_unix_ms,
const char *resolution, const char *horizon, const char *interval, /* ISO-8601 */
uint64_t count,
const double *percentiles_ptr, uint64_t percentiles_len,
int32_t dtype, uint64_t ndims, const uint64_t *dims_ptr,
const uint8_t *data_ptr, uint64_t data_byte_len,
const char *ext, /* optional */
const char *features_json, const char *units,
struct InfraStoreKey **out_key);
int32_t infrastore_store_transform_single_time_series(struct InfraStore *handle,
const char *horizon, const char *interval, /* ISO-8601 */
int32_t owner_category, /* <0 = all categories; else 0=Component, 1=SupplementalAttribute */
const char *resolution, /* NULL = all resolutions */
uint64_t *out_count);
infrastore_store_get_forecast is the forecast read function: it resolves a forecast by attributes
and returns the decoded data buffer, its out-dimensions, the metadata, and — for Probabilistic —
the percentile vector. A stored DeterministicSingleTimeSeries is synthesized into Deterministic
values (its dense windows are materialized from the backing SingleTimeSeries), but it remains a
distinct stored type for addressing purposes.
Its ts_type argument is a read request, not merely a stored-type filter:
- A concrete code (
2 = Deterministic,3 = DeterministicSingleTimeSeries,4 = Probabilistic,5 = Scenarios) matches only that exact stored type. Passing2does not find a storedDeterministicSingleTimeSeries, and passing3does not find a storedDeterministic. The non-forecast codes0and1are rejected withINFRASTORE_ERR_INVALID_PARAMETER. INFRASTORE_TYPE_ABSTRACT_DETERMINISTIC(100) is a request-only sentinel — never a stored type — for theAbstractDeterministicfamily: it matches a storedDeterministicor aDeterministicSingleTimeSeries. This is the only way to address a deterministic forecast whose concrete type the caller does not know in advance. The catalog resolves the family authoritatively (no client-side guess-and-retry) and reports the concrete type that matched through*out_matched_type. If both concrete types share the identity the request is ambiguous and returnsINFRASTORE_ERR_INVALID_PARAMETER; a genuine miss returns the usual not-found error.
*out_matched_type always receives the concrete TimeSeriesType that was matched — so a stored
DeterministicSingleTimeSeries reports 3, never 2, and the 100 sentinel is never returned.
When time_range_present is true, only the windows whose start timestamp falls in
[time_range_start_ms, time_range_end_ms) are returned; pass false to retrieve all windows. The
caller owns the returned buffers: free *out_data with infrastore_buffer_free_u8, *out_dims
with infrastore_buffer_free_u64, *out_percentiles (non-NULL only for Probabilistic) with
infrastore_buffer_free_f64, and each of the *out_resolution / *out_horizon / *out_interval
ISO-8601 strings with infrastore_string_free.
int32_t infrastore_store_get_forecast(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category,
const char *name,
int32_t ts_type, /* 2..5, or INFRASTORE_TYPE_ABSTRACT_DETERMINISTIC (100) */
const char *resolution, const char *interval, /* ISO-8601 filters; NULL = none */
const char *features_json,
bool time_range_present,
int64_t time_range_start_ms, int64_t time_range_end_ms,
int64_t *out_initial_ts_unix_ms,
char **out_resolution, char **out_horizon, char **out_interval, /* ISO-8601; infrastore_string_free */
uint64_t *out_count, uint64_t *out_scenario_count,
uint64_t *out_ndims, uint64_t **out_dims, /* dims: infrastore_buffer_free_u64 */
int32_t *out_dtype,
uint8_t **out_data, uint64_t *out_data_byte_len, /* infrastore_buffer_free_u8 */
double **out_percentiles, uint64_t *out_percentiles_len, /* infrastore_buffer_free_f64 */
int32_t *out_matched_type, /* concrete matched TimeSeriesType */
char **out_ext); /* optional (NULL skips); owned, infrastore_string_free */
infrastore_store_get_forecast_by_key is the key-based counterpart: it takes a InfraStoreKey
handle (the type comes from the key) instead of the
owner_id, name, ts_type, resolution, interval, features_json arguments, and produces identical
outputs with the same buffer-ownership rules. Because the key already names the concrete stored type
there is no family to resolve: *out_matched_type is simply the key's type (a
DeterministicSingleTimeSeries key reports 3, though its values are decoded into a dense
Deterministic window array). There is no key-level equivalent of the 100 sentinel — use
infrastore_store_get_forecast for a family request.
int32_t infrastore_store_get_forecast_by_key(const struct InfraStore *handle, const struct InfraStoreKey *key,
bool time_range_present,
int64_t time_range_start_ms, int64_t time_range_end_ms,
int64_t *out_initial_ts_unix_ms,
char **out_resolution, char **out_horizon, char **out_interval, /* ISO-8601; infrastore_string_free */
uint64_t *out_count, uint64_t *out_scenario_count,
uint64_t *out_ndims, uint64_t **out_dims, /* infrastore_buffer_free_u64 */
int32_t *out_dtype,
uint8_t **out_data, uint64_t *out_data_byte_len, /* infrastore_buffer_free_u8 */
double **out_percentiles, uint64_t *out_percentiles_len, /* infrastore_buffer_free_f64 */
int32_t *out_matched_type,
char **out_ext); /* optional (NULL skips); owned, infrastore_string_free */
Forecast metadata is read with the same infrastore_store_get_metadata_by_key as everything else —
build the key with infrastore_make_key_from_attrs, passing the forecast ts_type and, when a name
carries several forecasts differing only by interval, the interval. The returned row carries the
windowing parameters (horizon, interval, count), the content hash, and the percentiles of a
Probabilistic, without decoding the array.
int32_t infrastore_store_has_typed(const struct InfraStore *handle,
int64_t owner_id, int32_t owner_category, const char *name,
int32_t ts_type,
const char *resolution, const char *interval, /* ISO-8601; NULL = unset */
const char *features_json,
bool *out_present);
int32_t infrastore_store_remove_typed(struct InfraStore *handle,
int64_t owner_id, int32_t owner_category, const char *name,
int32_t ts_type,
const char *resolution, const char *interval, /* ISO-8601; NULL = unset */
const char *features_json);
infrastore_store_copy_time_series copies one association onto another owner (optionally under a
new name). Arrays are content-addressed, so only a new association row is written — no array data is
duplicated, and the stored type is preserved (a DeterministicSingleTimeSeries stays one rather
than being materialized into a dense Deterministic). The copy keeps the source's owner category.
The leading owner_id / owner_category / name / ts_type / resolution / interval /
features_json arguments identify the source series, exactly as for
infrastore_store_remove_typed; a NULL new_name keeps the source name.
int32_t infrastore_store_copy_time_series(struct InfraStore *handle,
/* source series: */
int64_t owner_id, int32_t owner_category, const char *name,
int32_t ts_type,
const char *resolution, const char *interval, /* ISO-8601; NULL = unset */
const char *features_json,
/* destination: */
int64_t dst_owner_id, const char *dst_owner_type,
const char *new_name); /* NULL = keep the source name */
Readers
The per-timestamp read path is exposed as two opaque reader handles — InfraStoreStaticReaderHandle
for SingleTimeSeries and InfraStoreForecastReaderHandle for forecasts. A reader is built once
over a filter (the same has_owner / owner_id / has_owner_category / owner_category / name
/ resolution / features_json convention as the attribute-based access, with a forecast reader
also taking a ts_type), then driven per timestamp. The lifecycle is: build → read the layout
once → *_read in a loop → fetch values per group/entry → free. Each reader pins one resolution
and owns reusable buffers that each read overwrites in place.
Ownership rules: the *_grid / *_timeline resolution/interval out-strings (char **) are owned —
free each with infrastore_string_free. Keys from *_group_key / *_entry_key are owned
InfraStoreKey * — free with infrastore_key_free. The *_values buffers (const uint8_t **) are
borrowed: they point into reader memory, stay valid only until the next read or *_free, and
must not be freed. Group/entry shapes follow the probe-then-fetch convention — call *_info with
shape_buf = NULL / shape_cap = 0 to learn *out_shape_len, then again with a buffer of that
length. *_read errors (never clamps) if at_unix_ms is off the reader's grid/timeline.
StaticReader
Reads every matching SingleTimeSeries at one timestamp, partitioned into (dtype, element_shape)
groups; each group's values are one dense [num_columns, *element_shape] little-endian buffer whose
column j is the key from infrastore_static_reader_group_key(reader, group_idx, j, …).
int32_t infrastore_store_build_static_reader(const struct InfraStore *handle,
bool has_owner, int64_t owner_id,
bool has_owner_category, int32_t owner_category,
const char *name, const char *resolution,
const char *features_json,
struct InfraStoreStaticReaderHandle **out_reader);
int32_t infrastore_static_reader_grid(const struct InfraStoreStaticReaderHandle *reader,
int64_t *out_initial_ms, char **out_resolution, /* free with infrastore_string_free */
uint64_t *out_length);
int32_t infrastore_static_reader_num_groups(const struct InfraStoreStaticReaderHandle *reader, uint64_t *out_n);
int32_t infrastore_static_reader_group_info(const struct InfraStoreStaticReaderHandle *reader, uint64_t group_idx,
int32_t *out_dtype, uint64_t *out_num_columns,
int64_t *shape_buf, uint64_t shape_cap, uint64_t *out_shape_len);
int32_t infrastore_static_reader_group_key(const struct InfraStoreStaticReaderHandle *reader,
uint64_t group_idx, uint64_t col_idx,
struct InfraStoreKey **out_key); /* free with infrastore_key_free */
int32_t infrastore_static_reader_read(struct InfraStoreStaticReaderHandle *reader,
const struct InfraStore *store, int64_t at_unix_ms);
int32_t infrastore_static_reader_group_values(const struct InfraStoreStaticReaderHandle *reader, uint64_t group_idx,
const uint8_t **out_ptr, /* borrowed; valid until next read/free */
uint64_t *out_byte_len);
void infrastore_static_reader_free(struct InfraStoreStaticReaderHandle *reader);
All matched series must share one grid (initial_timestamp + length); the build validates this
and errors on divergence, so every column has a value at every valid timestamp (no presence mask).
ForecastReader
Reads the forecast window at one timestamp for every matching forecast of one type. The build
ts_type names the forecast type; a Deterministic reader (2) is abstract and also includes
DeterministicSingleTimeSeries (3), read into identical [H, *E] windows. (Note the asymmetry
with infrastore_store_get_forecast, where 2 matches only a stored Deterministic and the family
request must be spelled INFRASTORE_TYPE_ABSTRACT_DETERMINISTIC; the reader build takes no such
sentinel.) All matched forecasts must share one window timeline (initial_timestamp + interval +
count). Each entry's window is a little-endian buffer of its *_entry_info shape.
int32_t infrastore_store_build_forecast_reader(const struct InfraStore *handle,
bool has_owner, int64_t owner_id,
bool has_owner_category, int32_t owner_category,
int32_t time_series_type,
const char *name, const char *resolution,
const char *features_json,
struct InfraStoreForecastReaderHandle **out_reader);
int32_t infrastore_forecast_reader_timeline(const struct InfraStoreForecastReaderHandle *reader,
int64_t *out_initial_ms,
char **out_resolution, char **out_interval, /* free each with infrastore_string_free */
uint64_t *out_count);
int32_t infrastore_forecast_reader_num_entries(const struct InfraStoreForecastReaderHandle *reader, uint64_t *out_n);
int32_t infrastore_forecast_reader_num_slots(const struct InfraStoreForecastReaderHandle *reader, uint64_t *out_n);
int32_t infrastore_forecast_reader_entry_slot(const struct InfraStoreForecastReaderHandle *reader,
uint64_t entry_idx, uint64_t *out_slot);
int32_t infrastore_forecast_reader_entry_info(const struct InfraStoreForecastReaderHandle *reader, uint64_t entry_idx,
int32_t *out_dtype,
int64_t *shape_buf, uint64_t shape_cap, uint64_t *out_shape_len);
int32_t infrastore_forecast_reader_entry_key(const struct InfraStoreForecastReaderHandle *reader,
uint64_t entry_idx, struct InfraStoreKey **out_key); /* free with infrastore_key_free */
int32_t infrastore_forecast_reader_read(struct InfraStoreForecastReaderHandle *reader,
const struct InfraStore *store, int64_t at_unix_ms);
int32_t infrastore_forecast_reader_entry_values(const struct InfraStoreForecastReaderHandle *reader, uint64_t entry_idx,
const uint8_t **out_ptr, /* borrowed; valid until next read/free */
uint64_t *out_byte_len);
void infrastore_forecast_reader_free(struct InfraStoreForecastReaderHandle *reader);
Window-read deduplication. Forecasts that reference the same backing array and read plan
(deduplicated identical data, or several DeterministicSingleTimeSeries over one
SingleTimeSeries) collapse to a single window slot. infrastore_forecast_reader_read performs
one backend read per slot, not per entry, so a forecast shared by N owners is read once per
timestamp. infrastore_forecast_reader_num_slots is that physical read count, and
infrastore_forecast_reader_entry_slot gives the 0-based slot backing each entry (entries that
share data report the same slot) — group entries by slot to also decode each unique window only
once.
Bulk Adds
A batch accumulates add requests client-side (no store I/O); infrastore_store_add_batch commits
them all in one metadata transaction, which is much faster than per-item adds when ingesting
many series. It is also the fast NetCDF write path: same-shaped SingleTimeSeries are packed into
batch-sized datasets so the timestamp-major chunks are filled whole rather than a column at a time.
The infrastore_batch_add_* functions take the same arguments as their infrastore_store_add_*
counterparts minus the store handle and out_key; data buffers are copied into the batch, so they
only need to stay valid for the call. The submit is all-or-nothing and drains the batch in either
case (on error nothing was committed and the batch is left empty). On success the caller owns the
key-handle array: free each key with infrastore_key_free, then the buffer with
infrastore_keys_buffer_free (same contract as infrastore_store_get_time_series_keys). The batch
handle itself is reusable after submit and must eventually be released with infrastore_batch_free.
struct InfraStoreBatch *infrastore_batch_new(void);
void infrastore_batch_free(struct InfraStoreBatch *batch);
int32_t infrastore_batch_add_single(struct InfraStoreBatch *batch, /* infrastore_store_add_single args sans handle/out_key */ ...);
int32_t infrastore_batch_add_non_sequential(struct InfraStoreBatch *batch, ...);
int32_t infrastore_batch_add_forecast(struct InfraStoreBatch *batch, ...); /* 2=Deterministic, 5=Scenarios */
int32_t infrastore_batch_add_probabilistic(struct InfraStoreBatch *batch, ...);
int32_t infrastore_store_add_batch(struct InfraStore *handle, struct InfraStoreBatch *batch,
struct InfraStoreKey ***out_keys, uint64_t *out_len);
Bulk Reads
infrastore_store_bulk_read_single reads many full SingleTimeSeries in one call, reading each
packed dataset's column span once instead of re-reading every chunk per series — the efficient way
to load many whole series (e.g. for exploration or plotting), where a single full-series read
otherwise touches every chunk under the timestamp-major layout. Every key must identify a
SingleTimeSeries; otherwise the call fails with INFRASTORE_ERR_INVALID_PARAMETER. The results
are held in a InfraStoreBulkReadHandle (input order preserved) and read out element-by-element
with infrastore_bulk_result_get_single, whose out-parameters match infrastore_store_get_single —
the caller owns the returned resolution string and the shape/data buffers and frees them with
infrastore_string_free, infrastore_buffer_free_i64, and infrastore_buffer_free_u8. The handle
is not consumed by a read (elements may be read more than once) and must be released with
infrastore_bulk_result_free.
int32_t infrastore_store_bulk_read_single(const struct InfraStore *handle,
const struct InfraStoreKey *const *keys, uint64_t n,
struct InfraStoreBulkReadHandle **out_result);
int64_t infrastore_bulk_result_len(const struct InfraStoreBulkReadHandle *result); /* -1 if null */
int32_t infrastore_bulk_result_get_single(const struct InfraStoreBulkReadHandle *result, uint64_t index,
/* same out-params as infrastore_store_get_single */ ...);
void infrastore_bulk_result_free(struct InfraStoreBulkReadHandle *result);
Store-Wide Operations
int32_t infrastore_store_counts(const struct InfraStore *handle, int64_t *out_components_with_time_series,
int64_t *out_static_time_series, int64_t *out_forecasts);
/* Association count per type as a JSON array of {time_series_type, count};
probe-then-fetch. */
int32_t infrastore_store_counts_by_type(const struct InfraStore *handle,
char *buf, uint64_t cap, uint64_t *out_len);
/* Distinct stored arrays (content hashes); shared arrays count once. */
int32_t infrastore_store_num_distinct_arrays(const struct InfraStore *handle, int64_t *out_count);
/* Grouped static / forecast summaries as JSON arrays (one object per group with a
`count` field); probe-then-fetch. */
int32_t infrastore_store_static_summary(const struct InfraStore *handle,
char *buf, uint64_t cap, uint64_t *out_len);
int32_t infrastore_store_forecast_summary(const struct InfraStore *handle,
char *buf, uint64_t cap, uint64_t *out_len);
/* Distinct owners per category + distinct arrays per kind (static/forecast). */
int32_t infrastore_store_counts_detailed(const struct InfraStore *handle, int64_t *out_components,
int64_t *out_supplemental_attributes,
int64_t *out_static_time_series, int64_t *out_forecasts);
/* Distinct owner ids of owner_category as a JSON array; optional type/resolution
filters (NULL resolution = none). Probe-then-fetch. */
int32_t infrastore_store_list_owner_ids(const struct InfraStore *handle, int32_t owner_category,
bool has_time_series_type, int32_t time_series_type,
const char *resolution, char *buf, uint64_t cap, uint64_t *out_len);
/* out_present = false when no matching forecast. The out_horizon/out_interval/
out_resolution ISO-8601 strings are NULL when absent (free with infrastore_string_free);
out_count and out_initial_ms are -1 when absent. filter_* NULL = no filter. */
int32_t infrastore_store_get_forecast_parameters(const struct InfraStore *handle,
const char *filter_resolution, const char *filter_interval,
bool *out_present, char **out_horizon,
char **out_interval, int64_t *out_count,
char **out_resolution, int64_t *out_initial_ms);
/* Per-resolution static grids as a JSON array of {"resolution","initial_timestamp_ms",
"length"} objects, ordered by resolution (empty array when no SingleTimeSeries);
error when the series at one resolution disagree. filter_resolution NULL = every
resolution, else scope to that ISO-8601 grid. Probe-then-fetch (buf=NULL, cap=0 to size). */
int32_t infrastore_store_check_static_consistency(const struct InfraStore *handle,
const char *filter_resolution, char *buf,
uint64_t cap, uint64_t *out_len);
/* Distinct resolutions as a JSON array of ISO-8601 duration strings, ascending;
optional type filter. Probe-then-fetch (buf=NULL, cap=0 to size). */
int32_t infrastore_store_get_resolutions(const struct InfraStore *handle,
bool has_time_series_type, int32_t time_series_type,
char *buf, uint64_t cap, uint64_t *out_len);
/* out_kind: 0 = none, 1 = DEFLATE (out_level 0-9 + out_shuffle). */
int32_t infrastore_store_get_compression(const struct InfraStore *handle, uint8_t *out_kind,
uint8_t *out_level, bool *out_shuffle);
int32_t infrastore_store_verify(const struct InfraStore *handle, uint64_t *out_error_count);
int32_t infrastore_store_compact(struct InfraStore *handle);
int32_t infrastore_store_flush(struct InfraStore *handle);
/* Cross-operation transactions. Adds, removals, and transforms between a begin and
its matching commit either all take effect or none do; removals are reversible
only inside one. Calls nest -- only the outermost commit is durable. This is
store state, not a borrowed guard, so nothing has to survive the ABI boundary.
Holds the SQLite write lock until the outermost commit/rollback. */
int32_t infrastore_store_begin_transaction(struct InfraStore *handle);
int32_t infrastore_store_commit_transaction(struct InfraStore *handle);
int32_t infrastore_store_rollback_transaction(struct InfraStore *handle);
int32_t infrastore_store_in_transaction(struct InfraStore *handle, bool *out);
/* Persist the store's data to `path` (NetCDF) and `<path>.sqlite` (metadata),
materializing an in-memory store to disk. Existing target files are overwritten. */
int32_t infrastore_store_persist(struct InfraStore *handle, const char *path);
/* has_owner=false clears all; when true, the owner is the pair (owner_id, owner_category). */
int32_t infrastore_store_clear(struct InfraStore *handle, bool has_owner, int64_t owner_id,
int32_t owner_category); /* owner_category: 0=Component, 1=SupplementalAttribute */
/* Reassign every time series owned by old_owner_id (in owner_category) to
new_owner_id; *out_updated (when non-NULL) receives the number of associations
changed. */
int32_t infrastore_store_replace_owner(struct InfraStore *handle,
int64_t old_owner_id, int64_t new_owner_id,
int32_t owner_category, uint64_t *out_updated);
/* List keys as a JSON array (identity + per-type descriptive snapshot, no physical
storage detail). The filters are independent; with none set the whole store is
listed. `interval` (ISO-8601; NULL = unset) matches forecasts only — static
rows carry no interval. Probe-then-fetch: call with buf=NULL, cap=0 to learn
the length via out_len, then again with len+1 bytes. */
int32_t infrastore_store_list_keys(const struct InfraStore *handle,
bool has_owner, int64_t owner_id,
bool has_owner_category, int32_t owner_category,
bool has_time_series_type, int32_t time_series_type,
const char *name, const char *resolution, const char *interval,
const char *features_json,
char *buf, uint64_t cap, uint64_t *out_len);
/* Like infrastore_store_list_keys, but each row is annotated with the hex content hash of
the array it resolves to (keys_to_json's shape plus a `data_hash` field); rows
that share a stored array share their `data_hash`, so a caller can group time
series by their underlying data in one query. Same filters and probe-then-fetch
convention as infrastore_store_list_keys. */
int32_t infrastore_store_list_array_groups(const struct InfraStore *handle,
bool has_owner, int64_t owner_id,
bool has_owner_category, int32_t owner_category,
bool has_time_series_type, int32_t time_series_type,
const char *name, const char *resolution, const char *interval,
const char *features_json,
char *buf, uint64_t cap, uint64_t *out_len);
Associations
Two catalogs of relationships between entities the store does not otherwise model. Both are independent of time series: there are no foreign keys and no cascade (both endpoints live in the caller's object graph, so a cascade could never fire), so removing a time series never removes an association and vice versa; a caller that wants both makes both calls.
Each family's query predicate crosses the boundary as one JSON object string rather than a set of positional arguments, because half its fields are string lists. Every field is optional and the set ones are ANDed:
{ "component_id": 1, "component_types": ["Generator", "Load"],
"attribute_id": 100, "attribute_types": ["GeographicInfo"] }
{ "parent_id": 1, "parent_types": ["Generator"],
"child_id": 7, "child_types": ["Bus"] }
A NULL (or empty) filter_json is the empty filter and matches every row — which is what
makes a bulk export/import round trip one call each way. The *_types lists hold concrete type
names, rendered as SQL IN (…); expanding an abstract type into its subtypes stays in the calling
language, and an empty list matches nothing. An unknown field or malformed JSON is
INFRASTORE_ERR_INVALID_PARAMETER. Every remove function reports its count through out_removed
and treats removing nothing as success, not an error.
The list functions follow the same probe-then-fetch convention as infrastore_store_list_keys: call
with buf = NULL, cap = 0 to learn the length via out_len, then again with a len + 1-byte
buffer.
Supplemental-attribute associations
Which supplemental attributes are attached to which components. Identity is the
(component_id, attribute_id) pair; the type names are denormalized labels, so re-attaching the
same pair under different type names is still a duplicate. One attribute may be attached to many
components.
/* Attach supplemental attribute (attribute_id, attribute_type) to component
(component_id, component_type). INFRASTORE_ERR_DUPLICATE_ASSOCIATION if that component
already carries that attribute, whatever type names are supplied. */
int32_t infrastore_store_add_supplemental_attribute_association(struct InfraStore *handle,
int64_t component_id,
const char *component_type,
int64_t attribute_id,
const char *attribute_type);
/* Attach many in one all-or-nothing transaction, from a JSON array of objects with
component_id, component_type, attribute_id, and attribute_type. The import half
of the round trip whose export is infrastore_store_list_supplemental_attribute_associations
with a NULL filter. *out_added (when non-NULL) receives the number inserted. */
int32_t infrastore_store_add_supplemental_attribute_associations(struct InfraStore *handle,
const char *associations_json,
uint64_t *out_added);
/* Whether any attachment matches filter_json (NULL = any). */
int32_t infrastore_store_has_supplemental_attribute_association(const struct InfraStore *handle,
const char *filter_json,
bool *out_found);
/* Matching attachments as a JSON array, in insertion order; each object carries
component_id, component_type, attribute_id, attribute_type. Probe-then-fetch. */
int32_t infrastore_store_list_supplemental_attribute_associations(const struct InfraStore *handle,
const char *filter_json,
char *buf, uint64_t cap,
uint64_t *out_len);
/* Distinct attribute ids of the matching rows, ascending, as a JSON array — the
attributes attached to a component when component_id is set. Probe-then-fetch. */
int32_t infrastore_store_list_supplemental_attribute_ids(const struct InfraStore *handle,
const char *filter_json,
char *buf, uint64_t cap,
uint64_t *out_len);
/* Distinct component ids of the matching rows, ascending, as a JSON array — the
components carrying an attribute when attribute_id is set. Probe-then-fetch. */
int32_t infrastore_store_list_components_with_attributes(const struct InfraStore *handle,
const char *filter_json,
char *buf, uint64_t cap,
uint64_t *out_len);
/* Remove every matching attachment; *out_removed (when non-NULL) receives the
count. Removing nothing is success, not an error. */
int32_t infrastore_store_remove_supplemental_attribute_associations(struct InfraStore *handle,
const char *filter_json,
uint64_t *out_removed);
/* Move every attachment from component old_id to new_id; *out_updated (when
non-NULL) receives the rows changed. INFRASTORE_ERR_DUPLICATE_ASSOCIATION if new_id
already carries one of the attributes being moved. */
int32_t infrastore_store_replace_supplemental_attribute_component_id(struct InfraStore *handle,
int64_t old_id, int64_t new_id,
uint64_t *out_updated);
/* Attachment counts through out_count. kind selects what is counted:
0 = rows matching the filter, 1 = distinct attributes among them,
2 = distinct components among them. */
int32_t infrastore_store_count_supplemental_attribute_associations(const struct InfraStore *handle,
const char *filter_json,
int32_t kind, int64_t *out_count);
/* Grouped counts as JSON arrays: by attribute type, as {"type": …, "count": …}
ordered by type; and by both type names, as
{"component_type": …, "attribute_type": …, "count": …} ordered by attribute type
then component type. Probe-then-fetch. */
int32_t infrastore_store_supplemental_attribute_counts_by_type(const struct InfraStore *handle,
char *buf, uint64_t cap,
uint64_t *out_len);
int32_t infrastore_store_supplemental_attribute_summary(const struct InfraStore *handle,
char *buf, uint64_t cap,
uint64_t *out_len);
/* The attribute ids attached to component 1. */
const char *filter = "{\"component_id\":1}";
uint64_t len = 0;
infrastore_store_list_supplemental_attribute_ids(store, filter, NULL, 0, &len);
char *json = malloc(len + 1);
infrastore_store_list_supplemental_attribute_ids(store, filter, json, len + 1, &len); /* e.g. "[100]" */
free(json);
Parent/child associations
Directed edges between components — a generator (parent) connected to a bus (child), say. Both
endpoints are always components. Identity is the ordered (parent_id, child_id) pair, so the
reversed pair is a different edge, and with no relationship-kind column one ordered pair may be
related at most once.
This family is deliberately narrower than the supplemental one — no counts-by-type and no grouped summary — because there is no consumer for them yet; both are additive if one appears.
/* Record a directed edge from component (parent_id, parent_type) to component
(child_id, child_type). INFRASTORE_ERR_DUPLICATE_ASSOCIATION if that ordered pair is
already related; the reversed pair is a different edge. */
int32_t infrastore_store_add_parent_child_association(struct InfraStore *handle,
int64_t parent_id, const char *parent_type,
int64_t child_id, const char *child_type);
/* Record many edges in one all-or-nothing transaction, from a JSON array of objects
with parent_id, parent_type, child_id, and child_type. *out_added (when non-NULL)
receives the number inserted. */
int32_t infrastore_store_add_parent_child_associations(struct InfraStore *handle,
const char *associations_json,
uint64_t *out_added);
/* Whether any edge matches filter_json (NULL = any). */
int32_t infrastore_store_has_parent_child_association(const struct InfraStore *handle,
const char *filter_json,
bool *out_found);
/* Matching edges as a JSON array, in insertion order; each object carries
parent_id, parent_type, child_id, child_type. Probe-then-fetch. */
int32_t infrastore_store_list_parent_child_associations(const struct InfraStore *handle,
const char *filter_json,
char *buf, uint64_t cap,
uint64_t *out_len);
/* Distinct ids on one end of the matching edges, ascending, as a JSON array.
endpoint is 0 for parents and 1 for children — so endpoint = 1 with parent_id
set is "the children of this component". Probe-then-fetch. */
int32_t infrastore_store_list_parent_child_ids(const struct InfraStore *handle,
const char *filter_json, int32_t endpoint,
char *buf, uint64_t cap, uint64_t *out_len);
/* Remove every matching edge; *out_removed (when non-NULL) receives the count.
Removing nothing is success, not an error. */
int32_t infrastore_store_remove_parent_child_associations(struct InfraStore *handle,
const char *filter_json,
uint64_t *out_removed);
/* Rewrite component old_id to new_id on BOTH ends of every edge; *out_updated
(when non-NULL) receives the rows changed. INFRASTORE_ERR_DUPLICATE_ASSOCIATION if the
rewrite would duplicate an edge new_id already has. */
int32_t infrastore_store_replace_parent_child_component_id(struct InfraStore *handle,
int64_t old_id, int64_t new_id,
uint64_t *out_updated);
/* Number of edges matching filter_json, through out_count. */
int32_t infrastore_store_count_parent_child_associations(const struct InfraStore *handle,
const char *filter_json,
int64_t *out_count);
/* The children of component 1. */
const char *filter = "{\"parent_id\":1}";
uint64_t len = 0;
infrastore_store_list_parent_child_ids(store, filter, 1 /* children */, NULL, 0, &len);
char *json = malloc(len + 1);
infrastore_store_list_parent_child_ids(store, filter, 1, json, len + 1, &len); /* e.g. "[7]" */
free(json);
Neither association catalog is exposed over the gRPC server or the
infrastore CLI.
Error Messages
int32_t infrastore_last_error_message(char *buf, uint64_t buf_len, uint64_t *needed);
Copies the thread-local error message (UTF-8, null-terminated) into buf. *needed receives the
length excluding the NUL. If buf_len is too small, the message is truncated but INFRASTORE_OK is
still returned — call once with buf = NULL, buf_len = 0 to learn the needed size, then again with
a buffer of *needed + 1. This is the pattern TimeSeries.jl uses.
Building
cargo build -p infrastore-ffi --release
# Library at: target/release/libinfrastore_ffi.{dylib,so,dll}
Point consumers at it via the INFRASTORE_LIB environment variable (see the
Julia how-to).
Tracing
int32_t infrastore_store_init_logging(const char *filter);
Initialize the Rust tracing subscriber. filter is a null-terminated UTF-8
EnvFilter
directive string (e.g. "debug" or "infrastore_core=debug"). Pass NULL to read the RUST_LOG
environment variable; if that variable is also unset, no output is produced.
The subscriber is initialized at most once per process — subsequent calls are no-ops. Returns
INFRASTORE_OK on success or INFRASTORE_ERR_INVALID_UTF8 if filter is not valid UTF-8.
The Julia binding calls this automatically from its __init__ hook when
RUST_LOG is set, and exposes init_logging for explicit control.