Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

infrastore is a Rust library for managing time-series data in power-systems and energy simulations. It separates persistence into two concerns: numerical arrays are stored in NetCDF4, while the metadata that associates each array with an owning component lives in SQLite. Identical arrays are stored once and shared through content addressing.

The library ships native Rust, Python (via PyO3), and Julia (via a C ABI) interfaces, plus the infrastore command-line tool and a read-only gRPC server with a Rust client.

flowchart TB
    subgraph clients["Language Interfaces"]
        RUST["Rust<br/>(native crate)"]
        PY["Python<br/>(PyO3 wheel)"]
        JL["Julia<br/>(C ABI)"]
        CLI["infrastore<br/>(CLI)"]
    end

    subgraph core["infrastore-core"]
        STORE["Store"]
        STORE --> NC[("NetCDF4<br/>arrays")]
        STORE --> SQL[("SQLite<br/>metadata")]
    end

    subgraph remote["Remote Access"]
        SRV["gRPC server<br/>(read-only)"]
        RC["Rust client"]
    end

    RUST --> STORE
    PY --> STORE
    JL --> STORE
    CLI --> STORE
    SRV --> STORE
    RC -->|"gRPC"| SRV

    style RUST fill:#4a9eff,color:#fff
    style PY fill:#17a2b8,color:#fff
    style JL fill:#9558b2,color:#fff
    style CLI fill:#fd7e14,color:#fff
    style STORE fill:#28a745,color:#fff
    style NC fill:#28a745,color:#fff
    style SQL fill:#28a745,color:#fff
    style SRV fill:#ffc107,color:#000
    style RC fill:#ffc107,color:#000

Key Features

  • One array, stored once — Arrays are addressed by a SHA-256 content hash, so identical series shared across components are written to disk a single time (content addressing)
  • NetCDF4 for arrays, SQLite for metadata — Numerical data lands in a compact, chunked NetCDF4 file; queryable associations live in a catalog SQLite database (storage model)
  • Feature-tagged associations — Each association carries an arbitrary map of typed features (int / float / bool / str) so multiple variants of a series can coexist under one owner
  • Typed, N-dimensional arrays — Store f64, f32, i64, i32, u64, or bool values, with an optional per-step element shape (e.g. the coefficient tuple of a cost curve)
  • Three language bindings — Use it from Rust, Python, or Julia with the same on-disk format
  • A infrastore command-line tool — Load time series from CSV, and list, read, and inspect a store straight from a terminal, with table / json / csv output (CLI how-to)
  • Read-only gRPC service — Serve a store over the network for remote readers, with optional API-key authentication
  • Designed for power-systems data — The data model maps onto InfrastructureSystems.jl and infrasys owners, categories, and time-series concepts

Who Should Read This

AudienceStart here
Python package developersPython Developer Guide
Julia package developersJulia Developer Guide
Rust developersRust Developer Guide
Command-line usersUse the infrastore CLI
Anyone deploying the servergRPC Server & Client
Tooling & forensicsOn-Disk File Format

Next Steps