PowerIO guide

Readers parse power system source formats into typed models. Explicit passes normalize, validate, and lower them, and writers emit supported target formats. The .pio.json document records how a source was interpreted: model kind, provenance, source maps, structured diagnostics, validation, and lowering history. Sparse matrices and graph views are built from the same models for solver and analysis code. Rustdoc covers API detail.

Public conventions:

• writing a case back to the format it was read from returns the original bytes when the reader kept them;
• cross format conversion keeps the electrical core and reports losses as warnings;
• lowering between model families is always an explicit, recorded pass;
• matrix builders state sign, tap, shift, shunt, and reference bus conventions;
• C, Python, and Julia bindings share the same Rust core.

Transmission readers cover MATPOWER, PSS/E revisions 33 through 35, PowerWorld AUX and PWB, PSLF EPC, PowerModels JSON, egret JSON, pandapower JSON, PyPSA CSV folders, GO Challenge 3 JSON, Surge JSON, DeepMind OPFData FullTop and N-1 dataset JSON, and GridFM Parquet datasets. PowerWorld PWD is a display artifact and uses the display API. Distribution readers and writers live in powerio-dist for OpenDSS, PowerModelsDistribution ENGINEERING JSON, and BMOPF JSON.

Where to look:

• Compiler model layers: the BalancedNetwork and MulticonductorNetwork model families and the .pio.json document.
• PIO JSON schema: the .pio.json field reference, metadata and model JSON versioning, and row identity.
• Format fidelity: numeric conventions, the validation oracles, known limits per format, and the missing generator cost policy.
• Matrix outputs and the DC OPF bundle.
• Language APIs and Python.
• Performance and testing and release checks.
• Julia bindings: https://github.com/eigenergy/PowerIO.jl.

Rendered API docs (rustdoc) for all crates: https://powerio.dev.

Crates

Adding a format means adding one reader or writer at the hub rather than pairwise converters. IndexedNetwork is the dense \([0,n)\) analysis view derived from a balanced BalancedNetwork; matrix builders work from that view. Code that maps source bus IDs to dense rows must use IndexedNetwork::bus_index; it must not clamp IDs or assume 1-based contiguous IDs.

Architecture

Source formats parse into typed network models. Normalization, lowering, matrix projection, package construction, and problem instance assembly consume those models without changing parser dependencies.

powerio             powerio-dist
   │                     │
   ├──────► powerio-matrix
   │
   ├──────► powerio-pkg ◄──── powerio-dist
   │
   └──────► powerio-prob
                 │
                 └── optional "matrix" ──► powerio-matrix

• powerio owns the balanced network model, format routing, indexing, normalization, and shared GOC3 document parsing.
• powerio-dist owns the multiconductor network model and distribution formats.
• powerio-matrix owns generic sparse matrix and graph projections from a balanced network. It does not depend on powerio-prob.
• powerio-pkg owns .pio.json packages, operating points, study commits, provenance, validation, and lowering between model families.
• powerio-prob owns complete numerical problem instances. Its default build depends on powerio; the optional matrix feature projects a DC OPF instance into sparse operators. It has no powerio-dist dependency because no distribution problem instance is implemented.
• powerio-cli, powerio-py, and powerio-capi depend on the layers they expose.

A problem instance contains the complete indexed input for a problem family: coefficients, bounds, mappings, units, and conventions. It is not a source network, matrix projection, solver formulation, or solution. The current crate provides DcOpfInstance and ScopfInstance.

Compiler model layers describes the balanced and multiconductor payloads. .pio.json format defines the package metadata and its independent payload versioning.

Compiler model layers

Readers parse source formats into typed models. Passes normalize or lower those models, and writers emit target artifacts. The .pio.json field reference is in the .pio.json format chapter.

PowerIO keeps balanced and multiconductor models as separate types. A .pio.json document stores one model payload with provenance, diagnostics, validation results, and lowering history.

Model families

PowerIO keeps two concrete static-grid IR families distinct. They share conventions while keeping separate types; code that needs both holds a .pio.json document rather than a union struct.

BalancedNetwork

powerio::BalancedNetwork is the scalar positive sequence model for transmission power flow, OPF, matrices, and graph analysis. Every electrical quantity is a single f64, with no phase or conductor dimension. Source bus IDs are not dense matrix indices; the dense solver view is derived separately and preserves source IDs. Loads and shunts have separate records rather than fields folded onto bus rows.

MulticonductorNetwork

powerio_dist::MulticonductorNetwork is the wire coordinate model for conductor level distribution. Bus IDs are strings; terminals are ordered string names; every element carries a terminal map; grounding is explicit; units are SI and radians. A neutral carries grounding and reduction semantics beyond a phase label. Format defaults and inferred facts are tracked, and unsupported objects are preserved rather than dropped.

A balanced model cannot represent conductor-level asymmetry; a multiconductor model carries terminal and grounding data that has no place in a positive sequence struct. The two families never merge into one struct.

BMOPF JSON is a strict case format for the distribution family. The .pio.json document uses the same MulticonductorNetwork model and wraps it with metadata: model kind, provenance, source maps, diagnostics, validation, and lowering history. The .pio.json chapter explains why the document is not a case format.

The .pio.json document

powerio_pkg::NetworkPackage is the implementation type for a .pio.json document. It records how a source was interpreted. Language bindings can pass the document without guessing whether it holds balanced or multiconductor data.

A .pio.json document always carries:

• powerio_version (semver), the powerio release that wrote the document;
• producer metadata;
• model_kind, explicit and authoritative;
• model, the typed model payload, tagged by kind;
• origin and sources;
• source_maps;
• diagnostics;
• validation;
• summary;
• lowering_history;
• optional operating_points;
• optional study commits;
• optional derived metadata.

operating_points is a format neutral series of replayable field updates over the document’s single static model payload. Materializing one point returns a static document with those updates applied and the series cleared. GO Challenge 3 document construction fills this block from time_series_input: the balanced model JSON holds the first interval, while every interval is available as an operating point.

For balanced model JSON, NetworkPackage::attach_normalized_solver_table_metadata records compact metadata for powerio::BalancedNetwork::to_normalized_solver_tables(): pass name, units, row counts, dense bus ids, reference/component indices, branch to arc indices, and source row provenance. The document does not duplicate the full table rows; it records enough metadata for a compiler cache or sidecar artifact to verify table identity.

Explicit model kind

model_kind is a standalone, authoritative field: a reader branches on it rather than inferring the model kind from which field is present. The reader requirements are in the .pio.json format chapter.

Model JSON stability

The model JSON changes are document changes, covered by the one powerio_version. Model rows carry stable uid identities that operating point updates resolve against. The bump rules are in the .pio.json format chapter.

Provenance and source maps

Origin distinguishes an in-memory model, a single file (with or without retained source), a folder dataset, a partially decoded binary, a derived product, or a composite. A SourceMapEntry points from a model field to its source with an element_path, a SourceRef into a declared source, a mapping_kind (exact, defaulted, inferred, converted_units, lowered, aggregated, split, synthetic, retained_extra), and a confidence. Balanced source_ref.field values use canonical model field names. Parser bookkeeping that should not live in the model JSON (retained source text, default-materialization records) is lifted into this layer rather than the raw model JSON.

Structured diagnostics

Every finding carries a stable dotted code, a severity (debug, info, warning, error, fatal; worst-last so a set’s dominant severity is its max), the stage it came from, a human message, and where known an element_path, a source_ref, a details object, and a suggested_action. The structured record is primary; human-readable warnings are rendered from it. Codes are namespaced by leading segment (PARSE, READ, IR, VALIDATE, FIDELITY, LOWER, EMIT, BINDING, PARTNER, PERF), with the conventional shape NAMESPACE.SOURCE_OR_TARGET.SPECIFIC.

Lowering

Each pass that transforms one model into another appends a LoweringRecord (input and output kind, options, assumptions, approximations, dropped fields, diagnostics, validation status) to lowering_history. The record makes the transformation explicit.

powerio_pkg::lower_multiconductor_to_balanced lowers transparent three phase MulticonductorNetwork values into BalancedNetwork using the FortescuePowerInvariant sequence convention. Neutral conductors are Kron reduced before the sequence transform. One wire and two wire inputs, transformers, untyped objects, missing phase references, and closed switches return structured LOWER.MULTI_TO_BALANCED.* diagnostics. NetworkPackage::lower_multiconductor_to_balanced returns a derived balanced document and appends the record. This pass is explicit only; readers, writers, matrix builders, bindings, and MCP operations do not run it implicitly.

Operating point materialization

NetworkPackage::materialize_operating_point(index) clones the document, applies one point’s field updates to the typed model JSON, clears operating_points, drops stale source maps and diagnostics for changed fields, recomputes validation, and records a LoweringRecord with pass = "materialize-operating-point". If the document already carried normalized solver table metadata, the metadata is rebuilt for the updated static model JSON.

Versioning

The metadata and model JSON versioning policies are in the .pio.json format chapter.

.pio.json format

A .pio.json file stores one typed network model payload and the record of how it was produced. The model field contains the JSON representation of either powerio::BalancedNetwork (balanced) or powerio_dist::MulticonductorNetwork (multiconductor). The document metadata records provenance, source maps, structured diagnostics, validation results, lowering history, optional operating points, and optional study commits. powerio_pkg::NetworkPackage is the implementation type; Compiler model layers describes the payload types.

Purpose

A MATPOWER or OpenDSS file states the case; it cannot state how a parser read it: which fields were defaulted or inferred, what validation found, or how a multiconductor model was lowered to a balanced one. The metadata records that work next to the model, so a downstream tool can audit a conversion instead of trusting it.

The .pio.json document is also the handoff object between PowerIO consumers: one artifact whose model kind is explicit, with provenance intact.

.pio.json is not a case format

Case formats move cases between tools: MATPOWER, PSS/E, OpenDSS, PMD JSON, BMOPF, GOC3, and the other rows in the conversion tables. PowerIO reads and writes those formats at converter boundaries. A .pio.json document is PowerIO’s compiled artifact: the model plus the record of how that model was produced.

Pick a case format by what the receiving tool reads. Use .pio.json when the receiving consumer is PowerIO or a binding that wants provenance, diagnostics, operating points, and the explicit model kind. Use BMOPF, OpenDSS, PMD JSON, or another supported case format when the next tool expects that format.

powerio-json is bare balanced BalancedNetwork JSON, without package metadata or source maps. Version 0.7 removes it from advertised CLI file formats. Use Network::to_json and Network::from_json for model JSON. The C ABI exposes the same operations as pio_to_json and pio_from_json.

ABI v4 continues to accept powerio-json in pio_parse_str and pio_to_format. Those format tokens are compatibility aliases. Removing them requires a future C ABI version change.

Versioning

Every document powerio authors states one number, powerio_version: the powerio release that wrote it. There is no separate schema number for the document, the payload, or the model JSON. A .pio.json file is a regenerable snapshot, so the reader’s only versioning job is telling the caller when a file needs regenerating.

• A reader accepts its own lineage: the major and minor pair while the major is 0, the major alone afterwards. That is what cargo and Pkg already mean by a 0.x bump. Anything else is rejected with an error naming the release that wrote the document and the release that must regenerate it.
• A document that states no powerio_version came from 0.8.x or earlier. The reader names that release rather than reporting a missing field.
• A reader tolerates unknown top-level fields (they are ignored without error), so a same lineage document from a newer producer still loads.

The generated JSON Schema for the document is served at https://powerio.dev/schema/pio-package/0.2/schema.json; the $id names that location. It embeds the model JSON types, so it validates complete .pio.json documents. It does not define a standalone case format.

Metadata Reference

Explicit model kind

model_kind is a standalone top-level field and is authoritative. A reader must branch on it and must not infer the model kind from which field is present. The model JSON is additionally self-describing: model is tagged by kind, so model.kind and model_kind carry the same value. NetworkPackage::kind_is_consistent asserts the two agree; a reader should reject a document where they disagree.

"model_kind": "balanced",
"model": { "kind": "balanced", "balanced_network": { "...": "..." } }

model_kind values: balanced, multiconductor (the enum is non-exhaustive; later families can be added).

The Model JSON

Each payload is what its Rust model serializes; changes to it are changes to the document and follow the powerio_version rules above. The generated JSON Schema is derived from the serde models and checked in CI against the committed docs/schema/**/schema.json files. The model’s rustdoc is the field reference, and the balanced payload’s serialized form is additionally held to a committed golden file by powerio/tests/snapshot_schema.rs.

The balanced model JSON

model.balanced_network is the serde form of powerio::BalancedNetwork, stamped when model_kind is balanced: the scalar positive sequence transmission model. The tables are buses, loads, shunts, branches, switches, generators, storage, hvdc, transformers_3w, and areas, alongside name, base_mva, base_frequency, source_format, and optional solver metadata. Units follow the MATPOWER conventions: MW and MVAr power, per unit voltage magnitudes and impedances on the system base, degree angles. Every element carries an extras map for source format fields the model does not name. The field reference is the powerio::BalancedNetwork rustdoc.

The multiconductor model JSON

model.multiconductor_network is the serde form of powerio_dist::MulticonductorNetwork, stamped when model_kind is multiconductor: the wire coordinate distribution model, in SI units with radian angles. Compiler IR describes the model family. The field reference is the powerio_dist::MulticonductorNetwork rustdoc. Do not extract this object as a distribution case file. Use .pio.json for PowerIO artifacts; when a receiving tool expects BMOPF, PMD JSON, or OpenDSS, write that case format through powerio convert.

Row identity

Every row of every balanced model table except areas carries a uid string: the source record uid where the format defines one (GOC3), and a {table}:{row} value synthesized at document build otherwise. A synthesized uid records the row the element had when the document was built and sticks to the element from then on. Uids are unique per table; a duplicate is a validation error. Operating point updates resolve against these identities (below). Rows in documents written before 0.1.1 carry no uid, which is what keeps their row-addressed operating points valid.

Operating points

operating_points records a time axis and an ordered list of model field updates. A point names a table, a row identity and/or a zero based row, and the fields to overwrite. Materializing a point clones the static model, applies those field updates, and clears operating_points in the returned document.

Updates resolve by identity first. When the referenced table carries uid values, element.source_uid is authoritative: it selects the row, a present element.row must agree with the resolved row, and an unknown or duplicated uid is an error (reported by validation and fatal to materialization). A producer that knows the identity can omit row entirely. When the table carries no uids (documents written before 0.1.1), source_uid is advisory and row addresses the update alone. An update may not overwrite uid itself, and an element ref with neither row nor source_uid does not parse.

The block shape is:

GO Challenge 3 documents use this block for the scheduling time series. The static model reflects the first interval that can be represented by BalancedNetwork; operating_points carries replayable updates for every interval. NetworkPackage::materialize_operating_point(index) returns a new static document with origin.kind = "derived" and origin.pass = "materialize-operating-point".

"operating_points": {
  "time_axis": { "periods": 2, "duration_hours": [1.0, 1.0], "labels": ["1", "2"] },
  "points": [
    { "index": 0, "updates": [] },
    { "index": 1,
      "updates": [
        { "element": { "table": "loads", "row": 0, "source_uid": "device_1" },
          "fields": { "p": 12.5, "q": 3.2 } }
      ] }
  ],
  "metadata": { "source_format": "goc3-json" }
}

Study commits

study stores ordered cumulative edits to a balanced model payload. Materializing commit k applies commits 0 through k, clears the study and operating point blocks, and returns a static package. Study commits differ from operating points, which are independent overlays. See Study blocks for edit kinds, identity resolution, materialization, and language APIs.

Derived metadata

derived.normalized_solver_tables records the compact identity metadata for powerio::BalancedNetwork::to_normalized_solver_tables() without embedding every table row in the document. The full tables are a derived artifact; this metadata lets a compiler cache prove it was built from the same lowering pass and row order.

The block carries:

• pass: "balanced-to-normalized-solver-tables";
• units: per unit power, per unit voltage, radian angles, per unit impedance and admittance, zero based dense indices;
• row_counts: counts for buses, loads, shunts, branches, switches, arcs, generators, storage, and HVDC rows;
• bus_ids, reference_bus_indices, and component_labels;
• branch_from_arc_indices and branch_to_arc_indices;
• source_rows: source row indices for rows that survived normalization, with null for synthetic rows such as 3-winding star buses and branches.

Diagnostics

Each diagnostic carries a stable dotted code, a severity (debug, info, warning, error, fatal; ordered worst-last), the stage it came from (parse, read, canonicalize, validate, lower, emit, bind, partner), a human message, and where known an element_path, a source_ref, a details object, a suggested_action, and a safe_to_ignore list. Code namespaces by leading segment: PARSE, READ, IR, VALIDATE, FIDELITY, LOWER, EMIT, BINDING, PARTNER, PERF.

Source maps

A source_map entry records where a canonical field came from: an element_path (a JSON pointer, or a best-effort locator in v0.1), a source_ref into a declared source, a mapping_kind (exact, defaulted, inferred, converted_units, lowered, aggregated, split, synthetic, retained_extra), and a confidence (exact, high, medium, low). Balanced documents emit source maps for stable bus, load, shunt, branch, and generator fields. Balanced source_ref.field values use the same canonical field names as the model JSON, so they can be compared directly with element_path. When a source format folds several canonical elements into one source row, the source map records that relation with another mapping kind; MATPOWER load and shunt fields use mapping_kind = split and point to the bus record while keeping fields such as p, q, g, and b. Values that the source format does not carry are not mapped as exact; MATPOWER base_frequency has no source map. When a multiconductor network is written as .pio.json, its defaulted fields lift into source maps with mapping_kind = defaulted, and its retained source becomes origin.retained_source. Validation diagnostics attach the matching source_ref when the document has a source map for the reported field.

NetworkPackage::lower_multiconductor_to_balanced(options) returns a new balanced document with origin.kind = derived and origin.pass = "multiconductor-to-balanced". It preserves the parent lowering_history and appends a LoweringRecord whose options, assumptions, approximations, dropped fields, diagnostics, and validation status describe the pass. Lowered balanced source maps use lowered, aggregated, converted_units, synthetic, and defaulted mapping kinds. The pass is never implicit during .pio.json readback, format conversion, matrix construction, bindings, or MCP operations.

Example

{
  "powerio_version": "0.9.0",
  "producer": { "tool": "powerio", "version": "0.8.0" },
  "model_kind": "multiconductor",
  "model": {
    "kind": "multiconductor",
    "multiconductor_network": {
      "base_frequency": 60.0,
      "loads": [
        { "name": "l1", "bus": "b1", "configuration": "wye",
          "voltage_model": { "model": "zip", "v_nom": [230.0], "alpha_z": [0.5], "...": "..." } }
      ]
    }
  },
  "origin": { "kind": "file", "format": "dss", "retained_source": true },
  "sources": [ { "id": "src0", "kind": "file", "format": "dss" } ],
  "source_maps": [
    { "element_path": "/model/multiconductor_network/vsource.source#basekv",
      "source_ref": { "source_id": "src0", "field": "basekv" },
      "mapping_kind": "defaulted", "confidence": "high" }
  ],
  "validation": { "status": "ok", "counts": { "fatal": 0, "error": 0, "warning": 0, "info": 0, "debug": 0 } },
  "summary": { "elements": { "buses": 1, "loads": 1 }, "units": { "power": "W/var", "angle": "radians" } }
}

Geographic and display data

PowerIO stores coordinates when a supported source provides them. Coordinates are optional; readers do not invent them, and network writers without a coordinate representation report the loss.

PowerWorld .pwd files are display data rather than network cases. Parse them with parse_display_file or parse_display_bytes rather than the network parser.

Coordinate fields

Both network model families expose the same JSON shape:

#![allow(unused)]
fn main() {
pub struct Location {
    /// Longitude for geographic coordinates.
    pub x: f64,
    /// Latitude for geographic coordinates.
    pub y: f64,
    /// Point provenance when it differs from the network default.
    pub kind: Option<CoordsKind>,
}

pub enum CoordsKind { Source, Synthetic, Manual, Derived }

pub struct GeoMeta {
    pub space: CoordinateSpace,
    pub kind: Option<CoordsKind>,
}

pub struct Canvas {
    pub width: Option<f64>,
    pub height: Option<f64>,
    pub units: Option<String>,
}

pub enum CoordinateSpace {
    Geographic { crs: Option<String> },
    Projected { crs: Option<String> },
    Diagram { canvas: Option<Canvas> },
    Unknown,
}
}

Balanced networks use powerio::geo::{Location, CoordsKind, CoordinateSpace, GeoMeta, Canvas} through BalancedNetwork.geo and Bus.location. Multiconductor networks use the matching powerio_dist::geo types through MulticonductorNetwork.geo and DistBus.location. A package serialization test keeps the two JSON shapes identical. Branches carry optional polyline routing (Branch.route, DistLine.route) when a source provides intermediate geometry; endpoint only rendering derives from the bus locations.

The coordinate space belongs to the network. For geographic coordinates, x is longitude and y is latitude in GeoJSON axis order. A missing CRS in a geographic space means EPSG:4326. kind records whether coordinates came from the source, a generated layout, a manual edit, or a derived transform.

Harvest and emit

Readers promote coordinates into location and stamp the space; promotion removes the raw keys from extras. Writers emit from location.

MATPOWER, PSS/E, PowerModels, egret, GOC3, PSLF, and Surge carry no geometry. Writing a located case to one of them reports the dropped locations, the same behavior base_frequency has; powerio geo extract writes the sidecar as the escape hatch.

The geographic document

Coordinates also arrive and leave as files of their own: a Buscoords CSV next to a DSS master, a GeoJSON export from a GIS tool, a layout computed by a renderer. The container is GeoLayer, surfaced as DisplayData::Geo beside the PowerWorld .pwd display path.

The canonical form is a GeoJSON FeatureCollection with one foreign member, suggested extension .geo.json:

{
  "type": "FeatureCollection",
  "powerio_geo": { "version": "0.1.0", "space": "geographic", "kind": "source" },
  "features": [
    { "type": "Feature",
      "geometry": { "type": "Point", "coordinates": [-80.05, 34.20] },
      "properties": { "target": "bus", "id": "312", "uid": "buses:11" } },
    { "type": "Feature",
      "geometry": { "type": "LineString", "coordinates": [[-80.05, 34.20], [-80.10, 34.30]] },
      "properties": { "target": "branch", "uid": "branches:4", "from": "312", "to": "410" } }
  ]
}

When the space is geographic this is valid RFC 7946 GeoJSON, so GIS tools open it directly.

Reading is tolerant; writing is canonical. GeoLayer::parse_bytes takes bytes plus a file name hint and touches no filesystem. It accepts headerless buscoords CSV (bus, x, y), CSV and JSON records with aliased field names (bus_i/bus/id, lat/latitude/y, lon/lng/longitude/x, branch endpoint pairs), and GeoJSON Point and LineString features. Features reference elements by up to three key fields, matched in order: uid, then id, then case insensitive name. Branch routes additionally fall back to the unordered (from, to) bus pair. A bare integer branch id (branch, branchid, branchnumber, catsid) is accepted on read as a 1-based positional row alias and never written; the durable identity is the payload uid. A branch key never reads from a bare id property, because GIS exports and RFC 7946 tooling put a feature row counter there.

Network::geo_layer() extracts, and Network::apply_geo_layer(&layer) applies and returns a GeoApplyReport with the matched and unmatched feature counts plus unlocated_buses and unlocated_branches, the elements that carry no geometry when the pass ends. The two together tell a layer that matched nothing from a model that needed nothing; report.require_located() is the strict caller’s one line check. The multiconductor equivalents attach through powerio-pkg (dist_geo_layer, apply_dist_geo_layer). The CLI wraps the same surface:

$ powerio geo extract case.aux -o case.geo.json
$ powerio geo apply case.m layout.csv -o placed.m
$ powerio geo convert buscoords.csv -o case.geo.json

PowerWorld display files

The .pwd reader returns DisplayData::PowerWorld with a PwdDisplay: canvas dimensions, a timestamp, and substation symbols with number, name, and diagram coordinates.

Four helpers connect it to the geo model. geo_layer_from_pwd lifts the substation symbols into a diagram space GeoLayer (also reachable as powerio geo extract case.pwd); geo_layer_from_aux_substations lifts the Latitude and Longitude columns of an aux Substation table into a geographic one; apply_substation_points joins either onto buses through the SubNum extras key; and pwd_mercator_to_lonlat is a documented, approximate inverse of the projection PowerWorld’s auto generated layouts use, for consumers that want to place a diagram on a map.

A bus row of a complete case export carries its own coordinates as well. The aux reader promotes the substation Latitude:1/Longitude:1 pair, and the bus’s own bare Latitude/Longitude pair, into Bus.location; a promoted pair leaves extras.

Rust uses parse_display_file and parse_display_bytes. Python exposes the same names and returns DisplayData(kind="powerworld", data=PwdDisplay(...)). Display files do not pass through BalancedNetwork, Conversion, or .pio.json.

Distribution graph projection

MulticonductorNetwork::graph() returns a bus and terminal graph without requiring coordinates. Python exposes dist_net.graph(), and the C dist feature exposes pio_dist_graph_json. Graph topology and geographic placement remain separate data.

PowerIO stores and transports coordinates; it does not compute them. Synthetic layout of a coordinate free case is renderer math and stays in the consumer, which can write the result back with kind = synthetic so the provenance survives.

The C ABI exposes the document as strings: pio_geo_parse normalizes a tolerant sidecar to the canonical form, pio_geo_extract and pio_geo_apply work on a parsed network handle (apply returns a new handle whose warnings carry the match report), and pio_dist_geo_extract/pio_dist_geo_apply are the multiconductor equivalents. Python mirrors the surface with parse_geo and geo_layer()/apply_geo_layer() on both network types.

Study blocks

A study block stores cumulative edits to a .pio.json package. Rust, C, and Python can read the block and materialize a study commit. The CLI, authoring helpers, Julia bindings, and geographic edits are tracked in #185.

Each study commit applies after every preceding commit. Materializing commit k applies commits 0 through k to a fresh copy of the base payload. This avoids numerical drift from repeatedly modifying an already materialized network.

Study commits and operating points

Operating points and study commits have different update rules:

• An operating point independently overwrites fields on existing payload rows. Materializing point k ignores every other point.
• A study commit applies deltas and field updates after all preceding commits. A demand delta can address a bus that has no load row.

The two blocks share ElementRef identity resolution. They do not share time axis or accumulation semantics.

Document shape

"study": {
  "label": "congestion sweep",
  "created_at": "2026-07-03T18:20:00Z",
  "commits": [
    { "edits": [
        { "kind": "demand_delta",
          "bus": { "table": "buses", "source_uid": "buses:1" }, "p_mw": 50.0 },
        { "kind": "rating_delta",
          "branch": { "table": "branches", "source_uid": "branches:2" },
          "delta_mw": -210.0 }
    ]}
  ],
  "app": { "tellegen": { "formulation": "dcopf", "options": { "shed": false } } }
}

StudyBlock contains optional label, author, created_at, and base_operating_point fields; an ordered commits array; an app map; and free form metadata. base_operating_point selects a snapshot from the package’s operating point series before applying the first commit.

Each StudyCommit contains optional label and created_at fields, its edits, and free form metadata. Packages without a study block retain their existing behavior and metadata schema version.

Edit kinds

StudyEdit supports these tagged variants:

• demand_delta { bus, p_mw, q_mvar? } adds demand at a bus, including a bus with no load rows.
• rating_delta { branch, delta_mw } adds to a branch thermal rating.
• set_fields { update } wraps an ElementUpdate and overwrites fields on one payload row.
• An unknown kind is retained during parsing. Materialization returns an error rather than ignoring the edit.

References resolve by identity before row number, as operating point updates do. Producers should set source_uid; row remains a compatibility fallback and consistency check. Package validation resolves every reference in every commit without modifying the package.

Materialization

NetworkPackage::materialize_study_commit(k) applies commits 0 through k to a copy of the model payload. It removes the study and operating point blocks from the result and records the operation in lowering_history. The returned package is static and can be converted, projected into matrices, or passed to a problem instance builder.

A demand delta is divided among the in service load rows at its bus in proportion to their existing demand. This preserves each load’s share and power factor. If the bus has no in service load or its total demand is zero, materialization appends a synthetic load with UID study:load:{bus_uid} and marks it as synthetic in metadata.

Study materialization accepts balanced model payloads. A multiconductor payload returns a validation error.

Application metadata

The app map stores application specific data that PowerIO retains without validation. A solver can keep its formulation and options under a private key, for example app["tellegen"]. Consumers that only need the network states can ignore this map.

Row identity

Study edits address payload rows by UID:

• Parsing the same source bytes produces the same UIDs.
• Generated UIDs use {table}:{row} when the source format has no UID.
• A source UID is retained rather than replaced by a generated value.
• UIDs survive a network JSON round trip.

ensure_payload_uids(&mut Network) adds missing UIDs before a consumer builds its own edit state. Use UIDs for stored references and row order for display.

APIs

• Rust: NetworkPackage::study, with_study, set_study, clear_study, materialize_study_commit, materialize_balanced_study_commit, and ensure_payload_uids.
• Python: pkg.study() and pkg.materialize_study_commit(k).
• C: pio_package_study_json and pio_package_materialize_study_commit.

The block, materialization rules, and UID behavior are tracked in #181.

Format fidelity and validation

PowerIO validates readers and writers against independent tools and committed round trip tests. The top level fidelity table summarizes the supported directions; the conventions and evidence are below.

Conventions

powerio’s numeric conventions match MATPOWER and PowerModels.jl. The reference implementations and the matching powerio code:

egret’s own MATPOWER parser uses the same reductions (bus type as matpower_bustype, polynomial coefficients reversed to a {degree: coefficient} map, piecewise to [[mw, cost], ...], impedances left per unit), which is why a MATPOWER case taken through powerio to egret JSON matches egret’s direct import.

Validation

The harness script benchmarks/run_validation.sh checks powerio against five independent tools. Every classic text reader and writer runs under an oracle: the conversion matrix covers MATPOWER, PSS/E, and egret sources against all five legacy text targets, every PowerWorld output is read back and bridged to PowerModels JSON, and the PMread leg covers the PowerModels JSON read side. pandapower JSON and PyPSA CSV folders have dedicated import validators because pandapower has its own JSON schema and PyPSA is a directory format; both validate the write direction only — the pandapower JSON and PyPSA readers have no external oracle. They, GO Challenge 3 JSON, Surge JSON, and the remaining source/target pairs (PowerModels JSON and PowerWorld sources into the non-PowerModels targets) rest on the Rust round trip suite.

• PowerModels.jl (validate_powermodels.jl, validate_psse.jl, core_json.jl). Reads MATPOWER, PowerModels JSON, and PSS/E. The MATPOWER to PowerModels JSON path is checked field by field after per unit normalization; the others by element counts and demand/generation/shunt totals.
• egret (validate_egret.py). The oracle for egret output, which PowerModels cannot read: it loads powerio’s egret JSON with egret.data.model_data.ModelData and compares counts, totals, and generator cost curves.
• ExaPowerIO.jl (validate_exapowerio.jl). Reads MATPOWER through powerio’s C ABI and compares value for value.
• pandapower (validate_pandapower.py, validate_pandapower_converter.py). Cross-checks MATPOWER parse/\(Y_{\mathrm{bus}}\) and imports powerio’s pandapower JSON output back into pandapower, comparing counts and \(Y_{\mathrm{bus}}\).
• PyPSA (validate_pypsa.py). Imports powerio’s PyPSA CSV folder output and checks counts, totals, line r/x/b rebased from ohms on the bus0 voltage, and transformer r/x/tap_ratio/s_nom rebased from the transformer s_nom base; a line/transformer split mismatch fails the case.

The conversion matrix

benchmarks/validate_matrix.py converts each source to every legacy text target and checks the electrical core of the output (bus/branch/generator counts and the per unit demand, generation, and shunt totals) against the source’s own core, read by an independent oracle. The diagonal is checked byte exact: writing back to the source format reproduces the file. Sources use the real native files where they exist (the vendored PSS/E .raw and egret .json) and representative MATPOWER cases otherwise: basic (case9), shunts and transformers (case14, case30), size (case118, case2869pegase), HVDC with a mixed piecewise/polynomial gencost (t_case9_dcline), and a piecewise-cost case (pglib_opf_case5_pjm).

All 65 legacy text cells pass (13 source cases × 5 targets). The core is preserved by every writer regardless of fidelity tier, so it is the invariant checked across the whole matrix; cost, HVDC, and angle limits are tier specific and covered by the dedicated checks above and the Rust suite. The pandapower JSON and PyPSA CSV validators run alongside this matrix and are reported as separate legs.

Running it

cargo build --release -p powerio-capi
python3.12 -m venv .venv
.venv/bin/python -m pip install --upgrade pip maturin -r benchmarks/requirements.txt
env VIRTUAL_ENV=$PWD/.venv .venv/bin/maturin develop --release
julia --project=benchmarks -e 'using Pkg; Pkg.instantiate()'
bash benchmarks/run_validation.sh

The oracle tools (PowerModels.jl, egret, ExaPowerIO.jl, pandapower, PyPSA) are benchmark scoped: they are declared only in benchmarks/Project.toml and benchmarks/requirements.txt, and the powerio package itself has no dependency on them. benchmarks/run_validation.sh requires the Python oracles to import in the selected Python 3.11+ environment; a missing PyPSA, pandapower, or egret import is a setup failure.

Known limits

Write side losses are reported in Conversion::warnings; the pandapower and PyPSA readers itemize what they ignore in Parsed::warnings (read_warnings in Python), naming the table and counting the affected rows. convert_file/convert_str fold the read warnings into Conversion::warnings. Package builders lift balanced reader warnings into structured diagnostics with code READ.TRANSMISSION.PARSE_WARNING. GridFM package reads use READ.GRIDFM.FIDELITY_WARNING; distribution packages already carry READ.DIST.PARSE_WARNING.

• PSS/E reads revisions 33, 34, and 35. 3-winding transformers are kept as typed records and star-lowered into \(Y_{\mathrm{bus}}\)/connectivity by the indexed view; two-terminal DC lines map to the neutral HVDC model. A switched shunt keeps its steady-state susceptance BINIT as the shunt b and carries its mode, voltage band, regulated bus, and step blocks. A 2-winding transformer’s magnetizing susceptance round-trips through MAG2 (\(\mathrm{CM} = 1\)). Impedances are assumed on the system base (\(\mathrm{CZ} = \mathrm{CW} = 1\)).
• PowerWorld .aux is read and written. .pwb binary cases are read only, and .pwd display files parse through the separate display API. .aux carries no system base, so the reader defaults to 100 MVA. No third party .aux reader exists, so that writer is validated by powerio’s own read back plus a PowerModels JSON bridge. The .pwb layouts are reverse engineered; the decode evidence and coverage matrix are maintainer notes at powerio/src/format/powerworld/FORMAT.md.
• PSLF .epc is read and written. The reader maps the static power flow core: buses, lines, two- and three-winding transformers, generators, loads, fixed shunts, controlled shunts at initial g/b, and limited two-terminal DC records. Three-winding transformers are kept as typed records and star-lowered into \(Y_{\mathrm{bus}}\)/connectivity by the indexed view. Unsupported sections stay in the retained source text and emit warnings.
• MATPOWER canonical output (for a case that did not originate as MATPOWER) omits dcline; the byte exact echo path keeps it when the case was read from MATPOWER. Storage is written as an mpc.storage block.
• egret output drops HVDC and storage. The reader takes the power flow ModelData subset (numeric bus ids, scalar values); unit commitment cases (system.time_keys) are rejected.
• pandapower JSON writes the power flow core as split oriented pandapowerNet tables. Line ohms are referred to the from bus voltage, as pandapower’s build_branch reads them; a bus with baseKV 0 writes vn_kv set to \(1\) (warned) so the per unit impedances survive. A branch with a tap, a shift, or terminals on two voltage levels becomes a trafo row with tap_changer_type = "Ratio"; its MATPOWER charging b rides as one bus shunt per terminal (warned, \(Y_{\mathrm{bus}}\) exact) because pandapower’s magnetizing model is inductive only. The file is labeled with f_hz set to \(50\) and c_nf_per_km compensated, so a 60 Hz source keeps its exact \(Y_{\mathrm{bus}}\). Reference buses without a generator get an ext_grid row, which reads back as a Ref generator. The writer also warns on dropped HVDC, storage, capability columns, angle limits, rate B/C, non-finite values (written as JSON null), and costs poly_cost cannot carry. The reader models ratio, ideal, and pandapower 2.x tap changers, off-nominal vn_hv_kv/vn_lv_kv, lv side taps, and shunt vn_kv scaling; ZIP load composition, line shunt conductance, magnetizing branches, tabular tap changers, reactive cost coefficients, and every other non-empty table warn with row counts.
• PyPSA CSV folders are canonicalized directory outputs rather than byte exact text conversions. Covered: static buses, generators, loads, lines (ohms on the bus0 voltage, as PyPSA computes them), transformers (rebased between the system base and the transformer s_nom), shunts, storage units, and base MVA. The reader maps links to HVDC with a warning, requires v_nom and balanced CSV quoting, and warns on stores, nonzero g, and every CSV it does not read (time series, carriers). The writer keys tables by bus name, falling back to the numeric id when names collide (warned), and warns on dropped HVDC, q limits, mbase, transformer angle limits, rate B/C, isolated buses, non-finite p limits, and slackless or normalized networks. Nonnumeric bus names read back as dense synthetic ids with the originals on Bus.name.
• GO Challenge 3 JSON reads ARPA-E GO Competition Challenge 3 input data into the balanced transmission model. BalancedNetwork is static, so the reader maps the first time interval into generator/load bounds and status fields, keeps the original JSON for byte exact source echo, and warns about scheduling data left in the retained source. There is no canonical GOC3 writer from an arbitrary BalancedNetwork; TargetFormat::Goc3Json only succeeds as a same format source echo. When a GOC3 BalancedNetwork is wrapped in .pio.json, powerio-pkg extracts the full input time axis into operating_points. Materializing one point applies those updates to the static model JSON and clears the series.
• Surge JSON reads and writes the versioned surge-json network document. The reader maps buses, loads, fixed shunts, branches, generators, storage, and HVDC links into BalancedNetwork, retains the original source for same format echo, and warns about source sections that stay only in the retained document. The writer emits a canonical Surge network body for the supported power flow core; richer MATPOWER generator capability or ramp columns and unsupported cost shapes are reported in Conversion::warnings.
• DeepMind OPFData JSON reads one raw JSON document from a FullTop or N-1 release into the balanced transmission model. Topology, limits, loads, shunts, and quadratic costs come from grid; solved bus voltages, generator dispatch, and branch flows come from solution. Powers and ratings are converted from per unit, angles from radians, link indices from zero based to one based, and flow columns from [pt, qt, pf, qf] into the canonical terminal order. Original bus IDs/names, areas/zones, and frequency are absent, and solver initial generator values are distinct from the solved snapshot, so cross-format reads report those facts. The adapter is driven by feature widths and the row/link counts in each file, not by a case name registry or expected element counts. The same path therefore covers all published grid families (14 through 13,659 buses) and both FullTop and N-1 examples; generator and branch outages are represented by absent rows and links and are validated against that example’s solution topology. The published releases are derived from PGLib-OPF cases, but the reader does not use PGLib case names or a case registry. A document from another source is accepted when it follows the same object layout, feature column order, units, and link rules. The paper’s Appendix A is the published format definition and the PyTorch Geometric loader is the executable reference. No separate JSON Schema or format version marker is published, so documents that change this contract are rejected by the reader’s shape and topology checks. Unrecognized object fields remain in the retained source and produce a projection warning instead of being silently discarded or making same format echo impossible. The raw source echoes byte exactly; there is no canonical writer, .pt cache reader, archive reader, downloader, or batch directory API.
• gridfm (read, the gridfm feature in powerio-matrix) reconstructs a BalancedNetwork from the gridfm-datakit Parquet dataset: lossy, but it recovers everything a power flow needs. That is bus types/voltages/limits, nodal load and shunt totals, generator dispatch and bounds, branch r/x/b/tap/shift/rate_a/angle limits, and baseMVA; it can’t recover original bus ids (synthesized 1..n), per element load/shunt granularity (folded one synthetic element per bus), piecewise/cubic gen costs (read as none), or HVDC/storage. Because the writer stores the effective tap, a branch with unit tap and no phase shift is read back as a line (raw \(\mathrm{tap} = 0\)); a unity ratio, zero shift transformer in the source is thus read as a line (the power flow is identical). The losses are returned as a warnings list on GridfmRead, mirroring Conversion::warnings. The same direction writer is documented in the top level README.

Missing generator costs

PSS/E .raw files carry no generator cost curves. Converting a PSS/E case to MATPOWER writes mpc.gen and omits mpc.gencost with a warning; powerio does not invent zero costs. A workflow that needs costs must pick an explicit policy:

powerio convert case.raw --from psse --to matpower --missing-gen-cost zero -o case.m
powerio dcopf case.m -o out --missing-gen-cost quadratic --default-gen-cost 0.01,2.0,0.0
powerio gridfm case.raw --from psse -o out --missing-gen-cost zero

• preserve: leave missing costs absent (default for conversion and GridFM export);
• require: fail on an in-service generator without cost (default for DC OPF export);
• zero: fill missing rows with a MATPOWER polynomial cost [0, 0, 0];
• quadratic: fill missing rows with --default-gen-cost C2,C1,C0.

--gen-cost-csv overrides costs by generator row before the missing-cost policy runs. The header is gen_index,bus,c2,c1,c0,startup,shutdown: gen_index is zero based in the current generator table, bus must match that generator’s bus id (catching stale tables after reordering), and startup/shutdown default to zero. GridFM stores cp0/cp1/cp2 columns; missing or unsupported costs still write zero columns, and the manifest separates missing_cost_gens, unsupported_cost_gens, zeroed_cost_gens, and synthesized_gen_costs.

Matrix outputs and conventions

powerio-matrix builds sparse matrices and graph views from a parsed BalancedNetwork. Builders take an IndexedNetwork, which maps source bus IDs to dense indices in \([0,n)\).

powerio-prob builds problem instances (matrix free DC OPF and AC OPF input data plus the GOC3 SCOPF instance) and its matrix feature projects those instances into sparse operators. The DC OPF bundle schema is in the DC OPF bundle guide. Per-builder API detail is in the crate docs.

Capabilities

Computing PTDF and LODF matrices requires a linear solve. The stable build_ptdf, build_lodf, and build_ptdf_lodf builders keep the dense grounded inverse path and remain the small case oracle. The option based build_ptdf_lodf_with_options path accepts SensitivityOptions: Dense forces the dense oracle path, Iterative uses preconditioned conjugate gradient on one grounded right hand side at a time, and Auto selects dense up to a reduced dimension of 512 and iterative above it. The iterative path avoids forming the \((n-r) \times (n-r)\) dense inverse; the PTDF/LODF outputs themselves can still be large. The iterative path requires positive finite branch susceptances, so the grounded DC bus susceptance matrix is positive definite after reference coverage is checked; the dense path remains the fallback for nonsingular indefinite cases. Every connected component must contain at least one reference bus. The DC OPF instance bundle (\(A\), \(b\), \(L\), costs, bounds, thermal limits, \(C_g\)) is produced by powerio-prob and documented in the DC OPF bundle guide.

Bp and Bpp are the fast decoupled power flow matrices from MATPOWER makeB. Solvers reduce Bp to PV+PQ buses for active power mismatch to voltage angle updates, and reduce Bpp to PQ buses for reactive power mismatch to voltage magnitude updates. PowerIO exports the full \(n \times n\) matrices so callers can apply their own bus type reduction.

GridFM datasets

The GridFM export is a Parquet dataset under <case>/raw/ with bus_data, gen_data, branch_data, and y_bus_data. A single parsed case writes one scenario. A scenario batch row stacks snapshots that share the same element set and uses the scenario column as the key.

GridFM reads recover bus types, voltages, limits, nodal load and shunt totals, generator dispatch and bounds, branch parameters, and base_mva. They cannot recover source bus IDs, per element load and shunt granularity, piecewise and cubic costs, HVDC, or storage. These losses are returned as warnings.

Conventions

• Weighted bus Laplacian matrices. Stored nonzero off-diagonal entries are negative; diagonals are nonnegative and positive for buses incident to a positive weight branch. For \(L = A \operatorname{diag}(w) A^\mathsf{T}\) with nonnegative branch weights \(w\), \(L_{ii} = \sum_j \lvert L_{ij} \rvert\). This is the M-matrix form an SDDM (symmetric diagonally dominant M-matrix) or Cholesky solver expects once the grounded matrix is positive definite; a consumer can recover an edge weight as \(-L_{ij} > 0\).

• Bus indexing. Source bus IDs are preserved on the model as a newtype and need not be contiguous. IndexedNetwork::bus_index(id) maps them into dense zero based indices in \([0,n)\). An unknown source ID returns Error::UnknownBus.

• Taps and shifts. \(\mathrm{tap} = 0\) means \(\mathrm{tap} = 1\) (Branch::effective_tap). MATPOWER Bp clears bus shunts and line charging, sets tap magnitudes to one, and keeps phase shifts. MATPOWER Bpp keeps bus shunts, line charging, and tap magnitudes while clearing phase shifts. \(Y_{\mathrm{bus}}\) keeps both tap magnitudes and phase shifts.

• Branch shunt admittance is stored per unit. Branch::charging is the stored per terminal admittance when present: g_fr, b_fr, g_to, and b_to are already per unit on the system base. Branch::b is the legacy MATPOWER BR_B total projection for formats that carry only one charging value. Matrix builders use Branch::terminal_charging(), so terminal values feed \(Y_{\mathrm{bus}}\) even when the legacy total is zero or stale.

• FDPF scheme. Scheme selects between the two MATPOWER fast decoupled variants. Xb clears resistance for Bp; Bx clears resistance for Bpp. The default is Bx.

• Zero impedance branches. BuildOptions::skip_zero_impedance controls the builders whose branch denominator can be zero. The default true skips the branch and records the skipped source branch rows in MatrixStats as skipped_zero_impedance and skipped_zero_impedance_branches; false returns Error::ZeroImpedance. Full AC admittance builders use \(r^2 + x^2\); DC incidence and reactance only FDPF variants use \(x\). The gridfm export still zeros its admittance and flow columns for these rows and records dropped_zero_impedance in gridfm_meta.json.

• Reference coverage. IndexedNetwork::check_reference_coverage verifies that every in-service island has a reference bus.

• Susceptance conventions for the DC approximation. DcConvention selects the branch susceptance vector \(b\) and, for the MATPOWER convention, the phase shift injection. The signed incidence matrix \(A\) combines with \(b\) to form the DC bus susceptance matrix \(L = A \operatorname{diag}(b) A^\mathsf{T}\), which feeds PTDF/LODF and the DC OPF matrix projection. \(b\) is positive for an inductive branch, the DC model convention MATPOWER makeBdc uses; the AC series susceptance \(\operatorname{Im}\left(1/(r + jx)\right)\) is its negation.

  The default SeriesImpedance uses \(b = x/(r^2 + x^2)\), so it reads the whole series impedance, plus the phase shift injection vector p_shift. A tap does not scale it. It reduces to \(b = 1/x\) when the branch has no resistance.

  Matpower reproduces MATPOWER’s makeBdc: \(b = 1/(x\tau)\) for a transformer with tap ratio \(\tau\), plus p_shift.

  ReactanceOnly is the textbook \(b = 1/x\) with resistance, taps, and shifts ignored. The resulting \(L\) matches MATPOWER Bp under Scheme::Xb when phase shifts are zero. Reproducing a published result needs it exactly as written, so it stays.

Output

Matrices write as Matrix Market files or stay in memory. A symmetric matrix is stored as its lower triangle with the symmetric header and 1-based indices (io::mtx::write_mtx). The sensitivities command writes <case>_ptdf.mtx, <case>_lodf.mtx, and <case>_sensitivity_meta.json. Use --solver dense|iterative|auto to choose the PTDF/LODF solve path and --drop-tolerance <value> to omit entries with absolute value at or below the tolerance. When the CLI uses the iterative path, it writes retained Matrix Market coordinates through temp files and does not hold the full sparse output in memory. The Rust build_ptdf_lodf_with_options API still returns CsMat values and is intended for outputs that fit in memory. The metadata records the requested solver, the actual solver path, matrix dimensions, nonzero counts, tolerance, and dropped entry counts. The dcopf CLI subcommand bundles its matrix family with a JSON manifest.

The standard case solver property fixture lives at powerio-matrix/tests/fixtures/solver_matrix_stats.json. It records bprime, bdoubleprime, and ybus_imag stats for case9, case14, case30, case57, and case118: n, nnz, min diagonal, M-matrix sign pattern, diagonal dominance margin, zero impedance skips, row sum checks, SPD checks, and a condition estimate when the solver input is SPD.

IndexedNetwork::to_petgraph returns the network as an undirected petgraph graph, one node per bus and one edge per in-service branch. The connectivity report and the radial check are built on it. Use the returned graph directly for other petgraph algorithms.

DC OPF bundle schema

powerio dcopf <case>.m -o <out> assembles a DcOpfInstance and writes <out>/<case>_dcopf/. Rust callers pass an assembled instance to powerio_prob::matrix::write_dcopf_bundle. The directory contains Matrix Market files and dcopf_meta.json.

Conventions

• Format. Matrix Market. Matrices are coordinate real; square symmetric ones (L, L_grounded) use the symmetric header and store the lower triangle only. Vectors are array real general, one value per line.
• Index base. .mtx row/column indices are 1-based (Matrix Market standard). reference_buses in the manifest are 0-based dense bus indices.
• Sign convention. The DC bus susceptance matrix \(L\) uses the positive M-matrix form: stored nonzero off-diagonal entries are negative, diagonals are nonnegative, and \(L_{ii} = \sum_j \lvert L_{ij} \rvert\). An off-diagonal entry is \(L_{ij} = -b_e\) for the branch between \(i\) and \(j\), so a consumer recovers the branch susceptance as \(-L_{ij} > 0\).
• Units. PerUnit by default: power divided by base_mva, cost scaled so it is a function of per unit power: \(q \leftarrow 2c_2 \cdot \mathrm{base}^2\) and \(c \leftarrow c_1 \cdot \mathrm{base}\). Native keeps MW / native cost. The choice is recorded in the manifest.
• Generator costs. The default DC OPF export policy is require: an in-service generator without cost data is an error. Use --missing-gen-cost to explicitly fill missing rows for feasibility tests.
• Reference buses. reference_buses in the manifest lists every grounded bus as a 0-based dense index. Each in-service island needs at least one reference. If several references lie in one island, the bundle fixes all of those voltage angles to zero; it is not a participation factor slack model.
• DC convention. b.mtx holds \(b_e\), positive for an inductive branch, the coefficient in \(f = b_e(\theta_f - \theta_t)\). SeriesImpedance by default: \(b_e = x/(r^2 + x^2)\) plus the phase shift injection p_shift, with no tap scaling. Matpower uses \(b_e = 1/(x \tau)\) plus p_shift. ReactanceOnly (\(b_e = 1/x\), taps and shifts ignored) stays: it is the textbook DC linearization, and reproducing a published result needs it exactly as written. Recorded in the manifest.

Matrices

Vectors

Bus-indexed (length \(n\)): pd (load), gs (shunt conductance, the constant real power a shunt draws at one per unit voltage; a nodal balance subtracts it beside pd), q/c/c0 (cost diag/linear/constant), pmax/pmin (generation bounds), e_r (reference indicator: \(1\) at every reference bus, else \(0\)), p_shift (phase shift injection, all zero unless Matpower + shifters). Branch-indexed (length \(m\)): b (susceptances), fmax (thermal limits; \(0\) means unlimited per MATPOWER), and the radian limits angle_min and angle_max. Generator space data (length \(n_{\mathrm{gen}}\)): q_gen, c_gen, c0_gen, pmax_gen, and pmin_gen.

The constant cost terms c0/c0_gen do not move the argmin; they exist so a consumer reporting objective values reconstructs the full cost.

Generator space is canonical. The nodal q, c, c0, pmax, and pmin files aggregate the generators at each bus. The bounds are the sum of the generator bounds. The cost curves combine by the parallel rule \(q = 1 / \sum_i 1/q_i\), which is the curve of the split that costs least, so it agrees with generator space only while that split stays inside the bound of each generator. A bus with one generator keeps that generator’s curve.

Manifest (dcopf_meta.json)

Schema powerio.dcopf version 0.4.0 writes Matrix Market files plus structured metadata:

• dimensions: n_buses, n_source_branches, n_branch_columns, n_generators, n_reference_buses, and n_grounded_buses.
• index_base: dense = 0 for manifest bus, branch, generator, and reference indices; matrix_market = 1 for .mtx coordinates.
• dc_convention, units, build_options, and zero_impedance. The zero impedance block records the skip flag, denominator rule, skipped count, and skipped source branch rows.
• grounding: reference buses, removed rows and columns, the grounded operator (L_grounded), and the reference selector (e_r).
• operators[]: one entry per emitted operator with name, file, kind, rows, cols, index_space, and units.

The legacy aliases n, m, n_gen, reference_buses, and convention remain for current readers. cost_policy, synthesized_gen_costs, patched_gen_costs, files[], and powerio_version remain top level fields.

Solving with it

The grounded system is the one to factor: L_grounded is SPD when every island has a reference. For DC power flow \(L\theta = p\) with net injection \(p = g - d\), drop all reference_buses entries from \(p\), solve \(L_{\mathrm{grounded}}\theta_{\mathrm{red}} = p_{\mathrm{red}}\), and set each reference angle to \(0\). e_r identifies the grounded buses without parsing the manifest. The full singular \(L\) can be used instead when the net injection sums to zero within each connected component.

An interior point DC OPF solver builds reweighted bus Laplacians each Newton step from the same A and b (only the edge weights change), so A is the durable operator to hand over.

Migrating to 0.7

DC OPF problem data moved from powerio-matrix to powerio-prob. powerio-matrix now owns generic network projections. It does not depend on or reexport powerio-prob.

Replace these 0.6 imports:

use powerio_matrix::{OpfInstance, Units, build_opf_instance};
use powerio_matrix::{DcOpfOptions, write_dcopf_bundle};

with:

use powerio_prob::{DcOpfInstance, DcOpfOptions, Units, build_dc_opf_instance};
use powerio_prob::matrix::{DcOpfBundleOptions, write_dcopf_bundle};

DcOpfInstance stores generators in generator space. Call DcOpfInstance::nodal_generator_data only when a bus space formulation is required; it aggregates the generators at each bus.

The default powerio-prob feature set contains no sparse matrix dependency. Enable matrix for incidence, Laplacian, flow, generator map, and bundle output:

[dependencies]
powerio-prob = { version = "0.7", features = ["matrix"] }

powerio_prob::matrix::write_dcopf_bundle accepts an assembled instance. Cost policy handling happens before assembly. The writer does not read the source network again.

Solver formulations and KKT operators are not part of powerio-prob. Build them in the solver from the indexed instance and optional matrix projections.

C ABI Arrow policy

The C ABI stays handle based. Parsed transmission cases use PioNetwork, distribution cases use PioDistNetwork, and .pio.json documents use PioPackage. Callers get full model transport through JSON, small copied arrays through dense extractors, and bulk typed tables through the Arrow C Data Interface.

Arrow tables

Arrow table ids are append only. Existing ids keep their meaning and column order. Matrix tables added axis metadata without changing their triplet columns:

Matrix schema metadata carries:

powerio.table
powerio.version
powerio.format
powerio.row_axis
powerio.col_axis
powerio.row_count
powerio.col_count
powerio.index_space   # legacy alias, still "solver_bus" for bus indexed matrices

matrix_bus gives bindings a dense matrix row and column map without inferring from solver_bus. It includes the dense index, source bus id, source row, reference flag, and component label. matrix_branch gives incidence column meaning: dense incidence column, source branch row, from bus id, and to bus id. Branches that do not contribute an incidence column, such as self-loops or skipped zero reactance rows, are not on this axis.

Arrow catalog JSON

pio_arrow_catalog_json(errbuf, errlen) returns compact JSON that lets a binding discover the Arrow tables compiled into the C library. It describes the build: available tells whether this library was built with the needed features, and a particular network’s row counts play no part.

Shape:

{
  "powerio_version": "0.9.0",
  "producer": "powerio-capi",
  "tables": [
    {
      "id": 17,
      "name": "bprime",
      "powerio_version": "0.9.0",
      "format": "coo",
      "feature_requirements": ["arrow", "matrix"],
      "available": true,
      "row_axis": "matrix_bus",
      "col_axis": "matrix_bus",
      "units": {
        "value": "per_unit",
        "matrix_index_base": "zero"
      },
      "columns": [
        {"name": "row_index", "type": "int64", "nullable": false},
        {"name": "col_index", "type": "int64", "nullable": false},
        {"name": "value", "type": "float64", "nullable": false}
      ]
    }
  ]
}

Bindings should read the catalog before assuming optional ids exist. The table ids are still exposed as C macros for callers that compile against powerio.h.

Binding policy

Julia keeps copy=true as the default for Arrow tables. That copies primitive columns into owned Julia vectors and releases the producer Arrow structs immediately. copy=false remains opt in and keeps the Arrow owner alive so zero copy views cannot outlive their buffers.

The Julia binding decodes the primitive table shapes listed in the catalog. A new Arrow table requires binding tests for copied and zero copy lifetime behavior.

Problem data boundary

PioNetwork Arrow tables describe a network or a generic matrix projection. They do not carry solver cost policy or a solver formulation. powerio-prob owns complete problem instances. The C prob feature currently exposes a matrix free SCOPF instance through a JSON document. DC OPF instances and bundles have no C entry points.

C ABI v5

v5 touches twelve symbols and renames none. Everything a caller wrote against v4 still compiles except the seven signatures listed below, which change because their v4 shape lost data.

This document has two halves. The first records what shipped. The second is a design study that proposed replacing the whole surface; it was cut down to the first half, and it stays here because the reasoning behind several of its pieces is worth having when a later version does break something.

What v5 changes

Five JSON documents also changed shape while their symbols kept their signatures, which is the reason the integer had to move at all. A binding built against 4 would pass the handshake and then read null for keys it mirrors.

The rename is the same edit everywhere: one release version now covers every document powerio authors, so a per-document schema_version frozen at 1.0.0 said nothing a caller could act on. pio_scopf_to_json keeps 1-based indices; the design study below proposed flipping them to 0-based and that did not ship.

pio_summary_json’s counts block stays the case file’s own inventory, so a 3-winding transformer counts once there rather than as the bus and three branches it lowers to. The two new topology fields are the lowered space, which is what every extractor reports.

Warnings. The seven conversion entry points used to fill a caller warnbuf and silently truncate when the fidelity-loss list outran it. finish_conversion discarded the length that would have told the caller. The out-pointer replaces both problems: NULL means the conversion lost nothing, any other value is an owned string the caller frees with pio_string_free, and passing NULL for the parameter itself discards them. The call writes the out-pointer before it does any work, so a stale value from an earlier call is never mistaken for this one’s. pio_warnings and pio_dist_warnings keep their caller buffer: they use the size-then-fill idiom and cannot truncate.

The bus space. A case with an in-service 3-winding transformer star-lowers before the dense extractors run, adding one bus per transformer. Through v4 pio_n_buses and pio_bus_ids reported the unexpanded table while pio_bus_demand, pio_bus_shunt and pio_n_islands reported the expansion, so a per-bus buffer sized from pio_n_buses read short and its trailing entries had no id. The v4 header documented the mismatch and said aligning it was a v5 change. This is that change. length(bus_ids) == n_buses is the migration test.

PIO_DIST_ABI_VERSION is frozen at 1 and no longer meaningful. It existed to absorb distribution volatility, and that volatility is in the BMOPF schema, which changes a reader, a writer and an emitted token but no C signature. The symbol stays because PowerIO.jl gates thirteen distribution call sites on resolving it. Foreign schema versions are reported at runtime by pio_build_info instead, which can express “I speak BMOPF 0.2” in a way an integer checked once at load cannot.

pio_parse_bytes takes (const uint8_t *, size_t) and accepts every pio_parse_str format name plus pwb. PowerWorld binary has no text form and a NUL truncates it, so before this the only way to read one was pio_parse_file, which means a consumer holding an upload or an archive member had to stage a temporary file. It opens nothing, which is a security property rather than a convenience: it is the entry point for untrusted input, and it is why the 0.7.3 advisory fix works. The Rust and Python surfaces gained the same entry point in the same change, so the symbol is not a promise the library cannot keep.

Design study: a full rewrite of the surface

Everything below was written as a v5 proposal and did not ship. It renames essentially every symbol in an 85-symbol surface. The estimate for PowerIO.jl alone was ~98 edit sites across 12 files, in exchange for consistency rather than for a defect fixed.

The premise that drove its scope was that v5 would be the last breaking change to the C ABI, so everything had to land at once. That premise is false. Every symbol resolves by dlsym behind an equality gate on pio_abi_version, powerio owns its one binding consumer, and that binding’s artifact repin is automated. A later bump is routine. Nothing here has to happen in one release, and most of it does not have to happen at all.

Read it as an argued menu. The pieces most worth revisiting are the options struct (rule 5), which is the only mechanism here that prevents future symbol bloat, and the conversion handle (rule 4), whose file-list argument identifies a live defect: OpenDSS sidecars are dropped today, so a written .dss can name a coordinates file the user does not have.

The grammar

Every symbol has this form:

pio_<subject>_<operation>[_<qualifier>]

Subjects. The subject is the handle the function takes or returns. An empty subject means the library itself.

Operations. There are six forms. Only four use a verb, and those verbs are a closed list.

Qualifiers. Two members: _check, a preflight for the operation it attaches to, and _bytes, which says the input is memory rather than a path. Nothing else may use the slot without being added to this list.

The rule that makes the grammar predictable:

There is one ingest verb, parse. It takes a path. A suffix appears only when the bytes come from somewhere other than a path.

So pio_balanced_parse(path, …) and pio_balanced_parse_bytes(data, len, …), and nothing else. _file, _str, _dir, _dataset and _scenario all disappear.

The suffix does not name the storage shape, which rule 7 rejects. It names who touches the filesystem. A path argument means the library opens files and may follow an OpenDSS Redirect tree. A buffer argument means it opens nothing, which is a security property and not a convenience: parse_bytes is the entry point for untrusted text, and it is why the 0.7.3 advisory fix works.

_bytes rather than _str, because a PowerWorld .pwb is binary and a NUL truncates it. v4 called this _str and could not accept half the formats it named.

This is the one piece of the study that shipped, under the existing names rather than the proposed ones. powerio::parse_bytes and pio_parse_bytes exist as of v5; the rename to pio_balanced_parse_bytes does not.

The precedent is libxml2: one verb, and a suffix for where the bytes came from — xmlReadFile, xmlReadMemory, xmlReadDoc.

The seven rules

1. Every symbol names its subject. v4 left the balanced model unnamed. pio_parse_str was balanced, pio_dist_parse_str was multiconductor. The reader had to know which omission meant what. Both models are peers. Both are named.

Names are spelled out. The reason is not that mc reads as Monte Carlo; that claim does not survive checking, and psspy has no Monte Carlo entry points. The reason is that balanced_network and multiconductor_network are already the .pio.json payload keys, the Rust type names, the Python class names and the Julia exported types. A C header you cannot grep with the word the file format uses is the one surface out of step. Spelling them out costs 3.9 characters of mean symbol length. v5 averages 22.7 characters. cairo averages 24.0.

2. dist is gone as a word. v4 used dist in 15 symbols and multiconductor in 4, in one header, for one model. The crate keeps the name powerio-dist, and that is fine: a package name is not a symbol prefix, and libcurl exports curl_easy_*. The word is also spent elsewhere. psspy ships 23 dist_* functions and every one means disturbance.

3. One buffer idiom. No raw pointer to library memory crosses the boundary. v4 had three idioms. 27 symbols returned an owned char * that the caller freed with pio_string_free. 5 filled a caller buffer. 9 filled caller arrays. v5 keeps the last two:

size_t pio_x(const PioHandle *h, char *out, size_t cap, char *errbuf, size_t errlen);

The return is the total available. NULL or 0 is a size query. pio_string_free is deleted with the class it freed.

Everything that crosses is either bytes copied into memory the caller owns, or an opaque handle with exactly one free function. The library still allocates handles. It no longer hands out pointers into its own memory.

These invariants hold for every buffer symbol:

• the return excludes the NUL
• the buffer is always NUL terminated
• a short buffer truncates on a UTF-8 boundary, so a C caller never receives a split codepoint
• a fallible symbol writes errbuf[0] = '\0' on entry, and a message only on failure

That last one matters. Several of these symbols can legitimately return nothing. Without it, a 0 return means both “empty” and “failed”, and the caller cannot tell.

Caching. A size query followed by a fill would run the serialization twice. Handles with no mutator may cache each payload in a OnceLock<Result<String, String>>. That is PioBalancedNetwork, PioMulticonductorNetwork, PioScopfInstance, PioConversion and PioSource. OnceLock<T> is Sync when T is, so the header’s promise that concurrent reads are safe still holds.

PioPackage is excluded. It has two mutators, pio_package_validate and pio_package_set_operating_points, and they rewrite the fields its accessors read. A cached read taken before validate and served after it would return pre-validation text from a const accessor. Its four document accessors recompute per call.

4. A conversion is a handle, because it has two outputs. v4’s seven warnbuf symbols truncated silently. finish_conversion discarded the needed length. There was no size query. A long fidelity-loss list was lost with no signal, and the header told callers that 256 bytes always sufficed.

A conversion has text and warnings. Both need a size query. Neither can attach to the network handle, because write-time warnings are produced at write time.

A PioConversion owns its text and warnings. It does not borrow the network. It stays valid after that handle is freed, so the two have no ordering requirement.

It also reports the files a write produced. pio_conversion_n_files and pio_conversion_file replace a newline-joined list, because a newline is a legal byte in a POSIX filename. This closes a live defect: OpenDSS sidecars are dropped today, so a written .dss can name a coordinates file the user does not have.

5. One extensible options struct. With no repr(C) structs, every new option needs a new symbol. Over three years that bloats the surface or forces a v6.

typedef struct PioNormalizeOptions {
  size_t  struct_size;
  int32_t clamp_angle_bounds;   /* 0 false, any nonzero true */
  double  angle_bound_pad;      /* radians; 0 is a valid explicit value */
} PioNormalizeOptions;

The precedent is the Linux extensible syscall convention: openat2, clone3, sched_setattr. Win32 cbSize is the older and coarser form. Vulkan is not a precedent; its sType and pNext chain solves a different problem.

The rules, all of which must be stated or the mechanism does not work:

• The library reads a field only if its offset and size fall within min(opts->struct_size, sizeof(PioNormalizeOptions)) as the library was compiled.
• A newer caller against an older library is safe by that same clause.
• If the caller’s tail beyond the library’s own size is nonzero, the call fails. The caller asked for something this build does not do. Silence would be wrong.
• Fields are append only. No reorder, no removal, no type change.
• The struct has no implicit padding. Reserved fields are named.
• The caller zero-fills before setting fields.
• NULL means all defaults. That is the only way to ask for defaults without stating a size.

Booleans cross as int32_t. The header contains no bool today, and pio_normalize_with_options already converts explicitly.

6. One ABI integer. PIO_DIST_ABI_VERSION existed to absorb distribution volatility. That volatility is in the BMOPF schema, which the IEEE task force owns and powerio reproduces. A BMOPF revision changes a reader, a writer and an emitted token. It changes no C signature.

So the second integer never fires for its stated reason. What it does instead is give one shared object two compatibility promises. No mature C library does that.

Foreign schema drift is reported at runtime by pio_build_info. An integer checked once at load cannot express “I speak BMOPF 0.2 but not 0.3”.

Removing the symbol costs the binding more than removing the constant. PowerIO.jl gates every distribution entry point on _ensure_dist_compatible, which resolves pio_dist_abi_version and reports a missing symbol as “use powerio-capi v0.3.1”. Thirteen call sites reach it. The gate has to be rebuilt on pio_has_feature("dist") in the same change that repins the artifact, or a v5 library that fully supports distribution refuses every distribution call.

7. Cardinality is the axis, not storage. v4 split reading by storage: pio_parse_file for documents, pio_read_dir for directories. That split does not survive the formats.

An OpenDSS .dss is one file that pulls in a tree. A PyPSA case is a directory. powerio’s own parse_file already dispatches a PyPSA directory before it looks at any extension. One file is not one case for much of the software powerio reads.

What does survive is how many cases a path yields. Almost every format yields one. A gridfm dataset directory yields N over one shared topology. A PowerModels JSON with multinetwork=true also yields N, and powerio reads the first and warns about the rest today.

So every path opens to a source:

PioSource *pio_source_open(const char *path, const char *from,
                           const PioReadOptions *opts, char *errbuf, size_t errlen);
size_t     pio_source_count(const PioSource *);
size_t     pio_source_entry_names(const PioSource *, char *out, size_t cap,
                                  char *errbuf, size_t errlen);   /* NUL separated */
size_t     pio_source_entry_name(const PioSource *, size_t index, char *out, size_t cap,
                                 char *errbuf, size_t errlen);
void       pio_source_free(PioSource *);

PioBalancedNetwork *pio_balanced_from_source(const PioSource *, size_t index,
                                             char *errbuf, size_t errlen);

A matpower file is a source with one entry. A PyPSA folder is a source with one entry. A gridfm dataset has N. pio_source_count may return 0 for an empty container.

Entries are named, not numbered. int64_t is gridfm’s Parquet key type, not a property of containers; PGLib and GOC3 entries have no integer key.

pio_balanced_parse survives as a documented composition of open plus entry 0, not as a second route. SQLite documents sqlite3_exec the same way. GDAL and libarchive make open-then- enumerate the only path, and they are the honest counterexample; the shortcut wins here on the ratio, since almost every case is a single entry. pio_balanced_parse on a multi-entry path fails and names pio_source_open, so the shortcut can never silently disagree with the container form.

The symbol table

= unchanged · R renamed · S re-signatured · X removed · N new

Handshake, 7 → 4

pio_abi_version and pio_version are both kept because neither derives from the other. The integer is the gate and changes only on a break. The string is identity and changes every release. PowerIO.jl resolves everything by dlsym from a pinned artifact, so pio_abi_version is powerio’s soname.

Balanced model

The remaining extractors are plain renames for rule 1: pio_n_branches, pio_n_switches, pio_n_islands, pio_base_mva, pio_is_radial, pio_ref_bus_index, pio_ref_bus_indices, pio_branches, pio_branch_charging, pio_switches, pio_bus_demand, pio_bus_shunt.

Source, new

pio_source_open, pio_source_count, pio_source_entry_names, pio_source_entry_name, pio_source_free. See rule 7.

Conversion, new

pio_conversion_text, pio_conversion_warnings, pio_conversion_n_files, pio_conversion_file, pio_conversion_free. See rule 4.

Multiconductor

Every pio_dist_* becomes pio_multiconductor_*. parse_file becomes parse, parse_str becomes parse_bytes, summary_json becomes summary, graph_json becomes graph. to_format folds into pio_multiconductor_write. pio_dist_convert_file and pio_dist_convert_str are removed, as their balanced twins are. warnings, free, to_json, from_json, geo_extract and geo_apply follow the balanced pattern.

parse is where rule 7 is clearest. An OpenDSS .dss is the format most associated with a file extension, and it is the one least likely to be a single file.

The EXPERIMENTAL banner is retired. The dist C signatures were never the unstable part. The BMOPF payload schema was, and it is versioned where it lives.

Package, 18 → 18

pio_package_parse_file becomes pio_package_parse, and pio_package_parse_str becomes pio_package_parse_bytes.

Four lose a repeated noun: from_balanced_network → from_balanced, from_multiconductor_network → from_multiconductor, and the two extraction twins. The handle is the network; saying it twice adds nothing.

The extraction direction is as_, so pio_package_to_balanced_network becomes pio_package_as_balanced. to_ already means emitter here: to_json returns bytes. A verb cannot mean “returns a handle you must free” in one symbol and “fills your buffer” in the next. as_ also matches what Rust and Python already ship (NetworkPackage::as_balanced, Package.as_balanced), so the rename moves the C ABI toward two surfaces rather than away from them. PowerIO.jl spells this from_package and dispatches on model_kind; that stays, because one Julia function over a tagged union is the Julia way to write it.

Two get shorter: pio_package_lower_multiconductor_to_balanced → pio_package_lower_to_balanced (44 → 29), and pio_package_multiconductor_to_balanced_preflight_json → pio_package_lower_to_balanced_check (53 → 35). lower stays because .pio.json publishes lowering_history. preflight goes because it is an internal stage name.

pio_package_lower_to_balanced and pio_package_as_balanced must not differ by one suffix. They return different handle types under mutually exclusive preconditions.

Five lose _json: to_json, validation_json, diagnostics_json, operating_points_json, study_json. The suffix named an encoding that no sibling symbol varies.

free, validate, set_operating_points, materialize_operating_point and materialize_study_commit keep their names.

Problem instances, 6 → 3

The freeze forbids breaking changes, not additions. Delete now. Re-cut additively when a C consumer exists and can say what shape it needs.

Geographic and Arrow

pio_geo_parse keeps its verb. It parses. normalize is already the per-unit transform in the same header, and in the geospatial domain GEOSNormalize_r canonicalizes a geometry that is already parsed.

Arrow stays balanced only. arrow_export.rs contains no multiconductor tables, and powerio-dist has no Arrow dependency. A table id is cheap to add later; a shipped table’s column order is frozen, so the cost of guessing lands on the columns.

Handles, structs, macros

New handles: PioConversion, PioSource. PioNetwork becomes PioBalancedNetwork. PioDistNetwork becomes PioMulticonductorNetwork. PioAcopfInstance is removed.

New structs: PioNormalizeOptions, PioReadOptions, PioWriteOptions, PioGeoApplyReport. All follow rule 5. PioWriteOptions ships with only struct_size, so a later write option is an appended field rather than a second symbol.

PIO_ABI_VERSION becomes 5. PIO_DIST_ABI_VERSION is removed. PIO_ERRBUF_MIN stays. The 21 PIO_ARROW_TABLE_* ids stay, and stay append only.

The symbols whose meaning moves

A rename is safe: the old name stops resolving. A re-signature is safe: the old call stops compiling. Neither protects a symbol whose payload changes while the call still works.

pio_balanced_n_buses and pio_balanced_bus_ids move to the star-lowered space. v4 returned fewer ids than the extractors had rows, so trailing rows had no id. v5 returns one id per row. A binding that asserts length(ids) == n fails on v4 and passes on v5. That assert is the migration test. The handle rename forces every C declaration to be edited, so no caller reaches the new behavior without touching the line.

pio_scopf_to_json keeps its name and changes 1-based indices to 0-based. The document carries index_base, but a field is only a mechanism if something reads it, and nothing does today. A 0-based index is still a valid 1-based index, so a missed conversion reads the wrong element rather than failing.

So v5 requires the binding to normalize: PowerIO.jl converts to 1-based at the boundary. The wire value is 0. The value a Julia caller sees is 1. Julia arrays are 1-based, and a binding should speak its own language. Python, whose lists are 0-based, passes it through.

What does not change

Arrow table ids and column order. The format tokens, which are strings and were never symbols. The opaque handle design. The panic guard on every entry point. errbuf and errlen last.

What this costs PowerIO.jl

Every symbol resolves by dlsym, so a rename fails at load rather than reading wrong. One v4 precedent is worth remembering: pio_convert_file kept its symbol, arity and types while two arguments were reordered. It linked, and it read the formats reversed.

Julia is where the C type system does not help. PowerIO.jl holds handles as Ptr{Cvoid}, so renaming a handle costs it nothing and protects it from nothing. The ccall signature changes carry the migration, and rule 3 changes most of the surface.

Roughly 98 symbol reference sites across 12 files. 22 _take_string sites become the size-query helper, and _take_string, _WARNLEN and the truncation guess are deleted. The two risk concentrations are the conversion handle lifetimes and the star-lowered bus space, which changes numbers rather than symbols. PIO_DIST_ABI_VERSION must be deleted from the binding, or the artifact repin parks forever.

Deleting that constant is not the whole of it. _ensure_dist_compatible, schema_versions, dist_capabilities and matrix_available all resolve symbols v5 removes. The first throws on a missing symbol, so every distribution call fails; the other three are guarded by _exports_symbol, so they report “unavailable” on a library that has the feature. All four move to pio_has_feature and pio_build_info in the same change as the repin.

Every exported PowerIO.jl name survives. open_source, entry_names and entry_name are added. BMOPFTools and ExaModelsPower see no API change, but BMOPFTools.from_dss reaches pio_dist_abi_version through parse_file(MulticonductorNetwork, …), so it breaks if the gate above is not rebuilt.

tellegen and PowerMCP are not affected. tellegen links the Rust crates. PowerMCP imports the Python wheel. Neither calls a pio_ symbol.

Open decisions

These were left unsettled when the study was cut down.

1. PioWriteOptions with no fields. Ship it empty so a later write option is an appended field, or omit it and accept that write gains options only through a second symbol.
2. Arrow generator cost tables. Additive, so nothing here gates them. They are what lets PowerIO.jl retire most of exa.jl, which rebuilds the ExaModelsPower payload from JSON because Arrow carries no cost.
3. The geo family. Five symbols, no C consumer today. tellegen’s Rust usage is a validated specification to build against, but shipping and deleting are both defensible.

What was deferred rather than rejected

Error codes. Every fallible symbol returns NULL or -1 and writes a message, so a caller that wants to branch on the failure has to match on English. ErrorCategory already exists in Rust with five variants and is deliberately not #[non_exhaustive], so adding one is a compile error at every binding. v5 publishes the category tokens in pio_build_info so a binding can build its map ahead of time; the int32_t return is near-free during a broad re-signature and expensive on its own, so it waits for one.

The options structs. Rule 5 is the only mechanism in this study that prevents the surface from growing a symbol per option. It costs nothing to adopt one struct at a time, on the next symbol that would otherwise need a _with_options twin.

Language APIs

PowerIO uses the same IO vocabulary across Rust, Python, Julia, and the C ABI, with language-specific spelling where needed. A new format or dataset appears as a format string or convenience wrapper rather than a new naming scheme.

Verb taxonomy:

• parse_*: bytes, paths, or text to typed parsed values. Transmission parsers return a balanced network handle; distribution parsers return a multiconductor network handle; display parsers return display data.
• to_*: BalancedNetwork to a new value
• convert_file: path to target text convenience
• write_*: filesystem outputs (write_gridfm, write_pypsa_csv_folder, write_dcopf_bundle); the Rust hub also keeps write_as and per-format write_* text builders, the internals behind to_format and the to_* writers, which the bindings do not mirror
• read_*: filesystem dataset inputs (read_gridfm, read_pypsa_csv_folder), the inverse of write_*. Datasets are multi-file directories, so they read and write; single documents parse and serialize (parse_*/to_*)
• export_*: handoff to external memory or interface protocols

Note: the C ABI carries no per-format symbols: matpower, powerio-json, PyPSA CSV directories, and gridfm datasets are all format strings into pio_to_format / pio_parse_str / pio_write_dir / pio_read_dir. Removing or changing a documented format token is a C behavior change even though the C signature stays the same. The language APIs keep their per-format conveniences (to_matpower, from_json, …) as wrappers over the same paths.

C ABI and binding compatibility

The C ABI is the stable boundary for non Rust callers. Handles own parsed networks. PioPackage handles own .pio.json documents. Callers free network handles with pio_network_free, package handles with pio_package_free, and SCOPF handles with pio_scopf_instance_free. They free returned text with pio_string_free, size output buffers before filling them, and treat every format name as a string routed through the same parser and writer hub.

C ABI review points:

• null handles must return documented defaults or errors and must never crash;
• optional output buffers must be safe to pass as null; required output structs such as Arrow exports must report an error when null;
• returned text and warning buffers must be NUL terminated when capacity permits;
• reported lengths must let callers allocate exact buffers;
• header declarations and exported Rust symbols must match;
• feature gated exports such as Arrow, GridFM, distribution, packages, and problem instances must be additive;
• ownership rules must be documented in the header, README, and binding code.

Julia’s PowerIO.jl uses the C ABI for handles, dense extractors, Arrow, GridFM, PyPSA CSV folders, distribution conversion, and .pio.json document construction. Programmatic whole-network JSON remains available through powerio-json; file handoffs should use .pio.json. The Julia binding checks pio_abi_version() against PIO_ABI_VERSION on first use. Distribution calls also check pio_dist_abi_version().

GOC3 document construction is the first .pio.json operating point path backed by a source format. The static balanced model JSON carries the first interval; the replayable series is exposed through the package APIs above.

During development, test the Julia binding against the local C ABI instead of a release artifact:

cargo build -p powerio-capi --release --features arrow,matrix,gridfm,dist,pkg,prob
POWERIO_CAPI=$PWD/target/release/libpowerio_capi.dylib \
  julia --project=../PowerIO.jl -e 'using Pkg; Pkg.test()'

Binding compatibility checks:

Distribution surface (powerio-dist)

The multiconductor distribution model follows the same taxonomy under its own handle type; the two families do not mix. The C distribution surface ships behind the optional dist feature (PIO_DIST); a consumer probes it with pio_has_feature("dist"), then checks pio_dist_abi_version() against PIO_DIST_ABI_VERSION. PowerIO.jl uses the same runtime check before calling the distribution C conversion helpers.

Python API

Install the base package for parsing, writing, JSON transport, and file conversion. It has no required third party Python packages:

pip install powerio

Install extras only for the outputs that need them:

pip install 'powerio[matrix]'   # numpy, scipy
pip install 'powerio[graph]'    # networkx
pip install 'powerio[gridfm]'   # polars
pip install 'powerio[pandas]'   # pandas and pyarrow compatibility reads (Python 3.10+)
pip install 'powerio[all]'      # matrix, graph, and gridfm reads

import powerio, parse_file, parse_str, convert_file, convert_str, to_matpower, and to_json do not import NumPy, SciPy, NetworkX, Polars, pandas, or pyarrow.

Transmission text and file format names accepted by parse_* and convert_* include matpower, psse, powerworld, pslf, powermodels-json, egret-json, pandapower-json, goc3-json, surge-json, opfdata-json, and powerio-json, plus their documented aliases. opfdata-json reads one extracted JSON document from a DeepMind OPFData FullTop or N-1 release without PyTorch. PyPSA CSV folders and GridFM Parquet datasets are directory formats; use read_pypsa_csv_folder, Network.write_pypsa_csv_folder, read_gridfm, Network.write_gridfm, or the conversion/package helpers that take a path.

Canonical use

import powerio as pio

net = pio.parse_file("case9.m")
same_text = net.to_matpower()
json_text = net.to_json()
pm = net.to_format("powermodels-json")
pp = net.to_format("pandapower-json")
raw = pio.convert_file("case9.m", "psse")
aux = pio.convert_str(json_text, "powerworld", format="powermodels-json")
pypsa_out = net.write_pypsa_csv_folder("case9-pypsa")
display = pio.parse_display_file("case.pwd")
pkg = pio.Package.from_file("goc3_case.json", from_="goc3-json")
points = pkg.operating_points()
period_1 = pkg.materialize_operating_point(1)

normalized = net.to_normalized()
dense = net.to_dense()       # needs powerio[matrix]
bprime = net.bprime()        # needs powerio[matrix]
graph = net.to_networkx()    # needs powerio[graph]
dist_graph = pio.dist.parse_file("feeder.dss").graph()
scopf = pio.parse_scopf(goc3_text, from_="goc3-json")

Model names

powerio.BalancedNetwork is the existing balanced transmission handle. v0.4 also exports powerio.BalancedNetwork as the long term family name for the same handle. The old powerio.Case compatibility alias was removed in v0.4.

For distribution models, use powerio.dist.MulticonductorNetwork or the existing powerio.dist.MulticonductorNetwork handle name. The old powerio.dist.DistCase alias was removed in v0.4. dist_net.graph() returns the collapsed bus and terminal graph as Python data.

parse_file(path, from_=None) reads network case files (inferred from the extension, or forced with from_); parse_str(text, format) reads in-memory case text. Display artifacts are not network cases, so they use the separate display API:

from pathlib import Path

display = pio.parse_display_file("case.pwd")
same = pio.parse_display_bytes(Path("case.pwd").read_bytes(), "pwd")

assert display.kind == "powerworld"
first = display.data.substations[0]
print(first.number, first.name, first.x, first.y)

display.data is a PwdDisplay with canvas_width, canvas_height, stamp, and substations.

Problem instances

parse_scopf(text, from_="goc3-json") assembles a matrix free SCOPF problem instance and returns its Julia compatibility document as a Python dictionary. The document declares its schema version and uses 1-based indices for language compatibility. Source UIDs and source bus IDs remain separate from those indices. Invalid JSON, duplicate identities, missing references, and period length mismatches raise PowerIOError subclasses.

PyPSA folders

PyPSA CSV folders are multi-file datasets, so they use explicit read and write helpers instead of Conversion.text.

import powerio as pio

case = pio.parse_file("case14.m")
out = case.write_pypsa_csv_folder("case14-pypsa")
round_trip = pio.read_pypsa_csv_folder(out["dir"])

The written folder can be imported with pypsa.Network().import_from_csv_folder(path). PyPSA itself is not a runtime dependency of powerio.

CSV folders are PyPSA’s native static component format and carry the network topology: buses, lines, transformers, generators, loads, shunts, storage units, and links (read as HVDC). NetCDF and HDF5 time series are not supported. They are tracked in #107.

GridFM reads

The native wheel includes the GridFM Parquet writer and reader.

read_gridfm(dir, scenario=0) rebuilds a BalancedNetwork from a dataset, the inverse of Network.write_gridfm, returning a GridfmRead(network, scenario, warnings) namedtuple. The read is lossy but recovers everything a power flow needs; warnings lists what the gridfm schema couldn’t round-trip (synthesized bus ids, folded per bus load/shunt, dropped HVDC/storage, piecewise costs). read_gridfm_scenarios(dir) returns one GridfmRead per scenario. dir resolves the raw/ leaf, a <case>/ directory, or a parent with one */raw/ child.

import powerio as pio

out = pio.parse_file("case14.m").write_gridfm("out")
net, scenario, warnings = pio.read_gridfm(out["dir"])
text = net.to_matpower()                 # gridfm → any classical format

To inspect the raw Parquet tables instead, the preferred read extra is Polars:

import polars as pl

bus = pl.read_parquet(f"{out['dir']}/bus_data.parquet")

Use powerio[pandas] only for downstream code that expects pandas DataFrames.

.pio.json documents

powerio.Package is the handle for .pio.json documents: it parses the document metadata once and every accessor reuses the handle. Package.from_file and Package.from_str build documents from case input, Package.from_json reads document text, and Package.from_balanced / Package.from_multiconductor wrap existing networks. pkg.model_kind names the document family; pkg.as_balanced() / pkg.as_multiconductor() rebuild typed network handles from the model JSON.

pkg.operating_points() returns a Python dict for the replayable operating point series, or None. pkg.materialize_operating_point(i) returns a new static Package with one point applied; updates resolve by the model rows’ uid identities, and an unknown identity or a row that contradicts one raises ValueError. GOC3 documents populate this series from the source time series while the static model JSON holds the first interval. Network table dicts (net.buses, net.loads, …) expose each row’s uid. pkg.study() returns a Python dict for the package study block, or None; pkg.materialize_study_commit(i) folds cumulative commits through i into a new static package and clears both replay blocks. pkg.validate(), pkg.validation(), and pkg.diagnostics() expose the document validation profile, and multiconductor documents lower through pkg.multiconductor_to_balanced_preflight() and pkg.lower_multiconductor_to_balanced().

pkg = pio.Package.from_file("goc3_case.json", from_="goc3-json")
series = pkg.operating_points()
static_pkg = pkg.materialize_operating_point(0)
net = static_pkg.as_balanced()

MCP path handling

MCP clients can request .pio.json document output from parse through the package transport and pass that same value back to the other network tools:

parsed = parse(path="case9.m", transport="package")
pkg = parsed["package_json"]
summary(package_json=pkg)
matrix("bprime", package_json=pkg)
save(out_path="case9.raw", to_format="psse", package_json=pkg)
diagnostics(pkg)

summary, normalize, matrix, and save also auto-detect .pio.json document JSON passed through the legacy json argument. The document metadata’s model_kind routes balanced and multiconductor model JSON.

The optional MCP server accepts local filesystem paths and file:// URIs for path and out_path arguments. Remote URI schemes are rejected. Deployments that need filesystem containment can set POWERIO_MCP_ALLOWED_ROOTS to an os.pathsep separated list of directories; all MCP reads and writes must resolve under one of those roots. POWERIO_MCP_ROOT is accepted as a single root alias.

Performance

PowerIO has five benchmark tiers. Keep them separate when publishing numbers.

The published table lives in the repository benchmark results, and this guide is the public reference for how those numbers are produced. Each refresh should update the snapshot environment there: machine model, chip, core count, memory, OS, Rust, C compiler, Julia, Python, and the package versions used by the comparison harnesses. Regenerate the JSON inputs first, then splice only the marked regions:

bash benchmarks/fetch_cases.sh
cargo build --release -p powerio-capi --features arrow,matrix
python3.12 -m venv .venv
.venv/bin/python -m pip install --upgrade pip maturin -r benchmarks/requirements.txt
env VIRTUAL_ENV=$PWD/.venv .venv/bin/maturin develop --release
julia --project=benchmarks benchmarks/bench_julia.jl --json
.venv/bin/python benchmarks/bench_parse.py --json \
  tests/data/case2869pegase.m \
  tests/data/large/case9241pegase.m \
  tests/data/large/case13659pegase.m \
  tests/data/large/case193k.m
python3 benchmarks/render_tables.py
python3 benchmarks/render_tables.py --check

The Julia benchmark writes rows for parse only and matrix_rows for parse plus Y bus construction. PowerIO measures pio_parse_file plus pio_to_arrow for table ybus; PowerModels measures parse_file, make_per_unit!, and calc_admittance_matrix; ExaPowerIO measures parse_matpower plus a sparse Y bus assembled from its parsed branch admittance rows.

PowerWorld .pwb and .aux parse timings are measured by the Rust Criterion benchmarks. Fetch the public fixtures, run cargo bench -p powerio --bench parse -- "parse_aux_|parse_pwb_", then run python3 benchmarks/extract_powerworld_bench.py before rendering the tables. If the Texas7k local row is published, pass its aux and pwb paths through POWERIO_BENCH_AUX and POWERIO_BENCH_PWB during the Criterion run.

Matrix builder timings are separate from parse timings. The matrix benchmark parses each fixture once, builds IndexedNetwork once, and times only derived matrix construction. Its pipeline row measures Pipeline::run for the paired \(Y_{\mathrm{bus}}\) export, including MTX, shunt, and metadata writes:

cargo bench -p powerio-matrix --bench matrix
python3 benchmarks/extract_matrix_bench.py
python3 benchmarks/render_tables.py

Use filtered runs while developing a focused change, for example:

cargo bench -p powerio-matrix --bench matrix -- 'matrix_bprime|matrix_ybus|dcopf_'

Criterion compares against the local target/criterion baseline. Treat a Performance has regressed line as a signal to investigate rather than a publishable claim by itself. A release note or benchmark page needs the commit, tree cleanliness, machine, toolchain, command, fixtures, and whether optional large cases were present.

Measure C ABI release size before publishing a C ABI change:

cargo build -p powerio-capi --release --no-default-features
cp target/release/libpowerio_capi.dylib /tmp/libpowerio_capi-core.dylib
cargo build -p powerio-capi --release --no-default-features --features arrow,matrix
cp target/release/libpowerio_capi.dylib /tmp/libpowerio_capi-arrow-matrix.dylib
cargo build -p powerio-capi --release --no-default-features --features arrow,matrix,gridfm,dist,pkg,prob
cp target/release/libpowerio_capi.dylib /tmp/libpowerio_capi-all.dylib
stat -f '%z %N' /tmp/libpowerio_capi-core.dylib \
  /tmp/libpowerio_capi-arrow-matrix.dylib \
  /tmp/libpowerio_capi-all.dylib

Testing and release checks

Keep changes reviewable. A numerical semantics change needs tests and a short reason in code or docs. A performance change needs before and after measurements. A documentation change should link to evidence instead of expanding the README into a second manual.

Baseline checks

These commands cover the Rust workspace, the Python extension build, the Python binding tests, and the book:

cargo fmt --all --check
bash scripts/ci-clippy.sh
cargo test
cargo test -p powerio-cli --test cli
cargo test -p powerio-capi
cargo build -p powerio-py
python3.12 -m venv .venv
.venv/bin/python -m pip install --upgrade pip maturin -r benchmarks/requirements.txt
env VIRTUAL_ENV=$PWD/.venv .venv/bin/maturin develop --release
.venv/bin/pytest python/tests
mdbook build docs
mdbook test docs

Route changes

Use the smallest gate set that covers the changed surface, then run the release gates before a release claim.

benchmarks/run_validation.sh requires the Python oracle stack in the same Python 3.11+ venv as the local wheel. Missing PyPSA, pandapower, or egret is a setup failure. benchmarks/run_rich_validation.sh treats the committed PowerModels rich oracle as strict; missing Julia is a setup failure.

Release gates

Run the full set below, in addition to the baseline checks, before publishing a release claim:

cargo test -p powerio-capi --no-default-features
cargo test -p powerio-capi --features arrow,matrix,gridfm,dist,pkg,prob
bash scripts/ci-clippy.sh capi-no-default
bash scripts/ci-clippy.sh capi-release
cargo build -p powerio-capi --release --features arrow,matrix,gridfm,dist,pkg,prob
scripts/capi-header-parity.sh
scripts/capi-smoke.sh
POWERIO_CAPI=$PWD/target/release/libpowerio_capi.dylib \
  julia --project=../PowerIO.jl -e 'using Pkg; Pkg.test()'
cargo bench -p powerio-matrix --bench matrix -- 'matrix_bprime|matrix_ybus|dcopf_'
(cd benchmarks/asv && ../../.venv/bin/asv check -E existing:../../.venv/bin/python)
(cd benchmarks/asv && ../../.venv/bin/asv run --quick --show-stderr -E existing:../../.venv/bin/python --dry-run)
for target in matpower psse pslf powerio_json powerworld_aux pwb pwd; do
  cargo +nightly fuzz run "$target" -- -runs=1
done
bash benchmarks/run_validation.sh
bash benchmarks/run_rich_validation.sh

run_validation.sh checks the classic transmission paths against PowerModels.jl, ExaPowerIO.jl, egret, pandapower, and the full legacy reader to writer matrix; run_rich_validation.sh covers fields outside the MATPOWER row shape (branch terminal admittance, switches, current ratings, solution values, HVDC costs, load voltage models). GOC3 and Surge have no external oracle in this harness; the Rust parser, writer, routing, package, and round trip tests cover them. What the oracle legs prove, per format, is in the format fidelity chapter.

The gates do not prove every source format field is lossless. Known losses are part of the public behavior and surface as warnings.

Benchmark updates

Regenerate benchmark JSON before changing published tables:

julia --project=benchmarks benchmarks/bench_julia.jl --json
.venv/bin/python benchmarks/bench_parse.py --json <cases>
cargo bench -p powerio --bench parse -- "parse_aux_|parse_pwb_"
python3 benchmarks/extract_powerworld_bench.py
cargo bench -p powerio-matrix --bench matrix
python3 benchmarks/extract_matrix_bench.py
python3 benchmarks/render_tables.py
python3 benchmarks/render_tables.py --check

The ASV suite tracks Python wheel parse and matrix timing across git history. For an uncommitted worktree, smoke test it against the local venv:

cd benchmarks/asv
../../.venv/bin/asv check -E existing:../../.venv/bin/python
../../.venv/bin/asv run --quick --show-stderr -E existing:../../.venv/bin/python --dry-run

Do not update generated benchmark tables by hand. Update the snapshot environment described in the performance guide when publishing new numbers: commit, tree cleanliness, machine, OS, toolchain, Python stack, Julia stack, commands, fixtures, and optional local data.

Broad local corpora stay local. Pass them through documented environment variables or --root flags, review the reports under benchmarks/results/, and do not commit corpus paths or generated outputs.