Data hub API

Connections, mappings, flows and runs — the engine that moves data between matgrk and your systems.

Status: in development. The data hub runs inside matgrk today and is operated from the dashboard. Public API access — with its own credentials — is planned; until it ships, the shapes on this page describe the engine as it runs and may still change.

The model

Connections

Where data lives: sftp, object_storage, database_read, or matgrk_drop. Credentials are write-only — they can be set, never read back. A connection must pass its test before it can be activated, and SFTP host keys are pinned on first contact.

Mappings

How source fields become matgrk fields, with transforms in between. Mappings are versioned: an update creates a new version, and any earlier version can be restored.

Flows

A connection plus a mapping plus a direction and a schedule. Flows name their files by template, can be paused or muted, alert when a scheduled run goes missing, and a shrink guard refuses a run that would deactivate an unexpectedly large share of records.

Runs

Every execution is recorded — refusals included — with its outcome and counts: read, changed, created, deactivated, rejected, unchanged, sent. The rejected, created and deactivated rows of a run can each be listed.

Reference data

  • GET/structuresThe record structures flows can carrydata.flows.read
  • GET/transformsThe transforms available to mappingsdata.flows.read

Connections

  • GET/connectionsList connections (credentials are never returned)data.flows.manage
  • POST/connectionsCreate a connectiondata.connections.manage
  • PUT/connections/{id}Change a connectiondata.connections.manage
  • DELETE/connections/{id}Delete a connectiondata.connections.manage
  • POST/connections/{id}/testTest the connectiondata.connections.manage
  • POST/connections/{id}/activateActivate a connection that passed its testdata.connections.manage
  • GET/connections/{id}/fieldsField names found at the sourcedata.flows.manage

Mappings

  • GET/mappingsList mappingsdata.flows.read
  • POST/mappingsCreate a mappingdata.flows.manage
  • PUT/mappings/{id}Update a mapping — a new version is createddata.flows.manage
  • GET/mappings/{id}/versionsThe mapping’s version historydata.flows.read
  • POST/mappings/{id}/restoreRestore an earlier versiondata.flows.manage

Flows and runs

  • GET/flowsList flowsdata.flows.read
  • POST/flowsCreate a flowdata.flows.manage
  • PUT/flows/{id}Change a flowdata.flows.manage
  • GET/flows/{id}/runsEvery run of the flow, refusals includeddata.flows.read
  • POST/flows/{id}/runsRun the flow now, or preview itdata.flows.manage
  • GET/runs/{id}One run with its counts and outcomedata.flows.read
  • GET/runs/{id}/rejectedThe rows a run rejecteddata.flows.read
  • GET/runs/{id}/createdThe rows a run createddata.flows.read
  • GET/runs/{id}/deactivatedThe rows a run deactivateddata.flows.read