Ingest API

• Base path /api/v1. All requests and responses are JSON.
• Two resources, projects and their benchmarks. Both writes are upserts keyed on slug: POST the same slug again to update in place.
• There is intentionally no delete endpoint. Deletion is a human, UI-only action; the API can only create, update, and read.
• Every benchmark spec is validated against the component whitelist before storage. Nothing partial is ever persisted.

Authenticate with a Hub-issued API key sent as a Bearer token on every request:

Authorization: Bearer <your-api-key>

The key is an opaque random token; the Hub stores only its SHA-256 hash, so a database read can never reconstruct a live credential. A missing or unrecognized token returns 401. Keys belong to one of two account types, library (automated publishers) or human, which the authorization rules below distinguish.

Authorization is enforced server-side per request. The rules:

This is a small, trusted set of key-holders by design: every human is an admin, and there is no per-owner isolation between library keys beyond the ownership check on update. A write you aren't allowed to make returns 403.

{project}in a path is the parent project's slug.

Request bodies are strict; any unexpected key is rejected with a 422. A ? marks an optional field.

Errors use conventional HTTP status codes and a JSON body with a message. Validation failures (422) additionally carry an errors array of issues, each pointing at the offending path.

A 422 from a spec that fails whitelist validation looks like:

{
  "message": "invalid render spec",
  "errors": [
    { "code": "custom", "path": ["3", "rows", 2],
      "message": "row 2 has 4 cells but there are 3 headers" }
  ]
}

• Request body: up to 8 MB. A larger declared Content-Length is rejected with 422 before parsing.
• Spec size: up to 2000 top-level nodes.
• Nesting depth: layout containers (row, columns, tabs) may nest up to 32 levels deep.
• Images: src must be an http(s):// URL or a data:image/URI, up to 5 MB of characters.
• Per-field caps (string lengths, array sizes) are listed with each node in the whitelist below.

A spec is a JSON array of nodes. Each node is an object discriminated on a literal type. Only the types below are accepted; any other type, or any extra key on a node, is rejected. There is no raw-HTML or script node; this whitelist is the injection boundary.

Fields marked ?are optional; every default shown is what the Hub applies when the field is omitted. Charts enforce cross-field consistency (e.g. a line series' x and y must be the same length) and report a 422 pointing at the mismatch.

Structured publishing registers metrics, creates a campaign and run, then appends iterations containing timestamped measurements and values.

cURL

# Register a metric, then retain the returned id.
curl -X POST https://benchmark.infinity.inc/api/v1/metrics \
  -H "Authorization: Bearer $HUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"decode-throughput","name":"Decode throughput",
       "unit":"token/s","direction":"higher_is_better"}'

curl -X POST https://benchmark.infinity.inc/api/v1/campaigns \
  -H "Authorization: Bearer $HUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"slug":"qwen-neuralchip-x1","name":"Qwen on NeuralChip X1",
       "model":"Qwen 3.5 9B","hardware":"NeuralChip X1"}'

# Use the returned campaign, run, and metric UUIDs in subsequent requests.
curl -X POST https://benchmark.infinity.inc/api/v1/campaigns/$CAMPAIGN_ID/runs \
  -H "Authorization: Bearer $HUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"July run","workload":"interactive-decode",
       "harness_version":"1.12.0"}'

curl -X POST https://benchmark.infinity.inc/api/v1/runs/$RUN_ID/iterations \
  -H "Authorization: Bearer $HUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"iteration_number":0,"phase":"baseline",
       "observed_at":"2026-07-22T12:00:00.000Z",
       "measurements":[{"name":"interactive-decode",
         "observed_at":"2026-07-22T12:00:00.000Z",
         "values":[{"metric_id":"'"$METRIC_ID"'","unit":"token/s","value":120}]}]}'

TypeScript SDK

@infinity/benchmark-hub-client is an ESM-only package exposing the typed, dependency-free HubClient. Identical iteration retries return the persisted iteration with created set to false.

import { HubClient } from "@infinity/benchmark-hub-client";

const hub = new HubClient("https://benchmark.infinity.inc", process.env.HUB_API_KEY!);
const metric = await hub.registerMetric({
  key: "decode-throughput", name: "Decode throughput",
  unit: "token/s", direction: "higher_is_better",
});
const campaign = await hub.resolveCampaign("qwen-neuralchip-x1")
  ?? await hub.createCampaign({
    slug: "qwen-neuralchip-x1", name: "Qwen on NeuralChip X1",
    model: "Qwen 3.5 9B", hardware: "NeuralChip X1",
  });
const run = await hub.createRun(campaign.id, {
  name: "July run", workload: "interactive-decode", harness_version: "1.12.0",
});
await hub.transitionRun(run.id, { status: "active" });

const observedAt = new Date().toISOString();
const { iteration, created } = await hub.appendIteration(run.id, {
  iteration_number: 0, phase: "baseline", observed_at: observedAt,
  measurements: [{ name: "interactive-decode", observed_at: observedAt,
    values: [{ metric_id: metric.id, unit: metric.unit, value: 120 }],
  }],
});
console.log(created ? "created" : "identical retry", iteration.id);

Python

Publishers on ML hardware don't need the SDK; the structured API accepts plain JSON over HTTP.

import os, httpx

HUB = "https://benchmark.infinity.inc"
h = {"Authorization": f"Bearer {os.environ['HUB_API_KEY']}",
     "Content-Type": "application/json"}

metric = httpx.post(f"{HUB}/api/v1/metrics", headers=h, json={
    "key": "decode-throughput", "name": "Decode throughput",
    "unit": "token/s", "direction": "higher_is_better",
}).raise_for_status().json()
campaign = httpx.post(f"{HUB}/api/v1/campaigns", headers=h, json={
    "slug": "qwen-neuralchip-x1", "name": "Qwen on NeuralChip X1",
    "model": "Qwen 3.5 9B", "hardware": "NeuralChip X1",
}).raise_for_status().json()
run = httpx.post(f"{HUB}/api/v1/campaigns/{campaign['id']}/runs", headers=h,
                 json={"name": "July run", "workload": "interactive-decode",
                       "harness_version": "1.12.0"}).raise_for_status().json()

observed_at = "2026-07-22T12:00:00.000Z"
iteration = httpx.post(f"{HUB}/api/v1/runs/{run['id']}/iterations", headers=h, json={
    "iteration_number": 0, "phase": "baseline", "observed_at": observed_at,
    "measurements": [{"name": "interactive-decode", "observed_at": observed_at,
                      "values": [{"metric_id": metric["id"],
                                  "unit": metric["unit"], "value": 120}]}],
}).raise_for_status().json()