platform
Everything between your application and the model
One API to reach every model, and everything you need around it: choosing models, seeing what they cost, searching your documents, building assistants, and testing before you switch. Each section names the endpoints behind it, so you can check every claim against the reference.
01 · gateway
One API for every model
Send one request and reach models from Anthropic, OpenAI, Google, Azure and your own servers. Switching the model behind a feature is a setting, not a rewrite.
curl https://app.aethergridlabs.com/v1/chat/completions \
-H "Authorization: Bearer $AETHERGRID_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "Summarise this incident."}],
"strategy": "cost",
"max_tokens": 512
}'{
"request_id": "req_01j9r2m8k4x7v3p6q1w5e8t2y0",
"content": "Two services lost quorum at 09:41 after…",
"finish_reason": "stop",
"model": "llama-3.3-70b-instruct",
"provider": "self-hosted",
"routing_strategy": "cost",
"routing_reason": "cheapest eligible candidate at $0.12/M input",
"did_fallback": false,
"attempt_count": 1,
"usage": {
"input_tokens": 1840,
"output_tokens": 212,
"is_estimated": false
},
"latency_ms": 318.4,
"cost_usd": 0.000284,
"cost_is_estimated": true,
"customer_cost_usd": 0.000341
}cost_usd is the provider's price; customer_cost_usd is what you pay, with the service markup added. An unknown cost is null, never zero.Six providers sit behind a single request shape: Anthropic, OpenAI, Google, Azure OpenAI, Experiential Labs, and any server that speaks the OpenAI format. Your code talks to AetherGrid; AetherGrid talks to whichever provider the model lives with.
A provider you have not connected is simply skipped. Starting with one provider key is normal, and adding more later changes nothing in your application.
Models are records you can see and manage, not a list baked into the software. When a provider ships a new model, it appears in your catalogue.
02 · routing
Smart routing
Tell AetherGrid what matters for each request, cost, speed, quality or a balance of all three, and it picks the model. You always get told which one it chose and why.
| model | adapter | p50 | your $/M in | tier | state |
|---|---|---|---|---|---|
| claude-opus-4.6 | anthropic | 1420 ms | 18.00 | 4 · frontier | same tier, 1420 ms at $18.00/M |
| gpt-5.1 | openai | 1180 ms | 1.50 | 4 · frontier | chosen |
| claude-sonnet-4.5 | anthropic | 940 ms | 3.60 | 3 · balanced | eligible |
| gemini-2.5-pro | 1260 ms | 1.50 | 3 · balanced | circuit breaker open | |
| deepseek-v3.2 | self-hosted | 640 ms | 0.34 | 3 · balanced | eligible |
| mistral-large-3 | self-hosted | 290 ms | 0.24 | 3 · balanced | eligible |
| claude-haiku-4.5 | experiential-labs | 460 ms | 1.20 | 2 · fast | eligible |
| gemini-2.5-flash | 520 ms | 0.36 | 2 · fast | circuit breaker open | |
| gpt-5-mini | azure-openai | 610 ms | 0.30 | 2 · fast | adapter unconfigured — no credential |
| llama-3.3-70b-instruct | self-hosted | 330 ms | 0.14 | 2 · fast | eligible |
rules from the project's routing policy: require_tools · min_context_window 128,000 · prices and response times are illustrative
{
"primary": {
"slug": "gpt-5.1",
"provider": "openai"
},
"fallbacks": [
{ "slug": "claude-opus-4.6", "provider": "anthropic" }
],
"strategy": "balanced",
"reason": "highest tier (frontier); then speed and cost together, 1180 ms at $1.50/M",
"considered": 7,
"eliminated": [
{ "slug": "gemini-2.5-pro", "reason": "circuit breaker open" },
{ "slug": "gemini-2.5-flash", "reason": "circuit breaker open" },
{ "slug": "gpt-5-mini", "reason": "adapter unconfigured — no credential" }
]
}Before any choice is made, models that cannot serve the request are removed: providers you have not connected, providers that are currently failing, and models without the abilities the request needs.
If a provider goes down, AetherGrid notices quickly and stops sending traffic there instead of waiting on every call. Requests fall back to the next best model automatically.
Every response says which strategy ran, why that model won, whether a fallback happened, and how many attempts it took.
03 · agents
Agents and workflows
Build assistants that use tools, and multistep workflows that chain models, searches and web calls together. They run through the same gateway, so they are logged and priced like everything else.
- 012 ms
triggertrigger
POST /v1/workflows/{id}/runs
- 02184 ms
fetchhttp_request
GET status-page incidents · 200
- 0396 ms
contextknowledge
runbooks · 4 chunks · pgvector hnsw
- 041,204 ms
draftmodelgpt-5.1
strategy: quality
req_01j3l9jp6c… · 3,860 / 412 tokens · $0.008945
- 051 ms
severitycondition
output.severity in [sev1, sev2] → true
- 063 ms
shapetransform
jsonpath · pick title, summary, owners
- 07221 ms
notifyhttp_request
POST on-call webhook · 202
- 081 ms
outputoutput
output_payload
- status
- succeeded
- latency_ms
- 1,712
- total_tokens
- 4,272
- total_cost_usd
- $0.008945
- 01succeeded88 ms
search_kbknowledge_search
help-centre · top 3 · score ≥ 0.72
- 02succeeded142 ms
lookup_orderhttp_request
GET /orders/{id} · 200
- 03succeeded4 ms
redactcode_transform
strip card digits before the model sees them
- 04succeeded318 ms
compose_replymodel_callllama-3.3-70b-instruct
cheapest eligible candidate
req_01j1u9f28g… · 2,140 / 186 tokens · $0.000313
- status
- succeeded
- latency_ms
- 552
- total_tokens
- 2,326
- total_cost_usd
- $0.000313
Agents can call four kinds of tools: web requests, searches over your documents, small code transforms, and further model calls. Every run is saved so you can see what the agent did.
Workflows are graphs built from nine step types, including model calls, conditions, web requests and document lookups, and are checked for mistakes before they are saved.
Runs execute while your request is open. A long workflow can take minutes, so give it a generous timeout. This is stated plainly because it is the fact that most changes how you use it.
04 · knowledge
Your documents, searchable
Upload documents and let models answer from them. AetherGrid extracts, splits and indexes the content, and every answer can cite where it came from.
| filename | status | pages | chunks | size |
|---|---|---|---|---|
| incident-response.pdf | ready | 42 | 187 | 1,840 kB |
| on-call-handbook.md | ready | — | 64 | 96 kB |
| postgres-failover.pdf | ready | 18 | 71 | 620 kB |
| q3-postmortems.docx | embedding | 27 | — | 1,210 kB |
| vendor-slas.pdf | extracting | — | — | 3,480 kB |
uploaded → extracting → chunking → embedding → ready. A document that is still being processed says so, rather than quietly returning no matches.
01incident-response.pdf · p.140.87
When a provider's 5xx rate exceeds the breaker threshold, the gateway opens the circuit and fails over to the next eligible candidate…
02on-call-handbook.md · §3.20.81
Page the on-call for the affected integration only if fallback traffic is also failing; a single-provider outage is routed around…
03postgres-failover.pdf · p.60.64
Provider-side incidents are distinct from datastore failover; see the incident-response runbook for the routing procedure…
retrieval_mode: vector · citation on every chunk
Knowledge bases accept files, extract the text, split it into passages and index them for fast similarity search.
A document that is still being processed says so, rather than quietly returning nothing.
The same search is available directly, as a step in a workflow, or as a tool for an agent, so you build the feature once.
05 · observability
See every request
Every call is recorded with its model, provider, response time, token counts, cost and the routing decision behind it, and you can look any of them up by id.
Requests
140,400
6 models · 4 adapters
p50 latency
405 ms
weighted by requests, from the log
Tokens in / out
324.4M / 24.7M
cached input not shown
Spend
$294.82
20% of a $1,500 advisory budget
| model | requests | spend | share |
|---|---|---|---|
| claude-haiku-4.5 | $143.52 | 49% | |
| gpt-5.1 | $72.00 | 24% | |
| claude-sonnet-4.5 | $32.04 | 11% | |
| llama-3.3-70b-instruct | $21.08 | 7% | |
| deepseek-v3.2 | $14.75 | 5% | |
| mistral-large-3 | $11.42 | 4% |
Spend is what you pay: the provider's reported cost plus the service markup. Every figure says whether the provider reported it or it was estimated. The cheapest model here carries 44% of requests for 7% of spend, which is exactly what routing by cost is for.
- mistral-large-3290
- llama-3.3-70b-instruct330
- claude-haiku-4.5460
- gemini-2.5-flash520
- gpt-5-mini610
- deepseek-v3.2640
- claude-sonnet-4.5940
- gpt-5.11,180
- gemini-2.5-pro1,260
- claude-opus-4.61,420
your own servers. Speed is measured from this workspace's own requests, not from a vendor's status page.
The request log, response time charts and time series are part of the product, not an export. The console shows you the same data the API gives you.
Usage, cost and evaluation results are all built from the request log, so the numbers agree with each other by construction.
Health checks report each provider's state, so you can see a failing provider before your users do.
06 · evaluations
Evaluations
Test models against your own examples, score the answers, and compare runs before you switch. Results are tied to the exact version of the test set they ran against.
| case | scorer | score | vs base |
|---|---|---|---|
| refund-window | contains | 1.00 | same |
| partial-refund-math | exact_match | 1.00 | ▲ improved |
| policy-json | json_equals | 1.00 | same |
| order-id-format | regex | 1.00 | same |
| tone-apology | llm_judge | 0.62 | ▼ regressed |
| escalation-path | contains | 1.00 | same |
- passed_cases
- 5 / 6
- average_score
- 0.94
- regressions
- tone-apology
- total_cost_usd
- $0.0069
1 improvement, 1 regression against the baseline run on the same version of the test set. Only runs on the same version are compared, so editing your examples can never look like a regression.
Four scorers give exact, deterministic marks: exact match, contains, pattern match and JSON equality. A fifth asks a model to judge the answer, and that call is priced like any other.
A run can be compared against a previous one, and only runs on the same version of the test set are comparable, so editing your examples can never look like a regression.
Every test case is a real request through the gateway, visible in the request log with its cost.
07 · cost
Cost control
Know what each request cost and where your money goes, by model, provider and day. Set a monthly budget per project, as a warning or a hard stop.
{
"plan": "pro",
"month_start": "2026-09-01",
"spend_usd": 245.68,
"customer_spend_usd": 294.82,
"budget_usd": 1500.00,
"percent_used": 16.4,
"is_over_budget": false,
"is_over_threshold": false,
"enforced": false,
"requests_this_month": 140400,
"request_limit": 250000,
"unpriced_requests": 0
}Over an enforced budget, further requests are refused until it is raised or the month rolls over. Over an advisory one, they are served and the console says so.
Cost is taken from the provider whenever the provider reports it, and estimated from the model's list price otherwise. Every figure says which of the two it is.
A cost that cannot be determined is shown as unknown, never as zero, so an incomplete price list can never make spending look smaller than it is.
Budgets are per project and per month. Over an enforced budget, further requests are refused; over an advisory one, they are served and flagged.
08 · api
Built for developers
One REST API with a published OpenAPI reference, and project keys for live and test environments so staging can never spend production budget.
| name | key | scopes | last_used | state |
|---|---|---|---|---|
| support-agent | ag_live_••••••••3f9c | inference:runrequests:read | 2s ago | active |
| search-rerank | ag_live_••••••••a71e | inference:run | 4s ago | active |
| triage | ag_live_••••••••c204 | inference:runrouting:read | 9s ago | active |
| eval-runner | ag_test_••••••••8d1b | inference:runevaluations:write | 1m ago | active |
| ci-smoke | ag_test_••••••••5e77 | inference:run | revoked 3d ago | revoked |
Every action is checked twice: against the key's permissions and against the role of the person who owns it (viewer, member, admin or owner). Both must allow it. A lost key is revoked and reissued; it cannot be recovered, because it was never stored.
Authenticate with a project API key as a bearer token. A key belongs to exactly one project, so requests never have to name one.
Keys are issued per environment, live or test, and can be scoped to only the actions an integration needs.
The OpenAPI reference is generated from the running service, so it always describes what is actually deployed.
09 · security
Security and access control
Four roles, scoped keys, and provider credentials that never leave the server. Every action is checked against both the key and the person behind it.
| permission | viewer | member | admin | owner |
|---|---|---|---|---|
| inference:run | ||||
| playground:use | ||||
| models:read | ||||
| models:write | ||||
| providers:read | ||||
| providers:write | ||||
| routing:read | ||||
| routing:write | ||||
| knowledge:read | ||||
| knowledge:write | ||||
| agents:read | ||||
| agents:write | ||||
| workflows:read | ||||
| workflows:write | ||||
| evals:read | ||||
| evals:write | ||||
| requests:read | ||||
| usage:read | ||||
| audit:read | ||||
| projects:read | ||||
| projects:write | ||||
| keys:read | ||||
| keys:write | ||||
| members:write | ||||
| org:read | ||||
| org:write | ||||
| billing:read | ||||
| billing:write |
minimum role · inherited. Every action is checked against this and against the API key's scopes; both must allow it.
Every action is checked twice: against the API key's scopes and against the role of the member who owns it. Both must allow it.
API keys are stored hashed and shown once, at creation. A lost key is revoked and reissued, never recovered.
Provider credentials live only on the server. They are never returned by any endpoint and never reach the browser.