Misata Studio: verified synthetic data

Stop a running start_generation job. It stops at the next stage boundary and keeps nothing.

The reference for the engine's full design language (the `blueprint` argument of generate_dataset, start_generation and validate_blueprint): roles, distributions, formulas that read parent columns and draw noise, per-group sequences (seq, lag, cumsum, ar1 drift, random walk), aggregates, event windows, cause-and-effect, lifecycles, with patterns, and an index of chapters. Read it before writing a blueprint. Args: topic: a chapter's id from the index (or several, comma-separated) for that chapter in full, with a world that builds as written: JSON payloads (webhooks, audit trails), threads of messages, one party's duplicate records, running balances, rosters made from demand, a source system's raw drops and schema changes, planted problems for an eval. Empty for the reference.

The dataset's ground truth: every number it is held to, and the business's standard numbers (MRR, churn, billed revenue, win rate...), by month, worked out when it was made. Use it to test whatever answers questions about the data -- an agent, a dashboard, a cube, a text-to-SQL model -- and to say exactly why a wrong answer is wrong. Call it four ways: 1. dataset_id alone: the key's numbers (`kpis`: id, title, unit, months) and `questions` to ask, drawn from the key ("What is MRR at each month's end for 2025-07?"), each with an id. Their answers are not shown. 2. answers={question_id: number}: each answer graded within its tolerance; a wrong one is named as the usual misreading it equals (voided invoices counted, churn over every account), when it is one. 3. values with kpi: numbers computed anywhere (ClickHouse, a semantic layer, a BI tile), as one number, a {"2025-07": 123.4, ...} series by month, or [[month, value], ...]. Says whether they are that KPI (a tile is read as its latest month, the year to date, ...), and what they are instead when not. 4. sql (and kpi): a query over this dataset's own tables (DuckDB, as query_dataset), run here; a wrong one is told which KPI it computes, or the one change after which it returns the key. The values are plain numbers, so an answer from any engine is checked the same way; export_dataset(format="demo", dialect="clickhouse") hands over the same data and key to load into ClickHouse.

Export a generated dataset as a file. Returns a `download_url` the person can open (or you can fetch, e.g. with curl) for as long as the dataset is held, about 2 hours. Give the person the link rather than pasting file contents into the chat. Args: dataset_id: from a prior generate_dataset call. format: data: csv, parquet, jsonl, json, avro, xlsx, feather, orc, sqlite, duckdb, sql. code and docs: dbt, notebook, dictionary, dbml, mermaid, prisma, sqlalchemy, typescript, jsonschema, expectations, django, openapi, mockapi. `sql` is schema.sql (DDL with keys) + data.sql (COPY/INSERT) — the way to seed a real database: run the returned SQL through your own database connection, since this server never holds a database credential itself. `demo` is the handover for a demo or an eval: the database with a one-step loader, the ER diagram, answer_key.json (every number by month), questions.jsonl and the proof of each request. dialect: for `sql`: postgres, mysql, sqlite, mssql, oracle, bigquery, snowflake, clickhouse (MergeTree tables; above 500k rows, typed Parquet with load.sql). For `demo`: postgres (psql) or clickhouse (Parquet loaded with clickhouse-client). inline: also return the file itself as `base64` (only for files under a few MB). Use it when you must write the file yourself and cannot fetch a URL. Returns: filename, content_type, bytes, download_url, expires_at (unix seconds), and `base64` when `inline` and small enough.

Make a verified relational dataset and wait for it: every foreign key checked, dates correctly ordered, declared aggregates and rates exact, before anything is returned. Deterministic for a seed. Use this when you give a `schema` or `ddl` (seconds). For a plain-English `request` the engine designs the tables with a model, which can take several minutes: call start_generation instead and poll get_status, so the call does not sit open and time out. Args: request: Plain-English description (needs an LLM key unless `schema`/`ddl` is also given). With no key, design it yourself instead: blueprint_guide, then `blueprint` below. schema: A Misata schema dict for exact structural control. No key needed for structure. Its `numbers` (revenue by month, churn, MRR, shares, averages) are held exactly: see the server instructions. A column's `moves_with` ({"column": "customers.income", "strength": 0.6}) pairs it with another number of its row or its parent's, by rank, each keeping its own values. ddl: CREATE TABLE statements. No key needed for structure. Beside it, schema={"numbers": [...], "window": {...}} holds the story's numbers on these tables. seed: Reproducibility seed — the same request/schema and seed always produce the same rows. research: Ground realistic numbers (prices, growth rates) in real published facts via a web search. Off by default (an anonymous caller's request should not trigger external calls unless asked for); needs a key regardless of `schema`/`ddl`. blueprint: The engine's full design language (blueprint_guide has the reference): readings around a parent's nominal, drift, autocorrelation, cause-and-effect, event windows, exact aggregates. Use it whenever the data must behave like the real process. No key needed; run validate_blueprint on it first. story: A built-in shape held to exact numbers, the way Studio's plan screen states them. No key needed; built by code in seconds. {"pack": "saas_sales", "window": {"start": "2024-09", "end": "2026-08"}, "curves": [{"lever": "mrr_curve", "points": {"2024-09": 50000, "2026-08": 200000}}], "rates": [{"lever": "monthly_logo_churn", "value": 0.02, "events": [{"name": "Q3 churn spike", "start": "2025-07-01", "end": "2025-09-30", "multiplier": 2.5}]}]}. Shapes: saas_sales, ecommerce, fintech, healthcare. Every stated number is held exactly and read back in SQL; get_certificate has the answer key. `numbers` sets plain sizes and shares ({"customers": 3000, "products": 400, "return_rate": 0.06}). With no model key the shape's content is generic (ecommerce sells electronics, books, toys): write it yourself as `content`, e.g. {"catalog": [{"name": "Linen Duvet Cover", "category": "Bedding", "brand": "Fernhill", "price": 89.0}, ...]} (30-100 items in this business's own categories, at its real prices). Returns: dataset_id: pass this to get_certificate / query_dataset / export_dataset. passed: whether every check held. A dataset that did not pass is still returned, with `certificate.findings` saying what failed — inspect before trusting it. verification: a short, plain-language account of what was checked and what held — show it to the person as the proof, in place of asking them to take the data on trust. tables: {name: {columns, preview_rows (first 20), total_rows}} — the preview only; every row is in the stored dataset, reachable by query_dataset/export_dataset. certificate: the short form (claims, requirements, findings). get_certificate returns the rest (patterns, realism scorecard, every proof chart).

See the tables, sizes and relationships the engine would build, before any rows exist. Free (no rows are made), so it is worth calling before generate_dataset on anything non-trivial: review what it understood and assumed, then adjust your request or schema before spending a real call. Args: request: Plain-English description (needs an LLM key, unless `schema`/`ddl` is also given — then it is still used to ground realism, e.g. locale and what columns mean). schema: A Misata schema dict (see the server instructions for the format). No key needed for structure. ddl: CREATE TABLE statements. No key needed for structure. Returns: route: "chat" (not a dataset request — see `reply`), "design" (the model designed the tables) or "pack" (matched a built-in shape). tables: name, estimated rows, columns, foreign keys for each table the engine would build. understanding: what the engine read the request as (business, archetype, assumptions). requirements: every specific thing the request asked for, so you can see what was understood.

Read-only SQL over a dataset's own tables (one SELECT/WITH statement; every table name is a view over that dataset's own files — nothing else on the server is reachable this way). Args: dataset_id: from a prior generate_dataset call. sql: a single SELECT (or WITH ... SELECT) statement. No semicolons, file paths, or statements that write or read outside the dataset (checked before running). limit: rows returned (capped at 5000). Returns: columns, rows, truncated (whether more rows existed than `limit`).

Start a generation in the background and return a job_id at once. Use it for any plain-English `request` (a model designs the tables, which takes minutes) — then call get_status(job_id) every 20-30 seconds until it says done. Arguments are the same as generate_dataset. Returns: job_id, status "running". get_status gives the stage and, when finished, the dataset.

The ready-made datasets Misata publishes: free sample databases (direct download, public domain) and premium datasets with a full answer key (a free preview, then a one-off price). Check this first when someone wants sample, demo, practice or teaching data for a common scenario (retail, e-commerce, SaaS, manufacturing SPC, predictive maintenance, insurance claims, fraud/AML, clinical, network security): handing over a dataset that already exists is instant. If none fits their tables, generate one instead. Args: query: what the person needs, in their words. Only orders the list (closest first); every dataset is still returned, so judge the fit yourself from the tables and summary. Returns: datasets: [{slug, kind (free|premium), title, summary, rows, tables, page_url, and download_url (free) or free_preview_url + buy_url + price_usd (premium)}].

The full certificate for a dataset made by generate_dataset: every claim (stated vs actual), every requirement's status and evidence, realism findings, planted defects/anomalies if any, and the pattern charts. This is the whole answer key, not the trimmed form generate_dataset returns.

Where a start_generation job is. While running: the stage it is in and how long it has run. When done: the same answer generate_dataset returns (dataset_id, verification, tables, certificate).

Check a blueprint before generating it: every error said as what to change, the design traps it falls into (a column that will round away, a copy of a value not made yet, a window that counts nothing), its size against your row cap, and a small preview run (a few thousand rows) with the verifier's findings and sample rows, so you can see the data behave before the real call. Free. Returns: valid, errors (fix these), warnings (read these), planned (sizes the stated numbers set; "keep_size" on a table keeps its row_count), estimated_rows, row_cap, preview: {passed, findings, tables: {name: first rows}} when `preview`.

Which account this connection is, and what it may do: signed in with a Studio MCP key or anonymous, the row cap, and whether a model key is available for plain-English requests.