A community BigQuery emulator, clearly labelled.
Datasets, tables, queries, DDL, scripting, loads, the Storage APIs and BigQuery ML in every Resource Manager project, on CloudBurrow's patched build of the community emulator.
Backed by: goccy/bigquery-emulator 0.8.1, built by CloudBurrow with patches and a validating front Community emulator MIT
Get startedWhat it covers
Start the community emulator with --services bigquery. CloudBurrow builds the emulator from pinned modules with patches to its SQL engine, embeds it in the CLI and puts a validating front before it, which holds IDs, schemas, rows and statements to BigQuery’s rules and answers 501 for what it cannot run. Every project Resource Manager has is served. Remote models, AI functions and image functions run on the instance’s own local AI, never on Google. The compatibility notes distinguish tested SQL shapes, Storage API operations and REST methods. Follow the named tests when deciding whether a query belongs in your local suite. The emulator keeps nothing across a restart.
- Datasets and tables, streaming inserts (
tabledata.insertAll) andSELECTqueries - Every ACTIVE Resource Manager project is a BigQuery project: datasets, tables and jobs per project, and queries, DML, loads, copies, clones and snapshots across projects (#1764, #1767, #1781)
- A name without a project resolves in the job's project: tables, views, functions, procedures and models (#1784)
- Per-project IAM for a caller with an access token this instance minted: jobs, datasets, tables, routines, models and authorized views, with inherited roles and IAM Conditions (#1767, #1774)
- DML, DDL, views, materialized views, scripts, procedures and multi-statement transactions
- Partitioned and clustered tables, time travel, change history and
INFORMATION_SCHEMAviews - Loads of CSV, JSON, Avro, Parquet and ORC from uploads or Cloud Storage, and extracts to Cloud Storage
- The Storage Read API (Arrow and Avro) and Storage Write API streams with protobuf or Arrow rows
- BigQuery ML: trained models from linear regression to ARIMA_PLUS, and imported XGBoost, TensorFlow Lite and TensorFlow SavedModel models (#1711, #1775)
- Remote models over the instance's local Vertex AI endpoint, never Google:
ML.GENERATE_TEXTandML.GENERATE_EMBEDDING, withmax_output_tokens,temperature,top_pandstop_sequencesapplied (#1760, #1797) - AI functions on the local endpoint:
AI.GENERATEwithoutput_schema,AI.GENERATE_BOOL,_INT,_DOUBLEandAI.GENERATE_TABLEthrough constrained decoding, andAI.GENERATE_EMBEDDING,AI.EMBED,AI.SIMILARITY,AI.CLASSIFY,AI.IFandAI.SCORE(#1771, #1772) - Object tables over the instance's Cloud Storage, with metadata caching and the connection's permissions checked (#1783)
- Image functions over object tables:
ML.DECODE_IMAGE,ML.RESIZE_IMAGE,ML.CONVERT_COLOR_SPACE,ML.CONVERT_IMAGE_TYPE, andML.PREDICTof an imported TensorFlow Lite or TensorFlow image model (#1798) - Vision text detection: a
CLOUD_AI_VISION_V1remote model andML.ANNOTATE_IMAGEwithTEXT_DETECTIONandDOCUMENT_TEXT_DETECTION, read locally with Tesseract (#1773) - The BigQuery Connection API, with connection IAM; remote models and object tables need
bigquery.connections.useon their connection (#1755, #1765) - Table, routine and row access policy IAM, stored, and generated BigQuery REST API coverage with named tests
In the console
What's verified
Follow the operation matrix to see what the backend serves and the tests linked to Verified calls.
BigQuery (REST API v2)
55 of 55 RPCs Verified in v0.3.0
Not supported
- 01 / Local boundary
BigQuery data is in memory and is gone after a restart, whatever
--modesays. A project Resource Manager does not have is 404; create or seed it first. - 02 / Local boundary
Imported ONNX models need the opt-in ONNX build. Vision features other than text detection, the other Cloud AI service types, and
safety_settingsanswer 501 with the reason. - 03 / Local boundary
Stored policies are not checked for requests without a minted token: the ADC fixture's token and unauthenticated calls are served as before.
Source: docs/compatibility.md at main 0ea2594.
Questions
Is this Google's BigQuery emulator?
Can I use more than one project?
Do remote models and AI functions call Google?
Build against Google Cloud APIs, locally
Free and open source under Apache-2.0. No account, no sign-up, no Google Cloud bill.