An embedding integration needs to send the right input parts and handle the vector and usage metadata that come back. CloudBurrow can exercise that request path locally with two pinned community ONNX conversions. This guide uses the development branch at 73443bd and the opt-in ONNX build from source. The v0.1.0 release supports local text generation. The embedding workflow below landed after that release. Local AI setup and availability · v0.1.0 Local AI notes
Build and enable the local runtime
- Run
make build-onnxfrom the pinned checkout to producebin/cloudburrow-onnx. The build uses cgo and fetches a pinned tokenizer library. ONNX build target - Set
ONNXRUNTIME_LIBto the path of your native ONNX Runtime library, then runbin/cloudburrow-onnx up -local-ai-embeddings. On first use the flag fetches the first model over HTTPS into the local state directory. CloudBurrow checks each file’s SHA-256 before use. Inference runs locally once the model files are available. Build and startup instructions - For the second model, open Vertex AI → Local models in the console. Choose Download, confirm the size, then choose Load. The startup flag fetches only the first model. Second model setup
Match the model to your inputs
embeddinggemma-300m-onnx-community accepts text through :predict or :embedContent and the corresponding gRPC methods. Start here if your fixtures contain only text. The SDK compatibility test checks that returned vectors have the requested dimensions and that their values are finite. Pinned model and endpoints · SDK embedding test
embeddinggemma-2-onnx-community accepts text, static PNG, JPEG, WebP or BMP images, and WAV audio up to 180 seconds per request through EmbedContent. One content item can combine these parts into a single vector. Pass supported media inline or reference an object in the same instance’s Cloud Storage if Storage is enabled. :predict embeds text. Video, PDF, MP3 and animated WebP fall outside this path. Media support and limits
Try an audio fixture from local Storage
With the second model loaded, upload a short WAV fixture to a local bucket and set the object’s content type to audio/wav. In Vertex AI → Embeddings, select the second model and enter the gs:// reference in Cloud Storage objects. Run EmbedContent and inspect the returned vector and its dimension. Try the same fixture through Audio to compare the two input paths. Console input controls
The compatibility test embeds the same WAV as inline bytes and as a local Storage object, then checks that the two vectors have a cosine similarity of at least 0.99999. It checks the result from the console’s request path against the SDK vector too. A missing object returns an error. Audio and Storage test
Keep those assertions tied to the model and fixtures you run. The published real-model evidence comes from developer-machine runs, including comparisons with Transformers.js on specific inputs. Their results give you a reference for local preprocessing and response handling. If the application depends on how a hosted model ranks search results, evaluate that separately with its own inputs and model. Local AI compatibility evidence · Vertex AI coverage on main
← All articles