FAQ

Local AI

What the local model is, and what it runs on.

Which model powers local text generation?

A community conversion of gemma-4-E2B-it powers one-turn text generation on CPU.

Source: docs/generation.md (3 model identity), README.md (what runs)

Link to this answer

Can I test embeddings locally?

An opt-in ONNX build serves community EmbeddingGemma conversions. The second model accepts text, images (including HEIC, HEIF and AVIF), WAV and MP3 audio, video and PDFs through EmbedContent, and reads scanned PDF pages with Tesseract OCR. Use the documented download and load steps. v0.1.0 predates this runtime; v0.2.0 has it.

Source: docs/embeddings.md, docs/compatibility.md (vertex ai predictionservice googlecloudaiplatformv1)

Link to this answer

Does it use a GPU?

No. CPU only.

Source: docs/compatibility.md (vertex ai custom prediction)

Link to this answer
Develop locally

Build against Google Cloud APIs, locally

Free and open source under Apache-2.0. No account, no sign-up, no Google Cloud bill.

Get started