Embed

Bring your own embedding model without changing the client wire. An autoscaler-served profile can name any Hugging Face checkpoint the configured inference provider supports, including a finetuned checkpoint. Clients still send Turbopuffer-compatible embed schema and Embed query expressions.

For a stock model on Turbopuffer, omit embed.serving or set prefer: native. Layer validates and forwards the native wire. Choose prefer: autoscaler for a BYO checkpoint or Layer extensions such as revision pins, instructions, and chunking. Choose prefer: local for the CPU-only erikkaum/lattice-retrieval model or a locally provisioned CLIP-family model; prefer: lattice remains an alias for local Lattice serving.

The modes are explicit. A provider failure returns an error; Layer does not switch a request to another mode.

Query with Embed

Embed is the query half of schema-attribute embedding:

// source attribute: infer the model from its schema
"rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]]

// derived vector attribute: name the model explicitly
"rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
  "model": "acme/clinical-retrieval-v3"
}]]

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10,
})
print(response.rows)

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"text", "ANN", []any{"Embed", "chest pain radiating to left arm"}},
    TopK:   10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10
  }'

Native mode forwards Embed unchanged on Turbopuffer. Autoscaler mode resolves it through the configured provider, then sends a normal ANN vector to the store. Local mode resolves it in the gateway process. Query vectors are cached for 60 seconds by default; set LAYER_EMBED_CACHE_TTL_MS to change the TTL. A missing provider returns 503 service_unavailable.

Embed with Auto

An inline Embed lets query routing execute a semantic or fused leg in one request:

{
  "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
    "vector": ["Embed", "how plants turn sunlight into food", {
      "field": "text"
    }]
  }],
  "top_k": 10
}

The first tuple field (title) owns the lexical legs. field inside Embed independently selects the source or derived attribute whose schema supplies the embedding profile and vector target (text above). Omit field when both are the same. Layer chooses the route before resolving Embed, so a short input that selects hybrid_text does not call the embedding provider. An executed semantic or fused response keeps the automatic decision (routing.policy: "v1", routing.executed: true) and merges embedding measurements into performance. Without a vector or an inline Embed, the router returns routing.executed: false and leaves embedding to the caller.

Lattice

Lattice is a compact static retriever for text workloads where CPU throughput and deployment size matter more than transformer-level retrieval quality. It is an explicit serving leg and never falls back to native or autoscaler.

Generate a deployment artifact with the upstream Lattice slicer, place its model.safetensors and tokenizer.json together, and set LAYER_LATTICE_MODEL_PATH to the model file before starting the gateway. The supported model id is erikkaum/lattice-retrieval; the requested embed.dims must match the loaded artifact and only text modality is supported.

uv run slicer slice \
  --dim 512 \
  --quant int4_row \
  --output-dir /var/lib/hevlayer/lattice
export LAYER_LATTICE_MODEL_PATH=/var/lib/hevlayer/lattice/model.safetensors

"text": {
  "type": "string",
  "embed": {
    "model": "erikkaum/lattice-retrieval",
    "dims": 512,
    "serving": { "prefer": "lattice" }
  }
}

The recommended operating point is an int4-per-row, 512-dimensional Lattice artifact. Int4 quantizes the model’s lookup-table weights only. Layer writes the resulting normalized vectors as [512]f32; Turbopuffer’s int8 minimum for quantized vector storage is a separate choice and is not used by this path.

End-to-end example

Declare the Lattice profile on a string attribute, write rows, and query with Embed. The gateway embeds both sides in-process — no external inference provider is involved.

Write two rows into a namespace whose text attribute carries the Lattice profile shown above:

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/write" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "title": "Planet",
       "text": "Jupiter is the biggest planet in the Solar System."},
      {"id": "photo-1", "title": "Photosynthesis",
       "text": "Plants turn sunlight, water, and carbon dioxide into food."}
    ],
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "erikkaum/lattice-retrieval",
          "dims": 512,
          "serving": { "prefer": "lattice" }
        }
      }
    }
  }'

Query by meaning rather than exact phrase:

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"]
  }'

{
  "rows": [
    { "id": "planet-1", "$dist": 0.137, "title": "Planet",
      "text": "Jupiter is the biggest planet in the Solar System." }
  ],
  "performance": {
    "embedding_tokens": 7,
    "embedding_ms": 1   // in-process lookup — no network hop to a provider
  }
}

A live example of exactly this contract is the Wikipedia × Lattice demo: all 283,997 Simple English Wikipedia articles (1.74M paragraph rows) embedded through prefer: lattice and searched on Turbopuffer, with the performance echo displayed beside each result. Source at github.com/hev/wiki.

When rank_by names the source attribute, Layer reads the model from the schema. When it names embed_<attr>, the {model} argument is required; omitting it returns 422 with a model name must be provided.

Local CLIP

prefer: local on a CLIP-family model runs both CLIP towers in the gateway process on CPU: the image tower embeds attribute values at write time, and the text tower resolves Embed at query time against the same vector column. No GPU worker, autoscaler pool, or store-native embedding service is involved. Like Lattice, local CLIP is an explicit serving leg — an unconfigured leg returns a validation error rather than falling back to another mode.

Declare it on a string attribute holding an image URL or base64 image:

"image_url": {"type": "string", "embed": {"model": "openai/clip-vit-base-patch32", "modality": "image",
  "serving": {"prefer": "local"}}}

CPU CLIP fits query-time text embeds (milliseconds) and write-time image embeds for small-to-medium corpora. Bulk image backfills stay on prefer: autoscaler; the legs compose per namespace.

Model provisioning

Set LAYER_LOCAL_CLIP_MODEL_PATH to a directory holding the checkpoint’s four files — model.safetensors, tokenizer.json, config.json, and preprocessor_config.json — before starting the gateway. Unset means the leg is unavailable for CLIP models: writes and queries that select it return a validation error, with no silent fallback. A directory that fails to load stops the gateway at startup.

On Kubernetes, the Helm chart provisions the model with the opt-in gateway.localClip values: an init container downloads the four-file directory from a private S3 artifact prefix, verifies a pinned SHA-256 checksum for each file before the gateway starts, and mounts the directory read-only at gateway.localClip.modelPath (default /var/lib/hevlayer/clip) with LAYER_LOCAL_CLIP_MODEL_PATH set to it. A checksum mismatch fails the pod rather than serving unverified weights. Lattice and CLIP artifacts mount side by side, so one gateway can serve both local legs.

Text→image query

Query the image column with a text Embed; the gateway embeds the query through CLIP’s text tower in-process:

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"]
  }'

{
  "rows": [
    { "id": "commons-131", "$dist": 0.412, "title": "Sunset at Porto Covo",
      "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" }
  ],
  "performance": {
    "embedding_tokens": 5,
    "embedding_ms": 134   // CLIP text tower on the gateway CPU — no provider hop
  }
}

Write responses report embedding_images instead of embedding_tokens. A live example of exactly this contract is the lens demo: Wikimedia Commons Quality images embedded and searched through prefer: local, with each result showing the fixed serving contract (compute: gateway-in-process-cpu) beside the gateway’s performance echo. Source at github.com/hev/lens.

Image input

A local CLIP image profile accepts an HTTP(S) URL or base64 image string and fetches at most 20 MiB per URL. A rate-limited image host returns HTTP 429 with error upstream_error; a server error from the image host returns HTTP 503 with error service_unavailable. Layer preserves the host’s Retry-After response header for both statuses. Malformed URLs or base64, non-retryable 4xx responses, oversize content, and undecodable images return 422 validation_error.

BYO model settings

Use a provider-namespaced Hugging Face repo id. The autoscaler path does not apply a gateway allowlist: model load or support errors come from the configured provider.

• embed.revision pins a stock or finetuned checkpoint revision.
• embed.instructions.document and embed.instructions.query add the prefixes required by asymmetric retrieval models. Both affect the query-cache key.
• embed.modality: image embeds writes with a CLIP-family image tower and query text with its text tower. It may use prefer: autoscaler, or prefer: local without revision, instruction, or chunking extensions.
• embed.chunk splits source text before write-time embedding.

These fields are never forwarded upstream. Client-side interoperability is unchanged: applications use the same schema, Embed expression, and response shape for stock and BYO models.

Performance accounting

Write and query responses report provider measurements under performance:

{
  "rows": [ /* ... */ ],
  "performance": {
    "embedding_tokens": 8,
    "embedding_ms": 42
  }
}

Queries omit embedding_tokens on a cache hit. Layer merges autoscaler provider measurements into the same object and exposes echoed work through hevlayer_embed_tokens_total and hevlayer_embed_compute_seconds_total, labeled by namespace, store kind, model, and serving mode.