feat(embeddings): store knowledge bases at five vector widths and add Ollama - #7472
Open
icecrasher321 wants to merge 1 commit into
Open
feat(embeddings): store knowledge bases at five vector widths and add Ollama#7472icecrasher321 wants to merge 1 commit into
icecrasher321 wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR expands knowledge-base embedding storage and search from one fixed vector width to five supported widths and adds Ollama as a local embedding provider.
Confidence Score: 5/5The PR appears safe to merge; no concrete blocking or independently actionable non-blocking defects were identified. Storage, indexing, embedding generation, and search consistently carry the persisted model-and-width target, while the migration follows the repository’s established replay-safe concurrent-index pattern and Ollama execution retains bounded, validated request handling.
|
| Filename | Overview |
|---|---|
| packages/db/migrations/0321_multi_width_embeddings.sql | Adds replay-safe multi-width vector storage, an exactly-one-width constraint, and concurrent HNSW indexes, including the required halfvec expression for 3,072 dimensions. |
| packages/db/schema.ts | Aligns the Drizzle schema with five vector columns, their indexes, and the width exclusivity constraint. |
| apps/sim/lib/knowledge/vector-columns.ts | Centralizes width-to-column writes and index-compatible distance expressions so storage and retrieval select the same representation. |
| apps/sim/lib/knowledge/search/queries.ts | Carries query vectors with their dimensions and consistently applies dimension-aware distance expressions across search modes. |
| apps/sim/lib/knowledge/embeddings.ts | Resolves deployment model/width configuration and validates every indexing and query embedding against the persisted target. |
| apps/sim/lib/embeddings/client.ts | Adds Ollama provider resolution while retaining request timeout, response limits, and vector-shape validation. |
| apps/sim/lib/embeddings/ollama-model-catalog.server.ts | Discovers local Ollama models with bounded concurrency, cancellation, response limits, capability filtering, and dimension metadata. |
| apps/sim/lib/knowledge/application/search.ts | Rejects mixed embedding targets and derives query generation and database selection from persisted knowledge-base metadata. |
| apps/sim/blocks/blocks/embeddings.ts | Adds a keyless Ollama block option backed by live model discovery and removes unsupported task-type and dimension inputs. |
| packages/deployment-config/src/env-capabilities.ts | Models provider readiness according to the embedding family selected by KB_EMBEDDING_MODEL and validates family-specific widths. |
| packages/sim-setup/src/capability-config.ts | Adds guided Gemini and Ollama knowledge-embedding configuration while preserving existing OpenAI-family setup. |
| apps/sim/app/api/v1/knowledge/search/route.ts | Propagates persisted vector dimensions through public API search and rejects incompatible cross-knowledge-base requests. |
Sequence Diagram
sequenceDiagram
participant Config as Deployment config
participant KB as Knowledge base
participant Embed as Embedding provider
participant DB as pgvector storage
participant Search as Search service
Config->>KB: Select model and supported width
KB->>KB: Persist model + embedding dimension
KB->>Embed: Embed document at stored target
Embed-->>KB: Validated vector
KB->>DB: Write exactly one width-specific column
Search->>KB: Load persisted model + width
Search->>Embed: Embed query at same target
Search->>DB: Query matching column/index
DB-->>Search: Ranked chunks
Reviews (1): Last reviewed commit: "feat(embeddings): store knowledge bases ..." | Re-trigger Greptile
… Ollama Knowledge bases were pinned to one 1536-dimension pgvector column, so only models that emit exactly that width could index one. The embedding table now carries a column per storable width — 384, 768, 1024, 1536, 3072 — and a base records which one it uses at creation, chosen with EMBEDDING_OUTPUT_DIMS. Ollama becomes an embedding provider on both paths: KB_EMBEDDING_MODEL accepts ollama/<model> for knowledge bases, and the Embeddings block offers the models installed on the configured server, with no API key and nothing billed.
icecrasher321
force-pushed
the
staging-v104
branch
from
September 4, 2026 02:32
af27f67 to
d1b7ba9
Compare
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
15 issues found across 77 files
Confidence score: 2/5
packages/db/migrations/0321_multi_width_embeddings.sqlleaves the originalembeddingcolumnNOT NULL, so inserts for 384-, 768-, 1024-, and 3072-dimensional knowledge bases still fail. Remove the column-level constraint before writing rows with the new widths.apps/sim/lib/knowledge/documents/service.tssends batches of up to 2,000 chunks, but 3072-dimensional bases reject batches over 1,064 before provider batching occurs. Cap the shared batch size at the aggregate limit.apps/sim/lib/embeddings/catalog.tsandapps/sim/lib/embeddings/client.tscan select the wrong width for Ollama or existing OpenAI bases after model/default changes, causing indexing or queries to fail. Resolve the installed model width and honor each base’s configured dimensions before generating embeddings.packages/deployment-config/src/env-capabilities.tstreats unsupported or case-variant Gemini IDs as configured even when runtime falls back to OpenAI, which can leave document and query paths without usable Gemini credentials. Align capability detection with the runtime model-resolution behavior.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/deployment-config/src/integrations.json">
<violation number="1" location="packages/deployment-config/src/integrations.json:7463">
P2: The description overstates Ollama support: chat or other non-embedding models on the server are not valid Embeddings models, and the block explicitly filters them out. Change this to “any embedding-capable model on a self-hosted Ollama.”</violation>
</file>
<file name="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx">
<violation number="1" location="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx:214">
P2: This walkthrough is not valid for every self-hosted deployment: `sim-setup add` does not manage Helm or Ollama-stack installs. Qualify it for supported production/local-development Compose installs and state that an Ollama-stack install must be configured separately.
(Based on your team's feedback about sim-setup scope.)</violation>
<violation number="2" location="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx:215">
P2: `sim-setup status` is the lifecycle health command and does not render capability/provider status. Use `sim-setup config` here, which invokes the capability report.</violation>
</file>
<file name="apps/sim/lib/knowledge/embedding-models.ts">
<violation number="1" location="apps/sim/lib/knowledge/embedding-models.ts:76">
P2: When `KB_EMBEDDING_MODEL` is `toString` or `constructor`, this predicate treats an inherited property as a supported model. `getConfiguredKbEmbedding` then crashes while reading `dimensions` instead of falling back; check for an own property here.</violation>
</file>
<file name="apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx">
<violation number="1" location="apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx:255">
P2: When an Ollama model emits a width outside the five storable widths, setting `EMBEDDING_OUTPUT_DIMS` to that width falls back to 1536, so recreating the base repeats the failure. Tell operators to use a model that emits one of the supported widths when the reported width is unsupported.</violation>
</file>
<file name="packages/db/migrations/0321_multi_width_embeddings.sql">
<violation number="1" location="packages/db/migrations/0321_multi_width_embeddings.sql:46">
P1: When a knowledge base uses 384, 768, 1024, or 3072 dimensions, inserts still fail because the original `embedding` column remains `NOT NULL`. Drop the column-level constraint before new-width rows are written; dropping the check constraint alone does not make the column nullable.</violation>
</file>
<file name="apps/sim/lib/knowledge/documents/service.ts">
<violation number="1" location="apps/sim/lib/knowledge/documents/service.ts:334">
P1: When a 3,072-dimensional base processes more than 1,064 chunks, this 2,000-item batch reaches `generateEmbeddings`, whose aggregate guard rejects it before provider batching. Cap the shared batch size at the maximum supported width or derive it from the base width.</violation>
</file>
<file name="packages/sim-setup/src/capability-config.ts">
<violation number="1" location="packages/sim-setup/src/capability-config.ts:752">
P2: When configuring Ollama at 384 dimensions, the wizard rejects the value before transition validation. Skip generic prompt validation for this provider and rely on the transition inspection, which applies Ollama's own width rule.</violation>
</file>
<file name="apps/docs/content/docs/integrations/embeddings.mdx">
<violation number="1" location="apps/docs/content/docs/integrations/embeddings.mdx:22">
P2: When Ollama omits capability or embedding-length metadata, the catalog keeps the model and the selector omits its width, so this sentence promises filtering and labels that the block cannot guarantee. Qualify the claims to say non-embedding capability metadata is filtered and widths are shown only when Ollama reports them.</violation>
</file>
<file name="apps/sim/lib/embeddings/catalog.ts">
<violation number="1" location="apps/sim/lib/embeddings/catalog.ts:272">
P1: When `KB_EMBEDDING_MODEL=ollama/nomic-embed-text` and `EMBEDDING_OUTPUT_DIMS` is unset, knowledge-base indexing always targets 1536 even though the model emits 768. Resolve the installed model's width before selecting the default, or require a verified width for Ollama instead of using the platform default.</violation>
</file>
<file name="apps/sim/lib/embeddings/ollama-model-catalog.server.ts">
<violation number="1" location="apps/sim/lib/embeddings/ollama-model-catalog.server.ts:109">
P2: When a configured Ollama server is unreachable, this fallback makes metadata resolution report the selected model as missing and returns HTTP 400 instead of 502. Preserve the empty-list behavior for catalog/selector requests, but propagate reachability failures separately when resolving a selected model so callers can classify the outage correctly.</violation>
</file>
<file name="apps/sim/lib/embeddings/client.ts">
<violation number="1" location="apps/sim/lib/embeddings/client.ts:981">
P1: When a deployment changes its default from `text-embedding-3-small` at 1536 to `text-embedding-3-large` at 3072, existing small-model bases fail before OpenAI is tried. Override `EMBEDDING_OUTPUT_DIMS` with the base’s `dimensions` (or omit it) when building `capabilityValues`, so capability validation follows the persisted target rather than the deployment default.</violation>
</file>
<file name="apps/sim/app/api/v1/knowledge/search/route.ts">
<violation number="1" location="apps/sim/app/api/v1/knowledge/search/route.ts:199">
P2: When a tag-only request targets a knowledge base with an unsupported persisted width, this unconditional `toKbEmbeddingDimensions` call returns a 500 before tag filtering runs. Build and validate embedding targets only when `hasQuery` is true.</violation>
</file>
<file name="packages/deployment-config/src/env-capabilities.ts">
<violation number="1" location="packages/deployment-config/src/env-capabilities.ts:1239">
P1: When `KB_EMBEDDING_MODEL` is an unsupported Gemini ID or case variant, this predicate reports Gemini as configured, but runtime falls back to `text-embedding-3-small`. With only Gemini credentials, document and query embedding then request OpenAI without an OpenAI key and fail; classify only IDs accepted by the runtime.</violation>
<violation number="2" location="packages/deployment-config/src/env-capabilities.ts:1297">
P2: When OpenRouter is the only configured OpenAI transport, unsupported `EMBEDDING_OUTPUT_DIMS` values still make status report the capability as configured because this provider skips the new width validation. Add the OpenAI model and dimension optional fields to OpenRouter, and validate the selected model rather than only the family.</violation>
</file>
| -- migration-safe: replaced by embedding_width_check, added above and in force before this runs. | ||
| -- The replacement rejects exactly what this rejected for the currently deployed application, which | ||
| -- only ever populates "embedding", so no write either app version makes is newly accepted. | ||
| ALTER TABLE "embedding" DROP CONSTRAINT IF EXISTS "embedding_not_null_check";--> statement-breakpoint |
Contributor
There was a problem hiding this comment.
P1: When a knowledge base uses 384, 768, 1024, or 3072 dimensions, inserts still fail because the original embedding column remains NOT NULL. Drop the column-level constraint before new-width rows are written; dropping the check constraint alone does not make the column nullable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/db/migrations/0321_multi_width_embeddings.sql, line 46:
<comment>When a knowledge base uses 384, 768, 1024, or 3072 dimensions, inserts still fail because the original `embedding` column remains `NOT NULL`. Drop the column-level constraint before new-width rows are written; dropping the check constraint alone does not make the column nullable.</comment>
<file context>
@@ -0,0 +1,72 @@
+-- migration-safe: replaced by embedding_width_check, added above and in force before this runs.
+-- The replacement rejects exactly what this rejected for the currently deployed application, which
+-- only ever populates "embedding", so no write either app version makes is newly accepted.
+ALTER TABLE "embedding" DROP CONSTRAINT IF EXISTS "embedding_not_null_check";--> statement-breakpoint
+-- Ends the runner's batch transaction (a redundant COMMIT is a WARNING, not an error). Every
+-- statement below runs in autocommit so that no scan or index build holds the batch's locks.
</file context>
Suggested change
| ALTER TABLE "embedding" DROP CONSTRAINT IF EXISTS "embedding_not_null_check";--> statement-breakpoint | |
| ALTER TABLE "embedding" ALTER COLUMN "embedding" DROP NOT NULL;--> statement-breakpoint | |
| ALTER TABLE "embedding" DROP CONSTRAINT IF EXISTS "embedding_not_null_check";--> statement-breakpoint |
| MAX_EMBEDDING_BATCH: Math.min( | ||
| envNumber(env.KB_CONFIG_BATCH_SIZE, 2000, { min: 1, integer: true }), | ||
| getEmbeddingAggregateItemLimit(EMBEDDING_DIMENSIONS) | ||
| getEmbeddingAggregateItemLimit(DEFAULT_KB_EMBEDDING_DIMENSIONS) |
Contributor
There was a problem hiding this comment.
P1: When a 3,072-dimensional base processes more than 1,064 chunks, this 2,000-item batch reaches generateEmbeddings, whose aggregate guard rejects it before provider batching. Cap the shared batch size at the maximum supported width or derive it from the base width.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/lib/knowledge/documents/service.ts, line 334:
<comment>When a 3,072-dimensional base processes more than 1,064 chunks, this 2,000-item batch reaches `generateEmbeddings`, whose aggregate guard rejects it before provider batching. Cap the shared batch size at the maximum supported width or derive it from the base width.</comment>
<file context>
@@ -318,9 +323,15 @@ const TIMEOUTS = {
MAX_EMBEDDING_BATCH: Math.min(
envNumber(env.KB_CONFIG_BATCH_SIZE, 2000, { min: 1, integer: true }),
- getEmbeddingAggregateItemLimit(EMBEDDING_DIMENSIONS)
+ getEmbeddingAggregateItemLimit(DEFAULT_KB_EMBEDDING_DIMENSIONS)
),
MAX_FILE_SIZE: 100 * 1024 * 1024,
</file context>
Suggested change
| getEmbeddingAggregateItemLimit(DEFAULT_KB_EMBEDDING_DIMENSIONS) | |
| getEmbeddingAggregateItemLimit(3072) |
| /** No entry in EMBEDDING_MODEL_PRICING: local inference costs Sim nothing. */ | ||
| pricingId: model, | ||
| tokenizerProvider: 'ollama', | ||
| nativeDimensions: DEFAULT_KB_EMBEDDING_DIMENSIONS, |
Contributor
There was a problem hiding this comment.
P1: When KB_EMBEDDING_MODEL=ollama/nomic-embed-text and EMBEDDING_OUTPUT_DIMS is unset, knowledge-base indexing always targets 1536 even though the model emits 768. Resolve the installed model's width before selecting the default, or require a verified width for Ollama instead of using the platform default.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/lib/embeddings/catalog.ts, line 272:
<comment>When `KB_EMBEDDING_MODEL=ollama/nomic-embed-text` and `EMBEDDING_OUTPUT_DIMS` is unset, knowledge-base indexing always targets 1536 even though the model emits 768. Resolve the installed model's width before selecting the default, or require a verified width for Ollama instead of using the platform default.</comment>
<file context>
@@ -175,26 +202,94 @@ export const DEFAULT_MODEL_BY_PROVIDER: Record<EmbeddingCatalogProvider, string>
+ /** No entry in EMBEDDING_MODEL_PRICING: local inference costs Sim nothing. */
+ pricingId: model,
+ tokenizerProvider: 'ollama',
+ nativeDimensions: DEFAULT_KB_EMBEDDING_DIMENSIONS,
+ supportedDimensions: KB_EMBEDDING_STORAGE_DIMENSIONS,
+ maxInputTokens: OLLAMA_MAX_INPUT_TOKENS,
</file context>
| * before the deployment default changed must still resolve its own family's | ||
| * transports. Substituting it evaluates the chain for the model at hand. | ||
| */ | ||
| KB_EMBEDDING_MODEL: model, |
Contributor
There was a problem hiding this comment.
P1: When a deployment changes its default from text-embedding-3-small at 1536 to text-embedding-3-large at 3072, existing small-model bases fail before OpenAI is tried. Override EMBEDDING_OUTPUT_DIMS with the base’s dimensions (or omit it) when building capabilityValues, so capability validation follows the persisted target rather than the deployment default.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/lib/embeddings/client.ts, line 981:
<comment>When a deployment changes its default from `text-embedding-3-small` at 1536 to `text-embedding-3-large` at 3072, existing small-model bases fail before OpenAI is tried. Override `EMBEDDING_OUTPUT_DIMS` with the base’s `dimensions` (or omit it) when building `capabilityValues`, so capability validation follows the persisted target rather than the deployment default.</comment>
<file context>
@@ -936,7 +969,18 @@ export async function embedKnowledgeForDeployment(
+ * before the deployment default changed must still resolve its own family's
+ * transports. Substituting it evaluates the chain for the model at hand.
+ */
+ KB_EMBEDDING_MODEL: model,
+ ...(workspaceKey ? { OPENAI_API_KEY: workspaceKey.apiKey } : {}),
+ }
</file context>
Suggested change
| KB_EMBEDDING_MODEL: model, | |
| KB_EMBEDDING_MODEL: model, | |
| EMBEDDING_OUTPUT_DIMS: String(dimensions), |
| export type KnowledgeEmbeddingFamily = 'openai' | 'gemini' | 'ollama' | ||
|
|
||
| export function knowledgeEmbeddingFamily(values: EnvCapabilityValues): KnowledgeEmbeddingFamily { | ||
| const model = String(readValue(values, 'KB_EMBEDDING_MODEL') ?? '') |
Contributor
There was a problem hiding this comment.
P1: When KB_EMBEDDING_MODEL is an unsupported Gemini ID or case variant, this predicate reports Gemini as configured, but runtime falls back to text-embedding-3-small. With only Gemini credentials, document and query embedding then request OpenAI without an OpenAI key and fail; classify only IDs accepted by the runtime.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/deployment-config/src/env-capabilities.ts, line 1239:
<comment>When `KB_EMBEDDING_MODEL` is an unsupported Gemini ID or case variant, this predicate reports Gemini as configured, but runtime falls back to `text-embedding-3-small`. With only Gemini credentials, document and query embedding then request OpenAI without an OpenAI key and fail; classify only IDs accepted by the runtime.</comment>
<file context>
@@ -1210,6 +1220,68 @@ export const OCR_CAPABILITY = defineCapability({
+export type KnowledgeEmbeddingFamily = 'openai' | 'gemini' | 'ollama'
+
+export function knowledgeEmbeddingFamily(values: EnvCapabilityValues): KnowledgeEmbeddingFamily {
+ const model = String(readValue(values, 'KB_EMBEDDING_MODEL') ?? '')
+ .trim()
+ .toLowerCase()
</file context>
| key: 'EMBEDDING_OUTPUT_DIMS', | ||
| input: 'text', | ||
| hint: 'optional vector width for new knowledge bases: 768, 1024, 1536 (default), or 3072', | ||
| validate: true, |
Contributor
There was a problem hiding this comment.
P2: When configuring Ollama at 384 dimensions, the wizard rejects the value before transition validation. Skip generic prompt validation for this provider and rely on the transition inspection, which applies Ollama's own width rule.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/sim-setup/src/capability-config.ts, line 752:
<comment>When configuring Ollama at 384 dimensions, the wizard rejects the value before transition validation. Skip generic prompt validation for this provider and rely on the transition inspection, which applies Ollama's own width rule.</comment>
<file context>
@@ -731,6 +731,28 @@ export const KNOWLEDGE_SETUP = defineCapabilitySetup(OCR_CAPABILITY, {
+ key: 'EMBEDDING_OUTPUT_DIMS',
+ input: 'text',
+ hint: 'optional vector width for new knowledge bases: 768, 1024, 1536 (default), or 3072',
+ validate: true,
+ },
+] as const satisfies readonly SetupPrompt[]
</file context>
Suggested change
| validate: true, | |
| validate: false, |
| Two things worth knowing before you build on it. Vectors are only comparable when they come from the same model at the same size, so changing either means re-embedding everything you intend to compare. And input longer than the model's limit is shortened to fit rather than rejected, with a warning in the run, so chunk long documents yourself when the tail matters. | ||
|
|
||
| Sim's knowledge bases embed separately, at a fixed vector width and from a smaller set of models. This block is for embedding text yourself inside a workflow. | ||
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and costs nothing, and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains — the block reads it live, filters it to models that can actually embed, and shows each one's vector width next to its name. Ollama accepts neither a task type nor a size reduction, so the block does not offer those controls for it. Self-hosted deployments configure the server with `OLLAMA_URL`; on Sim Cloud there is no Ollama to reach, so the list comes back empty. |
Contributor
There was a problem hiding this comment.
P2: When Ollama omits capability or embedding-length metadata, the catalog keeps the model and the selector omits its width, so this sentence promises filtering and labels that the block cannot guarantee. Qualify the claims to say non-embedding capability metadata is filtered and widths are shown only when Ollama reports them.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/docs/content/docs/integrations/embeddings.mdx, line 22:
<comment>When Ollama omits capability or embedding-length metadata, the catalog keeps the model and the selector omits its width, so this sentence promises filtering and labels that the block cannot guarantee. Qualify the claims to say non-embedding capability metadata is filtered and widths are shown only when Ollama reports them.</comment>
<file context>
@@ -13,19 +13,21 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
Two things worth knowing before you build on it. Vectors are only comparable when they come from the same model at the same size, so changing either means re-embedding everything you intend to compare. And input longer than the model's limit is shortened to fit rather than rejected, with a warning in the run, so chunk long documents yourself when the tail matters.
-Sim's knowledge bases embed separately, at a fixed vector width and from a smaller set of models. This block is for embedding text yourself inside a workflow.
+Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and costs nothing, and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains — the block reads it live, filters it to models that can actually embed, and shows each one's vector width next to its name. Ollama accepts neither a task type nor a size reduction, so the block does not offer those controls for it. Self-hosted deployments configure the server with `OLLAMA_URL`; on Sim Cloud there is no Ollama to reach, so the list comes back empty.
+
+Sim's knowledge bases embed separately: a base fixes one model and one vector width when it is created, from a smaller set of models. This block is for embedding text yourself inside a workflow.
</file context>
Suggested change
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and costs nothing, and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains — the block reads it live, filters it to models that can actually embed, and shows each one's vector width next to its name. Ollama accepts neither a task type nor a size reduction, so the block does not offer those controls for it. Self-hosted deployments configure the server with `OLLAMA_URL`; on Sim Cloud there is no Ollama to reach, so the list comes back empty. | |
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and costs nothing, and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains — the block reads it live, filters out models that report non-embedding capabilities, and shows each model's vector width when Ollama reports it. Ollama accepts neither a task type nor a size reduction, so the block does not offer those controls for it. Self-hosted deployments configure the server with `OLLAMA_URL`; on Sim Cloud there is no Ollama to reach, so the list comes back empty. |
| logger.info('Ollama is not reachable; offering no embedding models', { | ||
| error: getErrorMessage(error, 'Unknown error'), | ||
| }) | ||
| return [] |
Contributor
There was a problem hiding this comment.
P2: When a configured Ollama server is unreachable, this fallback makes metadata resolution report the selected model as missing and returns HTTP 400 instead of 502. Preserve the empty-list behavior for catalog/selector requests, but propagate reachability failures separately when resolving a selected model so callers can classify the outage correctly.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/lib/embeddings/ollama-model-catalog.server.ts, line 109:
<comment>When a configured Ollama server is unreachable, this fallback makes metadata resolution report the selected model as missing and returns HTTP 400 instead of 502. Preserve the empty-list behavior for catalog/selector requests, but propagate reachability failures separately when resolving a selected model so callers can classify the outage correctly.</comment>
<file context>
@@ -0,0 +1,160 @@
+ logger.info('Ollama is not reachable; offering no embedding models', {
+ error: getErrorMessage(error, 'Unknown error'),
+ })
+ return []
+ }
+
</file context>
| const embeddingTargets = new Map<string, KbEmbeddingTarget>( | ||
| accessibleKbs.map((kb) => [ | ||
| `${kb.embeddingModel}:${kb.embeddingDimension}`, | ||
| { model: kb.embeddingModel, dimensions: toKbEmbeddingDimensions(kb.embeddingDimension) }, |
Contributor
There was a problem hiding this comment.
P2: When a tag-only request targets a knowledge base with an unsupported persisted width, this unconditional toKbEmbeddingDimensions call returns a 500 before tag filtering runs. Build and validate embedding targets only when hasQuery is true.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/app/api/v1/knowledge/search/route.ts, line 199:
<comment>When a tag-only request targets a knowledge base with an unsupported persisted width, this unconditional `toKbEmbeddingDimensions` call returns a 500 before tag filtering runs. Build and validate embedding targets only when `hasQuery` is true.</comment>
<file context>
@@ -183,17 +188,28 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
+ const embeddingTargets = new Map<string, KbEmbeddingTarget>(
+ accessibleKbs.map((kb) => [
+ `${kb.embeddingModel}:${kb.embeddingDimension}`,
+ { model: kb.embeddingModel, dimensions: toKbEmbeddingDimensions(kb.embeddingDimension) },
+ ])
+ )
</file context>
| mode: 'any-present', | ||
| keys: ['AZURE_OPENAI_API_KEY', 'AZURE_OPENAI_ENDPOINT', 'AZURE_OPENAI_API_VERSION'], | ||
| }, | ||
| activeWhen: embeddingFamilyIs('openai'), |
Contributor
There was a problem hiding this comment.
P2: When OpenRouter is the only configured OpenAI transport, unsupported EMBEDDING_OUTPUT_DIMS values still make status report the capability as configured because this provider skips the new width validation. Add the OpenAI model and dimension optional fields to OpenRouter, and validate the selected model rather than only the family.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/deployment-config/src/env-capabilities.ts, line 1297:
<comment>When OpenRouter is the only configured OpenAI transport, unsupported `EMBEDDING_OUTPUT_DIMS` values still make status report the capability as configured because this provider skips the new width validation. Add the OpenAI model and dimension optional fields to OpenRouter, and validate the selected model rather than only the family.</comment>
<file context>
@@ -1222,6 +1294,7 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
mode: 'any-present',
keys: ['AZURE_OPENAI_API_KEY', 'AZURE_OPENAI_ENDPOINT', 'AZURE_OPENAI_API_VERSION'],
},
+ activeWhen: embeddingFamilyIs('openai'),
requires: allOf(
envField('AZURE_OPENAI_API_KEY'),
</file context>
Suggested change
| activeWhen: embeddingFamilyIs('openai'), | |
| activeWhen: embeddingFamilyIs('openai'), | |
| optionalFields: [ | |
| envField('KB_EMBEDDING_MODEL'), | |
| embeddingOutputDimsField(OPENAI_EMBEDDING_WIDTHS), | |
| ], |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
embeddingnow has a column per storable width — 384, 768, 1024, 1536, 3072 — and a base records which one it uses at creationEMBEDDING_OUTPUT_DIMSpicks that width. A value the deployment can't store, or one the configured model can't emit, warns and falls back to 1536 rather than failing knowledge-base creationKB_EMBEDDING_MODELacceptsollama/<model>, so a self-hosted deployment can index against its own Ollama with no API key and nothing billedvectorup to 2,000 dimensions, so the 3072 column is indexed through ahalfveccast. Queries have to repeat that exact cast to use the index, so the distance expression for every width comes from one placesim-setupgains Gemini and Ollama options, andsim-setup statusnow reports the one familyKB_EMBEDDING_MODELactually selects instead of every provider a key exists for. Gemini previously displayed as "OpenAI"Type of Change
Testing
Tested manually against a local Postgres (pgvector 0.8.0) and a real Ollama:
executeKnowledgeSearchin hybrid and vector mode — withollama/nomic-embed-text(768),ollama/all-minilm(384),text-embedding-3-large(3072 and 1024), andtext-embedding-3-small(1536)EXPLAINconfirms each width hits its own HNSW index, and that 3072 without thehalfveccast falls to a sequential scanembedding_width_checkrejects rows with no vector and rows with two; the migration applies from the pre-migration shape and is replay-safeapps/sim, all packages),turbo type-check,bun run lint,check:audits(45 audits),docs-manifest:check, block-registry check, andcheck:migrations origin/stagingChecklist