RAG + Knowledge Base

How Chatly grounds AI replies in your docs — hybrid retrieval, the indexing pipeline, and why an embedding key is not optional.

When the copilot suggests an article or the autonomous agent answers, it doesn't guess — it retrieves. The RAG (retrieval-augmented generation) pipeline finds the most relevant chunks from your knowledge base and includes them in the LLM call as grounded context.

Warning — Semantic search needs your own OpenAI key. Nothing indexes without one.

Embeddings are BYOK, like every other AI call here. With no OpenAI key on the workspace, indexing writes zero rows and returns code: "no_embedding_provider"; retrieval returns an empty list. That is deliberate. The alternative — writing hash-based placeholder vectors — produced an index that cosine-ranked unrelated articles into confident-looking citations, which is worse than returning nothing. Retrieval falls back to Postgres full-text search, which is genuinely useful, just not semantic. Add a key at Workspace → AI providers (/settings/ai).

Info — Quality of RAG = quality of KB

The single biggest lever on AI answer quality is the breadth and depth of your KB. Invest in writing well-structured articles before you invest in tuning models.

What gets indexed

kb_articles is the corpus. Which statuses the AI may retrieve from is a deliberate, narrow list:

Parameters

Macros, closed conversations and agent-only playbooks are not part of the KB index.

Indexing pipeline

publish / crawl / PDF ingest
    │
    ▼
extract plaintext   HTML → hand-written text extractor (output goes to
                    `body`, never `body_html` — a crawled page in a raw-HTML
                    sink would make every crawl a stored-XSS vector)
                    PDF  → in-process pure-JS extractor, no native code
    │
    ▼
chunk  ──  800 chars per chunk, 120-char overlap, cut at the last
           paragraph break or sentence end past the halfway mark
    │
    ├──► embed with the workspace's OpenAI key
    │      (text-embedding-3-small, 1536d)
    │            │
    │            ▼
    │     DELETE existing rows + INSERT into kb_embeddings, in ONE
    │     workspace-scoped transaction, stamped with the model that
    │     produced the vector
    │
    └──► search_tsv — a generated tsvector over title + body

Re-indexing an article is idempotent: prior embeddings are deleted and rewritten inside a single transaction, so a mid-way failure can never leave the article with its old vectors dropped and no new ones written.

Embedding spend is gated before the call and metered after it, on the workspace's BYOK meter at $0.02 / 1M input tokens.

Hybrid retrieval

The autonomous agent's retriever runs two sources and fuses them:

Parameters

Default topK is 5. There is a third fallback — ILIKE — used only when the vector path is unavailable and search_tsv is missing (a fresh database before the FTS migration ran). It logs a warning so operators notice.

The AI service's own RagService.retrieve is vector-only: it embeds the query and cosine-ranks kb_embeddings, returning [] when the workspace has no embedding provider so the caller can fall back.

Everything is scoped by workspace_id and, when a bot sets one, by collectionIds.

Ingestion: crawl your site

POST /v1/kb-ingest/sources/url — Bearer token

Requires kb:write and an Idempotency-Key header. Creating a source crawls it immediately — a source that reads "never ingested" until the next scheduled scan looks broken and is indistinguishable from one that is.

{
  "url": "https://docs.acme.com/guides",
  "label": "Acme docs",
  "collectionId": "0192...",
  "publishOnIngest": false,
  "syncIntervalMinutes": 1440,
  "maxPages": 200,
  "maxDepth": 3,
  "confineToPathPrefix": true,
  "followPdfLinks": true
}

Crawl bounds

The crawler:

• Honours robots.txt. Not optional and not a toggle. It implements the subset of RFC 9309 that decides real crawls: most-specific User-agent group with * fallback, Allow/Disallow with * and $ wildcards, longest-match-wins, and Crawl-delay. A 4xx on robots.txt means unrestricted; a 5xx is treated as fully disallowed.

• Sends a real User-Agent naming the product.

• Requests one at a time, one host, with a 1000 ms delay between them by default — the site's own Crawl-delay overrides it.

• De-duplicates by canonicalised URL.

Sitemap discovery is not implemented. The crawl starts from the seed URL you give it and follows links; pasting a sitemap.xml URL will not skip the discovery phase.

Manage sources with GET /v1/kb-ingest/sources, GET /v1/kb-ingest/sources/{id} (which returns the skipped-page report from the last run — this is what "why is this empty" reads), PATCH /v1/kb-ingest/sources/{id} to rename, pause, re-schedule or publish, POST /v1/kb-ingest/sources/{id}/ingest to run now, and DELETE /v1/kb-ingest/sources/{id} to remove the source while keeping the articles it created.

Ingestion: PDF upload

PDF only. There is no DOCX, Markdown, TXT or HTML upload path.

It is a two-step flow — presign, then register:

POST /v1/uploads/presign — Bearer token

POST /v1/kb-ingest/sources/pdf — Bearer token

Presign needs uploads:create; the ingest call needs kb:write. Both require an Idempotency-Key header. Presign returns a key — pass it as storageKey:

{
  "storageKey": "ws/0192.../handbook.pdf",
  "fileName": "handbook.pdf",
  "label": "Customer handbook",
  "collectionId": "0192...",
  "publishOnIngest": false
}

Warning — Two different size caps apply

Presign refuses anything over 50 MiB. The PDF text extractor refuses anything over 10 MiB, before it looks at a single byte — a PDF is a file a stranger uploaded, and every byte of it steers the parser. Decompressed output is capped independently at 64 MiB so a /FlateDecode zip bomb cannot exhaust the worker.

Extraction is a hand-written, pure-JS, in-process extractor: no native decoder, no eval, no recursion, every loop bounded by a count and a wall-clock deadline. Nothing throws — a malformed file returns a reason that lands on the source's last_error.

Info — Known limitation: CID-keyed fonts

Text drawn with an Identity-H encoding — common in subsetted or CJK documents — is stored as glyph indices, and recovering characters needs the font's /ToUnicode CMap. The extractor does not walk it. It detects the result and reports parse_failed rather than writing mojibake into a knowledge base a bot will later quote to a customer. There is no OCR fallback for scanned PDFs.

Embeddings

Embedding model

The model that produced a vector is stored on the row and is part of the retrieval predicate, so introducing a second model later cannot silently mix two vector spaces in one ranking.

Privacy + isolation

• RAG retrieval is strictly workspace-scoped. Every query filters on workspace_id in application code, and kb_articles / kb_embeddings are both FORCE RLS with the workspace GUC set inside the transaction.

• Retrieval is limited to published and indexed articles — see the status table above.

• Embeddings are vectors in your own pgvector column, generated with your own key.

Troubleshooting

Warning — Nothing is ever retrieved semantically

Check for an OpenAI key on /settings/ai first. Without one, indexArticle returns { indexed: false, code: "no_embedding_provider" } and writes nothing, so the index is genuinely empty however many articles you have published. Answers still work — they come from full-text search.

Warning — Citations point to the wrong articles

Under-tuned retrieval, usually. Make article titles descriptive (titles carry weight in the lexical retriever) and split heavy multi-topic articles into focused ones — the chunker cuts at 800 characters, so one article covering six topics produces chunks that each match weakly.

Info — The crawler stops after a few pages

Most often maxPages (default 50), maxDepth (default 2), confineToPathPrefix keeping it under the seed's path, or a robots.txt Disallow. The per-run skip report on GET /v1/kb-ingest/sources/{id} lists what was skipped and why.

Info — A crawled page is not on my help centre

That is publishOnIngest: false, the default, doing its job. The page is an indexed article: the bot can answer from it, your visitors cannot browse to it. PATCH the source to flip it.