* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents
The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.
Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
Keyword mode runs through the retrieval engine's keyword index instead of
the DB regex scan. Modes unavailable for the scope are refused with a
reason. Rerank, MMR and de-duplication are kept; already_seen markers,
retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
wiki_read_source_doc: id accepts a dN or cN handle, always returns the
document header, pages by exact offset (fixing the old floor-to-page
bug), finds passages with query (literal by default, regex opt-in) and
reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
literal queries are escaped so the model never reasons about JSON
escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
scope and its description/tips were trimmed.
Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
their successors by NormalizeAllowedTools at registration time; no data
migration. Stored histories with old names still decode, render and
redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
and document handle tables (decode-only), plus rendering for the
document header, chunk roles, in-document matches and next_offset.
Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
search -> read -> cite order; tool descriptions are short and contain no
workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
becomes keyword search, read_document gains query, wiki_search gains
regex), frontend editor/renderers/i18n, presets, website docs and the
changelog are updated accordingly.
Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.
* fix(agent): address review findings on read_document, list_documents and model rendering
- read_document: fetch the second page whenever the first page did not
reach the end of the document, so an unaligned window near the end no
longer drops the trailing chunks (also fixes context windows around a
chunk near the end); a query passed together with a cN handle now
searches the owning document instead of being ignored; query mode stops
collecting once the output budget is nearly spent and reports truncated;
the document row is authorized in place instead of being fetched twice;
the context read takes the total from the window fetch instead of a
separate count query.
- list_documents: resolve the page's document scope in one batched tag
lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
and budget-bound query results.
* fix(agent): close compatibility gaps in the consolidated retrieval tools
- Wiki-only scopes: read_document and list_documents are registered
whenever a wiki KB is in scope, since every KB stores chunks. Their
capability requirement includes wiki but is marked auxiliary, so they do
not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
of refusing the call: FAQ and vector-only bases answer keyword requests
semantically, keyword-only bases answer semantic requests by keyword.
Results carry requested_mode and mode_fallbacks, the empty-result text
names the fallback, and the call is refused only when no base in scope
has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
pushed into the database filter, pinned documents are loaded by ID, and
total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
sharing the index sequence no longer shift the neighbourhood; offset is
documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
only matches literally when the text is not a valid expression;
regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
fetch candidates from the keyword index (semantic for bases without
one), and every returned chunk is verified against the pattern.
* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics
* docs(mcp): note that grep_chunks recall is bounded by index candidates
Single SearchTarget must intersect document whitelists with tag scope
instead of treating tags as an independent authorization path. SQL scope
filters now AND document and tag constraints; Agent tools use an explicit
scopeEnforced flag so empty targets fail closed. Also add missing built-in
tool handle policies with drift tests, cache handle regexes at registration,
encode tool error text, and avoid wikiKBIDs slice aliasing.
Unify request-local model-handle handling behind a single
modelcontext.Registry (source citations, durable resource handles, wiki
issue handles) with per-tool argument policies and unresolved-handle
rejection before tool execution.
Also fixes issues found in review:
- decode wiki_read_issue's iN issue_id argument (was never resolved)
- preserve stored wiki page aliases when wiki_write_page omits aliases
- validate http/https scheme before registering url fields as web sources
- gate tool-role MultiContent text behind the tool result policy
- dedup KB IDs in NewWikiScopesFromKBIDs to avoid false ambiguity
- remove dead codec facades (decodeJSON, EncodeMessagesWithPolicy,
UnresolvedToolHandles)
Replace raw chunk/document/KB IDs in LLM context with short cN/dN/bN/wN
aliases, decode tool arguments on the way in, and expand compact <ref/>
citations to the public <kb>/<web> format before user-facing output.
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.
Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
- Added support for tracking the status of FAQ import tasks, including progress indicators for ongoing imports.
- Introduced new localization strings for processing tasks and batch operations in multiple languages.
- Updated the FAQ entry manager to display import progress and improved user feedback during the import process.
- Enhanced the KnowledgeBase component to reflect the processing state of knowledge items, improving user experience during document uploads.
- Implemented batch deletion functionality for chunks, streamlining the management of knowledge base content.
- Updated agent configuration to support separate system prompts for web search enabled and disabled states.
- Removed deprecated agent configuration parameters to streamline settings management.
- Enhanced UI components in AgentSettings.vue to allow configuration of custom prompts based on web search status.
- Improved localization in English, Russian, and Chinese for new prompt settings and UI elements.
- Refactored related API and service logic to accommodate changes in agent configuration structure.