* fix(sandbox): resolve a session's pinned sandbox in the workspace that owns it
tenant_sandbox_configs is keyed by (tenant_id, id), so a config id is not an
address on its own. sessions.sandbox_config_id stored only the id and every
reader took the workspace from the ambient request context.
That holds inside a chat turn, which already runs under the agent's workspace
(WithExecutionTenant in the QA handler). It breaks for a shared agent
everywhere else, because a shared agent runs on ITS OWNER's sandbox config
while the session belongs to the borrower, and these paths run from plain
requests whose only tenant is the session owner's:
* DeleteSession resolved nothing and returned after a warning. Cube and E2B
create session sandboxes with onTimeout=pause, so the MicroVM was not
reclaimed later either: it stayed paused, billing, with nobody holding its
id. ReapOrphanSandboxes has no production caller, so nothing recovered it.
* The terminal and desktop panels failed on both branches - attaching to the
pinned sandbox and provisioning a new one - with ErrSandboxConfigNotFound,
which terminalErrorFrame maps to a bare INTERNAL close reason. The panel is
offered unconditionally and already forwards agent_source_tenant_id, so
this was a dead end the user could reach in one click.
* Fork read the pin through PinnedSessionSandbox, got no manager, and
silently degraded to SNAPSHOT_UNSUPPORTED, losing the sandbox state.
* The fork snapshot lease recorded the session's workspace next to the
lending workspace's config id, so the reaper could not resolve it either.
Record the owning workspace beside the pin and make the pair travel together
as SandboxPin. Readers now take the workspace from the pin and fall back to the
session's own, which is what every non-borrowed pin (and every row written
before this column) resolves to - so behaviour for own agents is unchanged.
The provision path has no pin yet, so the WebSocket handler passes the agent's
workspace along with its config: terminalProvisionPin already resolved the
shared agent, it just discarded the tenant resolveAgent handed back.
Backfill in the migration is limited to sessions whose pinned config does not
exist in their own workspace - the only ones that can have borrowed it - using
the lending workspace recorded on their newest shared-agent message. Lite has
no shared spaces, so the SQLite mirror adds the column without a backfill.
* fix(sandbox): backfill the pin owner from the config row and self-heal legacy pins
Three gaps in the session sandbox pin owner:
Backfilling from messages.agent_tenant_id names the wrong workspace. The
pin is sticky to the session's FIRST sandbox, while the newest message
carrying agent_tenant_id may belong to a later shared agent from another
workspace. Writing that tenant leaves teardown resolving a config the
workspace does not own - the very failure the column exists to fix.
tenant_sandbox_configs.id is a global UUID primary key, so joining the
config row itself names the workspace that actually owns the pin.
The column was INTEGER while every other tenant id is BIGINT/uint64.
Postgres INTEGER is 32-bit signed, so a large tenant id would not fit.
Rows the SQL backfill cannot reach keep sandbox_config_tenant_id = 0
forever: the reader falls back to the caller's workspace but never
records what it learned. The next chat turn already runs under the
workspace that can resolve the config, so persisting it there is what
lets a later DELETE, panel open or fork find the sandbox. recordOwner
only writes when the lookup actually succeeded and the stored owner is
still unset - stamping the caller's tenant onto a failed lookup would
brand a sticky pin left by a previous shared agent with the wrong
workspace.
Also renames the lentSandboxConfg typo and corrects the fork degrade
reason in a comment: a lent config that resolves to nil fails
BoundSandboxID, so the fork degrades to SANDBOX_GONE, not
SNAPSHOT_UNSUPPORTED.
* fix(sandbox): require a named backend before recording a pin's owner
The self-heal wants to persist an owner "that actually resolved this config",
but a nil error does not prove that. resolveTenantSandboxForConfig returns a
disabled manager with no error twice before it ever reads the config row: for
the workspace kill switch, and for an empty or deployment-default config id.
That reopens the leak this column exists to close. The SQL backfill correctly
leaves sandbox_config_tenant_id at 0 whenever the pinned config belongs to the
session's own workspace, which is most existing rows. One turn of a shared
agent lent by a workspace whose admin has script execution turned off then
short-circuits on the kill switch, returns nil, and stamps that lending
workspace onto a config it does not own. The next DELETE resolves the pin there,
finds nothing, and strands the paused MicroVM.
The pin branch runs before agentConfigID is read, so the shared agent does not
even need a sandbox config of its own to trigger it.
A named backend (Cube/E2B/Docker) is the proof, and it is the same condition
that is already allowed to CREATE a pin further down: what may write a pin may
update whose it is. A config that exists but resolves to type "disabled" is
skipped as well - ownership is genuine there, but such a config never creates a
sandbox, so no teardown can go missing.
The deployment-default pin was inert either way (both workspaces short-circuit
to a manager with no DestroySession), but it is covered too so a later
relaxation of the proof cannot quietly reopen the kill-switch case.
Both tests fail without the guard.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
#3406 (000104_skill_served_version) and #3407
(000104_message_context_checkpoint) each took version 104 and were merged
eleven minutes apart. golang-migrate refuses to load a directory with a
duplicate version ("duplicate migration file"), so every PostgreSQL
deployment of main fails to migrate.
Renumber the later of the two. It is safe for every deployment that could
exist: one that migrated between the two merges applied the skill
migration as 104 and now applies this one as 105, and after the second
merge main could not load its migrations at all, so no deployment applied
this file as 104. The migration is ADD COLUMN IF NOT EXISTS either way.
Add a test that loads both migration directories. Pull request CI runs on
the merge with main, so a second PR taking an already merged number now
fails before it lands.
* feat(agent): persist compaction checkpoints across turns
Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.
A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.
History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.
Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.
* test(skill): wait for the background install in the name-conflict test
InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.
* feat(agent): size agent history by tokens instead of turn count
Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.
LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.
Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.
Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.
* Revert "feat(agent): size agent history by tokens instead of turn count"
This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.
Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.
Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
* feat(artifacts): add an artifacts library with its own message_artifacts table
Skill-generated files were only reachable from inside the conversation that
produced them. This adds a sidebar entry and a page that lists them across
every conversation the user can see.
Storage: artifacts move out of the messages.artifacts JSONB column into a new
message_artifacts table (PostgreSQL 000103, SQLite 000023), backfilled on
upgrade. The table is now the only source of truth; the old column is kept and
the down migration writes the table back into it. The message repository
loads artifacts for every message it returns and writes them on create,
update (when non-nil) and fork, so callers still use Message.Artifacts.
mod_time is stored as RFC 3339 text because the collector matches files on
nanosecond mtimes, which PostgreSQL timestamps would truncate.
API: GET /api/v1/artifacts lists the latest version of each file (grouped by
session + source path, with version_count), scoped like the session list,
with keyword / file type filters and paging. Downloads keep using the
per-session endpoint.
Frontend: "产物/Artifacts" sidebar entry (hidden without sandbox support), a
library page with type tabs, search, date groups, infinite loading, preview
drawer, download and jump-to-conversation; strings in all five locales. The
sidebar's top-menu paths are now one shared list.
* fix(artifacts): keep the restore hash guard and count versions by URL
RecordRestoredArtifactMtime again only stamps rows whose content hash matches
the restored content (and ignores empty hashes), and never rewrites the hash,
matching MessageArtifacts.WithRestoredMtime. Without the guard a restore of
v1's content also moved v2's mtime and relabelled v2 with v1's hash.
The library's version_count now counts distinct URLs per (session, source
path). A later answer that references an earlier file stores another row with
the same URL, which COUNT(*) reported as an extra version.
* fix(artifacts): scope the library to web chats and clear the legacy column
The library used the session list's default scope, which counts sessions
without an owner as legacy tenant-level rows. IM sessions are created without
an owner, so every member saw the file names and titles of every IM chat
(downloads then 404 for non-admins). The library now applies the same "web"
bucket predicate as the home sidebar, extracted from QueryPaged so the two
cannot drift: no IM mapping (including cleared ones), no embed marker, no
API-key owner.
Migrations 000103 / SQLite 000023 now clear messages.artifacts after the
backfill. Message loads still SELECT *, so the stale copy was read on every
history load. Only rows that carried artifacts are rewritten; the column stays
for instances on the old binary during a rolling upgrade, and the down
migration still rebuilds it from the table.
* fix(migrations): keep the 000103 backfill alive on malformed artifact JSON
A single file_size that is not an integer ("12.5", "") or a created_at that
does not parse ("garbage", month 13) raised a cast error and failed the whole
migration, blocking startup. Casts now go through pg_temp helpers that return
NULL on bad input, falling back to 0 and the message's created_at; valid rows
are unchanged. SQLite casts never raise; its migration test now covers the
same malformed values.
* docs(artifacts): describe the web-only library scope and the cleared legacy column
* feat(mcp): add built-in MCP server with per-workspace endpoints
WeKnora previously exposed itself to MCP clients only through the Python
package under mcp-server/, which binds one process to one API key and
therefore to one workspace, and mirrors the REST API tool-for-tool.
This adds a native MCP server surface to the Go backend:
- New mcp_endpoints table (PostgreSQL 000100, SQLite 000020). A workspace
can publish any number of endpoints, each with its own bearer token
(SHA-256 stored, rotatable, shown once), knowledge-base allowlist, tool
allowlist, default agent and per-minute rate limit.
- Management API under /api/v1/mcp-endpoints (Viewer reads, Admin writes,
API keys need manage_channels) plus a tool catalog endpoint.
- internal/mcpserver hosts one mcp-go Streamable HTTP server in stateless
mode at /mcp/:endpoint_id. tools/list is filtered per endpoint and
tools/call re-checks the allowlist, applies rate limiting and touches
last_used_at. MCPEndpointAuth resolves the token and injects the
workspace, an mcp_endpoint principal and an API-key style scope derived
from the endpoint, so existing knowledge-base and capability checks
apply unchanged.
- Redesigned tool set (12 tools in four groups): retrieval and reading
reuse the agent's native tools; ask runs the default or a named agent
synchronously and returns a cited answer with a session_id for
follow-ups; wiki tools cover search, read and index; write tools are
off by default. Knowledge bases accept ids or names.
- Frontend: new "MCP Server" integrations tab (tab id mcpserver, to
avoid clashing with the existing MCP-client settings section) with
endpoint cards, a create/edit drawer, a one-time token dialog and
ready-to-paste snippets for Cursor/VS Code/Claude Desktop, Claude Code
and mcp-remote. Locales updated in all five languages.
- Deployment capability integrations.mcpserver wired on both sides.
- Docs: website-docs MCP page now leads with the built-in server; the
Python mcp-server README carries a deprecation notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): wire the MCP server into DI/router and tighten authorization
The previous commit lost eight staged files to the repo pre-commit hook,
which resolves ROOT to the main checkout and re-adds the main-checkout
copies of tracked files into the worktree index. This commit restores them
(container providers, RouterParams and route registration, context key,
principal, clone table entry, deployment capability key, SQLite parity
test) and addresses the review findings:
- Knowledge-base scope is authorized, not trusted: endpoint allowlists and
every tool call resolve each knowledge base through access.ResolveKB for
the calling principal (ownership or organization share). Foreign IDs are
rejected on create/update and dropped at call time.
- Write tools mint explicit editor grants via access.KBAccess.WithGrant
before calling the knowledge service, so add/update/delete go through the
same RequireKBWrite path as the REST API. add_document also checks the
grant itself.
- ask no longer accepts agent_id. It runs only the endpoint's configured
agent; tenant-owned agents and user-facing builtins qualify, internal
builtins (wiki fixer, skill installer) are refused both on save and at
call time. Web search is forced off for MCP calls.
- Tool annotations match behaviour: ask is not read-only, add_document is
a mutation.
- CORS allows MCP-Protocol-Version / Mcp-Session-Id / Last-Event-ID and
exposes Mcp-Session-Id for browser-based MCP clients.
- Rate limiter runs its local cleanup goroutine; SQLite migration gains the
deleted_at index for parity with PostgreSQL; non-ASCII endpoint names get
unique mcpServers keys derived from the endpoint id.
- Swagger annotations on the management handler.
- Tests: KB isolation (service and server), write grants, endpoint-agent
restriction, tool annotations, and a router test that mounts the public
/mcp/:endpoint_id surface and checks the deployment capability.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main took 000100 / 000020 for chunks_index_diet, so the mcp_endpoints
migrations move to 000101 (PostgreSQL) and 000021 (SQLite); the SQLite
parity test and docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): run document tools under the knowledge base owner tenant
Shared knowledge bases resolved through access.ResolveKB were only half
usable: documents of an organization-shared knowledge base are stored under
the owner tenant, so the document-level tools that queried by the calling
tenant found nothing, and add_document would have created rows under the
caller's tenant inside someone else's knowledge base.
- knowledgeInScope loads documents tenant-agnostically and then requires
the document to belong to an allowed knowledge base and to that base's
tenant.
- scopedKBContext replaces grantContext: it resolves the grant, switches
the execution tenant to the knowledge base owner (KBAccess.Context) and
swaps TenantInfo to the owner so storage/model settings resolve there.
list_documents, read_document and every write tool run under it.
- Locale text and docs no longer claim clients can pass agent_id.
- Go SDK gains MCP endpoint methods (client/mcp_endpoint.go) with docs, and
the Swagger artifacts are regenerated.
- Tests: shared knowledge base read/write under the owner tenant, spoofed
document tenant rejected, viewer share cannot write.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main now carries 000101 / 000021 (knowledge_profiles), so the
mcp_endpoints migrations move to 000102 (PostgreSQL) and 000022 (SQLite);
the SQLite parity test expects version 22 and the docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): guard last_used touch without a repository and wrap SDK line
The transport tests build the server without a repository; the background
last_used_at write then dereferenced a nil interface and, on the slower CI
runner, took the test binary down. touchLastUsed now returns early when no
repository is wired, and the tests inject a recording repository and assert
the touch instead. Also wraps a 123-column SDK signature for the client
module's lll check.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
* perf(chunk): speed up chunk ingestion and batch updates
Chunk writes were dominated by avoidable work in the repository layer:
- UpdateChunks built one CASE WHEN id = ? chain per column, so a batch of
N rows evaluated 5N branches per row (O(N²)). On PostgreSQL it now joins
against a VALUES list keyed by the primary key (O(N), batched by 1000);
other dialects run one small UPDATE per row inside a single transaction.
- CreateChunks copied Content into SourceContent for every new row, doubling
insert volume. SourceContent is only needed once a chunk is edited and the
edit path already backfills it lazily, so it is left empty on create.
Batch size raised from 100 to 500 and the insert (including the SQLite
seq_id pre-assignment) now runs inside one transaction.
- New UpdateChunkFieldsByIDs applies uniform field values with a single
UPDATE ... WHERE id IN per batch. FAQ import uses it to flip status after
indexing instead of shipping every row's content back, and resolves tag
names per batch instead of per entry.
- Drop three chunks indexes that only slow down writes: chunk_type (low
cardinality), content_hash (never used as a filter) and
(tenant_id, knowledge_id) (covered by idx_chunks_knowledge_enabled).
Migrations 000100 (PostgreSQL) and 000020 (SQLite) with down scripts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(chunk): rewrite chunk batch writes and FAQ import status updates
Go changes belonging to the chunk write-path optimization. The previous
commit on this branch only carried the migrations because the repo's
pre-commit hook re-staged files from the main checkout when run inside a
git worktree, silently dropping the staged Go changes.
- UpdateChunks: O(N²) CASE chains replaced by a VALUES join on PostgreSQL
(batched by 1000) and per-row updates in one transaction elsewhere.
- CreateChunks: stop copying Content into SourceContent, batch 500 rows,
run the insert and the SQLite seq_id pre-assignment in one transaction.
- New UpdateChunkFieldsByIDs for uniform, tenant-scoped field updates.
- FAQ import flips status with UpdateChunkFieldsByIDs and resolves tag
names once per batch instead of once per entry.
- Drop the gorm index tag on ContentHash to match migration 000100;
SQLite schema test expects migration version 20.
- SQLite tests for the new and rewritten repository methods.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(session): add fork schema and history queries
为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。
* feat(sandbox): add git, fork snapshots, and bootstrap hook
沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。
* feat(session): checkpoint workspace at the end of each turn
每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。
* feat(session): fork conversations and reap orphan snapshots
支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。
* feat(chat): add session fork entry, prefill, and badges
在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。
* style: add missing newline so session_fork_delete_test.go is gofmt-clean
* fix(session): prune later git history when rolling a fork workspace back
reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。
* style(session): clear golangci-lint findings on the session-fork diff
Pre-push lints the whole PR against main; wrap long lines, check c.Error,
and fix gofumpt/revive/staticcheck so the merge can be pushed.
* fix(chat): show the fork button on hover without shifting the thread
Overlay the action so revealing it no longer expands the user bubble and janks the messages below.
* fix(chat): drop the fork landing banners
Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread.
* fix(agent): dedupe artifacts by content hash, not size
Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only.
* feat(session): allow forking from an assistant answer
User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists.
* fix(session): resolve parent attachments from a forked chat
Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss.
* fix(agent): rewrite sandbox refs after a content-hash skip
Fork restores change mtime on unchanged files, so Collect returns empty
and named citations were left as sandbox: paths. Resolve those names
against session artifacts, notify only real uploads, and stamp restored
mtime only on the matching content hash.
* fix(session): drop a dead fork snapshot and retry checkpoint rewrite
Create from a missing template now abandons bootstrap so the next
Resolve uses a normal sandbox. A failed checkpoint rewrite no longer
marks the bootstrap consumed, which previously made nested forks
SANDBOX_REPLACED.
* fix(session): load forked parent attachments into the prompt
ResolveForPrompt used session-scoped lookup, so copied parent attachment
IDs passed Get but never reached the model. Use the same resolveInSession
path as preview.
* fix(chat): keep the user-message fork button visible
The control was hover-only, which hid it on touch. Always show it, and
drop the unused degrade tooltip.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Long-term memory chose its candidates before it looked at the question.
Recall listed 400 items by `importance DESC, valid_from DESC`, loaded the
vectors belonging to that list, and scored those; extraction did the same
with a 200-item pool. Importance is uncorrelated with relevance, so the
vector stage could only re-order what importance had already picked, and a
memory below the cut was unreachable no matter how well it matched. The
capacity cap goes up to 2000, so everything past the 400th most important
memory was permanently unrecallable, and `memory_item_embeddings.vector`
being a BYTEA blob meant pgvector was never involved.
The store now answers "which of this person's memories is this question
about":
- Migration 000095 (and the SQLite mirror) add `embedding halfvec` plus a
`(tenant_id, subject_id, model_id, dims)` index. The column is
conditional on the vector extension, so a deployment whose retrieval
driver excludes postgres keeps working. No HNSW: an approximate index is
filtered after the fact, and one subject holds at most a couple of
thousand rows, so exact ordering inside the scope is both correct and a
few milliseconds.
- `SearchItemsByVector` ranks in SQL over every vector the subject has,
with the status, expiry and kind filters joined in so reaching past the
candidate list cannot reach past the filters. SQLite and pgvector-less
PostgreSQL fall back to scoring in process, still over the whole scope.
- Recall fuses the lexical ranking with those hits and grows the candidate
pool with matches the pool did not hold, which is what lets the search
escape it. The lexical pool is now the capacity cap, so it can no longer
be a second, invisible limit under the one that decides what stays active.
- Extraction picks the memories it shows the model the same way. This
matters more than recall: the memory a new statement contradicts is not
reliably an important one, so an importance-ordered window hid exactly
the collision that stops a duplicate being written.
- Vectors are written to the searchable column in the upsert's transaction;
rows predating the column are moved by `SyncVectorColumn` during
maintenance, which costs no model calls. The embedding backfill goes from
50 to 200 per run — at 50, a subject at the capacity cap took over a
month to become fully searchable.
- Traces and logs report `outside_pool`, the number of semantic matches the
lexical pool did not contain. It was structurally zero before, so it is
the number to watch when asking whether semantic recall does anything.
Tests cover whole-scope ranking, the lifecycle and kind filters, pool
growth, the pool-size invariant, and extraction seeing a low-importance
memory it would otherwise duplicate. A pgvector integration test
(WEKNORA_MEMORY_TEST_POSTGRES_DSN) exercises the migration, the halfvec
write, the SQL ranking and the column sync.
* feat(mcp): add per-tool enable toggle
* style: format migration test
* fix(mcp): keep enable and approval updates from overwriting each other
Partial policy writes now touch only the supplied columns, and a missing
tenant fails closed so a disabled tool cannot slip through.
---------
Co-authored-by: myysophia <myysophia@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Accumulate token usage across every LLM round of an agent turn and
expose the aggregate everywhere a turn is observable:
- AgentState.TurnUsage accumulates each round's reported usage,
including the final-answer synthesis call; TokenUsage.Accumulate
keeps the cache counters as prompt subsets and recomputes the
combined cache status (unsupported survives only while every call
was itself unsupported).
- The agent completion event carries the aggregate, and the complete
stream event exposes it both as data.usage and as a typed
StreamEvent.Usage that buildStreamResponse promotes into the
previously-unused StreamResponse.Usage wire field.
- The assistant message persists it as a nullable usage column on
both migration targets (postgres 000085 jsonb, sqlite 000012 text),
for web and IM channels alike, so history reads still carry the
turn's cost after the live stream is gone.
- "[LLM Usage]" log lines gain ", session_id=..., principal=type:id"
when the context carries them; calls outside sessions stay
byte-identical.
Usage from failed retry attempts is not accumulated: the per-call
usage log remains the ground truth, while the turn aggregate serves
as the operational signal.