Commit Graph

24 Commits (22cbcdfa5789b33dbb8d47532c21b24da2fdf5aa)

Author SHA1 Message Date
lyingbug b82fd6c0f9
fix(security): close the low-severity shared-space findings (#3416)
* fix(security): close the low-severity shared-space findings

Verified and fixed from the shared-space permission review:

- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
  and summary_model_id from anonymous visitors. Mentions are only scoped
  for shared agents, so a visitor could query any KB (and pick any model)
  of the channel workspace by ID. The payload now drops them; the widget
  never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
  by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
  workspace (or any user by ID) as its representative, whose username and
  email the roster then showed to all members. Direct adds attach no
  representative, and the roster returns emails only for the caller's own
  workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
  a path into an object key, while the tenant check scans every segment:
  cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
  object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
  the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
  its credentials) of a shared agent's workspace; only the agent's rules
  choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
  without a permission check. Each document is now resolved to its KB and
  checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
  Both must belong to the channel's workspace, the KB also to a restricted
  API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
  in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
  through. That switch is for roles inside a workspace; the guard now
  always denies another workspace's KB, since handlers behind it load by
  ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
  insert; it is now checked in the insert's transaction with the
  organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
  service and UI allow org admins. Comment and doc corrected.

Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.

* fix(security): address review of the low-severity shared-space fixes

- Embed payloads also drop question_origin. It selects a KB like an
  @mention for agents that retrieve only when mentioned, and the widget
  never sends it; the selection stays inside the agent's scope, but an
  anonymous visitor should not pick among its KBs. suggestion_attribution is
  kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
  suggestion instruction, a prompt like the others. Starter questions are
  shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
  types.WithExecutionTenant instead of a bare tenant rewrite, so the
  caller is pinned before the document permission check runs. The auth
  middleware and the shared run already pinned it; this removes the
  dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
  do); another bucket's path is a cross-backend copy.

Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
2026-09-18 22:58:46 +08:00
lyingbug 92e7c0ab37
refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents (#3381)
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents

The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.

Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
  mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
  Keyword mode runs through the retrieval engine's keyword index instead of
  the DB regex scan. Modes unavailable for the scope are refused with a
  reason. Rerank, MMR and de-duplication are kept; already_seen markers,
  retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
  wiki_read_source_doc: id accepts a dN or cN handle, always returns the
  document header, pages by exact offset (fixing the old floor-to-page
  bug), finds passages with query (literal by default, regex opt-in) and
  reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
  scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
  literal queries are escaped so the model never reasons about JSON
  escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
  scope and its description/tips were trimmed.

Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
  their successors by NormalizeAllowedTools at registration time; no data
  migration. Stored histories with old names still decode, render and
  redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
  and document handle tables (decode-only), plus rendering for the
  document header, chunk roles, in-document matches and next_offset.

Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
  search -> read -> cite order; tool descriptions are short and contain no
  workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
  the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
  becomes keyword search, read_document gains query, wiki_search gains
  regex), frontend editor/renderers/i18n, presets, website docs and the
  changelog are updated accordingly.

Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.

* fix(agent): address review findings on read_document, list_documents and model rendering

- read_document: fetch the second page whenever the first page did not
  reach the end of the document, so an unaligned window near the end no
  longer drops the trailing chunks (also fixes context windows around a
  chunk near the end); a query passed together with a cN handle now
  searches the owning document instead of being ignored; query mode stops
  collecting once the output budget is nearly spent and reports truncated;
  the document row is authorized in place instead of being fetched twice;
  the context read takes the total from the window fetch instead of a
  separate count query.
- list_documents: resolve the page's document scope in one batched tag
  lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
  through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
  and budget-bound query results.

* fix(agent): close compatibility gaps in the consolidated retrieval tools

- Wiki-only scopes: read_document and list_documents are registered
  whenever a wiki KB is in scope, since every KB stores chunks. Their
  capability requirement includes wiki but is marked auxiliary, so they do
  not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
  bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
  of refusing the call: FAQ and vector-only bases answer keyword requests
  semantically, keyword-only bases answer semantic requests by keyword.
  Results carry requested_mode and mode_fallbacks, the empty-result text
  names the fallback, and the call is refused only when no base in scope
  has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
  always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
  pushed into the database filter, pinned documents are loaded by ID, and
  total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
  ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
  sharing the index sequence no longer shift the neighbourhood; offset is
  documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
  only matches literally when the text is not a valid expression;
  regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
  fetch candidates from the keyword index (semantic for bases without
  one), and every returned chunk is verified against the pattern.

* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics

* docs(mcp): note that grep_chunks recall is bounded by index candidates
2026-09-18 14:12:22 +08:00
lyingbug 5574ebc011
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles

Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.

Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.

Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.

Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.

Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.

Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.

* fix(knowledge): keep generated descriptions out of duplicates and refresh clones

DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.

CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.

* fix(knowledge): refresh the loaded clone target and carry document profiles

ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.

cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 22:21:55 +08:00
wizardchen 2514e42420 docs: remove unsupported Homebrew installation guidance 2026-09-17 19:57:21 +08:00
wizardchen fa56b5b932 fix(docker): upgrade ParadeDB for CPUs without AVX2 2026-09-17 17:17:04 +08:00
nullptr-error 6ccc5cedf1
feat(sandbox): add a graphical desktop to session sandboxes (#3245) 2026-09-14 20:37:37 +08:00
lyingbug ef9cbf456d
docs: update website guides and improve writing style (#3095)
* docs: sync website guides with current product capabilities

* docs: improve website guide structure and writing style
2026-09-08 10:54:41 +08:00
HUANG Cheng 6a55860431
feat(ui): multi-stage Docker frontend build (no host npm) (#3074)
* feat(ui): multi-stage Docker build without host npm

Build frontend assets inside the image with node:24-bookworm, then
copy into the existing digest-pinned nginx runtime so docker compose
and CI no longer require a host-side dist step.

* fix(ui): copy local xlsx package and raise Node heap in Docker build

npm ci needs packages/xlsx before install, and vite OOM'd under the
default Node heap during the multi-stage image build.

* fix(ui): native multi-arch frontend Docker build and layer cache

Keep Vite on BUILDPLATFORM so CI does not qemu-emulate Node, pin the
builder image, and move the commit ARG after npm ci so dependency
layers stay cacheable.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 19:33:57 +08:00
lyingbug d5cb842580
feat(agent): unify sandbox tools and drop host preloaded skills (#3033)
* feat(agent): unify sandbox tools and drop host preloaded skills

Read workspace and skill files through read_file, and run installed or staged skills through shell_exec(skill_name). Remove the unused skills/preloaded tree and keep generated-file previews stable across re-renders.

* fix(frontend): do not assume classList in markdown sanitize hooks

Node unit tests stub anchors without a DOM classList. Detect download cards from class/download attributes instead.
2026-09-05 16:45:59 +08:00
wizardchen d75cf72fc6 chore: prepare v0.8.0 release notes and version bump
Bump VERSION, frontend, and Helm to 0.8.0; document sandbox, skill catalog, long-term memory, and related API/client changes.
2026-09-03 14:46:44 +08:00
wizardchen 56d3086da1 fix(sandbox): grant appuser the docker.sock group inside the app container
The app drops to appuser via gosu, so compose group_add never reaches the
process. Matching the socket GID before the drop lets a mounted Engine API
socket work without chmod 666 on the host.
2026-09-01 17:03:40 +08:00
lyingbug 02269bd8f1
feat: require opt-in for the Docker sandbox backend (#2936)
* feat: require opt-in for the Docker sandbox backend

Docker engine access is host-root equivalent, so keep the backend off
until System Settings or WEKNORA_SANDBOX_DOCKER_ENABLED says otherwise.
Also move system-settings help into the title icon and add i18n for
tenant.auto_accept_invitation.

* fix(i18n): drop tenant wording from Docker host-risk copy

The workspace terminology check rejects 租户 / tenant in user-facing strings.
2026-09-01 15:42:28 +08:00
Prodman Devokadev db05a64caa
feat: add LiteLLM as a model provider (#2923)
* feat(provider): add LiteLLM as an OpenAI-compatible provider

* docs: use revive-compliant comment for ProviderLiteLLM const

* fix(provider): stop defaulting LiteLLM to localhost under SSRF

The catalog URL localhost:4000 is blocked by default SSRF and does not
match DetectProvider, so UI/CLI could neither save nor auto-detect it.
Use a placeholder host, keep thinking aligned with generic, and sync
ModelSource across types, Swagger, and docs.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-01 12:21:03 +08:00
wizardchen 7fda02637c feat(sandbox): remove the Local host-process backend
Local was a host-subprocess fallback that skill install, snapshots, and
session sandboxes already excluded. Drop it so Cube, E2B, and Docker are
the only backends; leftover local configs can only be deleted.
2026-08-26 17:48:21 +08:00
wizardchen 9c5c52f0a4 feat(sandbox): Docker 后端支持会话级长驻容器与远程守护进程
将 Docker 沙箱后端从一次性 exec 容器改为会话级长驻容器,使其会话与文件
系统语义与 E2B / Cube 后端对齐:

- 新增 docker_engine / docker_remote_client / docker_rpc_timeout,直连
  Docker Engine API,支持 TCP+TLS 远程守护进程,并为拉镜像等慢操作单独
  设置超时;daemon 地址留空时跟随本机 docker context。
- 新增空闲清理器 docker_idle_sweeper,按 exec 刷新的活跃标记回收长驻
  容器;标记文件对沙箱非 root 账号可写。
- 会话容器补 PID1 以回收僵尸进程;标准镜像补入 curl。
- 镜像即模板:docker_template_catalog 只上报可识别的沙箱镜像,标准模板
  的拉取在后台进行并报告 building。
- 设置页与租户配置支持守护进程地址、TLS 与 CPU/内存限额,四种语言文案
  同步;沙箱健康检查覆盖 Docker 后端。
- 补充 docs/sandbox-docker-backend.md 与 POC,说明能力边界与快照语义;
  git 钩子的变更包检测跳过嵌套 module,避免拿 POC 的包去跑主 module 测试。
2026-08-24 12:22:58 +08:00
wizardchen 9b4f792a04 fix(docker): 默认在 app 镜像中链接 anydoc 解析引擎
Hub / compose 打包的二进制此前走 stub,设置页会显示引擎未编入。WITH_ANYDOC 改为默认开启,并补上 builder 所需的 curl。
2026-08-16 13:31:51 +08:00
wizardchen 738fe1ec53 feat(sandbox): 沙箱后端改为按空间配置,并补齐 E2B 模板与检测诊断
沙箱的后端类型、凭据、模板、超时和私网访问策略此前分散在部署级
WEKNORA_SANDBOX_* 环境变量里,一个实例只能有一套,租户之间无法各用
自己的 E2B 账号或自建 Cube 集群。现在这些配置全部落到空间维度的具名
沙箱配置上:部署环境不再提供覆盖项,未选择配置的智能体直接不启用脚本
执行,而不是静默回落到某个部署默认值——后者会让「这个智能体到底跑在
哪」无从判断。

同时把连接检测从「能不能连上控制面」扩展成可解释的诊断:
- E2B 标准模板显式带上 default 标签构建。创建沙箱时裸模板名/ID 只解析
  default 标签,构建带了别的标签就永远拿不到,表现为一个与「模板不存在」
  无法区分的 404。
- 模板状态不再从历史构建或 spawn 次数反推就绪,只认当前构建;构建完成
  却没有 default 标签的模板单独报成 untagged,因为它在列表里和「仍在构建」
  长得一样,但等待永远不会有结果。
- 创建失败带上 HTTP 状态,404 走模板解释路径,给出具体到「重建模板」还是
  「Key 属于其他团队」的结论。
- 探针脚本改在进程工作目录下暂存。系统临时目录在 macOS 上位于
  /var/folders,Docker 虚拟机并不共享,挂载进容器后是空目录,真实技能执行
  (脚本在 skills/ 下)却不会踩到,检测因此报假失败。
- 检测失败不再只说「命令输出与预期不符」,带上退出码与 stderr 首行。

前端把沙箱设置页与配置抽屉重做:凭据按「是否已配置」展示而非回显掩码,
删除与停用改为就地确认,占用情况走右侧抽屉,列表卡片展示模板状态、凭据、
超时等可判断健康度的事实。
2026-08-11 15:42:02 +08:00
wizardchen 89221ba015 docs(release): prepare v0.7.2 release notes and version bump
Update version numbers, CHANGELOG, multilingual READMEs, QA docs,
architecture diagram, and showcase screenshots for the v0.7.2 release.
2026-08-07 11:37:05 +08:00
wizardchen 267c847fb8 docs: complete docker compose pull guidance across deployment docs
Follow up on #2567 by documenting the upgrade workflow, syncing JA/KO
READMEs and installation guide, and clarifying that WEKNORA_VERSION
changes require docker compose pull to avoid stale cached images.
2026-08-06 19:58:02 +08:00
wizardchen f780f81fc0 docs(website-docs): add comprehensive product documentation site
Import and update WeKnora documentation with VitePress: getting started,
architecture, features, API reference, clients, and development guides.
Includes landing page with inline SVG illustrations, theme components,
doc check scripts, and product screenshots.
2026-08-06 19:54:40 +08:00
lyingbug 8d9309e8fc Revert "docs: add comprehensive WeKnora documentation"
This reverts commit 4aeef20b46.
2026-08-06 19:52:25 +08:00
lyingbug fefc2c1dc0 Revert "docs: complete docker compose pull guidance across deployment docs"
This reverts commit 485454a9ea.
2026-08-06 19:51:39 +08:00
wizardchen 485454a9ea docs: complete docker compose pull guidance across deployment docs
Follow up on #2567 by documenting the upgrade workflow, syncing JA/KO
READMEs and installation guide, and clarifying that WEKNORA_VERSION
changes require docker compose pull to avoid stale cached images.
2026-08-06 19:30:55 +08:00
wizardchen 4aeef20b46 docs: add comprehensive WeKnora documentation
This commit introduces a complete set of documentation for WeKnora, organized into six main sections: Getting Started, Architecture, Features, API Reference, Clients, and Development Guide. Key highlights include:

- Detailed installation instructions and quick start guides.
- In-depth architectural overview and backend design documentation.
- Comprehensive feature descriptions, including multi-tenant support, document parsing, and agent capabilities.
- API reference covering approximately 360 endpoints with usage examples.
- Client documentation for various interfaces including web, CLI, and SDK.

This documentation aims to facilitate user onboarding and provide a thorough understanding of the WeKnora system.
2026-08-06 18:29:35 +08:00