WeKnora/internal/handler/session/agent_stream_handler.go

1002 lines
35 KiB
Go
Raw Normal View History

package session
import (
"context"
feat(agent): human-in-the-loop approval for MCP tool calls (#1173) Add an opt-in human approval gate so Agent runs pause before executing MCP tools that operators flag as dangerous, surface an approval card in the chat UI, and only resume after the user approves (optionally with edited args) or rejects. Backend - New mcp_tool_approvals table + repo/service to mark per-tool approval required (PG migration 000042 + sqlite init). - approval.Gate coordinates RequestAndWait / Resolve with sync.Once delivery, configurable timeout, and Redis Pub/Sub fan-out so multi- replica deployments work without sticky sessions. - MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx (without the per-tool 60s timeout) for the wait, and re-derives a fresh 60s exec ctx after approval so CallTool keeps a full window. - New SSE response types (tool_approval_required / _resolved) and EventBus events plumb approval state to AgentStreamDisplay. - REST: list/set per-tool approval flag, resolve pending approval. - Configurable via agent.tool_approval_timeout_seconds (yaml) or WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go duration). Frontend - MCP settings: per-tool "require approval" switch on the test panel. - Chat: ToolApprovalCard renders the pause point with editable JSON args, validation feedback, mm:ss countdown that turns warning/danger near deadline, and a resolved state that retains context. - i18n strings added for zh-CN / en-US / ko-KR / ru-RU. Docs - docs/zh/mcp-approval.md covering behavior, config, API, deployment considerations (Redis cross-instance, restart limitations).
2026-05-10 13:23:49 +00:00
"encoding/json"
"fmt"
"strings"
"sync"
"time"
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
"github.com/Tencent/WeKnora/internal/agent/skills"
agenttools "github.com/Tencent/WeKnora/internal/agent/tools"
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
"github.com/Tencent/WeKnora/internal/application/service"
"github.com/Tencent/WeKnora/internal/event"
"github.com/Tencent/WeKnora/internal/logger"
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
// SandboxIDLookup reports the sandbox currently bound to a session without
// provisioning one. A session with no live sandbox reports ok=false, which the
// completion path treats as "nothing to check point".
type SandboxIDLookup interface {
BoundSandboxID(ctx context.Context, sessionID string) (string, bool)
}
var _ SandboxIDLookup = (*sandbox.SessionBoundManager)(nil)
var _ SandboxIDLookup = (*service.PinnedSessionSandbox)(nil)
// AgentStreamHandler handles agent events for SSE streaming
// It uses a dedicated EventBus per request to avoid SessionID filtering
// Events are appended to StreamManager without accumulation
type AgentStreamHandler struct {
ctx context.Context
sessionID string
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
tenantID uint64 // Tenant that owns this session; used when persisting skill artifacts.
assistantMessageID string
requestID string
receivedAt time.Time // Handler entry timestamp, used for TTFB logging
ttfbLogged bool // Guards one-shot TTFB log on first answer chunk
assistantMessage *types.Message
streamManager interfaces.StreamManager
completionEvent *interfaces.StreamEvent // Published only after the final message is persisted.
eventBus *event.EventBus
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
// artifactCollector drains skill-generated files from the session
// sandbox after the agent completes. Nil when the sandbox backend
// doesn't support artifact collection or WeKnora was built without it.
artifactCollector *service.ArtifactCollector
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
// checkpointer commits the sandbox's /workspace at the end of the turn so
// session fork can roll a forked sandbox back to this exact message. Nil
// when the deployment has no sandbox backend; handleComplete checks.
checkpointer *service.WorkspaceCheckpointer
// sandboxIDLookup resolves the session's currently bound sandbox ID. The
// ID is stored next to the commit SHA because a SHA is only meaningful
// within one sandbox's git repository.
sandboxIDLookup SandboxIDLookup
// State tracking
knowledgeRefs []*types.SearchResult
finalAnswer string
answerSegments []*answerSegment // Per-answer-event-ID accumulation, so superseded preambles can be dropped
eventStartTimes map[string]time.Time // Track start time for duration calculation
mu sync.Mutex
}
// answerSegment accumulates the streamed content of a single final-answer event
// ID. A non-terminal round may stream a preamble ("let me search…") under its
// own answer ID and then be marked superseded once the round turns out to call
// tools; tracking segments separately lets us exclude that preamble from the
// persisted assistant message instead of leaking it into the final answer.
type answerSegment struct {
id string
content string
superseded bool
}
// findAnswerSegment returns the segment for an answer event ID, or nil.
// Callers must hold h.mu.
func (h *AgentStreamHandler) findAnswerSegment(id string) *answerSegment {
for _, seg := range h.answerSegments {
if seg.id == id {
return seg
}
}
return nil
}
// composeFinalAnswer rebuilds the persisted answer from all non-superseded
// segments in arrival order. Callers must hold h.mu.
func (h *AgentStreamHandler) composeFinalAnswer() string {
var b strings.Builder
for _, seg := range h.answerSegments {
if !seg.superseded {
b.WriteString(seg.content)
}
}
return b.String()
}
// NewAgentStreamHandler creates a new handler for agent SSE streaming
func NewAgentStreamHandler(
ctx context.Context,
sessionID, assistantMessageID, requestID string,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
tenantID uint64,
receivedAt time.Time,
assistantMessage *types.Message,
streamManager interfaces.StreamManager,
eventBus *event.EventBus,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
artifactCollector *service.ArtifactCollector,
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
checkpointer *service.WorkspaceCheckpointer,
sandboxIDLookup SandboxIDLookup,
) *AgentStreamHandler {
return &AgentStreamHandler{
ctx: ctx,
sessionID: sessionID,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
tenantID: tenantID,
assistantMessageID: assistantMessageID,
requestID: requestID,
receivedAt: receivedAt,
assistantMessage: assistantMessage,
streamManager: streamManager,
eventBus: eventBus,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
artifactCollector: artifactCollector,
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
checkpointer: checkpointer,
sandboxIDLookup: sandboxIDLookup,
knowledgeRefs: make([]*types.SearchResult, 0),
eventStartTimes: make(map[string]time.Time),
}
}
// Subscribe subscribes to all agent streaming events on the dedicated EventBus
// No SessionID filtering needed since we have a dedicated EventBus per request
func (h *AgentStreamHandler) Subscribe() {
// Subscribe to all agent streaming events on the dedicated EventBus
h.eventBus.On(event.EventAgentThought, h.handleThought)
h.eventBus.On(event.EventAgentToolCall, h.handleToolCall)
h.eventBus.On(event.EventAgentToolResult, h.handleToolResult)
h.eventBus.On(event.EventAgentCommandOutput, h.handleCommandOutput)
h.eventBus.On(event.EventAgentReferences, h.handleReferences)
h.eventBus.On(event.EventMemoryRecalled, h.handleMemoryRecalled)
h.eventBus.On(event.EventContextCompacted, h.handleContextCompacted)
feat(chat): append messages to a running agent turn (#3123) * feat(stream): persist live-run markers and per-run steer queues Mid-run sends need a cross-replica pointer to the generating assistant and a control-plane list that never hits the SSE stream. * feat(agent): inject queued user messages at ReAct round boundaries The engine drains the steer sink after each round, persists accepted rows, and delays Done until after a loop-end inject so the turn can continue. * feat(session): expose /steer and hand leftover messages to a follow-up turn Owner-scoped steer routes queue or inject into the live run, single-flight AgentQA via the live-run marker, and history replay keeps multiple user rows inside one turn. * feat(frontend): queue mid-run messages while an agent turn streams Stop and Send stay visible together, the overlay follows server delivery, and history splits the transcript around injected user bubbles. * fix(frontend): type pending and client_id on SteerQueueItem vue-tsc failed because the overlay sets those fields while POST /steer is in flight. * fix(session): harden steer persist, handoff, and new_run Keep SetLiveRun ahead of SSE and roll back orphan turns, make inject persist idempotent, append-then-delete on rebind, cap overrun rounds, and stop aborting a live stream when /steer returns new_run. * fix(session): satisfy pre-push lint on steer paths Wrap long lines, check discarded Error/Emit results, and keep context as the first argument so the fork push hook can pass. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-09 08:43:33 +00:00
h.eventBus.On(event.EventUserMessageInjected, h.handleUserMessageInjected)
h.eventBus.On(event.EventAgentFinalAnswer, h.handleFinalAnswer)
h.eventBus.On(event.EventAgentReflection, h.handleReflection)
h.eventBus.On(event.EventError, h.handleError)
h.eventBus.On(event.EventSessionTitle, h.handleSessionTitle)
h.eventBus.On(event.EventAgentComplete, h.handleComplete)
feat(agent): human-in-the-loop approval for MCP tool calls (#1173) Add an opt-in human approval gate so Agent runs pause before executing MCP tools that operators flag as dangerous, surface an approval card in the chat UI, and only resume after the user approves (optionally with edited args) or rejects. Backend - New mcp_tool_approvals table + repo/service to mark per-tool approval required (PG migration 000042 + sqlite init). - approval.Gate coordinates RequestAndWait / Resolve with sync.Once delivery, configurable timeout, and Redis Pub/Sub fan-out so multi- replica deployments work without sticky sessions. - MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx (without the per-tool 60s timeout) for the wait, and re-derives a fresh 60s exec ctx after approval so CallTool keeps a full window. - New SSE response types (tool_approval_required / _resolved) and EventBus events plumb approval state to AgentStreamDisplay. - REST: list/set per-tool approval flag, resolve pending approval. - Configurable via agent.tool_approval_timeout_seconds (yaml) or WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go duration). Frontend - MCP settings: per-tool "require approval" switch on the test panel. - Chat: ToolApprovalCard renders the pause point with editable JSON args, validation feedback, mm:ss countdown that turns warning/danger near deadline, and a resolved state that retains context. - i18n strings added for zh-CN / en-US / ko-KR / ru-RU. Docs - docs/zh/mcp-approval.md covering behavior, config, API, deployment considerations (Redis cross-instance, restart limitations).
2026-05-10 13:23:49 +00:00
h.eventBus.On(event.EventToolApprovalRequired, h.handleToolApprovalRequired)
h.eventBus.On(event.EventToolApprovalResolved, h.handleToolApprovalResolved)
h.eventBus.On(event.EventMCPOAuthRequired, h.handleMCPOAuthRequired)
h.eventBus.On(event.EventMCPOAuthResolved, h.handleMCPOAuthResolved)
}
// handleThought handles agent thought events
func (h *AgentStreamHandler) handleThought(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentThoughtData)
if !ok {
return nil
}
h.mu.Lock()
// Track start time on first chunk
if _, exists := h.eventStartTimes[evt.ID]; !exists {
h.eventStartTimes[evt.ID] = time.Now()
}
// Calculate duration if done
var metadata map[string]interface{}
if data.Done {
startTime := h.eventStartTimes[evt.ID]
duration := time.Since(startTime)
metadata = map[string]interface{}{
"event_id": evt.ID,
"duration_ms": duration.Milliseconds(),
"completed_at": time.Now().Unix(),
}
delete(h.eventStartTimes, evt.ID)
} else {
metadata = map[string]interface{}{
"event_id": evt.ID,
}
}
h.mu.Unlock()
// Append this chunk to stream (no accumulation - frontend will accumulate)
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeThinking,
Content: data.Content, // Just this chunk
Done: data.Done,
Timestamp: time.Now(),
Data: metadata,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append thought event to stream failed", "error", err)
}
return nil
}
// handleToolCall handles tool call events
func (h *AgentStreamHandler) handleToolCall(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentToolCallData)
if !ok {
return nil
}
h.mu.Lock()
_, first := h.eventStartTimes[data.ToolCallID]
if !first {
h.eventStartTimes[data.ToolCallID] = time.Now()
// Any answer text streamed before this tool call was a non-terminal round's
// preamble, not the final answer (the agent only ends by stopping naturally
// with plain text and no tool calls). Drop those segments from the persisted
// answer so the preamble never leaks into Message.Content.
supersededAny := false
for _, seg := range h.answerSegments {
if !seg.superseded && seg.content != "" {
seg.superseded = true
supersededAny = true
}
}
if supersededAny {
h.finalAnswer = h.composeFinalAnswer()
}
}
h.mu.Unlock()
metadata := map[string]interface{}{
"tool_name": data.ToolName,
"arguments": agenttools.SanitizeSandboxFileCallArgs(data.ToolName, data.Arguments),
"tool_call_id": data.ToolCallID,
}
// Append event to stream
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeToolCall,
Content: fmt.Sprintf("Calling tool: %s", data.ToolName),
Done: false,
Timestamp: time.Now(),
Data: metadata,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append tool call event to stream failed", "error", err)
}
return nil
}
// handleToolResult handles tool result events
func (h *AgentStreamHandler) handleToolResult(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentToolResultData)
if !ok {
return nil
}
h.mu.Lock()
// Calculate duration from start time if available, otherwise use provided duration
var durationMs int64
if startTime, exists := h.eventStartTimes[data.ToolCallID]; exists {
durationMs = time.Since(startTime).Milliseconds()
delete(h.eventStartTimes, data.ToolCallID)
} else if data.Duration > 0 {
// Fallback to provided duration if start time not tracked
durationMs = data.Duration
}
h.mu.Unlock()
// Send SSE response (both success and failure). A failed tool execution is
// still a tool result: response_type=error is reserved for internal agent
// failures (handleError), while tool failures are reported through
// response_type=tool_result with success=false in the metadata.
responseType := types.ResponseTypeToolResult
content := agenttools.StreamContentForToolResult(data.ToolName, data.Success, data.Error, data.Data)
if !data.Success && content == "" && data.Error != "" {
content = data.Error
}
// Build metadata including tool result data for rich frontend rendering
metadata := map[string]interface{}{
"tool_name": data.ToolName,
"success": data.Success,
"error": data.Error,
"duration_ms": durationMs,
"tool_call_id": data.ToolCallID,
}
clientData := agenttools.SanitizeToolResultForClient(data.ToolName, &types.ToolResult{
Success: data.Success,
Output: data.Output,
Error: data.Error,
Data: data.Data,
})
for k, v := range clientData {
metadata[k] = v
}
// Append event to stream
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: responseType,
Content: content,
Done: false,
Timestamp: time.Now(),
Data: metadata,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append tool result event to stream failed", "error", err)
}
return nil
}
feat(agent): human-in-the-loop approval for MCP tool calls (#1173) Add an opt-in human approval gate so Agent runs pause before executing MCP tools that operators flag as dangerous, surface an approval card in the chat UI, and only resume after the user approves (optionally with edited args) or rejects. Backend - New mcp_tool_approvals table + repo/service to mark per-tool approval required (PG migration 000042 + sqlite init). - approval.Gate coordinates RequestAndWait / Resolve with sync.Once delivery, configurable timeout, and Redis Pub/Sub fan-out so multi- replica deployments work without sticky sessions. - MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx (without the per-tool 60s timeout) for the wait, and re-derives a fresh 60s exec ctx after approval so CallTool keeps a full window. - New SSE response types (tool_approval_required / _resolved) and EventBus events plumb approval state to AgentStreamDisplay. - REST: list/set per-tool approval flag, resolve pending approval. - Configurable via agent.tool_approval_timeout_seconds (yaml) or WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go duration). Frontend - MCP settings: per-tool "require approval" switch on the test panel. - Chat: ToolApprovalCard renders the pause point with editable JSON args, validation feedback, mm:ss countdown that turns warning/danger near deadline, and a resolved state that retains context. - i18n strings added for zh-CN / en-US / ko-KR / ru-RU. Docs - docs/zh/mcp-approval.md covering behavior, config, API, deployment considerations (Redis cross-instance, restart limitations).
2026-05-10 13:23:49 +00:00
func toolApprovalDataToMap(v interface{}) map[string]interface{} {
b, err := json.Marshal(v)
if err != nil {
return map[string]interface{}{}
}
var m map[string]interface{}
if err := json.Unmarshal(b, &m); err != nil {
return map[string]interface{}{}
}
return m
}
// handleToolApprovalRequired persists MCP tool human-approval prompts for SSE / replay (issue #1173).
func (h *AgentStreamHandler) handleToolApprovalRequired(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.ToolApprovalRequiredData)
if !ok {
return nil
}
meta := toolApprovalDataToMap(data)
meta["pending_id"] = data.PendingID
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeToolApprovalRequired,
Content: "MCP tool requires human approval",
Done: true,
Timestamp: time.Now(),
Data: meta,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append tool approval required event failed", "error", err)
}
return nil
}
// handleToolApprovalResolved persists the outcome of a tool approval (issue #1173).
func (h *AgentStreamHandler) handleToolApprovalResolved(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.ToolApprovalResolvedData)
if !ok {
return nil
}
meta := toolApprovalDataToMap(data)
meta["pending_id"] = data.PendingID
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeToolApprovalResolved,
Content: "MCP tool approval resolved",
Done: true,
Timestamp: time.Now(),
Data: meta,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append tool approval resolved event failed", "error", err)
}
return nil
}
// handleMCPOAuthRequired forwards an in-conversation "authorize this MCP
// service" prompt to the SSE stream so the UI can render an Authorize card.
func (h *AgentStreamHandler) handleMCPOAuthRequired(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.MCPOAuthRequiredData)
if !ok {
return nil
}
meta := toolApprovalDataToMap(data)
meta["pending_id"] = data.PendingID
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeMCPOAuthRequired,
Content: "MCP service requires OAuth authorization",
Done: true,
Timestamp: time.Now(),
Data: meta,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append mcp oauth required event failed", "error", err)
}
return nil
}
// handleMCPOAuthResolved forwards the outcome of an in-conversation OAuth prompt.
func (h *AgentStreamHandler) handleMCPOAuthResolved(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.MCPOAuthResolvedData)
if !ok {
return nil
}
meta := toolApprovalDataToMap(data)
meta["pending_id"] = data.PendingID
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeMCPOAuthResolved,
Content: "MCP OAuth authorization resolved",
Done: true,
Timestamp: time.Now(),
Data: meta,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append mcp oauth resolved event failed", "error", err)
feat(agent): human-in-the-loop approval for MCP tool calls (#1173) Add an opt-in human approval gate so Agent runs pause before executing MCP tools that operators flag as dangerous, surface an approval card in the chat UI, and only resume after the user approves (optionally with edited args) or rejects. Backend - New mcp_tool_approvals table + repo/service to mark per-tool approval required (PG migration 000042 + sqlite init). - approval.Gate coordinates RequestAndWait / Resolve with sync.Once delivery, configurable timeout, and Redis Pub/Sub fan-out so multi- replica deployments work without sticky sessions. - MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx (without the per-tool 60s timeout) for the wait, and re-derives a fresh 60s exec ctx after approval so CallTool keeps a full window. - New SSE response types (tool_approval_required / _resolved) and EventBus events plumb approval state to AgentStreamDisplay. - REST: list/set per-tool approval flag, resolve pending approval. - Configurable via agent.tool_approval_timeout_seconds (yaml) or WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go duration). Frontend - MCP settings: per-tool "require approval" switch on the test panel. - Chat: ToolApprovalCard renders the pause point with editable JSON args, validation feedback, mm:ss countdown that turns warning/danger near deadline, and a resolved state that retains context. - i18n strings added for zh-CN / en-US / ko-KR / ru-RU. Docs - docs/zh/mcp-approval.md covering behavior, config, API, deployment considerations (Redis cross-instance, restart limitations).
2026-05-10 13:23:49 +00:00
}
return nil
}
// handleReferences handles knowledge references events
func (h *AgentStreamHandler) handleReferences(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentReferencesData)
if !ok {
return nil
}
h.mu.Lock()
defer h.mu.Unlock()
// Extract knowledge references
// Try to cast directly to []*types.SearchResult first
if searchResults, ok := data.References.([]*types.SearchResult); ok {
h.knowledgeRefs = append(h.knowledgeRefs, searchResults...)
} else if refs, ok := data.References.([]interface{}); ok {
// Fallback: convert from []interface{}
for _, ref := range refs {
if sr, ok := ref.(*types.SearchResult); ok {
h.knowledgeRefs = append(h.knowledgeRefs, sr)
} else if refMap, ok := ref.(map[string]interface{}); ok {
// Parse from map if needed
h.knowledgeRefs = append(h.knowledgeRefs, searchResultFromMap(refMap))
}
}
}
// Update assistant message references
h.assistantMessage.KnowledgeReferences = h.knowledgeRefs
// Append references event to stream
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeReferences,
Content: "",
Done: false,
Timestamp: time.Now(),
Data: map[string]interface{}{
"references": types.References(h.knowledgeRefs),
},
}); err != nil {
logger.GetLogger(h.ctx).Error("Append references event to stream failed", "error", err)
}
return nil
}
// handleMemoryRecalled records the long-term memories injected into this turn.
// The list is both persisted on the assistant message and streamed, so the
// panel is present live and after a reload.
func (h *AgentStreamHandler) handleMemoryRecalled(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.MemoryRecalledData)
if !ok {
return nil
}
used, ok := data.Memories.(types.UsedMemories)
if !ok || len(used) == 0 {
return nil
}
h.mu.Lock()
h.assistantMessage.UsedMemories = used
h.mu.Unlock()
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeMemoryRecalled,
Done: false,
Timestamp: time.Now(),
Data: map[string]interface{}{"memories": used},
}); err != nil {
logger.GetLogger(h.ctx).Error("Append memory recalled event to stream failed", "error", err)
}
return nil
}
// handleContextCompacted forwards a compaction to the UI.
//
// The summary itself is carried so the user can expand it and see exactly what
// the agent kept, which is the only way to tell an agent that forgot something
// from an agent that never had it.
func (h *AgentStreamHandler) handleContextCompacted(_ context.Context, evt event.Event) error {
data, ok := evt.Data.(event.ContextCompactedData)
if !ok {
return nil
}
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeContextCompacted,
Done: true,
Timestamp: time.Now(),
Data: map[string]interface{}{
"reason": data.Reason,
"round": data.Round,
"tokens_before": data.TokensBefore,
"tokens_after": data.TokensAfter,
"messages_before": data.MessagesBefore,
"messages_after": data.MessagesAfter,
"summary": data.Summary,
"degraded": data.Degraded,
"split_turn": data.SplitTurn,
},
}); err != nil {
logger.GetLogger(h.ctx).Error("Append context compacted event to stream failed", "error", err)
}
return nil
}
// handleFinalAnswer handles final answer events
func (h *AgentStreamHandler) handleFinalAnswer(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentFinalAnswerData)
if !ok {
return nil
}
h.mu.Lock()
// Track start time on first chunk
if _, exists := h.eventStartTimes[evt.ID]; !exists {
h.eventStartTimes[evt.ID] = time.Now()
}
// Emit a one-shot TTFB log the first time *any* answer chunk reaches
// the stream handler. This lets us compare the backend's "request in →
// first token out" timing against the frontend-observed TTFB and pin
// down where latency lives (network vs server vs LLM).
if !h.ttfbLogged && !h.receivedAt.IsZero() {
h.ttfbLogged = true
ttfb := time.Since(h.receivedAt)
logger.GetLogger(h.ctx).Infof("TTFB:first_answer_chunk request_id=%s, session_id=%s, ttfb_ms=%d",
h.requestID, h.sessionID, ttfb.Milliseconds())
}
// Accumulate final answer locally for assistant message (database). Track
// per event ID so a later supersede can subtract this segment's content.
if data.Content != "" {
seg := h.findAnswerSegment(evt.ID)
if seg == nil {
seg = &answerSegment{id: evt.ID}
h.answerSegments = append(h.answerSegments, seg)
}
seg.content += data.Content
h.finalAnswer = h.composeFinalAnswer()
}
if data.IsFallback {
h.assistantMessage.IsFallback = true
}
// Calculate duration if done
var metadata map[string]interface{}
if data.Done {
startTime := h.eventStartTimes[evt.ID]
duration := time.Since(startTime)
metadata = map[string]interface{}{
"event_id": evt.ID,
"duration_ms": duration.Milliseconds(),
"completed_at": time.Now().Unix(),
}
delete(h.eventStartTimes, evt.ID)
} else {
metadata = map[string]interface{}{
"event_id": evt.ID,
}
}
if data.IsFallback {
metadata["is_fallback"] = true
}
fix(agent): end the turn when the completion cap truncates an answer (#3449) * fix(agent): end the turn when the completion cap truncates an answer A round that came back with finish_reason=length and no tool calls used to fall through analyzeResponse as non-terminal. With no tool calls to run, appendToolResults pushed the half-written answer back into the history as a plain assistant message with nothing telling the model to continue, so the next round rewrote the answer from the top, hit the same cap, and repeated until the round budget ran out or the user cancelled. Three guards, each independently load-bearing: - analyzeResponse gains a truncation case: deliver the partial text and end the turn. A cap hit that produced no text at all has nothing to hand over and defers to the existing bounded empty-content retry. No continuation nudge is sent — it only grows the prompt with the discarded fragment, and assistant prefill, the reliable form, is unavailable on most OpenAI-compatible endpoints. - A consecutive-length breaker stops the loop when truncation keeps landing inside tool-call arguments, the one shape that still legitimately loops. - The stuck-loop detector now counts an identical lack of content. A model burning its whole budget on reasoning returns empty every round, which used to land in the else branch and reset the counter, so the guard could never fire on the case that needed it most. The answer event and its Done marker carry a truncated flag through the SSE metadata, and the chat UI shows the same style of notice it already uses for fallback answers. Fixes #3446 * fix(agent): treat unterminated reasoning as no answer and keep the truncation marker Review follow-up on three ways the truncation path could still mislead. An inline <think> block that the cap cut off never gets its closing tag, so the pair regex left it whole and the round looked like it had produced an answer. The truncation case then skipped the empty-content retry it exists for and emitted the chain of thought as the final answer. StripThinkBlocks now drops a dangling <think> through end of input, which is what the streaming splitter already does with the same text, so the two agree on what counts as an answer. The truncation marker only rode on the SSE metadata, and a finished conversation is rebuilt from agent_steps rather than replayed from stream events. A reloaded, shared or re-opened turn therefore lost the notice and its half-written answer looked complete. AgentStep carries the flag now — it is a jsonb column, so no migration — and the frontend marks the rebuilt answer event and message from it. The consecutive-length guard ran before the round was recorded, so a final round of "let me look that up" plus a truncated tool call handed that preamble over as the final answer, with its tool calls never refused and the preamble never retracted from the answer area. The guard now records the round and fails its tool calls the way every other truncated round does, and a response carrying tool calls never donates its text as the answer.
2026-09-20 05:17:50 +00:00
// The completion cap cut this answer off. Carried on every chunk and on
// the Done marker: a live-streamed answer only learns of the cap at the
// close, and the metadata is persisted with the stream event so a replay
// still shows the notice.
if data.Truncated {
metadata["truncated"] = true
}
h.mu.Unlock()
// Append this chunk to stream (frontend will accumulate by event ID)
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeAnswer,
Content: data.Content, // Just this chunk
Done: data.Done,
Timestamp: time.Now(),
Data: metadata,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append answer event to stream failed", "error", err)
}
return nil
}
// handleReflection handles agent reflection events
func (h *AgentStreamHandler) handleReflection(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentReflectionData)
if !ok {
return nil
}
// Append this chunk to stream (frontend will accumulate by event ID)
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeReflection,
Content: data.Content, // Just this chunk
Done: data.Done,
Timestamp: time.Now(),
}); err != nil {
logger.GetLogger(h.ctx).Error("Append reflection event to stream failed", "error", err)
}
return nil
}
// handleError handles error events
func (h *AgentStreamHandler) handleError(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.ErrorData)
if !ok {
return nil
}
// Build error metadata
metadata := map[string]interface{}{
"stage": data.Stage,
"error": data.Error,
}
// Append error event to stream
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeError,
Content: data.Error,
Done: true,
Timestamp: time.Now(),
Data: metadata,
}); err != nil {
logger.GetLogger(h.ctx).Error("Append error event to stream failed", "error", err)
}
return nil
}
// handleSessionTitle handles session title update events
func (h *AgentStreamHandler) handleSessionTitle(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.SessionTitleData)
if !ok {
return nil
}
// Use background context for title event since it may arrive after stream completion
bgCtx := context.Background()
// Append title event to stream
if err := h.streamManager.AppendEvent(bgCtx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeSessionTitle,
Content: data.Title,
Done: true,
Timestamp: time.Now(),
Data: map[string]interface{}{
"session_id": data.SessionID,
"title": data.Title,
},
}); err != nil {
logger.GetLogger(h.ctx).Warn("Append session title event to stream failed (stream may have ended)", "error", err)
}
return nil
}
feat(chat): append messages to a running agent turn (#3123) * feat(stream): persist live-run markers and per-run steer queues Mid-run sends need a cross-replica pointer to the generating assistant and a control-plane list that never hits the SSE stream. * feat(agent): inject queued user messages at ReAct round boundaries The engine drains the steer sink after each round, persists accepted rows, and delays Done until after a loop-end inject so the turn can continue. * feat(session): expose /steer and hand leftover messages to a follow-up turn Owner-scoped steer routes queue or inject into the live run, single-flight AgentQA via the live-run marker, and history replay keeps multiple user rows inside one turn. * feat(frontend): queue mid-run messages while an agent turn streams Stop and Send stay visible together, the overlay follows server delivery, and history splits the transcript around injected user bubbles. * fix(frontend): type pending and client_id on SteerQueueItem vue-tsc failed because the overlay sets those fields while POST /steer is in flight. * fix(session): harden steer persist, handoff, and new_run Keep SetLiveRun ahead of SSE and roll back orphan turns, make inject persist idempotent, append-then-delete on rebind, cap overrun rounds, and stop aborting a live stream when /steer returns new_run. * fix(session): satisfy pre-push lint on steer paths Wrap long lines, check discarded Error/Emit results, and keep context as the first argument so the fork push hook can pass. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-09 08:43:33 +00:00
// handleUserMessageInjected forwards a mid-run message injection to the
// user-visible stream so the frontend can flip its optimistic "queued"
// bubble into a normal message of the running turn. The event carries the
// steer ID the client generated queue time, so correlation is exact.
func (h *AgentStreamHandler) handleUserMessageInjected(_ context.Context, evt event.Event) error {
data, ok := evt.Data.(event.UserMessageInjectedData)
if !ok {
return nil
}
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeUserMessageInjected,
Done: true,
Timestamp: time.Now(),
Data: map[string]interface{}{
"steer_id": data.SteerID,
"message_id": data.MessageID,
"content": data.Content,
"user_message_id": data.UserMessageID,
},
}); err != nil {
logger.GetLogger(h.ctx).Error("Append user message injected event to stream failed", "error", err)
}
return nil
}
// handleComplete handles agent complete events
func (h *AgentStreamHandler) handleComplete(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentCompleteData)
if !ok {
return nil
}
h.mu.Lock()
defer h.mu.Unlock()
// Update assistant message with final data
if data.MessageID == h.assistantMessageID {
// h.assistantMessage.Content = data.FinalAnswer
h.assistantMessage.IsCompleted = true
h.assistantMessage.AgentDurationMs = data.TotalDurationMs
// Update knowledge references if provided
if len(data.KnowledgeRefs) > 0 {
knowledgeRefs := make([]*types.SearchResult, 0, len(data.KnowledgeRefs))
for _, ref := range data.KnowledgeRefs {
if sr, ok := ref.(*types.SearchResult); ok {
knowledgeRefs = append(knowledgeRefs, sr)
}
}
h.assistantMessage.KnowledgeReferences = knowledgeRefs
}
h.assistantMessage.Content += data.FinalAnswer
// Update agent steps if provided
if data.AgentSteps != nil {
if steps, ok := data.AgentSteps.([]types.AgentStep); ok {
h.assistantMessage.AgentSteps = agenttools.SanitizeAgentStepsForStorage(steps)
}
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
// Persist the turn's aggregated LLM usage with the message so history
// reads still carry it after the live stream is gone.
if usage, ok := data.Usage.(*types.TokenUsage); ok && usage != nil {
h.assistantMessage.Usage = usage
}
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
// Check point /workspace before draining artifacts. Every tool has
// finished, and the turn's sandbox lease is still held, so the sandbox
// is guaranteed to still be here.
//
// This runs on every exit path, including cancelled and errored turns.
// Skipping unsuccessful turns would fold their file changes into the
// NEXT turn's commit, so forking at that next turn would silently pick
// up the cancelled turn's edits.
//
// Best-effort throughout: a nil checkpoint just means this message
// cannot serve as a fork point.
if h.checkpointer != nil && h.sandboxIDLookup != nil {
checkpointCtx := context.WithoutCancel(h.ctx)
if sandboxID, ok := h.sandboxIDLookup.BoundSandboxID(checkpointCtx, h.sessionID); ok {
h.assistantMessage.SandboxCheckpoint = h.checkpointer.Checkpoint(
checkpointCtx, h.sessionID, sandboxID, h.assistantMessageID,
)
}
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
// Drain skill-generated files from the sandbox into persistent
// storage. Best-effort: any failure is logged and the turn is
// persisted without artifacts. Collect is a no-op when either the
// collector wasn't wired in, no sandbox is bound, or no files were
// produced — those cases must not disturb the completion path.
feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) * feat(sandbox): resolve workspace paths through WorkspaceLayout Keep remote sessions on the /workspace contract, while file, shell, and artifact code follow the session layout instead of hardcoded paths. * fix(sandbox): fail closed on host layout lookup and restore the shell blacklist Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots, and skip collection when the output tree is missing or equals the project root. * fix(sandbox): close leftover fail-open holes in the layout path Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots. * fix(sandbox): satisfy golangci-lint on the layout helpers Wrap new copy and test lines to 120 columns, and add the exported comments revive requires. * fix(sandbox): gofmt the read_file layout copy wrap * feat(sandbox): add host as an OS-enforced backend type Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B. * feat(session): persist Lite host workspace directory Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects. * feat(localsandbox): adapt the host OS sandbox to session interfaces File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace. * feat(sandbox): run unpinned Lite sessions on the host backend Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed. * feat(desktop): pick and persist host project directories The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it. * fix(sandbox): fail closed on Lite host layout lookup misses Unpinned host sessions were described as /workspace when pin read or sandbox resolve failed. Prompts and attachment staging now share the same fallback so Lite never pretends the remote tree exists. * fix(frontend): keep settings.sandbox.host in sync with the backend keys TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI. * fix(lint): satisfy golangci-lint on the host sandbox wiring Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-22 11:35:03 +00:00
//
// Layouts without a separate output tree must not be collected
// (scanning Root would upload the whole project). Remote backends keep
// skills.ArtifactOutputDir(), or layout.OutputDir when advertised.
var previous types.MessageArtifacts
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
if h.artifactCollector != nil {
collectCtx := context.WithoutCancel(h.ctx)
feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) * feat(sandbox): resolve workspace paths through WorkspaceLayout Keep remote sessions on the /workspace contract, while file, shell, and artifact code follow the session layout instead of hardcoded paths. * fix(sandbox): fail closed on host layout lookup and restore the shell blacklist Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots, and skip collection when the output tree is missing or equals the project root. * fix(sandbox): close leftover fail-open holes in the layout path Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots. * fix(sandbox): satisfy golangci-lint on the layout helpers Wrap new copy and test lines to 120 columns, and add the exported comments revive requires. * fix(sandbox): gofmt the read_file layout copy wrap * feat(sandbox): add host as an OS-enforced backend type Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B. * feat(session): persist Lite host workspace directory Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects. * feat(localsandbox): adapt the host OS sandbox to session interfaces File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace. * feat(sandbox): run unpinned Lite sessions on the host backend Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed. * feat(desktop): pick and persist host project directories The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it. * fix(sandbox): fail closed on Lite host layout lookup misses Unpinned host sessions were described as /workspace when pin read or sandbox resolve failed. Prompts and attachment staging now share the same fallback so Lite never pretends the remote tree exists. * fix(frontend): keep settings.sandbox.host in sync with the backend keys TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI. * fix(lint): satisfy golangci-lint on the host sandbox wiring Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-22 11:35:03 +00:00
var artifacts types.MessageArtifacts
if collectDir, skip := h.artifactCollector.CollectTarget(collectCtx, h.sessionID); !skip {
if collectDir == "" {
collectDir = skills.ArtifactOutputDir()
}
var err error
artifacts, err = h.artifactCollector.CollectWithNotify(
collectCtx,
h.sessionID,
h.assistantMessageID,
h.tenantID,
collectDir,
h.emitArtifactsPending,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
)
feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) * feat(sandbox): resolve workspace paths through WorkspaceLayout Keep remote sessions on the /workspace contract, while file, shell, and artifact code follow the session layout instead of hardcoded paths. * fix(sandbox): fail closed on host layout lookup and restore the shell blacklist Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots, and skip collection when the output tree is missing or equals the project root. * fix(sandbox): close leftover fail-open holes in the layout path Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots. * fix(sandbox): satisfy golangci-lint on the layout helpers Wrap new copy and test lines to 120 columns, and add the exported comments revive requires. * fix(sandbox): gofmt the read_file layout copy wrap * feat(sandbox): add host as an OS-enforced backend type Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B. * feat(session): persist Lite host workspace directory Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects. * feat(localsandbox): adapt the host OS sandbox to session interfaces File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace. * feat(sandbox): run unpinned Lite sessions on the host backend Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed. * feat(desktop): pick and persist host project directories The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it. * fix(sandbox): fail closed on Lite host layout lookup misses Unpinned host sessions were described as /workspace when pin read or sandbox resolve failed. Prompts and attachment staging now share the same fallback so Lite never pretends the remote tree exists. * fix(frontend): keep settings.sandbox.host in sync with the backend keys TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI. * fix(lint): satisfy golangci-lint on the host sandbox wiring Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-22 11:35:03 +00:00
if err != nil {
logger.GetLogger(h.ctx).Warnf(
"artifact collect failed session=%s message=%s: %v",
h.sessionID, h.assistantMessageID, err,
)
}
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
}
// Resolve the files the answer names against this turn's artifacts
// AND the ones already recorded for the session.
//
// A turn can legitimately reference a file it did not regenerate.
// The first turn after a session fork is restored to the fork point,
// so every unchanged file is de-duplicated out of `artifacts` — yet
// the model still writes `sandbox:<name>` for it. Gating the
// rewrite on `artifacts` alone would leave those references
// unnormalized, and the client resolves names only against the
// message's own artifact list, so they would render as missing
// files.
// KnownArtifacts is oldest-first. Name matching is first-wins, so
// this turn shadows a regenerated file, then the latest known
// version (the fork-point file) beats earlier ones of the same name.
known := h.artifactCollector.SessionArtifacts(collectCtx, h.sessionID)
referenced := referencedArtifacts(
h.assistantMessage.Content,
mergeArtifactLists(artifacts, artifactsNewestFirst(known)),
)
previous = historyOnlyArtifacts(referenced, artifacts)
if attached := mergeArtifactLists(h.assistantMessage.Artifacts, artifacts, referenced); len(attached) > 0 {
h.assistantMessage.Artifacts = attached
feat(agent): preview skill artifacts and verify installs without running them (#2823) * feat(agent): preview skill artifacts and verify installs without running them Generated files open in the chat drawer with overlay/Esc returning to the list, and install verification no longer executes guessed skill entry points. A failed install can be retried from the saved archive. * ui(chat): use a folder icon and corner count for skill artifacts The eye and document glyphs read as preview or copy, and TDesign's loading spinner sat beside the icon. A folder with a top-right count matches the generated-files drawer without covering the toolbar action. * feat(artifact): implement inline artifact references with previews Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic. * fix(client): document AgentResponseType constants for revive Touching the const block made golangci-lint require comments on every exported value. Match the rest of the SDK and describe artifacts_pending. * fix(agent): wrap artifact-reference prompt lines for lll Pre-push golangci-lint rejected the branch on three lines over 120 characters in the sandbox: reference guidance. * fix(agent): scope HTML scripts and tighten install verification Keep scripted HTML preview on skill artifacts only, recreate sandbox workspace dirs after they are deleted, and treat nested skill files as dependencies rather than first-party imports. * refactor(artifact): enhance artifact reference handling and metadata Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 02:30:32 +00:00
// The answer text names generated files the way the model saw
// them in the sandbox. Bind those names to artifact indices now
// that the index space is final, so a reloaded conversation
// renders them instead of showing a broken link.
h.assistantMessage.Content = rewriteArtifactReferences(
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
h.assistantMessage.Content, attached,
feat(agent): preview skill artifacts and verify installs without running them (#2823) * feat(agent): preview skill artifacts and verify installs without running them Generated files open in the chat drawer with overlay/Esc returning to the list, and install verification no longer executes guessed skill entry points. A failed install can be retried from the saved archive. * ui(chat): use a folder icon and corner count for skill artifacts The eye and document glyphs read as preview or copy, and TDesign's loading spinner sat beside the icon. A folder with a top-right count matches the generated-files drawer without covering the toolbar action. * feat(artifact): implement inline artifact references with previews Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic. * fix(client): document AgentResponseType constants for revive Touching the const block made golangci-lint require comments on every exported value. Match the rest of the SDK and describe artifacts_pending. * fix(agent): wrap artifact-reference prompt lines for lll Pre-push golangci-lint rejected the branch on three lines over 120 characters in the sandbox: reference guidance. * fix(agent): scope HTML scripts and tighten install verification Keep scripted HTML preview on skill artifacts only, recreate sandbox workspace dirs after they are deleted, and treat nested skill files as dependencies rather than first-party imports. * refactor(artifact): enhance artifact reference handling and metadata Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 02:30:32 +00:00
)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
logger.GetLogger(h.ctx).Infof(
"artifact collect attached %d file(s) to message=%s session=%s",
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
len(attached), h.assistantMessageID, h.sessionID,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
)
}
feat(session): fork conversations from a historical user message (#3304) * feat(session): add fork schema and history queries 为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。 * feat(sandbox): add git, fork snapshots, and bootstrap hook 沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。 * feat(session): checkpoint workspace at the end of each turn 每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。 * feat(session): fork conversations and reap orphan snapshots 支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。 * feat(chat): add session fork entry, prefill, and badges 在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。 * style: add missing newline so session_fork_delete_test.go is gofmt-clean * fix(session): prune later git history when rolling a fork workspace back reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。 * style(session): clear golangci-lint findings on the session-fork diff Pre-push lints the whole PR against main; wrap long lines, check c.Error, and fix gofumpt/revive/staticcheck so the merge can be pushed. * fix(chat): show the fork button on hover without shifting the thread Overlay the action so revealing it no longer expands the user bubble and janks the messages below. * fix(chat): drop the fork landing banners Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread. * fix(agent): dedupe artifacts by content hash, not size Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only. * feat(session): allow forking from an assistant answer User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists. * fix(session): resolve parent attachments from a forked chat Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss. * fix(agent): rewrite sandbox refs after a content-hash skip Fork restores change mtime on unchanged files, so Collect returns empty and named citations were left as sandbox: paths. Resolve those names against session artifacts, notify only real uploads, and stamp restored mtime only on the matching content hash. * fix(session): drop a dead fork snapshot and retry checkpoint rewrite Create from a missing template now abandons bootstrap so the next Resolve uses a normal sandbox. A failed checkpoint rewrite no longer marks the bootstrap consumed, which previously made nested forks SANDBOX_REPLACED. * fix(session): load forked parent attachments into the prompt ResolveForPrompt used session-scoped lookup, so copied parent attachment IDs passed Get but never reached the model. Use the same resolveInSession path as preview. * fix(chat): keep the user-message fork button visible The control was hover-only, which hid it on touch. Always show it, and drop the unused degrade tooltip. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 06:20:18 +00:00
// A reused reference is owned by this message too, so deleting the
// message that first produced the file cannot invalidate it.
h.artifactCollector.BindArtifactsToMessage(collectCtx, h.assistantMessageID, previous)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
}
h.assistantMessage.Content = types.ClarifyArtifactVersions(h.assistantMessage.Content,
h.assistantMessage.Artifacts, previous, types.LanguageFromContextOrDefault(h.ctx))
}
// Fallback: if no answer events were streamed but we have a final answer,
// emit it as answer events so the frontend can render it properly.
// This guards against edge cases where the LLM stops without calling final_answer.
if h.finalAnswer == "" && data.FinalAnswer != "" {
logger.GetLogger(h.ctx).Warnf(
"No answer events were streamed, emitting fallback answer (len=%d). "+
"This typically happens when: (1) model stopped naturally and content was sent as thought events, "+
"or (2) Ollama model returned tool calls non-incrementally. "+
"total_steps=%d, total_duration_ms=%d",
len(data.FinalAnswer), data.TotalSteps, data.TotalDurationMs,
)
fallbackID := fmt.Sprintf("answer-fallback-%d", time.Now().UnixMilli())
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: fallbackID,
Type: types.ResponseTypeAnswer,
Content: data.FinalAnswer,
Done: false,
Timestamp: time.Now(),
Data: map[string]interface{}{
"event_id": fallbackID,
"is_fallback": true,
},
}); err != nil {
logger.GetLogger(h.ctx).Errorf("Append fallback answer event failed: %v", err)
}
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: fallbackID,
Type: types.ResponseTypeAnswer,
Content: "",
Done: true,
Timestamp: time.Now(),
Data: map[string]interface{}{
"event_id": fallbackID,
"is_fallback": true,
},
}); err != nil {
logger.GetLogger(h.ctx).Errorf("Append fallback answer done event failed: %v", err)
}
}
// Prepare completion metadata. Publishing must wait for the caller to save
// Content/AgentSteps/Artifacts: message-scoped file requests authorize against
// that persisted output as soon as the client receives this event.
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
completeData := map[string]interface{}{
"total_steps": data.TotalSteps,
"total_duration_ms": data.TotalDurationMs,
"final_content": h.assistantMessage.Content,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
}
// Attach the freshly-collected artifacts so the frontend can render the
// download button without waiting for a page refresh. We strip the
// storage URL and any other server-only fields via publicArtifactViews
// — clients only ever download through /artifacts/:index which enforces
// tenant ownership.
if len(h.assistantMessage.Artifacts) > 0 {
completeData["artifacts"] = publicArtifactViews(h.assistantMessage.Artifacts)
}
// Carry the turn's aggregated LLM usage both inside data (map consumers)
// and on the typed event field, which buildStreamResponse promotes to the
// response's top-level usage.
turnUsage, _ := data.Usage.(*types.TokenUsage)
if turnUsage != nil {
completeData["usage"] = turnUsage
}
h.completionEvent = &interfaces.StreamEvent{
ID: evt.ID,
Type: types.ResponseTypeComplete,
Content: "",
Done: true,
Timestamp: time.Now(),
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
Data: completeData,
Usage: turnUsage,
}
return nil
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
// publishCompletion is called only after UpdateMessage succeeds. Keep it
// separate from handleComplete so steering handoff and final persistence retain
// their existing order, including on cancellation and quick-answer turns.
func (h *AgentStreamHandler) publishCompletion(ctx context.Context) error {
h.mu.Lock()
defer h.mu.Unlock()
if h.completionEvent == nil {
return nil
}
if err := h.streamManager.AppendEvent(ctx, h.sessionID, h.assistantMessageID, *h.completionEvent); err != nil {
return err
}
h.completionEvent = nil
return nil
}
feat(agent): preview skill artifacts and verify installs without running them (#2823) * feat(agent): preview skill artifacts and verify installs without running them Generated files open in the chat drawer with overlay/Esc returning to the list, and install verification no longer executes guessed skill entry points. A failed install can be retried from the saved archive. * ui(chat): use a folder icon and corner count for skill artifacts The eye and document glyphs read as preview or copy, and TDesign's loading spinner sat beside the icon. A folder with a top-right count matches the generated-files drawer without covering the toolbar action. * feat(artifact): implement inline artifact references with previews Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic. * fix(client): document AgentResponseType constants for revive Touching the const block made golangci-lint require comments on every exported value. Match the rest of the SDK and describe artifacts_pending. * fix(agent): wrap artifact-reference prompt lines for lll Pre-push golangci-lint rejected the branch on three lines over 120 characters in the sandbox: reference guidance. * fix(agent): scope HTML scripts and tighten install verification Keep scripted HTML preview on skill artifacts only, recreate sandbox workspace dirs after they are deleted, and treat nested skill files as dependencies rather than first-party imports. * refactor(artifact): enhance artifact reference handling and metadata Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 02:30:32 +00:00
// emitArtifactsPending tells the live UI that sandbox files exist and are
// being uploaded. It must not take h.mu — Collect calls it while
// handleComplete already holds the lock.
func (h *AgentStreamHandler) emitArtifactsPending(count int) {
if h == nil || h.streamManager == nil || count <= 0 {
return
}
if err := h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: fmt.Sprintf("artifacts-pending-%d", time.Now().UnixMilli()),
Type: types.ResponseTypeArtifactsPending,
Timestamp: time.Now(),
Data: map[string]interface{}{
"count": count,
},
}); err != nil {
logger.GetLogger(h.ctx).Warnf(
"append artifacts_pending failed session=%s message=%s: %v",
h.sessionID, h.assistantMessageID, err,
)
}
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
// publicArtifactViews returns a redacted view of the artifact list suitable
feat(agent): preview skill artifacts and verify installs without running them (#2823) * feat(agent): preview skill artifacts and verify installs without running them Generated files open in the chat drawer with overlay/Esc returning to the list, and install verification no longer executes guessed skill entry points. A failed install can be retried from the saved archive. * ui(chat): use a folder icon and corner count for skill artifacts The eye and document glyphs read as preview or copy, and TDesign's loading spinner sat beside the icon. A folder with a top-right count matches the generated-files drawer without covering the toolbar action. * feat(artifact): implement inline artifact references with previews Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic. * fix(client): document AgentResponseType constants for revive Touching the const block made golangci-lint require comments on every exported value. Match the rest of the SDK and describe artifacts_pending. * fix(agent): wrap artifact-reference prompt lines for lll Pre-push golangci-lint rejected the branch on three lines over 120 characters in the sandbox: reference guidance. * fix(agent): scope HTML scripts and tighten install verification Keep scripted HTML preview on skill artifacts only, recreate sandbox workspace dirs after they are deleted, and treat nested skill files as dependencies rather than first-party imports. * refactor(artifact): enhance artifact reference handling and metadata Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 02:30:32 +00:00
// for direct serialization onto the SSE stream. The physical storage path is
// stripped; the resource handle is kept because it is what the answer body
// references, and the frontend needs it to tie an inline reference to the file
// it names. Bytes are fetched through /artifacts/:index/download.
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
func publicArtifactViews(list types.MessageArtifacts) []map[string]interface{} {
out := make([]map[string]interface{}, 0, len(list))
for i, a := range list {
out = append(out, map[string]interface{}{
"index": i,
feat(agent): preview skill artifacts and verify installs without running them (#2823) * feat(agent): preview skill artifacts and verify installs without running them Generated files open in the chat drawer with overlay/Esc returning to the list, and install verification no longer executes guessed skill entry points. A failed install can be retried from the saved archive. * ui(chat): use a folder icon and corner count for skill artifacts The eye and document glyphs read as preview or copy, and TDesign's loading spinner sat beside the icon. A folder with a top-right count matches the generated-files drawer without covering the toolbar action. * feat(artifact): implement inline artifact references with previews Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic. * fix(client): document AgentResponseType constants for revive Touching the const block made golangci-lint require comments on every exported value. Match the rest of the SDK and describe artifacts_pending. * fix(agent): wrap artifact-reference prompt lines for lll Pre-push golangci-lint rejected the branch on three lines over 120 characters in the sandbox: reference guidance. * fix(agent): scope HTML scripts and tighten install verification Keep scripted HTML preview on skill artifacts only, recreate sandbox workspace dirs after they are deleted, and treat nested skill files as dependencies rather than first-party imports. * refactor(artifact): enhance artifact reference handling and metadata Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 02:30:32 +00:00
"handle": artifactHandle(a),
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
"file_name": a.FileName,
"file_type": a.FileType,
"file_size": a.FileSize,
"source_path": a.SourcePath,
"mod_time": a.ModTime,
"created_at": a.CreatedAt,
})
}
return out
}
// handleCommandOutput forwards UI-only progress without adding partial output
// to the model conversation or completing the tool call.
func (h *AgentStreamHandler) handleCommandOutput(_ context.Context, evt event.Event) error {
data, ok := evt.Data.(event.CommandOutputData)
if !ok || data.ToolCallID == "" {
return nil
}
return h.streamManager.AppendEvent(h.ctx, h.sessionID, h.assistantMessageID, interfaces.StreamEvent{
ID: evt.ID, Type: types.ResponseTypeCommandOutput, Timestamp: time.Now(),
Data: map[string]interface{}{
"tool_call_id": data.ToolCallID, "command": data.Command,
"started_at": data.StartedAt, "output": data.Output, "done": data.Done,
},
})
}