WeKnora/internal/agent/tools/shell_exec_test.go

624 lines
23 KiB
Go
Raw Normal View History

feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
package tools
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/Tencent/WeKnora/internal/agent/skills"
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/Tencent/WeKnora/internal/types"
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type fakeShellExecutor struct {
result *sandbox.ExecuteResult
err error
timeout time.Duration
calls int
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
command string
workDir string
env map[string]string
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
}
func (f *fakeShellExecutor) ExecShellCommand(
_ context.Context,
_ string,
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
command string,
workDir string,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
timeout time.Duration,
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
env map[string]string,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
) (*sandbox.ExecuteResult, error) {
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
f.command, f.workDir, f.env = command, workDir, env
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
f.timeout = timeout
f.calls++
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
if f.err != nil {
return nil, f.err
}
if f.result == nil {
return &sandbox.ExecuteResult{ExitCode: 0}, nil
}
return f.result, nil
}
func shellExecTestContext() context.Context {
return WithToolExecContext(context.Background(), &ToolExecContext{SessionID: "session-1"})
}
func TestShellExecAllowsWorkDirOutsideWorkspace(t *testing.T) {
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
executor := &fakeShellExecutor{}
tool := NewShellExecTool(executor, nil)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"pwd","work_dir":"../tmp/task"}`,
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
))
require.NoError(t, err)
require.True(t, result.Success, result.Error)
require.Equal(t, "/tmp/task", executor.workDir)
require.Equal(t, 1, executor.calls)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
}
func TestShellExecTimeoutHonorsAndCapsRequestedValue(t *testing.T) {
executor := &fakeShellExecutor{}
tool := NewShellExecTool(executor, nil)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"sleep 1","timeout_sec":999}`,
))
require.NoError(t, err)
require.True(t, result.Success)
assert.Equal(t, shellExecMaxTimeout, executor.timeout)
}
type fakeInstallShellExecutor struct {
timeout time.Duration
workDir string
calls int
}
func (f *fakeInstallShellExecutor) ExecShellCommandWithOptions(
_ context.Context, _ string, _ string, opts sandbox.ShellExecOptions,
) (*sandbox.ExecuteResult, error) {
f.timeout = opts.Timeout
f.workDir = opts.WorkDir
f.calls++
return &sandbox.ExecuteResult{ExitCode: 0}, nil
}
// installShellSkillDir is the directory an install owns, and the working
// directory its shell must default to.
const installShellSkillDir = sandbox.SkillsImageRoot + "/pdf-tools"
func TestInstallShellExecDescriptionDoesNotPointAtWriteSandboxFile(t *testing.T) {
description := NewInstallShellExecTool(&fakeInstallShellExecutor{}, installShellSkillDir).Description()
assert.Contains(t, description, ToolWriteSkillFile,
"the writer for this tree is write_skill_file, not a heredoc")
assert.Contains(t, description, installShellSkillDir)
assert.NotContains(t, description, "Do NOT dump large files through")
assert.NotContains(t, NewShellExecTool(&fakeShellExecutor{}, nil).Description(),
"write_sandbox_file is not available")
}
// The model reached for `cd <skill-dir> && …` on command after command because
// the default landed it in /workspace — a directory an install is told not to
// touch, since it is wiped before the snapshot. Naming the skill directory as
// the default is what removes the prefix.
func TestInstallShellExecDefaultsToTheSkillDirectory(t *testing.T) {
inner := &fakeInstallShellExecutor{}
tool := NewInstallShellExecTool(inner, installShellSkillDir)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"ls -la scripts/"}`,
))
require.NoError(t, err)
require.True(t, result.Success, result.Error)
assert.Equal(t, installShellSkillDir, inner.workDir)
assert.Equal(t, installShellSkillDir, result.Data["work_dir"],
"the transcript must show where the command actually ran")
assert.Contains(t, tool.Description(), "Do NOT prefix",
"the default alone did not stop the model; it has to be told")
}
// An explicit work_dir still wins: an install occasionally has to leave its own
// directory, and the allowed roots have not changed.
func TestInstallShellExecStillHonoursAnExplicitWorkDir(t *testing.T) {
inner := &fakeInstallShellExecutor{}
result, err := NewInstallShellExecTool(inner, installShellSkillDir).Execute(
shellExecTestContext(),
json.RawMessage(`{"command":"ls","work_dir":"/workspace"}`),
)
require.NoError(t, err)
require.True(t, result.Success, result.Error)
assert.Equal(t, "/workspace", inner.workDir)
}
// A run that somehow carries no valid skill directory must not invent one, and
// must not send commands to a path outside the allowed roots.
func TestInstallShellExecFallsBackToWorkspaceWithoutAValidSkillDir(t *testing.T) {
for _, dir := range []string{"", sandbox.SkillsImageRoot, "/etc", "/opt/weknora/tenant/skills/a/b"} {
inner := &fakeInstallShellExecutor{}
result, err := NewInstallShellExecTool(inner, dir).Execute(
shellExecTestContext(), json.RawMessage(`{"command":"ls"}`))
require.NoError(t, err)
require.True(t, result.Success, result.Error)
assert.Equal(t, "/workspace", inner.workDir, "skillDir %q must not become a default", dir)
}
}
func TestInstallShellExecDefaultsToTheTenMinuteBudget(t *testing.T) {
inner := &fakeInstallShellExecutor{}
tool := NewInstallShellExecTool(inner, installShellSkillDir)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"pip install -r requirements.txt"}`,
))
require.NoError(t, err)
require.True(t, result.Success)
assert.Equal(t, shellExecMaxTimeout, inner.timeout,
"install-mode shell_exec must not keep the ordinary 120s default")
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
func TestShellExecConfigurableOutputAndRegistryOverride(t *testing.T) {
content := strings.Repeat("x", 24*1024)
executor := &fakeShellExecutor{result: &sandbox.ExecuteResult{
Stdout: content,
ExitCode: 0,
}}
registry := NewToolRegistry()
registry.RegisterTool(NewShellExecTool(executor, nil))
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
result, err := registry.ExecuteTool(shellExecTestContext(), ToolShellExec, json.RawMessage(
`{"command":"cat large.txt","max_output_bytes":32768}`,
))
require.NoError(t, err)
require.True(t, result.Success)
assert.Contains(t, result.Output, content)
assert.NotContains(t, result.Output, "output truncated")
assert.Equal(t, 32768, result.Data["max_output_bytes"])
assert.Equal(t, "shell_exec", result.Data["display_type"])
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
}
func TestShellExecOutputLimitIsHardCapped(t *testing.T) {
tool := NewShellExecTool(&fakeShellExecutor{}, nil)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
limit := tool.OutputLimitChars(json.RawMessage(`{"max_output_bytes":999999}`))
assert.Equal(t, maxShellExecVisibleBytes, limit)
}
func TestShellExecSuppressesBinaryStreams(t *testing.T) {
binary := "prefix\x00\x01payload"
executor := &fakeShellExecutor{result: &sandbox.ExecuteResult{
Stdout: binary,
Stderr: "text error",
ExitCode: 0,
}}
tool := NewShellExecTool(executor, nil)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(`{"command":"cat image.bin"}`))
require.NoError(t, err)
require.True(t, result.Success)
assert.NotContains(t, result.Output, binary)
assert.Contains(t, result.Output, "Binary Output Suppressed")
assert.Equal(t, "", result.Data["stdout"])
assert.Equal(t, true, result.Data["stdout_binary"])
assert.Equal(t, len(binary), result.Data["stdout_bytes"])
assert.Equal(t, "text error", result.Data["stderr"])
}
func TestShellExecDescriptionDefinesOneExecutionEntry(t *testing.T) {
description := NewShellExecTool(&fakeShellExecutor{}, nil).Description()
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
for _, fact := range []string{
"/workspace", "skill_name", "virtualenv", "as root", "die with this session",
"write_sandbox_file", "edit_sandbox_file", "not automatically saved",
} {
require.Contains(t, description, fact)
}
require.NotContains(t, description, "execute_skill_script")
require.Less(t, len(description), 2500)
}
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
func TestShellExecBoundsStdoutStderrErrorAndTotal(t *testing.T) {
executor := &fakeShellExecutor{result: &sandbox.ExecuteResult{
Stdout: strings.Repeat("o", 100*1024),
Stderr: strings.Repeat("s", 40*1024) + "TAIL_ERROR",
Error: strings.Repeat("e", 8*1024),
}}
result, err := NewShellExecTool(executor, nil).Execute(
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
shellExecTestContext(),
json.RawMessage(`{"command":"noisy","max_output_bytes":999999,"max_stderr_bytes":999999}`),
)
require.NoError(t, err)
require.False(t, result.Success)
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
assert.LessOrEqual(t, len(result.Output), maxShellExecVisibleBytes)
assert.LessOrEqual(t, result.Data["stdout_returned_bytes"].(int), maxShellExecOutputBytes)
assert.LessOrEqual(t, result.Data["stderr_returned_bytes"].(int), maxShellExecStderrBytes)
assert.LessOrEqual(t, result.Data["error_returned_bytes"].(int), maxShellExecErrorBytes)
assert.Equal(t, true, result.Data["stdout_truncated"])
assert.Equal(t, true, result.Data["stderr_truncated"])
assert.Equal(t, true, result.Data["error_truncated"])
assert.Contains(t, result.Data["stderr"], "TAIL_ERROR")
}
func TestShellExecBoundsExecutorErrors(t *testing.T) {
executor := &fakeShellExecutor{err: errors.New(strings.Repeat("network failure ", 1024))}
result, err := NewShellExecTool(executor, nil).Execute(
feat(sandbox) (#2301) * cubesandbox support * feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system - Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination - sandbox_read: read sandbox file content with offset-based chunked reading - Add Artifacts system: - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI - DB migration 000066_message_artifacts: add artifacts column to messages table - Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access - Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules - Extend Agent Observe: include agent steps in Langfuse observability tracing - Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries - Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales - Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc. * feat(agent):add remote sandbox client abstract & client errors * feat(sandbox): implement Redis binding store with distributed lifecycle lock; * feat(sandbox): add remote session lifecycle with ownership context Implement remoteSessionLifecycle to coordinate persistent remote sandboxes across multi-instance deployments. Supports Resolve (lazy provision, recovery, race-safe create) and Destroy with compare-delete semantics. Add ownership context isolation to decouple remote resource cleanup from request cancellation. * feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient implementation. Error normalization, timeout mapping, and metadata support are fully integrated. SDK details remain private to the adapter. * refactor(sandbox): decouple SessionBoundManager from Cube Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add RemoteSandbox as provider-neutral Sandbox interface implementation. Rewrite SessionBoundManager to depend on RemoteSandboxClient + SessionSandboxBindingStore + SessionExistenceChecker, with all lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry with time.Time ModTime. Cleanup delegated to provider idle-timeout. * refactor(agent): enhance session capability handling in sandbox manager - Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution. - Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends. - Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types. - Removed Cube-specific checks, promoting a more provider-neutral approach to session management. * refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding - Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove) - Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager) - Dependency: matiasinsaurralde/go-e2b v0.1.0 - Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go, cube_defaults.go) and replace with direct *cubesandbox.Client usage - Parse CubeProxy URL directly in adapter (parseProxyURL), removing the old hand-rolled envd transport layer - Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra - Target Cube deployment v0.5.1 - Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess, AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters - Both adapters default to permissive (internet+public on) for backwards compat - Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE - Add repositorySessionExistenceChecker — checks session existence via DB (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding) - newSandboxManager() now accepts redisClient + sessionRepo; deprecated hardcoded NewMemorySessionSandboxBindingStore() in production path - buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume (Cube and E2B). Sessions survive brief idle periods; providers still handle TTL-based reaping. - buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized by RemoteProvider (was hardcoded to Cube Config fields) - NewSessionBoundManager validates provider and delegates defaults to per-provider apply*Defaults helpers - Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go, cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines) - Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal, IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding - Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds - Dead ExecuteResult method: GetOutput - Unused test: session_artifacts_integration_test.go - Preloaded skill: sandbox-test-python - Extract shared httpErrorKind() from per-adapter normalize functions - Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore advertisement (E2B returns true post-reconnect, Cube always true) - Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future" - Bump session sandbox TTL defaults: 604800s → 300s (.env.example) - Update docs/agent-skills.md: full multi-instance deployment guide Total: 29 files, +1188 / -3721 * feat(agent): bind collected skill artifacts to owning message in resource registry When ArtifactCollector persists a skill-generated file it now registers a resource:// binding (owner_type=message, owner_id=assistantMessageID, relation=artifact) in addition to the messages.artifacts JSONB record. This mirrors how knowledge uploads and chat attachments bind their resources, so artifacts can be enumerated and garbage-collected through the resource registry instead of only via the message JSONB column. - ArtifactCollector: inject ResourceCatalog, add messageID to Collect/ maybePersist, and add best-effort bindArtifactResource (binding failures are logged, never drop the artifact; only resource:// refs are bound). - agent_stream_handler: pass assistantMessageID into Collect. - Renumber message_artifacts migration 000066 -> 000077 to resolve the version collision with existing migrations (upstream now uses 000075/000076). - Tests: update collector signatures and add coverage for the binding path and for graceful degradation when Bind fails. Note: binding applies to newly generated artifacts only; existing rows are not backfilled. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sandbox): harden remote sandbox security and artifact collection Disable Cube local fallback so skill scripts never run on the host when the remote provider is unavailable. Require Redis for cube/e2b binding unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev. Collect artifacts with WithoutCancel so user stop does not drop files, reject shell_exec work_dir outside /workspace, and refuse session-scoped Execute when a local fallback would be engaged. * fix(ci): gofmt sandbox files and resolve artifact drawer type error Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer props in AgentStreamDisplay so session-id is always a string. * fix(sandbox): adapt to upstream main after rebase Renumber the message_artifacts migration 000077 -> 000079: upstream now uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata. Declare empty model-handle policies for shell_exec / list_sandbox_files / read_sandbox_file. Upstream added a coverage test requiring every UI-exposed tool to declare one; sandbox output is arbitrary skill-authored content, so it gets the same treatment as execute_skill_script. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant. * feat(sandbox): add sandbox config entity, cordon lease and session pin field * feat(sandbox): add sandbox config repository with tenant scoping and cordon lease * feat(sandbox): stamp sandbox config ID onto sandbox metadata * feat(sandbox): scope sandbox listing, release and reaping to a single config * feat(sandbox): resolve managers per config and refuse cordoned or missing configs * feat(sandbox): implement session pinning and config identity change detection - Add SessionSandboxPinner with read/pin/clear to bind session to config - Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config - Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL) - Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts - Fix artifact collection for default backend and destroy for legacy empty-pin sessions * feat(sandbox): add multi-tenant sandbox config management and HTTP APIs Service layer: - Add sandbox config business service supporting CRUD and inventory listing - For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions - For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side, and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting - Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config - Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation HTTP interfaces and routing: - Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory) - Mask sensitive credentials (API key, etc.) in all read responses - Return 409 with structured inventory details on rejection scenarios - sandbox-check probing supports specifying an existing config_id directly - Remove old handler file, consolidate validation logic into service layer - DI wiring and route registration for all six endpoints * feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config - Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults - Validate config existence on create/update; return 400 if not found - Inject sandboxConfigService into CustomAgentHandler - Add unit tests covering field serialization and validation logic * feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default. * feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch - Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios. - Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned. - Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace. * feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities - Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked - Add named backend type validation, only cube/e2b can be stored as workspace named config - Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default - Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents - Adapt artifact collector and other callers to fill missing resolve signature parameter * fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube Session teardown & deletion ordering - Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins. - Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions. ## Config update edge cases - When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409. - UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results. Artifact collection for Cube skills - Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`. HTTP pool sharing for named Cube configs - Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request. - Add `CubeTransportPool` and `cubeSplitTransport`: - Control plane traffic reuses the same `SharedTransport` as E2B. - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header. - Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes. * feat(sandbox): sandbox execution DefaultUser change from root to user * fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults - Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars instead of DefaultConfig() hardcoded values, allowing empty fields to inherit from .env rather than Go constants - Centralize deployment baseline parsing in sandbox/defaults.go with DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv - Delegate container startup to shared helpers to avoid drift - Propagate tenant env_vars through ResolveEffectiveConfig into buildSessionCreateRequest for both Cube and E2B session sandboxes * fix(multi-sandbox-config-per-tenant): fix some bugs frontend. * fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant Three defects found while reviewing the remote sandbox work: - The Cube connectivity probe and the deployment-default Cube/E2B managers built clients without a guarded transport, so an admin-supplied api_url or proxy_url could reach link-local addresses via DNS rebinding even though save-time validation passed. All three now dial through the same outbound guard the per-tenant resolver enforces. - Attachment staging was gated on the process-wide manager, so a workspace pointing an agent at a named Cube/E2B config while the deployment default stayed docker/local would run its tools remotely against an empty /workspace/input. The gate now resolves the manager the session actually runs on. - Session to sandbox bindings were keyed by the tenant in context, which a shared agent deliberately swaps for the agent owner's so models, KBs and named configs resolve in the sharing workspace. Session deletion only ever knows the session's own tenant, so teardown looked up a key nobody wrote and abandoned a paused MicroVM that keeps billing. The binding tenant is now carried separately and survives CloneContext. * chore(sandbox): renumber migrations above upstream and de-collide test stub Upstream main took 000079 (knowledge_folder_path) and 000080 (knowledge_base_auto_tag_config) while this branch was open, so the three sandbox migrations move to 000081-000083. golang-migrate requires unique, ordered versions, and the RAISE NOTICE labels move with the file names. Main also added a handler test declaring stubSessionService in the same package; the artifact download test's configurable variant is renamed rather than merged, since the two stubs answer GetSession differently. --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 13:28:20 +00:00
shellExecTestContext(),
json.RawMessage(`{"command":"echo hi"}`),
)
require.NoError(t, err)
require.False(t, result.Success)
assert.LessOrEqual(t, len(result.Error), maxShellExecErrorBytes)
assert.Contains(t, result.Error, "truncated")
}
func TestTruncateShellStreamIncludesMarkerWithinLimit(t *testing.T) {
output, truncated := truncateShellStream(strings.Repeat("x", 10000), 100)
require.True(t, truncated)
assert.LessOrEqual(t, len(output), 100)
assert.Contains(t, output, "truncated")
}
type recordedCapture struct {
skillName string
pairs map[string]string
calls int
}
func (r *recordedCapture) capture(_ context.Context, skillName string, pairs map[string]string) {
r.calls++
r.skillName = skillName
r.pairs = pairs
}
// stubEnvResolver stands in for the service-layer resolver: resolved is what
// the workspace or this caller already has stored, missing is what a required
// declaration still needs.
type stubEnvResolver struct {
err error
resolved map[string]string
missing []string
}
func (r stubEnvResolver) ResolveEnv(
_ context.Context, _ string,
) (map[string]string, []string, error) {
return r.resolved, r.missing, r.err
}
func TestShellExecCapturesUsedEnvAfterSuccessfulCommand(t *testing.T) {
recorder := &recordedCapture{}
tool := NewShellExecTool(&fakeShellExecutor{}, nil).WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"export USER_TOKEN=from-command; python x.py","skill_name":"pdf-tools","env":{"EXTRA_TOKEN":"from-tool"}}`,
))
require.NoError(t, err)
require.True(t, result.Success)
require.Equal(t, 1, recorder.calls)
require.Equal(t, "pdf-tools", recorder.skillName)
require.Equal(t, "from-command", recorder.pairs["USER_TOKEN"])
require.Equal(t, "from-tool", recorder.pairs["EXTRA_TOKEN"])
}
// Guessing the skill from a path would let any successful command that merely
// mentions a skill directory write into that skill's credentials.
func TestShellExecDoesNotCaptureWithoutAnExplicitSkillName(t *testing.T) {
recorder := &recordedCapture{}
tool := NewShellExecTool(&fakeShellExecutor{}, nil).WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"export USER_TOKEN=from-command; cd /opt/weknora/tenant/skills/pdf-tools && python x.py"}`,
))
require.NoError(t, err)
require.True(t, result.Success)
require.Zero(t, recorder.calls)
}
// A stored value belongs to whoever entered it. Capture may fill a blank, so a
// name that resolved is never handed to the write path.
func TestShellExecDoesNotCaptureAlreadyResolvedNames(t *testing.T) {
recorder := &recordedCapture{}
resolver := stubEnvResolver{resolved: map[string]string{"USER_TOKEN": "stored"}}
tool := NewShellExecTool(&fakeShellExecutor{}, resolver).WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"python x.py","skill_name":"pdf-tools","env":{"USER_TOKEN":"model-made-this-up","NEW_TOKEN":"fresh"}}`,
))
require.NoError(t, err)
require.True(t, result.Success)
require.Equal(t, 1, recorder.calls)
require.NotContains(t, recorder.pairs, "USER_TOKEN")
require.Equal(t, "fresh", recorder.pairs["NEW_TOKEN"])
}
// Refusing a required variable the call itself carries would strand every user
// without a settings page, since the value can only arrive through the chat.
func TestShellExecRunsWhenTheCallSuppliesTheMissingRequiredValue(t *testing.T) {
recorder := &recordedCapture{}
resolver := stubEnvResolver{missing: []string{"USER_TOKEN"}}
tool := NewShellExecTool(&fakeShellExecutor{}, resolver).WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"python x.py","skill_name":"pdf-tools","env":{"USER_TOKEN":"typed-in-chat"}}`,
))
require.NoError(t, err)
require.True(t, result.Success)
require.Equal(t, "typed-in-chat", recorder.pairs["USER_TOKEN"],
"the value that made the run possible is the one worth storing")
}
func TestShellExecStillRefusesAMissingRequiredValueNobodySupplied(t *testing.T) {
resolver := stubEnvResolver{missing: []string{"USER_TOKEN"}}
tool := NewShellExecTool(&fakeShellExecutor{}, resolver).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"python x.py","skill_name":"pdf-tools"}`,
))
require.NoError(t, err)
require.False(t, result.Success)
require.Contains(t, result.Error, "USER_TOKEN")
}
func TestShellExecDoesNotCaptureWhenCommandFails(t *testing.T) {
recorder := &recordedCapture{}
tool := NewShellExecTool(&fakeShellExecutor{result: &sandbox.ExecuteResult{ExitCode: 1}}, nil).
WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"export USER_TOKEN=x; false","skill_name":"pdf-tools"}`,
))
require.NoError(t, err)
require.True(t, result.Success, "non-zero exit is still a successful tool call")
require.Zero(t, recorder.calls)
}
func TestShellExecDoesNotCaptureWhenExecutorErrors(t *testing.T) {
recorder := &recordedCapture{}
tool := NewShellExecTool(&fakeShellExecutor{err: errors.New("sandbox down")}, nil).
WithEnvCapture(recorder.capture).WithSkillEnvironment(shellTestSkillEnvironment(t))
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"export USER_TOKEN=x; true","skill_name":"pdf-tools"}`,
))
require.NoError(t, err)
require.False(t, result.Success)
require.Zero(t, recorder.calls)
}
func TestInstallShellExecDoesNotCapture(t *testing.T) {
recorder := &recordedCapture{}
tool := NewInstallShellExecTool(&fakeInstallShellExecutor{}, installShellSkillDir).WithEnvCapture(recorder.capture)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"export USER_TOKEN=x; pip install x"}`,
))
require.NoError(t, err)
require.True(t, result.Success)
require.Zero(t, recorder.calls)
}
func TestShellExecOversizeCommandPointsAtWriteSandboxFile(t *testing.T) {
command := strings.Repeat("a", shellExecMaxCommandBytes+1)
raw, err := json.Marshal(map[string]string{"command": command})
require.NoError(t, err)
result, err := NewShellExecTool(&fakeShellExecutor{}, nil).Execute(
shellExecTestContext(), raw,
)
require.NoError(t, err)
require.False(t, result.Success)
assert.Contains(t, result.Error, "command too long")
assert.Contains(t, result.Error, "write_sandbox_file")
}
func TestShellExecHintsWhenTreeCommandIsMissing(t *testing.T) {
tool := NewShellExecTool(&fakeShellExecutor{result: &sandbox.ExecuteResult{
ExitCode: 127,
Stderr: "/bin/bash: line 1: tree: command not found\n",
}}, nil)
result, err := tool.Execute(
shellExecTestContext(),
json.RawMessage(`{"command":"tree -L 2 /workspace/output"}`),
)
require.NoError(t, err)
require.True(t, result.Success)
assert.Equal(t, 127, result.Data["exit_code"])
assert.Contains(t, result.Output, "not in the default sandbox image")
assert.Contains(t, result.Output, "Use find/ls")
assert.Contains(t, result.Output, "Do not apt-get install")
}
func TestInferredMissingCommandFromBashStderr(t *testing.T) {
assert.Equal(t, "file", inferredMissingCommand(
"file /tmp/x",
"/bin/bash: line 1: file: command not found\n",
))
assert.Equal(t, "tree", inferredMissingCommand("tree -L 2", "tree: command not found"))
}
func TestShellExecHintsWhenSystemPythonMissesASkillModule(t *testing.T) {
script := `cd /workspace/output && python3 -c "
from docx import Document
doc = Document('brief.docx')
print(len(doc.paragraphs))
"`
tool := NewShellExecTool(&fakeShellExecutor{result: &sandbox.ExecuteResult{
ExitCode: 1,
Stderr: "ModuleNotFoundError: No module named 'docx'\n",
}}, nil)
raw, err := json.Marshal(map[string]string{"command": script})
require.NoError(t, err)
result, err := tool.Execute(shellExecTestContext(), raw)
require.NoError(t, err)
require.True(t, result.Success)
assert.Contains(t, result.Output, "Do not pip install")
assert.Contains(t, result.Output, "write_sandbox_file")
assert.Contains(t, result.Output, "shell_exec")
assert.Contains(t, result.Output, ".venv/bin/python -c")
}
func TestSkillNameFromShellCommandExtractsImageSkill(t *testing.T) {
assert.Equal(t, "meeting-and-brief", skillNameFromShellCommand(
`/opt/weknora/tenant/skills/meeting-and-brief/.venv/bin/python -c "print(1)"`,
))
assert.Empty(t, skillNameFromShellCommand(`python3 -c "print(1)"`))
}
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
func TestShellExecAllowsAnInstallThatMentionsTheSkillTree(t *testing.T) {
// Previously an up-front command blacklist rejected this recovery path
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
// because the line contained both `pip install` and the skills root. It is
// now the recommended one: a missing package goes into the skill's own venv.
executor := &fakeShellExecutor{result: &sandbox.ExecuteResult{ExitCode: 0}}
tool := NewShellExecTool(executor, nil)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
`{"command":"/opt/weknora/tenant/skills/foo/.venv/bin/python -m pip install -r `+
`/opt/weknora/tenant/skills/foo/requirements.txt"}`,
))
require.NoError(t, err)
require.True(t, result.Success, result.Error)
assert.Equal(t, 1, executor.calls)
}
func TestInstallShellExecStillPipsIntoTheSkillTree(t *testing.T) {
inner := &fakeInstallShellExecutor{}
tool := NewInstallShellExecTool(inner, installShellSkillDir)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"pip install python-docx","work_dir":"/opt/weknora/tenant/skills/律师助手"}`,
))
require.NoError(t, err)
require.True(t, result.Success, result.Error)
require.Equal(t, 1, inner.calls)
}
func TestShellExecHintsWhenVenvHasNoPip(t *testing.T) {
tool := NewShellExecTool(&fakeShellExecutor{result: &sandbox.ExecuteResult{
ExitCode: 1,
Stderr: "/opt/weknora/tenant/skills/律师助手/.venv/bin/python: No module named pip\n",
}}, nil)
result, err := tool.Execute(shellExecTestContext(), json.RawMessage(
`{"command":"/opt/weknora/tenant/skills/律师助手/.venv/bin/python /opt/weknora/tenant/skills/律师助手/scripts/install_deps.py --word --yes"}`,
))
require.NoError(t, err)
require.True(t, result.Success)
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
assert.Contains(t, result.Output, skillPythonPackageInstallCommand)
assert.NotContains(t, result.Output, "/workspace/.skill-packages")
assert.NotContains(t, result.Output, "write_sandbox_file")
}
func shellTestSkillEnvironment(t *testing.T) *skills.Manager {
t.Helper()
manager := skills.NewManager(&skills.ManagerConfig{Enabled: true}, nil)
manager.WithTenantSource(skills.NewTenantSkillSource([]*types.TenantSkillEntity{
{Name: "pdf-tools", Status: types.SkillStatusReady, Enabled: true},
}, nil))
require.NoError(t, manager.Initialize(context.Background()))
return manager
}
func TestShellExecNeverSilentlyFallsBackFromNamedSkill(t *testing.T) {
executor := &fakeShellExecutor{}
result, err := NewShellExecTool(executor, nil).Execute(shellExecTestContext(), json.RawMessage(`{"command":"python3 report.py","skill_name":"missing"}`))
require.NoError(t, err)
require.False(t, result.Success)
require.Zero(t, executor.calls)
require.Contains(t, result.Error, "no skill environment")
}
feat(sandbox): run sandboxed execs as root by default (#3067) * feat(sandbox): run sandboxed execs as root by default Every chat session owns its own sandbox, so the in-container account is not a tenant boundary and root is the least surprising default for an agent that installs packages and writes wherever it needs. Host and cross-tenant isolation stay at the container boundary. DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox image ends with USER root. The three backends keep naming the account explicitly on every exec so the account a call lands on does not depend on which backend a workspace selected; the adapter default, the file operations and the artifact-directory bootstrap all follow the constant. The template still provisions a uid 1000 "user" account for interop with E2B/Cube tooling that addresses it by name. Root also makes the skill-install flow's account hand-over pointless, so it goes: verification no longer runs after a chown/chmod lock-down and a repair round no longer reopens the tree (normalizeSkillPermissions and reopenSkillDirForRepair are gone), and the prepare and scratch commands stop chowning. The scratch wipe now also clears /tmp, and the cache budget guard sizes only root's caches, since no second account can grow one. Verification is asserted to stay on the ordinary executor (no AsRoot, no AllowSkillsRoot) rather than on which account it ran as. * refactor(skills): install chat-time extras into the skill's own env PYTHONPATH / NODE_PATH pointed at a per-session overlay (/workspace/.skill-packages/<skill>) because the skill tree was frozen after install: root-owned, mode 555, and often built by `uv venv` with no pip, so `pip install --target` somewhere else was the only way a chat session could add a package. None of that holds now. The lock-down is gone and execs run as root, so a missing package goes straight into the skill's own environment, and the write is already session-local: a chat sandbox is built from the config's image and its filesystem dies with the session, so nothing reaches the image other sessions start from. The overlay bought nothing over that while splitting one skill's packages across two locations. SessionSkillPackageDir goes, and applySessionPackagePath becomes applySkillNodePath: it appends the skill's own node_modules after whatever the caller supplied, replacing a `+=` that prepended a bare separator once the overlay stopped seeding the variable. No PYTHONPATH is injected at all. Python dependencies are reached through the skill's venv interpreter, which the shell wrapper already puts first on PATH; the overlay-era value named the venv root, which is not an importable path. PYTHONPATH stays on the injected-name blacklist so a stored value cannot shadow the venv. The runtime hints and every prompt that described the overlay now name `<skillDir>/.venv/bin/python -m pip install` and `npm install` in the skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are renamed for what they detect now that nothing is frozen. * docs(prompts): reserve /workspace/output for finished deliverables /workspace/output is the one directory an artifact collector reads, so everything written there is offered to the user as a deliverable. The prompts described it as the place for "deliverables" or "downloadable artifacts" without saying it is exclusively that, which reads as "the directory you write results to" — and a model doing multi-step work has results at every step. Drafts, intermediate renders and debug dumps land next to the finished file and the user is handed all of them. The four places that describe the workspace now say the same thing: the collector reads only /workspace/output, so finished deliverables go there and everything in progress goes anywhere else under /workspace. The attachment block gains the matching half for the other direction — /workspace/input holds the user's own files and is not ours to write to, which it previously implied by calling the paths "read-only inputs" while the tools that enforce it say so plainly. Only the /workspace/input half is enforced (write_sandbox_file refuses that tree). Whether a file is finished is a judgement no tool can make, so the output half stays guidance and is stated wherever the model is already reading about the workspace rather than in one place it might not see. * fix(skills): restore executable bundles and package recovery * style(sandbox): wrap prompt text for repository lint checks * fix(sandbox): guard recovery paths and document root execution --------- Co-authored-by: hailongzhao <hailongzhao@tencent.com> Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 05:03:35 +00:00
func TestShellExecPackageRecoveryKeepsWorkspaceCWD(t *testing.T) {
for _, command := range []string{skillNodePackageInstallCommand, skillPythonPackageInstallCommand} {
executor := &fakeShellExecutor{}
tool := NewShellExecTool(executor, nil).WithSkillEnvironment(shellTestSkillEnvironment(t))
args, err := json.Marshal(ShellExecInput{
SkillName: "pdf-tools", Command: strings.ReplaceAll(command, "<package>", "test-package"),
})
require.NoError(t, err)
result, err := tool.Execute(shellExecTestContext(), args)
require.NoError(t, err)
require.True(t, result.Success, "%+v", result)
require.Equal(t, 1, executor.calls)
require.Equal(t, "/workspace", executor.workDir)
require.Equal(t, sandbox.SkillsImageRoot+"/pdf-tools", executor.env["WEKNORA_SKILL_DIR"])
require.Contains(t, executor.command, "${WEKNORA_SKILL_DIR:?}")
}
}
func TestShellExecPackageRecoveryUsesNamedEnvironment(t *testing.T) {
tool := NewShellExecTool(&fakeShellExecutor{}, nil).WithSkillEnvironment(shellTestSkillEnvironment(t))
for _, name := range []string{"pdf-tools", "host-skill"} {
for _, stderr := range []string{
"No module named pip", "ModuleNotFoundError: No module named 'docx'", "Cannot find module 'example'",
} {
hint := tool.recoveryHint(name, 1, "python3 script.py", stderr)
require.Contains(t, hint, `skill_name="`+name+`"`)
require.Contains(t, hint, skillPythonPackageInstallCommand)
require.Contains(t, hint, skillPythonVenvCreateCommand)
require.Contains(t, hint, skillNodePackageInstallCommand)
require.NotContains(t, hint, sandbox.SkillsImageRoot)
require.NotContains(t, hint, "work_dir=", "package installation must keep the default workspace CWD")
}
}
}
func TestShellExecVenvAccessFailuresPrecedeGenericPermissionHints(t *testing.T) {
for _, withEnvironment := range []bool{false, true} {
tool := NewShellExecTool(&fakeShellExecutor{}, nil)
if withEnvironment {
tool.WithSkillEnvironment(shellTestSkillEnvironment(t))
}
for _, message := range []string{"Permission denied", "Operation not permitted", "EPERM"} {
hint := tool.recoveryHint("pdf-tools", 1, "python3 script.py",
message+": /skill/.venv/lib/site-packages")
require.Contains(t, hint, "skill virtualenv denied access")
require.Contains(t, hint, "Once the environment is writable")
require.Contains(t, hint, skillPythonPackageInstallCommand)
}
for _, message := range []string{"Read-only file system", "Read-only filesystem", "EROFS"} {
hint := tool.recoveryHint("pdf-tools", 1, "python3 script.py",
message+": /skill/.venv/lib/site-packages")
require.Contains(t, hint, "skill virtualenv is on a read-only filesystem")
require.Contains(t, hint, "cannot write through a read-only mount")
require.NotContains(t, hint, "pip install")
unrelated := tool.recoveryHint("pdf-tools", 1, "touch report.txt",
message+": /workspace/output/report.txt")
require.NotContains(t, unrelated, "skill virtualenv")
require.NotContains(t, unrelated, skillPythonPackageInstallCommand)
}
}
}