feat(sandbox): 双模式沙箱分配的基础(门控关闭,线上行为不变) (#4507)
* refactor(sandbox): move workspace layout and path rules into a leaf package
internal/sandbox/workspace owns the remote layout (root, input, output,
GitDir, skills root) and the lexical path rules. The sandbox root package
re-exports them through aliases, so callers keep their imports. No
behaviour change.
* feat(sandbox): add the allocation domain model
Pure types for placing a session: session/user mode, the user-mode
admission whitelist (web account or API external user only, with a fixed
fallback reason otherwise), the shared-machine identity and its digest,
the empty-retention bounds and the deployment gate.
* feat(sandbox): add machine and session admission leases
Registry of in-flight operations per shared machine and per session, with
Redis and in-memory implementations. Leases renew only while still live,
report loss at a local deadline measured from the request send time, and
Bind cancels a context when any held lease or barrier is lost. Holding or
renewing only ever extends the scope key's lifetime.
* feat(sandbox): add allocation_mode and empty retention to sandbox configs
Configs gain allocation_mode (session|user, default session) and
user_sandbox_empty_retention_days (0 = default 7, up to 90). Updates that
omit them keep the stored values. Switching a config to user requires the
deployment gate (WEKNORA_SANDBOX_USER_ALLOCATION plus Redis), exposed as
the settings.sandbox.user_allocation capability.
The gate stays closed in this build regardless of the env var: the session
manager does not place sessions by their assignment yet, so a user-mode
placement would be recorded for a session that still runs on its own
machine.
* feat(session): record the server-verified owner auth source
sessions.owner_auth_source records how user_id was authenticated when the
session was created or its owner first bound: web, API direct header or
signed token, API key, MCP endpoint, embed, IM, internal. The auth
middleware puts the source on the context, CreateSession stamps it and
ignores client input, owner binding (MCP ask, embed backfill) records the
entry's own source, and a fork inherits it. Rows from before the column
stay empty, which means unknown.
Per-user sandbox allocation reads it to tell a verified person from a
shared entry; the owner's kind follows from the source and user_id, so it
is not stored separately.
* feat(sandbox): persist session placements and shared-machine registry
session_sandbox_assignments records each session's placement once: its
config pair, the shared machine digest ('' for a machine of its own),
when it was released and when its directory cleanup is due. Mode and
layout follow from the digest. sandbox_user_machines holds the identity
behind each digest and the empty-retention clock.
ClaimPlacement writes the assignment and projects the session pin in one
transaction that locks the session row, then the machine row, before
inserting. RetireSessions releases assignments under the same locks and
writes a tombstone for a session that was never placed, so no placement
lands after a deletion began. The repository is also exercised on the
schema the shipped SQLite migrations create.
* feat(sandbox): place sessions through the allocation service
sandboxalloc decides a session's placement on first remote use and records
it with the pin: sessions that already used a per-session sandbox stay on
it, maintenance sessions are always dedicated, and only an eligible owner
on a user-mode config with the gate open shares a machine. A placed
session resolves straight from its assignment; a released one is refused.
Skill maintenance sessions record a maintenance placement and are removed
if that fails.
* fix(sandbox): retry placement when the sandbox config cannot be read
A failed config load was stored as session mode, so the session could never join a shared machine after the error cleared.
* fix(sandbox): satisfy golangci-lint on the allocation foundations
Wrap lines past 120 characters, comment exported names, and clear the two staticcheck findings so the pull request lint job can pass.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
main
parent
eecfe4b3c8
commit
8dab7550f5
|
|
@ -101,6 +101,24 @@ test('remote sandbox capability fails closed', () => {
|
|||
)
|
||||
})
|
||||
|
||||
test('user allocation stays hidden unless the deployment explicitly enables it', () => {
|
||||
assert.equal(isDeploymentCapabilitySupported({}, 'settings.sandbox.user_allocation'), false)
|
||||
assert.equal(
|
||||
isDeploymentCapabilitySupported(
|
||||
{ 'settings.sandbox.user_allocation': { supported: false, reason: 'deployment_gate_off' } },
|
||||
'settings.sandbox.user_allocation',
|
||||
),
|
||||
false,
|
||||
)
|
||||
assert.equal(
|
||||
isDeploymentCapabilitySupported(
|
||||
{ 'settings.sandbox.user_allocation': { supported: true } },
|
||||
'settings.sandbox.user_allocation',
|
||||
),
|
||||
true,
|
||||
)
|
||||
})
|
||||
|
||||
test('skill settings need a remote sandbox or the host sandbox', () => {
|
||||
assert.equal(skillSettingsSupported({ 'settings.sandbox': { supported: true }, 'settings.sandbox.remote': { supported: true } }), true)
|
||||
assert.equal(skillSettingsSupported({ 'settings.sandbox': { supported: true }, 'settings.sandbox.host': { supported: true } }), true)
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ export const DEPLOYMENT_CAPABILITY_KEYS = [
|
|||
'settings.sandbox.docker',
|
||||
'settings.sandbox.host',
|
||||
'settings.sandbox.remote',
|
||||
'settings.sandbox.user_allocation',
|
||||
] as const
|
||||
|
||||
export type DeploymentCapabilityKey = typeof DEPLOYMENT_CAPABILITY_KEYS[number]
|
||||
|
|
@ -45,10 +46,13 @@ export function isDeploymentCapabilitySupported(
|
|||
// Host sandbox is Lite-desktop-only; keep the same fail-closed gate so a
|
||||
// missed probe does not show the new-session open-project UI on other deployments.
|
||||
// Remote sandbox is unavailable on Lite desktop; fail closed like docker/host.
|
||||
// User allocation is a deployment gate (env opt-in plus Redis). A missed
|
||||
// probe must not show the switch.
|
||||
if (
|
||||
key === 'settings.sandbox.docker'
|
||||
|| key === 'settings.sandbox.host'
|
||||
|| key === 'settings.sandbox.remote'
|
||||
|| key === 'settings.sandbox.user_allocation'
|
||||
) {
|
||||
return capabilities[key]?.supported === true
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,419 @@
|
|||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
||||
type sandboxAllocationRepository struct{ db *gorm.DB }
|
||||
|
||||
// NewSandboxAllocationRepository builds the allocation repository.
|
||||
func NewSandboxAllocationRepository(db *gorm.DB) interfaces.SandboxAllocationRepository {
|
||||
return &sandboxAllocationRepository{db: db}
|
||||
}
|
||||
|
||||
// forUpdate adds a row lock where the dialect has one. SQLite serializes
|
||||
// writers already.
|
||||
func forUpdate(tx *gorm.DB) *gorm.DB {
|
||||
switch tx.Name() {
|
||||
case "postgres", "mysql":
|
||||
return tx.Clauses(clause.Locking{Strength: "UPDATE"})
|
||||
default:
|
||||
return tx
|
||||
}
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) GetAssignment(
|
||||
ctx context.Context, sessionID string,
|
||||
) (*types.SessionSandboxAssignment, error) {
|
||||
return getAssignment(r.db.WithContext(ctx), sessionID)
|
||||
}
|
||||
|
||||
func getAssignment(tx *gorm.DB, sessionID string) (*types.SessionSandboxAssignment, error) {
|
||||
var a types.SessionSandboxAssignment
|
||||
err := tx.Where("session_id = ?", sessionID).Take(&a).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &a, nil
|
||||
}
|
||||
|
||||
// InsertSandboxAssignmentTx inserts a unless the session already has one, and
|
||||
// for a user-mode row registers (or re-opens) machine. The fork path shares it
|
||||
// so a forked session and its assignment commit together.
|
||||
func InsertSandboxAssignmentTx(
|
||||
tx *gorm.DB, a *types.SessionSandboxAssignment, machine *types.SandboxUserMachine,
|
||||
) (bool, error) {
|
||||
if (a.MachineDigest == "") != (machine == nil) || (machine != nil && machine.MachineDigest != a.MachineDigest) {
|
||||
return false, errors.New("sandbox assignment and its machine disagree")
|
||||
}
|
||||
if machine != nil {
|
||||
// Lock the machine row before the assignment exists. A reclaimer
|
||||
// holding it counts live assignments under READ COMMITTED and cannot
|
||||
// see an uncommitted insert, so inserting first would let it delete a
|
||||
// machine this session is joining. Waiting here means: the reclaimer
|
||||
// finishes and the upsert below registers a fresh machine, or the join
|
||||
// commits first and the reclaimer counts it.
|
||||
if err := lockMachineRow(tx, machine.MachineDigest); err != nil {
|
||||
return false, err
|
||||
}
|
||||
}
|
||||
res := tx.Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "session_id"}}, DoNothing: true,
|
||||
}).Create(a)
|
||||
if res.Error != nil {
|
||||
return false, res.Error
|
||||
}
|
||||
if res.RowsAffected == 0 || machine == nil {
|
||||
return res.RowsAffected > 0, nil
|
||||
}
|
||||
// Joining always clears the empty clock, after the insert, so a release
|
||||
// that counted zero concurrently cannot leave it set.
|
||||
row := *machine
|
||||
row.EmptySince = nil
|
||||
err := tx.Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "machine_digest"}},
|
||||
DoUpdates: clause.Assignments(map[string]any{"empty_since": nil}),
|
||||
}).Create(&row).Error
|
||||
return true, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) ClaimPlacement(
|
||||
ctx context.Context, a *types.SessionSandboxAssignment, machine *types.SandboxUserMachine,
|
||||
expected interfaces.PlacementPin,
|
||||
) (*types.SessionSandboxAssignment, error) {
|
||||
var out *types.SessionSandboxAssignment
|
||||
err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
// Lock the session row: the pin lives on it, and an older replica's
|
||||
// conditional pin update waits on this lock instead of interleaving.
|
||||
var row struct {
|
||||
SandboxConfigID *string
|
||||
SandboxConfigTenantID uint64
|
||||
}
|
||||
res := forUpdate(tx.Model(&types.Session{})).
|
||||
Select("sandbox_config_id", "sandbox_config_tenant_id").
|
||||
Where("id = ?", a.SessionID).Take(&row)
|
||||
if res.Error != nil {
|
||||
return res.Error // gorm.ErrRecordNotFound for a missing or soft-deleted session
|
||||
}
|
||||
current := interfaces.PlacementPin{TenantID: row.SandboxConfigTenantID}
|
||||
if row.SandboxConfigID != nil {
|
||||
current.ConfigID = *row.SandboxConfigID
|
||||
}
|
||||
|
||||
existing, err := getAssignment(tx, a.SessionID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing != nil && existing.ReleasedAt != nil {
|
||||
// Released, or a tombstone: the session is being deleted.
|
||||
return interfaces.ErrSessionRetired
|
||||
}
|
||||
if existing != nil {
|
||||
out = existing
|
||||
return projectPin(tx, existing, current)
|
||||
}
|
||||
if current != expected {
|
||||
return interfaces.ErrPlacementConflict
|
||||
}
|
||||
created, err := InsertSandboxAssignmentTx(tx, a, machine)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !created {
|
||||
return interfaces.ErrPlacementConflict
|
||||
}
|
||||
out = a
|
||||
return projectPin(tx, a, current)
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// projectPin writes the assignment's config pair onto an unpinned session.
|
||||
// A pin naming another config — including the same id in another tenant — is
|
||||
// a conflict, never overwritten: the per-session machine it names may still
|
||||
// exist. A legacy pin that has the id but not yet its tenant is completed.
|
||||
func projectPin(tx *gorm.DB, a *types.SessionSandboxAssignment, current interfaces.PlacementPin) error {
|
||||
want := interfaces.PlacementPin{ConfigID: a.ConfigID, TenantID: a.ConfigTenantID}
|
||||
switch {
|
||||
case current == want:
|
||||
return nil
|
||||
case current.ConfigID == "", current.ConfigID == want.ConfigID && current.TenantID == 0:
|
||||
return tx.Model(&types.Session{}).Where("id = ?", a.SessionID).Updates(map[string]any{
|
||||
"sandbox_config_id": want.ConfigID, "sandbox_config_tenant_id": want.TenantID,
|
||||
}).Error
|
||||
default:
|
||||
return interfaces.ErrPlacementConflict
|
||||
}
|
||||
}
|
||||
|
||||
func lockMachineRow(tx *gorm.DB, digest string) error {
|
||||
var row types.SandboxUserMachine
|
||||
err := forUpdate(tx.Model(&types.SandboxUserMachine{})).
|
||||
Select("machine_digest").Where("machine_digest = ?", digest).Take(&row).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func countLive(tx *gorm.DB, digest string) (int64, error) {
|
||||
var n int64
|
||||
err := tx.Model(&types.SessionSandboxAssignment{}).
|
||||
Where("machine_digest = ? AND released_at IS NULL", digest).Count(&n).Error
|
||||
return n, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) RetireSessions(
|
||||
ctx context.Context, sessionIDs []string, at time.Time,
|
||||
) ([]interfaces.RetiredSession, error) {
|
||||
var out []interfaces.RetiredSession
|
||||
err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var err error
|
||||
out, err = RetireSessionsTx(tx, sessionIDs, at)
|
||||
return err
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
|
||||
// RetireSessionsTx is RetireSessions inside the caller's transaction (T14
|
||||
// retires a deleted account's sessions with it). Locks are taken in a fixed
|
||||
// order — every session row by id, then every machine row by digest — which
|
||||
// is the order ClaimPlacement uses for one session, so the two cannot
|
||||
// deadlock.
|
||||
func RetireSessionsTx(tx *gorm.DB, sessionIDs []string, at time.Time) ([]interfaces.RetiredSession, error) {
|
||||
at = at.UTC().Truncate(time.Microsecond)
|
||||
seen := make(map[string]bool, len(sessionIDs))
|
||||
ordered := make([]string, 0, len(sessionIDs))
|
||||
for _, id := range sessionIDs {
|
||||
if id != "" && !seen[id] {
|
||||
seen[id] = true
|
||||
ordered = append(ordered, id)
|
||||
}
|
||||
}
|
||||
locked := append([]string(nil), ordered...)
|
||||
sort.Strings(locked)
|
||||
exists := make(map[string]bool, len(locked))
|
||||
for _, id := range locked {
|
||||
// Unscoped: a session already soft-deleted by an older replica must
|
||||
// still be retired and its assignment released.
|
||||
var ids []string
|
||||
if err := forUpdate(tx.Unscoped().Model(&types.Session{})).
|
||||
Where("id = ?", id).Pluck("id", &ids).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
exists[id] = len(ids) > 0
|
||||
}
|
||||
|
||||
byID := make(map[string]*types.SessionSandboxAssignment, len(ordered))
|
||||
var digests []string
|
||||
for _, id := range ordered {
|
||||
a, err := getAssignment(tx, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if a == nil {
|
||||
if !exists[id] {
|
||||
continue
|
||||
}
|
||||
// Never placed: a tombstone makes any later placement fail.
|
||||
tomb := &types.SessionSandboxAssignment{SessionID: id, ReleasedAt: &at}
|
||||
if err := tx.Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "session_id"}}, DoNothing: true,
|
||||
}).Create(tomb).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if a.IsTombstone() {
|
||||
continue
|
||||
}
|
||||
byID[id] = a
|
||||
if a.ReleasedAt == nil && a.MachineDigest != "" {
|
||||
digests = append(digests, a.MachineDigest)
|
||||
}
|
||||
}
|
||||
sort.Strings(digests)
|
||||
for i, d := range digests {
|
||||
if i > 0 && digests[i-1] == d {
|
||||
continue
|
||||
}
|
||||
if err := lockMachineRow(tx, d); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
out := make([]interfaces.RetiredSession, 0, len(ordered))
|
||||
released := make([]bool, 0, len(ordered))
|
||||
for _, id := range ordered {
|
||||
res := interfaces.RetiredSession{SessionID: id, Assignment: byID[id]}
|
||||
a := byID[id]
|
||||
if a == nil || a.ReleasedAt != nil {
|
||||
out = append(out, res)
|
||||
released = append(released, false)
|
||||
continue
|
||||
}
|
||||
updates := map[string]any{"released_at": at}
|
||||
if a.MachineDigest != "" {
|
||||
// The directory on the shared machine goes with the release.
|
||||
updates["cleanup_due_at"] = at
|
||||
a.CleanupDueAt = &at
|
||||
}
|
||||
if err := tx.Model(&types.SessionSandboxAssignment{}).
|
||||
Where("session_id = ? AND released_at IS NULL", id).
|
||||
Updates(updates).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.ReleasedAt = &at
|
||||
out = append(out, res)
|
||||
released = append(released, true)
|
||||
}
|
||||
// Empty clocks last, once every release of this batch is visible.
|
||||
for i := range out {
|
||||
a := out[i].Assignment
|
||||
if !released[i] || a.MachineDigest == "" {
|
||||
continue
|
||||
}
|
||||
live, err := countLive(tx, a.MachineDigest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if live > 0 {
|
||||
continue
|
||||
}
|
||||
out[i].MachineEmpty = true
|
||||
if err := tx.Model(&types.SandboxUserMachine{}).
|
||||
Where("machine_digest = ? AND empty_since IS NULL", a.MachineDigest).
|
||||
Update("empty_since", at).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) HasSandboxCheckpoint(ctx context.Context, sessionID string) (bool, error) {
|
||||
var ids []string
|
||||
err := r.db.WithContext(ctx).Model(&types.Message{}).
|
||||
Where("session_id = ? AND sandbox_checkpoint IS NOT NULL", sessionID).
|
||||
Limit(1).Pluck("id", &ids).Error
|
||||
return len(ids) > 0, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) ListUserMachinesByConfig(
|
||||
ctx context.Context, configTenantID uint64, configID string,
|
||||
) ([]*types.SandboxUserMachine, error) {
|
||||
var rows []*types.SandboxUserMachine
|
||||
err := r.db.WithContext(ctx).
|
||||
Where("config_tenant_id = ? AND config_id = ?", configTenantID, configID).
|
||||
Order("machine_digest").Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) ListUserMachinesByOwner(
|
||||
ctx context.Context, ownerTenantID uint64, ownerID string,
|
||||
) ([]*types.SandboxUserMachine, error) {
|
||||
q := r.db.WithContext(ctx).Where("owner_id = ?", ownerID)
|
||||
if ownerTenantID != 0 {
|
||||
q = q.Where("owner_tenant_id = ?", ownerTenantID)
|
||||
}
|
||||
var rows []*types.SandboxUserMachine
|
||||
return rows, q.Order("machine_digest").Find(&rows).Error
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) ListEmptyUserMachines(
|
||||
ctx context.Context, emptyBefore time.Time, after interfaces.EmptyMachineCursor, limit int,
|
||||
) ([]*types.SandboxUserMachine, error) {
|
||||
q := r.db.WithContext(ctx).Where("empty_since IS NOT NULL AND empty_since <= ?", emptyBefore)
|
||||
if !after.EmptySince.IsZero() {
|
||||
q = q.Where("(empty_since > ? OR (empty_since = ? AND machine_digest > ?))",
|
||||
after.EmptySince, after.EmptySince, after.Digest)
|
||||
}
|
||||
var rows []*types.SandboxUserMachine
|
||||
err := q.Order("empty_since").Order("machine_digest").Limit(limit).Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) GetUserMachine(
|
||||
ctx context.Context, digest string,
|
||||
) (*types.SandboxUserMachine, error) {
|
||||
var m types.SandboxUserMachine
|
||||
err := r.db.WithContext(ctx).Where("machine_digest = ?", digest).Take(&m).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &m, nil
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) CountLiveAssignments(ctx context.Context, digest string) (int64, error) {
|
||||
return countLive(r.db.WithContext(ctx), digest)
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) DeleteUserMachineIfEmptySince(
|
||||
ctx context.Context, digest string, emptySince time.Time,
|
||||
) (bool, error) {
|
||||
var deleted bool
|
||||
err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := lockMachineRow(tx, digest); err != nil {
|
||||
return err
|
||||
}
|
||||
live, err := countLive(tx, digest)
|
||||
if err != nil || live > 0 {
|
||||
return err
|
||||
}
|
||||
res := tx.Where("machine_digest = ? AND empty_since = ?", digest, emptySince).
|
||||
Delete(&types.SandboxUserMachine{})
|
||||
deleted = res.RowsAffected > 0
|
||||
return res.Error
|
||||
})
|
||||
return deleted, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) DeleteUserMachinesByConfig(
|
||||
ctx context.Context, configTenantID uint64, configID string,
|
||||
) error {
|
||||
return r.db.WithContext(ctx).
|
||||
Where("config_tenant_id = ? AND config_id = ?", configTenantID, configID).
|
||||
Delete(&types.SandboxUserMachine{}).Error
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) ListDueWorkspaceCleanups(
|
||||
ctx context.Context, now time.Time, limit int,
|
||||
) ([]*types.SessionSandboxAssignment, error) {
|
||||
var rows []*types.SessionSandboxAssignment
|
||||
err := r.db.WithContext(ctx).Where("cleanup_due_at IS NOT NULL AND cleanup_due_at <= ?", now).
|
||||
Order("cleanup_due_at").Order("session_id").Limit(limit).Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) CompleteWorkspaceCleanup(ctx context.Context, sessionID string) error {
|
||||
return r.db.WithContext(ctx).Model(&types.SessionSandboxAssignment{}).
|
||||
Where("session_id = ?", sessionID).Update("cleanup_due_at", nil).Error
|
||||
}
|
||||
|
||||
func (r *sandboxAllocationRepository) RetryWorkspaceCleanup(
|
||||
ctx context.Context, sessionID string, next time.Time,
|
||||
) error {
|
||||
// Only a cleanup still pending moves; a completed one stays done.
|
||||
return r.db.WithContext(ctx).Model(&types.SessionSandboxAssignment{}).
|
||||
Where("session_id = ? AND cleanup_due_at IS NOT NULL", sessionID).
|
||||
Update("cleanup_due_at", next.UTC().Truncate(time.Microsecond)).Error
|
||||
}
|
||||
|
||||
var _ interfaces.SandboxAllocationRepository = (*sandboxAllocationRepository)(nil)
|
||||
|
|
@ -0,0 +1,592 @@
|
|||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/database"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
||||
func newAllocationRepo(t *testing.T) (*sandboxAllocationRepository, *gorm.DB) {
|
||||
t.Helper()
|
||||
// SQLite allows one writer. Cap the pool at one connection and wait on a
|
||||
// busy lock so the first-writer test queues instead of failing with
|
||||
// "database table is locked". Real row-lock races run on Postgres in Task 20.
|
||||
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared&_busy_timeout=5000", uuid.NewString())
|
||||
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
sqlDB, err := db.DB()
|
||||
require.NoError(t, err)
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
require.NoError(t, db.AutoMigrate(
|
||||
&types.Session{}, &types.SessionSandboxAssignment{}, &types.SandboxUserMachine{}, &types.Message{},
|
||||
))
|
||||
return NewSandboxAllocationRepository(db).(*sandboxAllocationRepository), db
|
||||
}
|
||||
|
||||
func seedSession(t *testing.T, db *gorm.DB, id string, pin interfaces.PlacementPin) {
|
||||
t.Helper()
|
||||
s := &types.Session{
|
||||
ID: id, TenantID: 7, UserID: "u-1",
|
||||
SandboxConfigID: pin.ConfigID, SandboxConfigTenantID: pin.TenantID,
|
||||
}
|
||||
require.NoError(t, db.Session(&gorm.Session{SkipHooks: true}).Create(s).Error)
|
||||
}
|
||||
|
||||
func userAssignment(sessionID, digest string) *types.SessionSandboxAssignment {
|
||||
return &types.SessionSandboxAssignment{
|
||||
SessionID: sessionID, ConfigTenantID: 7, ConfigID: "cfg", MachineDigest: digest,
|
||||
}
|
||||
}
|
||||
|
||||
func userMachine(digest string) *types.SandboxUserMachine {
|
||||
if digest == "" {
|
||||
return nil
|
||||
}
|
||||
return &types.SandboxUserMachine{
|
||||
MachineDigest: digest, OwnerTenantID: 7, OwnerID: "u-1",
|
||||
ConfigTenantID: 7, ConfigID: "cfg",
|
||||
}
|
||||
}
|
||||
|
||||
// claim places a with the machine its digest names.
|
||||
func claim(ctx context.Context, repo *sandboxAllocationRepository, a *types.SessionSandboxAssignment,
|
||||
pin interfaces.PlacementPin,
|
||||
) (*types.SessionSandboxAssignment, error) {
|
||||
return repo.ClaimPlacement(ctx, a, userMachine(a.MachineDigest), pin)
|
||||
}
|
||||
|
||||
// retireOne retires one session and returns its outcome.
|
||||
func retireOne(ctx context.Context, repo *sandboxAllocationRepository, id string, at time.Time) (
|
||||
*types.SessionSandboxAssignment, bool, error,
|
||||
) {
|
||||
out, err := repo.RetireSessions(ctx, []string{id}, at)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
return out[0].Assignment, out[0].MachineEmpty, nil
|
||||
}
|
||||
|
||||
func readPin(t *testing.T, db *gorm.DB, id string) interfaces.PlacementPin {
|
||||
t.Helper()
|
||||
var s types.Session
|
||||
require.NoError(t, db.Session(&gorm.Session{SkipHooks: true}).Where("id = ?", id).Take(&s).Error)
|
||||
return interfaces.PlacementPin{ConfigID: s.SandboxConfigID, TenantID: s.SandboxConfigTenantID}
|
||||
}
|
||||
|
||||
func TestClaimPlacementWritesAssignmentAndPinTogether(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
|
||||
got, err := claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "d-1", got.MachineDigest)
|
||||
require.Equal(t, interfaces.PlacementPin{ConfigID: "cfg", TenantID: 7}, readPin(t, db, "s-1"))
|
||||
m, err := repo.GetUserMachine(ctx, "d-1")
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, m)
|
||||
}
|
||||
|
||||
func TestClaimPlacementFirstWriterWinsAndPinFollowsTheWinner(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
var wg sync.WaitGroup
|
||||
results := make(chan string, 8)
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
a := userAssignment("s-1", "d-1")
|
||||
if i%2 == 1 {
|
||||
a.MachineDigest, a.ConfigID = "", "cfg-b"
|
||||
}
|
||||
w, err := claim(ctx, repo, a, interfaces.PlacementPin{})
|
||||
if errors.Is(err, interfaces.ErrPlacementConflict) {
|
||||
// The winner already pinned the row; a racer that re-reads adopts it.
|
||||
w, err = claim(ctx, repo, a, readPin(t, db, "s-1"))
|
||||
}
|
||||
require.NoError(t, err)
|
||||
results <- w.ConfigID
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
close(results)
|
||||
winner := ""
|
||||
for c := range results {
|
||||
if winner == "" {
|
||||
winner = c
|
||||
}
|
||||
require.Equal(t, winner, c, "every racer ends on one persisted placement")
|
||||
}
|
||||
require.Equal(t, winner, readPin(t, db, "s-1").ConfigID, "the pin is the winner's projection")
|
||||
}
|
||||
|
||||
func TestClaimPlacementRefusesAStalePinRead(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{ConfigID: "cfg-old", TenantID: 7})
|
||||
_, err := claim(context.Background(), repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, interfaces.ErrPlacementConflict)
|
||||
}
|
||||
|
||||
func TestClaimPlacementOnADeletedSession(t *testing.T) {
|
||||
repo, _ := newAllocationRepo(t)
|
||||
_, err := claim(context.Background(), repo, userAssignment("missing", "d-1"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, gorm.ErrRecordNotFound)
|
||||
}
|
||||
|
||||
func TestClaimPlacementComparesTheWholePin(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{ConfigID: "cfg", TenantID: 9})
|
||||
_, err := claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{ConfigID: "cfg", TenantID: 9})
|
||||
require.ErrorIs(t, err, interfaces.ErrPlacementConflict, "same config id in another tenant is another config")
|
||||
|
||||
seedSession(t, db, "s-2", interfaces.PlacementPin{ConfigID: "cfg"})
|
||||
_, err = claim(ctx, repo, userAssignment("s-2", "d-1"), interfaces.PlacementPin{ConfigID: "cfg"})
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, interfaces.PlacementPin{ConfigID: "cfg", TenantID: 7}, readPin(t, db, "s-2"),
|
||||
"a legacy pin without its tenant is completed from the assignment")
|
||||
}
|
||||
|
||||
func TestRetireWithoutAnAssignmentBlocksALaterPlacement(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
out, err := repo.RetireSessions(ctx, []string{"s-1"}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.Nil(t, out[0].Assignment)
|
||||
tomb, err := repo.GetAssignment(ctx, "s-1")
|
||||
require.NoError(t, err)
|
||||
require.True(t, tomb.IsTombstone(), "an unplaced session is retired with a tombstone")
|
||||
|
||||
again, err := repo.RetireSessions(ctx, []string{"s-1", "never-existed"}, time.Now().UTC())
|
||||
require.NoError(t, err, "retiring is idempotent")
|
||||
require.Nil(t, again[0].Assignment)
|
||||
missing, err := repo.GetAssignment(ctx, "never-existed")
|
||||
require.NoError(t, err)
|
||||
require.Nil(t, missing, "no tombstone for a session that does not exist")
|
||||
|
||||
_, err = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, interfaces.ErrSessionRetired)
|
||||
live, _ := repo.CountLiveAssignments(ctx, "d-1")
|
||||
require.Zero(t, live)
|
||||
}
|
||||
|
||||
func TestRetireAndPlacementInEitherOrderLeaveNothingLive(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
|
||||
placedFirst := uuid.NewString()
|
||||
seedSession(t, db, placedFirst, interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment(placedFirst, "d-race"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
out, err := repo.RetireSessions(ctx, []string{placedFirst}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, out[0].Assignment.CleanupDueAt, "a placement that won is released with its cleanup")
|
||||
|
||||
retiredFirst := uuid.NewString()
|
||||
seedSession(t, db, retiredFirst, interfaces.PlacementPin{})
|
||||
_, err = repo.RetireSessions(ctx, []string{retiredFirst}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
_, err = claim(ctx, repo, userAssignment(retiredFirst, "d-race"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, interfaces.ErrSessionRetired, "a placement that lost never lands")
|
||||
|
||||
live, err := repo.CountLiveAssignments(ctx, "d-race")
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, live)
|
||||
}
|
||||
|
||||
func TestRetireSessionsDedupesAndKeepsOrder(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
for _, id := range []string{"s-2", "s-1"} {
|
||||
seedSession(t, db, id, interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment(id, "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
out, err := repo.RetireSessions(ctx, []string{"s-2", "s-1", "s-2"}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.Len(t, out, 2, "duplicates are retired once")
|
||||
require.Equal(t, "s-2", out[0].SessionID, "results keep the caller's order")
|
||||
require.True(t, out[1].MachineEmpty)
|
||||
}
|
||||
|
||||
func TestListEmptyUserMachinesPagesPastSkippedCandidates(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
base := time.Now().UTC().Add(-48 * time.Hour).Truncate(time.Second)
|
||||
for i, d := range []string{"d-a", "d-b", "d-c"} {
|
||||
id := "s-" + d
|
||||
seedSession(t, db, id, interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment(id, d), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
_, _, err = retireOne(ctx, repo, id, base.Add(time.Duration(i)*time.Minute))
|
||||
require.NoError(t, err)
|
||||
}
|
||||
page1, err := repo.ListEmptyUserMachines(ctx, time.Now().UTC(), interfaces.EmptyMachineCursor{}, 2)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, []string{"d-a", "d-b"}, []string{page1[0].MachineDigest, page1[1].MachineDigest})
|
||||
last := page1[1]
|
||||
page2, err := repo.ListEmptyUserMachines(ctx, time.Now().UTC(),
|
||||
interfaces.EmptyMachineCursor{EmptySince: *last.EmptySince, Digest: last.MachineDigest}, 2)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, page2, 1)
|
||||
require.Equal(t, "d-c", page2[0].MachineDigest)
|
||||
}
|
||||
|
||||
func TestReleaseRecordsCleanupIntentAndEmptiesTheLastSession(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
for _, id := range []string{"s-1", "s-2"} {
|
||||
seedSession(t, db, id, interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment(id, "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
now := time.Now().UTC().Truncate(time.Second)
|
||||
|
||||
a, empty, err := retireOne(ctx, repo, "s-1", now)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, a.ReleasedAt)
|
||||
require.False(t, empty)
|
||||
require.NotNil(t, a.CleanupDueAt, "the cleanup intent is written with the release")
|
||||
|
||||
_, empty, err = retireOne(ctx, repo, "s-2", now)
|
||||
require.NoError(t, err)
|
||||
require.True(t, empty)
|
||||
m, _ := repo.GetUserMachine(ctx, "d-1")
|
||||
require.NotNil(t, m.EmptySince)
|
||||
|
||||
require.NoError(t, repo.CompleteWorkspaceCleanup(ctx, "s-2"))
|
||||
_, _, err = retireOne(ctx, repo, "s-2", now.Add(time.Hour))
|
||||
require.NoError(t, err)
|
||||
row, _ := repo.GetAssignment(ctx, "s-2")
|
||||
require.Nil(t, row.CleanupDueAt, "a second release records nothing new")
|
||||
require.True(t, row.ReleasedAt.Equal(now), "nor moves the release time")
|
||||
require.NoError(t, repo.RetryWorkspaceCleanup(ctx, "s-2", now))
|
||||
row, _ = repo.GetAssignment(ctx, "s-2")
|
||||
require.Nil(t, row.CleanupDueAt, "a completed cleanup is not revived by a late retry")
|
||||
|
||||
due, err := repo.ListDueWorkspaceCleanups(ctx, now.Add(time.Second), 10)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, due, 1)
|
||||
require.Equal(t, "s-1", due[0].SessionID)
|
||||
}
|
||||
|
||||
func TestReleaseOfASessionModeAssignmentHasNoCleanup(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
a := userAssignment("s-1", "")
|
||||
_, err := claim(ctx, repo, a, interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
released, empty, err := retireOne(ctx, repo, "s-1", time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, released.ReleasedAt)
|
||||
require.Nil(t, released.CleanupDueAt)
|
||||
require.False(t, empty)
|
||||
}
|
||||
|
||||
func TestDeleteUserMachineIfEmptySinceComparesTheClock(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
_, _ = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
first := time.Now().UTC().Add(-10 * 24 * time.Hour).Truncate(time.Second)
|
||||
_, _, err := retireOne(ctx, repo, "s-1", first)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Someone joined and left after the reclaimer read the row: a new clock.
|
||||
seedSession(t, db, "s-2", interfaces.PlacementPin{})
|
||||
_, _ = claim(ctx, repo, userAssignment("s-2", "d-1"), interfaces.PlacementPin{})
|
||||
second := time.Now().UTC().Truncate(time.Second)
|
||||
_, _, _ = retireOne(ctx, repo, "s-2", second)
|
||||
|
||||
deleted, err := repo.DeleteUserMachineIfEmptySince(ctx, "d-1", first)
|
||||
require.NoError(t, err)
|
||||
require.False(t, deleted, "the verified clock is gone; the new empty period must run its full window")
|
||||
|
||||
deleted, err = repo.DeleteUserMachineIfEmptySince(ctx, "d-1", second)
|
||||
require.NoError(t, err)
|
||||
require.True(t, deleted)
|
||||
}
|
||||
|
||||
func TestWorkspaceCleanupRetryAndComplete(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
_, _ = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
_, _, err := retireOne(ctx, repo, "s-1", time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
|
||||
require.NoError(t, repo.RetryWorkspaceCleanup(ctx, "s-1", time.Now().UTC().Add(time.Hour)))
|
||||
due, _ := repo.ListDueWorkspaceCleanups(ctx, time.Now().UTC(), 10)
|
||||
require.Empty(t, due)
|
||||
later, _ := repo.ListDueWorkspaceCleanups(ctx, time.Now().UTC().Add(2*time.Hour), 10)
|
||||
require.Len(t, later, 1)
|
||||
require.Equal(t, "d-1", later[0].MachineDigest, "the cleanup still knows its machine")
|
||||
require.NoError(t, repo.CompleteWorkspaceCleanup(ctx, "s-1"))
|
||||
later, _ = repo.ListDueWorkspaceCleanups(ctx, time.Now().UTC().Add(2*time.Hour), 10)
|
||||
require.Empty(t, later)
|
||||
}
|
||||
|
||||
func TestHasSandboxCheckpoint(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
ok, err := repo.HasSandboxCheckpoint(ctx, "s-1")
|
||||
require.NoError(t, err)
|
||||
require.False(t, ok)
|
||||
require.NoError(t, db.Create(&types.Message{
|
||||
ID: uuid.NewString(), SessionID: "s-1", Role: "assistant",
|
||||
SandboxCheckpoint: &types.SandboxCheckpoint{SandboxID: "sb", CommitSHA: "a"},
|
||||
}).Error)
|
||||
ok, err = repo.HasSandboxCheckpoint(ctx, "s-1")
|
||||
require.NoError(t, err)
|
||||
require.True(t, ok)
|
||||
}
|
||||
|
||||
// A session already soft-deleted by an older replica must still be retired and
|
||||
// its live assignment released. ClaimPlacement hides soft-deleted rows, so
|
||||
// retirement is the path that reaches them.
|
||||
func TestRetireSoftDeletedSession(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, db.Where("id = ?", "s-1").Delete(&types.Session{}).Error)
|
||||
|
||||
out, err := repo.RetireSessions(ctx, []string{"s-1"}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, out[0].Assignment)
|
||||
require.NotNil(t, out[0].Assignment.ReleasedAt)
|
||||
require.NotNil(t, out[0].Assignment.CleanupDueAt)
|
||||
|
||||
var row types.Session
|
||||
require.NoError(t, db.Unscoped().Where("id = ?", "s-1").Take(&row).Error)
|
||||
require.True(t, row.DeletedAt.Valid)
|
||||
|
||||
_, err = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, gorm.ErrRecordNotFound)
|
||||
}
|
||||
|
||||
// sqlCapture records the SQL GORM executes, in order.
|
||||
type sqlCapture struct {
|
||||
logger.Interface
|
||||
mu sync.Mutex
|
||||
sqls []string
|
||||
}
|
||||
|
||||
func (l *sqlCapture) Trace(_ context.Context, _ time.Time, fc func() (string, int64), _ error) {
|
||||
sql, _ := fc()
|
||||
l.mu.Lock()
|
||||
l.sqls = append(l.sqls, sql)
|
||||
l.mu.Unlock()
|
||||
}
|
||||
|
||||
func (l *sqlCapture) reset() {
|
||||
l.mu.Lock()
|
||||
l.sqls = nil
|
||||
l.mu.Unlock()
|
||||
}
|
||||
|
||||
func (l *sqlCapture) snapshot() []string {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
out := make([]string, len(l.sqls))
|
||||
copy(out, l.sqls)
|
||||
return out
|
||||
}
|
||||
|
||||
func firstIndexContaining(sqls []string, needle string) int {
|
||||
for i, sql := range sqls {
|
||||
if strings.Contains(sql, needle) {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// ClaimPlacement locks the session row before it writes the machine row.
|
||||
func TestClaimPlacementTouchesSessionBeforeMachine(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
captured := &sqlCapture{Interface: logger.Discard}
|
||||
db.Logger = captured
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
captured.reset()
|
||||
_, err := claim(context.Background(), repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
sqls := captured.snapshot()
|
||||
sessionIdx := firstIndexContaining(sqls, "FROM `sessions`")
|
||||
machineIdx := firstIndexContaining(sqls, "sandbox_user_machines")
|
||||
require.GreaterOrEqual(t, sessionIdx, 0)
|
||||
require.Greater(t, machineIdx, sessionIdx, "session row is locked before the machine row")
|
||||
}
|
||||
|
||||
// The machine row is locked before the assignment is inserted, so a
|
||||
// reclaimer counting live assignments can never miss a join in flight.
|
||||
func TestClaimPlacementLocksTheMachineBeforeInsertingTheAssignment(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-0", interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment("s-0", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
|
||||
captured := &sqlCapture{Interface: logger.Discard}
|
||||
db.Logger = captured
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
captured.reset()
|
||||
_, err = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
sqls := captured.snapshot()
|
||||
lock := -1
|
||||
for i, sql := range sqls {
|
||||
if strings.Contains(sql, "SELECT") && strings.Contains(sql, "FROM `sandbox_user_machines`") {
|
||||
lock = i
|
||||
break
|
||||
}
|
||||
}
|
||||
insert := firstIndexContaining(sqls, "INSERT INTO `session_sandbox_assignments`")
|
||||
require.GreaterOrEqual(t, lock, 0, "sql:\n%s", strings.Join(sqls, "\n"))
|
||||
require.Greater(t, insert, lock, "machine row is locked before the assignment insert")
|
||||
}
|
||||
|
||||
// RetireSessions locks every session id in sorted order, then every machine
|
||||
// digest in sorted order — the same sessions-then-machines order as ClaimPlacement.
|
||||
func TestRetireLocksSessionsBeforeMachines(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
captured := &sqlCapture{Interface: logger.Discard}
|
||||
db.Logger = captured
|
||||
ctx := context.Background()
|
||||
for _, pair := range []struct{ id, digest string }{{"s-2", "d-b"}, {"s-1", "d-a"}} {
|
||||
seedSession(t, db, pair.id, interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment(pair.id, pair.digest), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
captured.reset()
|
||||
_, err := repo.RetireSessions(ctx, []string{"s-2", "s-1"}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
sqls := captured.snapshot()
|
||||
|
||||
var sessionLocks, machineLocks []string
|
||||
lastSessionLock := -1
|
||||
for i, sql := range sqls {
|
||||
if strings.Contains(sql, "FROM `sessions`") && strings.Contains(sql, "SELECT") {
|
||||
sessionLocks = append(sessionLocks, sql)
|
||||
lastSessionLock = i
|
||||
}
|
||||
if strings.Contains(sql, "FROM `sandbox_user_machines`") && strings.Contains(sql, "SELECT") {
|
||||
machineLocks = append(machineLocks, sql)
|
||||
}
|
||||
}
|
||||
require.GreaterOrEqual(t, len(sessionLocks), 2, "sql:\n%s", strings.Join(sqls, "\n"))
|
||||
require.GreaterOrEqual(t, len(machineLocks), 2, "sql:\n%s", strings.Join(sqls, "\n"))
|
||||
require.Contains(t, sessionLocks[0], `"s-1"`)
|
||||
require.Contains(t, sessionLocks[1], `"s-2"`)
|
||||
require.Contains(t, machineLocks[0], `"d-a"`)
|
||||
require.Contains(t, machineLocks[1], `"d-b"`)
|
||||
require.Greater(t, firstIndexContaining(sqls, "sandbox_user_machines"), lastSessionLock,
|
||||
"every session row is locked before any machine row")
|
||||
}
|
||||
|
||||
// Postgres timestamptz equality is microsecond precision. Retirement truncates
|
||||
// before writing so a clock read back compares equal to the value that was stored.
|
||||
func TestRetireTruncatesTimestampsToMicroseconds(t *testing.T) {
|
||||
repo, db := newAllocationRepo(t)
|
||||
ctx := context.Background()
|
||||
seedSession(t, db, "s-1", interfaces.PlacementPin{})
|
||||
_, err := claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
|
||||
at := time.Date(2026, 10, 13, 10, 11, 12, 123456789, time.UTC)
|
||||
want := at.Truncate(time.Microsecond)
|
||||
released, _, err := retireOne(ctx, repo, "s-1", at)
|
||||
require.NoError(t, err)
|
||||
require.True(t, released.CleanupDueAt.Equal(want))
|
||||
|
||||
var stored types.SessionSandboxAssignment
|
||||
require.NoError(t, db.Where("session_id = ?", "s-1").Take(&stored).Error)
|
||||
require.NotNil(t, stored.ReleasedAt)
|
||||
require.True(t, stored.ReleasedAt.Equal(want), "released_at = %s", stored.ReleasedAt)
|
||||
|
||||
deleted, err := repo.DeleteUserMachineIfEmptySince(ctx, "d-1", at)
|
||||
require.NoError(t, err)
|
||||
require.False(t, deleted, "nanoseconds below a microsecond must not match the stored clock")
|
||||
|
||||
deleted, err = repo.DeleteUserMachineIfEmptySince(ctx, "d-1", want)
|
||||
require.NoError(t, err)
|
||||
require.True(t, deleted)
|
||||
}
|
||||
|
||||
// The tests above build their schema with AutoMigrate from the entities. This
|
||||
// one runs the repository on the schema the shipped SQLite migrations create,
|
||||
// so an entity column the migrations do not have fails here.
|
||||
func TestSandboxAllocationOnTheMigratedSQLiteSchema(t *testing.T) {
|
||||
root, err := filepath.Abs(filepath.Join("..", "..", ".."))
|
||||
require.NoError(t, err)
|
||||
t.Chdir(root)
|
||||
path := filepath.Join(t.TempDir(), "migrated.db")
|
||||
require.NoError(t, database.RunMigrationsWithOptions("sqlite3://unused",
|
||||
database.MigrationOptions{SQLiteDBPath: path}))
|
||||
db, err := gorm.Open(sqlite.Open(path+"?_busy_timeout=5000"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
repo := NewSandboxAllocationRepository(db).(*sandboxAllocationRepository)
|
||||
ctx := context.Background()
|
||||
|
||||
for _, id := range []string{"s-1", "s-2", "s-3"} {
|
||||
require.NoError(t, db.Exec(
|
||||
"INSERT INTO sessions (id, tenant_id, user_id, owner_auth_source) VALUES (?, 7, 'u-1', 'web')", id).Error)
|
||||
}
|
||||
_, err = claim(ctx, repo, userAssignment("s-1", "d-1"), interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
session := userAssignment("s-2", "")
|
||||
_, err = claim(ctx, repo, session, interfaces.PlacementPin{})
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, interfaces.PlacementPin{ConfigID: "cfg", TenantID: 7}, readPin(t, db, "s-1"))
|
||||
|
||||
out, err := repo.RetireSessions(ctx, []string{"s-1", "s-2", "s-3"}, time.Now().UTC())
|
||||
require.NoError(t, err)
|
||||
require.True(t, out[0].MachineEmpty)
|
||||
require.NotNil(t, out[0].Assignment.CleanupDueAt)
|
||||
require.Nil(t, out[1].Assignment.CleanupDueAt)
|
||||
require.Nil(t, out[2].Assignment)
|
||||
_, err = claim(ctx, repo, userAssignment("s-3", "d-1"), interfaces.PlacementPin{})
|
||||
require.ErrorIs(t, err, interfaces.ErrSessionRetired)
|
||||
|
||||
due, err := repo.ListDueWorkspaceCleanups(ctx, time.Now().UTC().Add(time.Second), 10)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, due, 1)
|
||||
require.NoError(t, repo.RetryWorkspaceCleanup(ctx, "s-1", time.Now().UTC().Add(time.Hour)))
|
||||
require.NoError(t, repo.CompleteWorkspaceCleanup(ctx, "s-1"))
|
||||
|
||||
m, err := repo.GetUserMachine(ctx, "d-1")
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, m.EmptySince)
|
||||
empty, err := repo.ListEmptyUserMachines(
|
||||
ctx, time.Now().UTC().Add(time.Second), interfaces.EmptyMachineCursor{}, 10,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, empty, 1)
|
||||
deleted, err := repo.DeleteUserMachineIfEmptySince(ctx, "d-1", *m.EmptySince)
|
||||
require.NoError(t, err)
|
||||
require.True(t, deleted)
|
||||
}
|
||||
|
|
@ -345,8 +345,9 @@ func (r *sessionRepository) SetOwnerID(ctx context.Context, tenantID uint64, id,
|
|||
Model(&types.Session{}).
|
||||
Where("tenant_id = ? AND id = ?", tenantID, id).
|
||||
Updates(map[string]interface{}{
|
||||
"user_id": ownerID,
|
||||
"updated_at": time.Now(),
|
||||
"user_id": ownerID,
|
||||
"owner_auth_source": types.AuthSourceForBoundOwnerID(ownerID),
|
||||
"updated_at": time.Now(),
|
||||
})
|
||||
return res.RowsAffected, res.Error
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,40 @@
|
|||
package sandboxalloc
|
||||
|
||||
import (
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// ToEntity converts an assignment into its row, and for a user-mode one the
|
||||
// registry row of its machine (nil otherwise). digest names that machine.
|
||||
func ToEntity(a allocation.Assignment, digest string) (*types.SessionSandboxAssignment, *types.SandboxUserMachine) {
|
||||
row := &types.SessionSandboxAssignment{
|
||||
SessionID: a.SessionID, ConfigTenantID: a.ConfigTenantID, ConfigID: a.ConfigID,
|
||||
ReleasedAt: a.ReleasedAt,
|
||||
}
|
||||
m, ok := a.UserMachine()
|
||||
if !ok {
|
||||
return row, nil
|
||||
}
|
||||
row.MachineDigest = digest
|
||||
return row, &types.SandboxUserMachine{
|
||||
MachineDigest: digest, OwnerTenantID: m.OwnerTenantID, OwnerID: m.OwnerID,
|
||||
ConfigTenantID: m.ConfigTenantID, ConfigID: m.ConfigID,
|
||||
}
|
||||
}
|
||||
|
||||
// FromEntity converts a row back. machine is the registry row its digest
|
||||
// names; it is required for a user-mode row and ignored otherwise.
|
||||
func FromEntity(e *types.SessionSandboxAssignment, machine *types.SandboxUserMachine) allocation.Assignment {
|
||||
a := allocation.Assignment{
|
||||
SessionID: e.SessionID, ConfigTenantID: e.ConfigTenantID, ConfigID: e.ConfigID,
|
||||
ReleasedAt: e.ReleasedAt,
|
||||
}
|
||||
if e.MachineDigest != "" && machine != nil {
|
||||
a.Machine = allocation.UserMachine{
|
||||
OwnerTenantID: machine.OwnerTenantID, OwnerID: machine.OwnerID,
|
||||
ConfigTenantID: machine.ConfigTenantID, ConfigID: machine.ConfigID,
|
||||
}
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
package sandboxalloc
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// ownerFromSession maps the session's persisted owner onto allocation.Owner.
|
||||
// Only the authentication source is stored; the kind follows from it, and is
|
||||
// cross-checked against the owner id's shape so a row whose source disagrees
|
||||
// with its owner can never look like a verified person.
|
||||
func ownerFromSession(s *types.Session) allocation.Owner {
|
||||
src := allocation.AuthSource(strings.TrimSpace(s.OwnerAuthSource))
|
||||
if src == "" {
|
||||
src = allocation.SourceUnknown
|
||||
}
|
||||
o := allocation.Owner{TenantID: s.TenantID, ID: s.UserID, Kind: allocation.KindUnknown, Source: src}
|
||||
id := strings.TrimSpace(s.UserID)
|
||||
switch src {
|
||||
case allocation.SourceWeb:
|
||||
if id != "" && !strings.Contains(id, ":") {
|
||||
o.Kind = allocation.KindWebUser
|
||||
}
|
||||
case allocation.SourceAPIDirectHeader, allocation.SourceAPISignedToken:
|
||||
if strings.HasPrefix(id, types.SessionOwnerAPIExternalUserPrefix) {
|
||||
o.Kind = allocation.KindAPIExternalUser
|
||||
}
|
||||
case allocation.SourceAPITenantKey:
|
||||
o.Kind = allocation.KindAPITenantKey
|
||||
case allocation.SourceAPIPlatformKey:
|
||||
o.Kind = allocation.KindAPIPlatformKey
|
||||
case allocation.SourceMCPEndpoint:
|
||||
o.Kind = allocation.KindMCPEndpoint
|
||||
case allocation.SourceEmbed:
|
||||
o.Kind = allocation.KindEmbed
|
||||
case allocation.SourceIM:
|
||||
o.Kind = allocation.KindIM
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
|
@ -0,0 +1,300 @@
|
|||
// Package sandboxalloc decides and records how sessions are placed on remote
|
||||
// sandboxes. It is the only writer of session_sandbox_assignments and, once
|
||||
// wired, of the session's sandbox pin.
|
||||
package sandboxalloc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
||||
// ConfigReader loads a sandbox config by its owning workspace.
|
||||
type ConfigReader interface {
|
||||
GetByID(ctx context.Context, tenantID uint64, id string) (*types.TenantSandboxConfigEntity, error)
|
||||
}
|
||||
|
||||
// SessionReader loads a session row.
|
||||
type SessionReader interface {
|
||||
GetByID(ctx context.Context, tenantID uint64, id string) (*types.Session, error)
|
||||
}
|
||||
|
||||
// Service is the allocation service.
|
||||
type Service struct {
|
||||
repo interfaces.SandboxAllocationRepository
|
||||
configs ConfigReader
|
||||
sessions SessionReader
|
||||
namespace string
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// ErrMachineUnregistered is returned when a user-mode assignment names a
|
||||
// machine whose registry row is gone (its config was deleted). The session
|
||||
// cannot run, and must never be mistaken for one with a machine of its own.
|
||||
var ErrMachineUnregistered = errors.New("sandboxalloc: shared machine is not registered")
|
||||
|
||||
// New wires the service. namespace must be the Redis binding namespace.
|
||||
func New(
|
||||
repo interfaces.SandboxAllocationRepository, configs ConfigReader,
|
||||
sessions SessionReader, namespace string,
|
||||
) *Service {
|
||||
return &Service{
|
||||
repo: repo, configs: configs, sessions: sessions, namespace: namespace,
|
||||
now: func() time.Time { return time.Now().UTC().Truncate(time.Microsecond) },
|
||||
}
|
||||
}
|
||||
|
||||
// Namespace is the deployment namespace machine digests use.
|
||||
func (s *Service) Namespace() string { return s.namespace }
|
||||
|
||||
// Digest is the shared-machine digest of m in this deployment.
|
||||
func (s *Service) Digest(m allocation.UserMachine) string { return m.Digest(s.namespace) }
|
||||
|
||||
// Repo is the allocation repository.
|
||||
func (s *Service) Repo() interfaces.SandboxAllocationRepository { return s.repo }
|
||||
|
||||
// Now is the clock placements and releases use.
|
||||
func (s *Service) Now() time.Time { return s.now() }
|
||||
|
||||
// PlaceRequest names the session and the config address the caller resolved
|
||||
// as its candidate.
|
||||
type PlaceRequest struct {
|
||||
SessionTenantID uint64
|
||||
SessionID string
|
||||
ConfigTenantID uint64
|
||||
ConfigID string
|
||||
Purpose allocation.Purpose
|
||||
}
|
||||
|
||||
const placeAttempts = 3
|
||||
|
||||
// Place returns the session's placement, recording it on first use together
|
||||
// with the session's pin. The stored row always wins: neither the config
|
||||
// switch, the candidate, nor the request identity can move a placed session.
|
||||
func (s *Service) Place(ctx context.Context, req PlaceRequest) (allocation.Assignment, error) {
|
||||
a, _, err := s.place(ctx, req)
|
||||
return a, err
|
||||
}
|
||||
|
||||
// place is Place that also reports why a placement it recorded differs from
|
||||
// the config's mode. The reason is logged, not stored.
|
||||
func (s *Service) place(
|
||||
ctx context.Context, req PlaceRequest,
|
||||
) (allocation.Assignment, allocation.FallbackReason, error) {
|
||||
for attempt := 0; attempt < placeAttempts; attempt++ {
|
||||
existing, ok, err := s.Lookup(ctx, req.SessionID)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", err
|
||||
}
|
||||
if ok && existing.Released() {
|
||||
// Lookup keeps released rows for cleanup and inventory; nothing
|
||||
// may run on them again.
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: session %s is being deleted: %w",
|
||||
req.SessionID, sandbox.ErrSandboxSessionDeleted)
|
||||
}
|
||||
if ok {
|
||||
return existing, allocation.ReasonNone, nil
|
||||
}
|
||||
sess, err := s.sessions.GetByID(ctx, req.SessionTenantID, req.SessionID)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: load session %s: %w", req.SessionID, err)
|
||||
}
|
||||
if sess == nil {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: session %s not found", req.SessionID)
|
||||
}
|
||||
pin := interfaces.PlacementPin{ConfigID: sess.SandboxConfigID, TenantID: sess.SandboxConfigTenantID}
|
||||
a, reason, err := s.decide(ctx, req, sess, pin)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", err
|
||||
}
|
||||
row, machine := ToEntity(a, s.digestOf(a))
|
||||
winner, err := s.repo.ClaimPlacement(ctx, row, machine, pin)
|
||||
if errors.Is(err, interfaces.ErrPlacementConflict) {
|
||||
continue
|
||||
}
|
||||
if errors.Is(err, interfaces.ErrSessionRetired) {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: session %s is being deleted: %w",
|
||||
req.SessionID, sandbox.ErrSandboxSessionDeleted)
|
||||
}
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: record placement: %w", err)
|
||||
}
|
||||
if winner.ConfigTenantID != row.ConfigTenantID || winner.ConfigID != row.ConfigID ||
|
||||
winner.MachineDigest != row.MachineDigest || winner.ReleasedAt != nil {
|
||||
// Someone else placed the session first; their row is the answer.
|
||||
got, _, err := s.Lookup(ctx, req.SessionID)
|
||||
return got, allocation.ReasonNone, err
|
||||
}
|
||||
logger.Infof(ctx, "[sandbox-alloc] session=%s mode=%s reason=%s config=%d/%s",
|
||||
a.SessionID, a.Mode(), reason, a.ConfigTenantID, a.ConfigID)
|
||||
return a, reason, nil
|
||||
}
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: placement of %s kept conflicting", req.SessionID)
|
||||
}
|
||||
|
||||
// digestOf names a's shared machine, or "" for a machine of its own.
|
||||
func (s *Service) digestOf(a allocation.Assignment) string {
|
||||
m, ok := a.UserMachine()
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
return s.Digest(m)
|
||||
}
|
||||
|
||||
// decide picks the mode and config for a session without a placement. A
|
||||
// session that already used a per-session sandbox — it has a pin, an
|
||||
// unopened snapshot fork, or a checkpoint — stays per-session on that config.
|
||||
func (s *Service) decide(
|
||||
ctx context.Context, req PlaceRequest, sess *types.Session, pin interfaces.PlacementPin,
|
||||
) (allocation.Assignment, allocation.FallbackReason, error) {
|
||||
purpose := req.Purpose
|
||||
if purpose == "" {
|
||||
purpose = allocation.PurposeChat
|
||||
}
|
||||
if types.IsSkillMaintenanceDescription(sess.Description) {
|
||||
purpose = allocation.PurposeMaintenance
|
||||
}
|
||||
configID, configTenant := req.ConfigID, req.ConfigTenantID
|
||||
if pin.ConfigID != "" {
|
||||
configID = pin.ConfigID
|
||||
if pin.TenantID != 0 {
|
||||
configTenant = pin.TenantID
|
||||
} else if req.ConfigID != pin.ConfigID {
|
||||
configTenant = 0
|
||||
}
|
||||
}
|
||||
if configTenant == 0 {
|
||||
configTenant = sess.TenantID
|
||||
}
|
||||
configID = sandbox.NormalizeConfigID(configID)
|
||||
owner := ownerFromSession(sess)
|
||||
|
||||
legacy := pin.ConfigID != "" || (sess.ForkBootstrap != nil && !sess.ForkBootstrap.Consumed())
|
||||
if !legacy {
|
||||
has, err := s.repo.HasSandboxCheckpoint(ctx, sess.ID)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: check history of %s: %w", sess.ID, err)
|
||||
}
|
||||
legacy = has
|
||||
}
|
||||
decision := allocation.Decision{Mode: allocation.ModeSession, Reason: allocation.ReasonLegacySessionSandbox}
|
||||
if !legacy {
|
||||
mode, err := s.configMode(ctx, configTenant, configID)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, "", err
|
||||
}
|
||||
decision = allocation.Decide(allocation.DecideInput{
|
||||
ConfigMode: mode,
|
||||
Purpose: purpose,
|
||||
Owner: owner,
|
||||
SessionID: sess.ID,
|
||||
UserModeAvailable: allocation.UserModeAvailable(),
|
||||
})
|
||||
}
|
||||
a := allocation.Assignment{SessionID: sess.ID, ConfigTenantID: configTenant, ConfigID: configID}
|
||||
if decision.Mode != allocation.ModeUser {
|
||||
return a, decision.Reason, nil
|
||||
}
|
||||
a.Machine = allocation.UserMachine{
|
||||
OwnerTenantID: sess.TenantID, OwnerID: sess.UserID,
|
||||
ConfigTenantID: configTenant, ConfigID: configID,
|
||||
}
|
||||
if err := a.Machine.Validate(); err != nil {
|
||||
return allocation.Assignment{}, "", fmt.Errorf("sandboxalloc: %w", err)
|
||||
}
|
||||
return a, decision.Reason, nil
|
||||
}
|
||||
|
||||
// configMode reads the config's switch. A missing row or an unparsable value
|
||||
// is session: a failure to load must never widen sharing. A transport error
|
||||
// is not a decision — the caller retries instead of storing session mode.
|
||||
func (s *Service) configMode(ctx context.Context, tenantID uint64, configID string) (allocation.Mode, error) {
|
||||
if s.configs == nil || configID == "" || configID == types.SandboxConfigIDGlobalDefault {
|
||||
return allocation.ModeSession, nil
|
||||
}
|
||||
entity, err := s.configs.GetByID(ctx, tenantID, configID)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("sandboxalloc: read config %d/%s: %w", tenantID, configID, err)
|
||||
}
|
||||
if entity == nil || entity.Config == nil {
|
||||
return allocation.ModeSession, nil
|
||||
}
|
||||
mode, err := allocation.ParseMode(entity.Config.AllocationMode)
|
||||
if err != nil {
|
||||
return allocation.ModeSession, nil
|
||||
}
|
||||
return mode, nil
|
||||
}
|
||||
|
||||
// Lookup returns the stored assignment without creating one. It always reads
|
||||
// the database: callers use it to verify validity. A session retired before
|
||||
// it was ever placed has no assignment.
|
||||
func (s *Service) Lookup(ctx context.Context, sessionID string) (allocation.Assignment, bool, error) {
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return allocation.Assignment{}, false, nil
|
||||
}
|
||||
row, err := s.repo.GetAssignment(ctx, sessionID)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, false, fmt.Errorf("sandboxalloc: read assignment %s: %w", sessionID, err)
|
||||
}
|
||||
if row == nil || row.IsTombstone() {
|
||||
return allocation.Assignment{}, false, nil
|
||||
}
|
||||
var machine *types.SandboxUserMachine
|
||||
if row.MachineDigest != "" {
|
||||
machine, err = s.repo.GetUserMachine(ctx, row.MachineDigest)
|
||||
if err != nil {
|
||||
return allocation.Assignment{}, false, fmt.Errorf("sandboxalloc: read machine of %s: %w", sessionID, err)
|
||||
}
|
||||
if machine == nil {
|
||||
return allocation.Assignment{}, false, fmt.Errorf("%w: session %s", ErrMachineUnregistered, sessionID)
|
||||
}
|
||||
}
|
||||
return FromEntity(row, machine), true, nil
|
||||
}
|
||||
|
||||
// Retire is the first step of deleting sessions (C5): one transaction, under
|
||||
// the lock first placement takes, releases each session's assignment and
|
||||
// schedules its workspace cleanup. A session without an assignment gets a
|
||||
// tombstone, so no placement can land after the deletion began.
|
||||
func (s *Service) Retire(ctx context.Context, sessionIDs []string) ([]interfaces.RetiredSession, error) {
|
||||
out, err := s.repo.RetireSessions(ctx, sessionIDs, s.now())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("sandboxalloc: retire sessions: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Release retires one session.
|
||||
func (s *Service) Release(ctx context.Context, sessionID string) (interfaces.RetiredSession, error) {
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return interfaces.RetiredSession{}, errors.New("sandboxalloc: release needs a session id")
|
||||
}
|
||||
out, err := s.Retire(ctx, []string{sessionID})
|
||||
if err != nil {
|
||||
return interfaces.RetiredSession{}, err
|
||||
}
|
||||
if len(out) != 1 {
|
||||
return interfaces.RetiredSession{}, fmt.Errorf(
|
||||
"sandboxalloc: retiring %s returned %d results", sessionID, len(out),
|
||||
)
|
||||
}
|
||||
return out[0], nil
|
||||
}
|
||||
|
||||
// ForkAssignment is the child's assignment: same mode, machine and config,
|
||||
// fresh session id. The caller persists it with the forked session.
|
||||
func (s *Service) ForkAssignment(source allocation.Assignment, targetSessionID string) allocation.Assignment {
|
||||
child := source
|
||||
child.SessionID = targetSessionID
|
||||
child.ReleasedAt = nil
|
||||
return child
|
||||
}
|
||||
|
|
@ -0,0 +1,301 @@
|
|||
package sandboxalloc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/application/repository"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
type fakeConfigs map[string]string // config id -> allocation_mode
|
||||
|
||||
func (f fakeConfigs) GetByID(_ context.Context, tenantID uint64, id string) (*types.TenantSandboxConfigEntity, error) {
|
||||
return &types.TenantSandboxConfigEntity{
|
||||
ID: id, TenantID: tenantID,
|
||||
Config: &types.TenantSandboxConfig{AllocationMode: f[id]},
|
||||
}, nil
|
||||
}
|
||||
|
||||
type testEnv struct {
|
||||
svc *Service
|
||||
db *gorm.DB
|
||||
}
|
||||
|
||||
func newEnv(t *testing.T, configs fakeConfigs, sessions ...*types.Session) *testEnv {
|
||||
t.Helper()
|
||||
// Same SQLite pool as the allocation repository tests: one connection and
|
||||
// a busy timeout, so concurrent first-writers queue instead of failing
|
||||
// with "database table is locked".
|
||||
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared&_busy_timeout=5000", uuid.NewString())
|
||||
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
sqlDB, err := db.DB()
|
||||
require.NoError(t, err)
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
require.NoError(t, db.AutoMigrate(&types.Session{}, &types.SessionSandboxAssignment{},
|
||||
&types.SandboxUserMachine{}, &types.Message{}))
|
||||
for _, s := range sessions {
|
||||
require.NoError(t, db.Session(&gorm.Session{SkipHooks: true}).Create(s).Error)
|
||||
}
|
||||
svc := New(repository.NewSandboxAllocationRepository(db), configs, repository.NewSessionRepository(db), "weknora")
|
||||
return &testEnv{svc: svc, db: db}
|
||||
}
|
||||
|
||||
func webSession(id string) *types.Session {
|
||||
return &types.Session{ID: id, TenantID: 7, UserID: "u-1", OwnerAuthSource: types.AuthSourceWeb}
|
||||
}
|
||||
|
||||
func (e *testEnv) place(t *testing.T, sessionID, configID string) allocation.Assignment {
|
||||
t.Helper()
|
||||
a, _ := e.placeWithReason(t, sessionID, configID)
|
||||
return a
|
||||
}
|
||||
|
||||
func (e *testEnv) placeWithReason(
|
||||
t *testing.T, sessionID, configID string,
|
||||
) (allocation.Assignment, allocation.FallbackReason) {
|
||||
t.Helper()
|
||||
a, reason, err := e.svc.place(context.Background(), PlaceRequest{
|
||||
SessionTenantID: 7, SessionID: sessionID, ConfigTenantID: 7, ConfigID: configID,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
return a, reason
|
||||
}
|
||||
|
||||
func (e *testEnv) reason(t *testing.T, sessionID, configID string) allocation.FallbackReason {
|
||||
t.Helper()
|
||||
_, r := e.placeWithReason(t, sessionID, configID)
|
||||
return r
|
||||
}
|
||||
|
||||
func (e *testEnv) pin(t *testing.T, sessionID string) (string, uint64) {
|
||||
t.Helper()
|
||||
var s types.Session
|
||||
require.NoError(t, e.db.Session(&gorm.Session{SkipHooks: true}).Where("id = ?", sessionID).Take(&s).Error)
|
||||
return s.SandboxConfigID, s.SandboxConfigTenantID
|
||||
}
|
||||
|
||||
func enableUserMode(t *testing.T) {
|
||||
t.Helper()
|
||||
allocation.SetUserModeAvailable(true)
|
||||
t.Cleanup(func() { allocation.SetUserModeAvailable(false) })
|
||||
}
|
||||
|
||||
type errConfigs struct{ err error }
|
||||
|
||||
func (e errConfigs) GetByID(context.Context, uint64, string) (*types.TenantSandboxConfigEntity, error) {
|
||||
return nil, e.err
|
||||
}
|
||||
|
||||
func TestPlaceDoesNotRecordAModeWhenTheConfigReadFails(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
e.svc.configs = errConfigs{err: errors.New("timeout")}
|
||||
|
||||
_, err := e.svc.Place(context.Background(), PlaceRequest{
|
||||
SessionTenantID: 7, SessionID: id, ConfigTenantID: 7, ConfigID: "cfg-a",
|
||||
})
|
||||
require.Error(t, err)
|
||||
row, err := e.svc.repo.GetAssignment(context.Background(), id)
|
||||
require.NoError(t, err)
|
||||
require.Nil(t, row, "a failed config read must be retried, not stored as session mode")
|
||||
}
|
||||
|
||||
func TestPlacePutsAVerifiedWebUserOnASharedMachineAndPinsIt(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
|
||||
a := e.place(t, id, "cfg-a")
|
||||
require.Equal(t, allocation.ModeUser, a.Mode())
|
||||
m, ok := a.UserMachine()
|
||||
require.True(t, ok)
|
||||
require.Equal(t, allocation.UserMachine{OwnerTenantID: 7, OwnerID: "u-1", ConfigTenantID: 7, ConfigID: "cfg-a"}, m)
|
||||
cfg, tenant := e.pin(t, id)
|
||||
require.Equal(t, "cfg-a", cfg)
|
||||
require.EqualValues(t, 7, tenant)
|
||||
|
||||
// The stored row wins over a later request for another config, and reads
|
||||
// back the same machine.
|
||||
again := e.place(t, id, "cfg-b")
|
||||
require.Equal(t, a, again)
|
||||
}
|
||||
|
||||
// The owner's kind is derived from the stored source and the id's shape; a
|
||||
// source that disagrees with the id never makes a verified person.
|
||||
func TestPlaceDerivesTheOwnerKindFromItsSource(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
ext, webWithColon, unknown := uuid.NewString(), uuid.NewString(), uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"},
|
||||
&types.Session{
|
||||
ID: ext, TenantID: 7, UserID: "api_external_user:7:alice",
|
||||
OwnerAuthSource: types.AuthSourceAPISignedToken,
|
||||
},
|
||||
&types.Session{
|
||||
ID: webWithColon, TenantID: 7, UserID: "api_tenant_key:7:3",
|
||||
OwnerAuthSource: types.AuthSourceWeb,
|
||||
},
|
||||
&types.Session{ID: unknown, TenantID: 7, UserID: "system-7"})
|
||||
|
||||
require.Equal(t, allocation.ModeUser, e.place(t, ext, "cfg-a").Mode())
|
||||
require.Equal(t, allocation.ReasonOwnerIdentityUnverified, e.reason(t, webWithColon, "cfg-a"))
|
||||
require.Equal(t, allocation.ReasonOwnerIdentityUnverified, e.reason(t, unknown, "cfg-a"),
|
||||
"a row from before the column existed keeps a machine per session")
|
||||
}
|
||||
|
||||
func TestLookupRefusesAUserAssignmentWithoutItsMachine(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
e.place(t, id, "cfg-a")
|
||||
require.NoError(t, e.svc.Repo().DeleteUserMachinesByConfig(context.Background(), 7, "cfg-a"))
|
||||
|
||||
_, _, err := e.svc.Lookup(context.Background(), id)
|
||||
require.ErrorIs(t, err, ErrMachineUnregistered, "never read back as a machine of its own")
|
||||
}
|
||||
|
||||
func TestPlaceConcurrentCandidatesConvergeOnOneConfig(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user", "cfg-b": "session"}, webSession(id))
|
||||
var wg sync.WaitGroup
|
||||
got := make(chan string, 8)
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
cfg := "cfg-a"
|
||||
if i%2 == 1 {
|
||||
cfg = "cfg-b"
|
||||
}
|
||||
got <- e.place(t, id, cfg).ConfigID
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
close(got)
|
||||
winner := ""
|
||||
for c := range got {
|
||||
if winner == "" {
|
||||
winner = c
|
||||
}
|
||||
require.Equal(t, winner, c, "every caller is told the same config")
|
||||
}
|
||||
cfg, _ := e.pin(t, id)
|
||||
require.Equal(t, winner, cfg, "the pin is the assignment's projection")
|
||||
}
|
||||
|
||||
func TestPlaceFallbacks(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
apiKey, legacy, maint, fork := uuid.NewString(), uuid.NewString(), uuid.NewString(), uuid.NewString()
|
||||
legacyRow := webSession(legacy)
|
||||
legacyRow.SandboxConfigID, legacyRow.SandboxConfigTenantID = "cfg-a", 7
|
||||
maintRow := webSession(maint)
|
||||
maintRow.Description = types.SkillMaintenanceSessionMarker + "install"
|
||||
forkRow := webSession(fork)
|
||||
forkRow.ForkBootstrap = &types.ForkBootstrap{SnapshotID: "snap"}
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"},
|
||||
&types.Session{
|
||||
ID: apiKey, TenantID: 7, UserID: "api_tenant_key:7:3",
|
||||
OwnerAuthSource: types.AuthSourceAPITenantKey,
|
||||
},
|
||||
legacyRow, maintRow, forkRow)
|
||||
|
||||
require.Equal(t, allocation.ReasonSharedAPIKey, e.reason(t, apiKey, "cfg-a"))
|
||||
got, reason := e.placeWithReason(t, legacy, "cfg-a")
|
||||
require.Equal(t, allocation.ModeSession, got.Mode(),
|
||||
"a session that already has a pin keeps its per-session sandbox")
|
||||
require.Equal(t, allocation.ReasonLegacySessionSandbox, reason)
|
||||
require.Equal(t, allocation.ReasonMaintenanceDedicated, e.reason(t, maint, "cfg-a"))
|
||||
require.Equal(t, allocation.ReasonLegacySessionSandbox, e.reason(t, fork, "cfg-a"),
|
||||
"an unopened snapshot fork keeps its per-session bootstrap")
|
||||
}
|
||||
|
||||
func TestPlaceWithTheGateOffFallsBack(t *testing.T) {
|
||||
allocation.SetUserModeAvailable(false)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
a, reason := e.placeWithReason(t, id, "cfg-a")
|
||||
require.Equal(t, allocation.ModeSession, a.Mode())
|
||||
require.Equal(t, allocation.ReasonUserModeUnavailable, reason)
|
||||
}
|
||||
|
||||
func TestReleaseAndForkAssignment(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
a := e.place(t, id, "cfg-a")
|
||||
|
||||
target := uuid.NewString()
|
||||
child := e.svc.ForkAssignment(a, target)
|
||||
require.Equal(t, target, child.SessionID)
|
||||
require.Equal(t, a.Machine, child.Machine)
|
||||
require.Equal(t, a.ConfigID, child.ConfigID)
|
||||
require.Nil(t, child.ReleasedAt)
|
||||
|
||||
r, err := e.svc.Release(context.Background(), id)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, r.Assignment)
|
||||
require.True(t, r.MachineEmpty)
|
||||
require.NotNil(t, r.Assignment.ReleasedAt)
|
||||
require.NotNil(t, r.Assignment.CleanupDueAt)
|
||||
|
||||
released, ok, err := e.svc.Lookup(context.Background(), id)
|
||||
require.NoError(t, err)
|
||||
require.True(t, ok, "cleanup and inventory still find a released session's machine")
|
||||
require.True(t, released.Released())
|
||||
require.Equal(t, a.Machine, released.Machine)
|
||||
}
|
||||
|
||||
func TestPlaceOnARetiredSessionIsRefused(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
out, err := e.svc.Retire(context.Background(), []string{id})
|
||||
require.NoError(t, err)
|
||||
require.Nil(t, out[0].Assignment, "retiring works before any placement")
|
||||
|
||||
_, err = e.svc.Place(types.WithSandboxTenantID(context.Background(), 7), PlaceRequest{
|
||||
SessionTenantID: 7, SessionID: id, ConfigTenantID: 7, ConfigID: "cfg-a",
|
||||
})
|
||||
require.ErrorIs(t, err, sandbox.ErrSandboxSessionDeleted)
|
||||
_, ok, err := e.svc.Lookup(context.Background(), id)
|
||||
require.NoError(t, err)
|
||||
require.False(t, ok)
|
||||
}
|
||||
|
||||
// Lookup keeps a released row for cleanup; Place must refuse it, or a turn
|
||||
// already in flight would rebuild the workspace of a deleted session.
|
||||
func TestPlaceOnAReleasedAssignmentIsRefused(t *testing.T) {
|
||||
enableUserMode(t)
|
||||
id := uuid.NewString()
|
||||
e := newEnv(t, fakeConfigs{"cfg-a": "user"}, webSession(id))
|
||||
e.place(t, id, "cfg-a")
|
||||
_, err := e.svc.Release(context.Background(), id)
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = e.svc.Place(context.Background(), PlaceRequest{
|
||||
SessionTenantID: 7, SessionID: id, ConfigTenantID: 7, ConfigID: "cfg-a",
|
||||
})
|
||||
require.ErrorIs(t, err, sandbox.ErrSandboxSessionDeleted)
|
||||
}
|
||||
|
||||
func TestReleaseWithoutASessionIDIsAnError(t *testing.T) {
|
||||
e := newEnv(t, fakeConfigs{})
|
||||
_, err := e.svc.Release(context.Background(), " ")
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
|
@ -244,6 +244,9 @@ func (s *sessionService) CreateSession(ctx context.Context, session *types.Sessi
|
|||
|
||||
logger.Infof(ctx, "Creating session, tenant ID: %d", session.TenantID)
|
||||
|
||||
// Owner type and source are server facts, never client input.
|
||||
types.StampSessionOwnerSource(ctx, session)
|
||||
|
||||
// Create session in repository
|
||||
createdSession, err := s.sessionRepo.Create(ctx, session)
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -259,6 +259,7 @@ func (s *SessionForkService) Fork(
|
|||
TenantID: source.TenantID,
|
||||
Kind: source.Kind,
|
||||
UserID: source.UserID,
|
||||
OwnerAuthSource: source.OwnerAuthSource,
|
||||
Title: forkTitle(title, source.Title),
|
||||
Description: source.Description,
|
||||
LastRequestState: source.LastRequestState,
|
||||
|
|
|
|||
|
|
@ -506,6 +506,7 @@ func TestForkHappyPathTakesSnapshotAndCopiesHistory(t *testing.T) {
|
|||
}
|
||||
port := &fakeForkSandboxPort{boundID: "sbx-1", bound: true, snapshotID: "snap-1"}
|
||||
svc, sessions, _ := newForkFixture(t, port, append(turn, forkPoint))
|
||||
sessions.source.OwnerAuthSource = "web"
|
||||
|
||||
got, err := svc.Fork(context.Background(), 1, "u1", "src", "u-msg-2", "")
|
||||
|
||||
|
|
@ -520,6 +521,7 @@ func TestForkHappyPathTakesSnapshotAndCopiesHistory(t *testing.T) {
|
|||
require.Equal(t, "u-msg-2", created.ForkedFromMessageID)
|
||||
require.Equal(t, "cfg-1", created.SandboxConfigID, "必须继承 sandbox_config_id,否则可能落到别的 backend")
|
||||
require.Equal(t, "原会话(分支)", created.Title)
|
||||
require.Equal(t, "web", created.OwnerAuthSource)
|
||||
require.NotNil(t, created.ForkBootstrap)
|
||||
require.Equal(t, "snap-1", created.ForkBootstrap.SnapshotID)
|
||||
require.Equal(t, "sha1", created.ForkBootstrap.CommitSHA)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,36 @@
|
|||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
||||
func TestCreateSessionStampsOwnerSourceServerSide(t *testing.T) {
|
||||
repo := &recordingSessionRepo{}
|
||||
svc := &sessionService{sessionRepo: repo}
|
||||
ctx := context.WithValue(context.Background(), types.UserIDContextKey, "u-1")
|
||||
ctx = types.WithPrincipal(ctx, types.Principal{Type: types.PrincipalWebUser, ID: "u-1"})
|
||||
ctx = types.WithPrincipalAuthSource(ctx, types.AuthSourceWeb)
|
||||
|
||||
_, err := svc.CreateSession(ctx, &types.Session{
|
||||
TenantID: 1, UserID: "u-1", OwnerAuthSource: "api_signed_token",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, types.AuthSourceWeb, repo.created.OwnerAuthSource)
|
||||
}
|
||||
|
||||
type recordingSessionRepo struct {
|
||||
interfaces.SessionRepository
|
||||
created *types.Session
|
||||
}
|
||||
|
||||
func (r *recordingSessionRepo) Create(_ context.Context, s *types.Session) (*types.Session, error) {
|
||||
copied := *s
|
||||
r.created = &copied
|
||||
return &copied, nil
|
||||
}
|
||||
|
|
@ -25,8 +25,10 @@ import (
|
|||
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/application/service/sandboxalloc"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
|
|
@ -65,10 +67,27 @@ func (p SandboxPin) TenantOr(sessionTenantID uint64) uint64 {
|
|||
return sessionTenantID
|
||||
}
|
||||
|
||||
// SessionPlacer records a session's placement and its pin on first use, and
|
||||
// reads it back without placing.
|
||||
type SessionPlacer interface {
|
||||
Place(ctx context.Context, req sandboxalloc.PlaceRequest) (allocation.Assignment, error)
|
||||
Lookup(ctx context.Context, sessionID string) (allocation.Assignment, bool, error)
|
||||
}
|
||||
|
||||
// SessionSandboxPinner is the only place session sandbox pins are read or
|
||||
// written. Centralising it is what keeps the four entry points consistent.
|
||||
type SessionSandboxPinner struct {
|
||||
db *gorm.DB
|
||||
db *gorm.DB
|
||||
placer SessionPlacer
|
||||
}
|
||||
|
||||
// WithPlacer attaches the allocation service. Without one the pinner behaves
|
||||
// exactly as before (no assignment rows; every session per-session).
|
||||
func (p *SessionSandboxPinner) WithPlacer(placer SessionPlacer) *SessionSandboxPinner {
|
||||
if p != nil {
|
||||
p.placer = placer
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// NewSessionSandboxPinner returns a pinner over the sessions table.
|
||||
|
|
@ -79,11 +98,53 @@ func NewSessionSandboxPinner(db *gorm.DB) *SessionSandboxPinner {
|
|||
// Read returns the pinned backend, or the zero pin when the session has no
|
||||
// live sandbox. Note the pin is an optimistic hint: the sandbox may have been
|
||||
// reclaimed out of band (see Clear).
|
||||
//
|
||||
// Once a placer is attached, a recorded assignment is the address (C15). The
|
||||
// pin column is only its projection and may still lack a legacy owner tenant.
|
||||
// A released assignment is returned too: cleanup and inventory still have to
|
||||
// find the machine at its original address.
|
||||
func (p *SessionSandboxPinner) Read(ctx context.Context, sessionID string) (SandboxPin, error) {
|
||||
if p != nil && p.placer != nil {
|
||||
a, ok, err := p.placer.Lookup(ctx, sessionID)
|
||||
if err != nil {
|
||||
return SandboxPin{}, err
|
||||
}
|
||||
if ok {
|
||||
return SandboxPin{ConfigID: a.ConfigID, TenantID: a.ConfigTenantID}, nil
|
||||
}
|
||||
}
|
||||
pinned, _, err := p.read(ctx, sessionID)
|
||||
return pinned, err
|
||||
}
|
||||
|
||||
// placeAndResolve records the placement for candidate and returns the
|
||||
// manager of the WINNING config (C1). candidateMgr is reused only when the
|
||||
// winner is exactly the candidate.
|
||||
func (p *SessionSandboxPinner) placeAndResolve(
|
||||
ctx context.Context, resolver sandbox.TenantSandboxResolver, fallback sandbox.Manager,
|
||||
policy WorkspaceSandboxPolicy, sessionID string, candidate SandboxPin, candidateMgr sandbox.Manager,
|
||||
) (sandbox.Manager, SandboxPin, error) {
|
||||
sessionTenant, ok := types.SandboxTenantIDFromContext(ctx)
|
||||
if !ok || sessionTenant == 0 {
|
||||
return nil, SandboxPin{}, fmt.Errorf("sandbox: placing session %s needs its tenant", sessionID)
|
||||
}
|
||||
a, err := p.placer.Place(ctx, sandboxalloc.PlaceRequest{
|
||||
SessionTenantID: sessionTenant, SessionID: sessionID,
|
||||
ConfigTenantID: candidate.TenantID, ConfigID: candidate.ConfigID,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, SandboxPin{}, err
|
||||
}
|
||||
winner := SandboxPin{ConfigID: a.ConfigID, TenantID: a.ConfigTenantID}
|
||||
if winner.ConfigID == candidate.ConfigID && winner.TenantOr(sessionTenant) == candidate.TenantOr(sessionTenant) {
|
||||
return candidateMgr, winner, nil
|
||||
}
|
||||
mgr, err := resolveTenantSandboxForConfig(
|
||||
ctx, resolver, fallback, winner.TenantOr(sessionTenant), winner.ConfigID, policy,
|
||||
)
|
||||
return mgr, winner, err
|
||||
}
|
||||
|
||||
// read additionally reports whether the session row exists at all, which Pin
|
||||
// needs and Read deliberately hides: a caller asking "does this session have a
|
||||
// live sandbox" gets the same "no" either way, but a caller that just created
|
||||
|
|
@ -236,8 +297,28 @@ func resolveSandboxForExecution(
|
|||
return resolveLiteSandbox(ctx, policy, tenantID, agentConfigID, o), SandboxPin{}, nil
|
||||
}
|
||||
|
||||
if pinner != nil && pinner.placer != nil && strings.TrimSpace(sessionID) != "" {
|
||||
// A placed session runs on its assignment's config (C1). Resolving it
|
||||
// here, instead of reading the pin and placing again, keeps the common
|
||||
// path at one query.
|
||||
a, placed, err := pinner.placer.Lookup(ctx, sessionID)
|
||||
if err != nil {
|
||||
return nil, SandboxPin{}, err
|
||||
}
|
||||
if placed {
|
||||
if a.Released() {
|
||||
return nil, SandboxPin{}, fmt.Errorf("sandbox: session %s is being deleted: %w",
|
||||
sessionID, sandbox.ErrSandboxSessionDeleted)
|
||||
}
|
||||
pin := SandboxPin{ConfigID: a.ConfigID, TenantID: a.ConfigTenantID}
|
||||
mgr, err := resolveTenantSandboxForConfig(ctx, resolver, fallback, pin.TenantID, pin.ConfigID, policy)
|
||||
return mgr, pin, err
|
||||
}
|
||||
}
|
||||
|
||||
if pinner != nil && strings.TrimSpace(sessionID) != "" {
|
||||
pinned, err := pinner.Read(ctx, sessionID)
|
||||
// Not placed: Read would only repeat the lookup above.
|
||||
pinned, _, err := pinner.read(ctx, sessionID)
|
||||
if err != nil {
|
||||
return nil, SandboxPin{}, err
|
||||
}
|
||||
|
|
@ -269,6 +350,10 @@ func resolveSandboxForExecution(
|
|||
}
|
||||
pinned.TenantID = owner
|
||||
}
|
||||
if proven && pinner.placer != nil {
|
||||
return pinner.placeAndResolve(ctx, resolver, fallback, policy, sessionID,
|
||||
SandboxPin{ConfigID: pinned.ConfigID, TenantID: owner}, mgr)
|
||||
}
|
||||
return mgr, pinned, err
|
||||
}
|
||||
}
|
||||
|
|
@ -300,6 +385,11 @@ func resolveSandboxForExecution(
|
|||
if pinner == nil || strings.TrimSpace(sessionID) == "" {
|
||||
return mgr, pin, nil
|
||||
}
|
||||
if pinner.placer != nil {
|
||||
// The placement transaction writes the pin; the legacy Pin CAS below
|
||||
// is only for deployments without the allocation service.
|
||||
return pinner.placeAndResolve(ctx, resolver, fallback, policy, sessionID, pin, mgr)
|
||||
}
|
||||
|
||||
winner, err := pinner.Pin(ctx, sessionID, pin)
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -3,12 +3,15 @@ package service
|
|||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/application/service/sandboxalloc"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
|
|
@ -377,6 +380,158 @@ func TestResolveSandboxForExecutionDoesNotPersistOwnerForTheDefaultConfig(t *tes
|
|||
require.Zero(t, stored.TenantID, "the deployment default has no owning workspace")
|
||||
}
|
||||
|
||||
// pinFixture is a fresh session plus named remote backends the resolver
|
||||
// distinguishes by config id. Allocation tests use it to prove the winning
|
||||
// assignment, not the caller's candidate, is what the request runs on.
|
||||
type pinFixture struct {
|
||||
db *gorm.DB
|
||||
pinner *SessionSandboxPinner
|
||||
resolver sandbox.TenantSandboxResolver
|
||||
managers map[string]sandbox.Manager
|
||||
ctx context.Context
|
||||
sessionID string
|
||||
sessionTenantID uint64
|
||||
configTenantID uint64
|
||||
}
|
||||
|
||||
func newPinFixture(t *testing.T) *pinFixture {
|
||||
t.Helper()
|
||||
db := newPinTestDB(t)
|
||||
managers := map[string]sandbox.Manager{
|
||||
"cfg-a": &pinTestManager{typ: sandbox.SandboxTypeCube},
|
||||
"cfg-b": &pinTestManager{typ: sandbox.SandboxTypeE2B},
|
||||
}
|
||||
return &pinFixture{
|
||||
db: db,
|
||||
pinner: NewSessionSandboxPinner(db),
|
||||
resolver: configMapResolver{byID: managers},
|
||||
managers: managers,
|
||||
ctx: context.Background(),
|
||||
sessionID: "s-1",
|
||||
sessionTenantID: 7,
|
||||
configTenantID: 7,
|
||||
}
|
||||
}
|
||||
|
||||
func (fx *pinFixture) managerFor(configID string) sandbox.Manager {
|
||||
return fx.managers[configID]
|
||||
}
|
||||
|
||||
// setPin writes the two pin columns directly. SkipHooks keeps the session id
|
||||
// the fixture created.
|
||||
func (fx *pinFixture) setPin(t *testing.T, pin SandboxPin) {
|
||||
t.Helper()
|
||||
require.NoError(t, fx.db.Session(&gorm.Session{SkipHooks: true}).
|
||||
Model(&types.Session{}).
|
||||
Where("id = ?", fx.sessionID).
|
||||
Updates(map[string]any{
|
||||
"sandbox_config_id": pin.ConfigID,
|
||||
"sandbox_config_tenant_id": pin.TenantID,
|
||||
}).Error)
|
||||
}
|
||||
|
||||
// configMapResolver returns a distinct manager per config id so a test can
|
||||
// tell which backend a request actually resolved.
|
||||
type configMapResolver struct {
|
||||
byID map[string]sandbox.Manager
|
||||
}
|
||||
|
||||
func (r configMapResolver) Resolve(_ context.Context, _ uint64, configID string) (sandbox.Manager, error) {
|
||||
m, ok := r.byID[configID]
|
||||
if !ok || m == nil {
|
||||
return nil, sandbox.ErrSandboxConfigNotFound
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
type recordingPlacer struct {
|
||||
calls []sandboxalloc.PlaceRequest
|
||||
answer func(sandboxalloc.PlaceRequest) allocation.Assignment
|
||||
lookup map[string]allocation.Assignment
|
||||
}
|
||||
|
||||
func (r *recordingPlacer) Place(_ context.Context, req sandboxalloc.PlaceRequest) (allocation.Assignment, error) {
|
||||
r.calls = append(r.calls, req)
|
||||
return r.answer(req), nil
|
||||
}
|
||||
|
||||
func (r *recordingPlacer) Lookup(_ context.Context, id string) (allocation.Assignment, bool, error) {
|
||||
a, ok := r.lookup[id]
|
||||
return a, ok, nil
|
||||
}
|
||||
|
||||
func TestResolveSandboxForExecutionUsesTheAssignmentsConfig(t *testing.T) {
|
||||
fx := newPinFixture(t) // named remote backends cfg-a and cfg-b, fresh session row
|
||||
placer := &recordingPlacer{answer: func(req sandboxalloc.PlaceRequest) allocation.Assignment {
|
||||
// Another replica won with cfg-b.
|
||||
return allocation.Assignment{SessionID: req.SessionID, ConfigTenantID: req.ConfigTenantID, ConfigID: "cfg-b"}
|
||||
}}
|
||||
fx.pinner.WithPlacer(placer)
|
||||
ctx := types.WithSandboxTenantID(fx.ctx, fx.sessionTenantID)
|
||||
|
||||
mgr, pin, err := resolveSandboxForExecution(ctx, fx.resolver, nil, fx.pinner,
|
||||
fx.configTenantID, fx.sessionID, "cfg-a", nil)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "cfg-b", pin.ConfigID)
|
||||
require.Equal(t, fx.managerFor("cfg-b"), mgr, "the request runs on the winner's config, not its own candidate")
|
||||
require.Len(t, placer.calls, 1)
|
||||
}
|
||||
|
||||
// A placed session resolves straight from its assignment: no second
|
||||
// placement round trip, and a released one never runs again.
|
||||
func TestResolveSandboxForExecutionUsesAnExistingAssignmentWithoutPlacing(t *testing.T) {
|
||||
fx := newPinFixture(t)
|
||||
placer := &recordingPlacer{lookup: map[string]allocation.Assignment{
|
||||
fx.sessionID: {SessionID: fx.sessionID, ConfigTenantID: fx.configTenantID, ConfigID: "cfg-b"},
|
||||
}}
|
||||
fx.pinner.WithPlacer(placer)
|
||||
ctx := types.WithSandboxTenantID(fx.ctx, fx.sessionTenantID)
|
||||
|
||||
mgr, pin, err := resolveSandboxForExecution(ctx, fx.resolver, nil, fx.pinner,
|
||||
fx.configTenantID, fx.sessionID, "cfg-a", nil)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, SandboxPin{ConfigID: "cfg-b", TenantID: fx.configTenantID}, pin)
|
||||
require.Equal(t, fx.managerFor("cfg-b"), mgr)
|
||||
require.Empty(t, placer.calls)
|
||||
|
||||
released := placer.lookup[fx.sessionID]
|
||||
at := time.Now()
|
||||
released.ReleasedAt = &at
|
||||
placer.lookup[fx.sessionID] = released
|
||||
_, _, err = resolveSandboxForExecution(ctx, fx.resolver, nil, fx.pinner,
|
||||
fx.configTenantID, fx.sessionID, "cfg-a", nil)
|
||||
require.ErrorIs(t, err, sandbox.ErrSandboxSessionDeleted)
|
||||
require.Empty(t, placer.calls)
|
||||
}
|
||||
|
||||
func TestPinnerReadPrefersTheAssignmentsConfig(t *testing.T) {
|
||||
fx := newPinFixture(t)
|
||||
// The pin column still names the legacy owner tenant; the assignment is authoritative.
|
||||
fx.setPin(t, SandboxPin{ConfigID: "cfg-a"})
|
||||
placer := &recordingPlacer{lookup: map[string]allocation.Assignment{
|
||||
fx.sessionID: {
|
||||
SessionID: fx.sessionID, ConfigTenantID: 9, ConfigID: "cfg-a",
|
||||
Machine: allocation.UserMachine{OwnerTenantID: 7, OwnerID: "u-1", ConfigTenantID: 9, ConfigID: "cfg-a"},
|
||||
},
|
||||
}}
|
||||
fx.pinner.WithPlacer(placer)
|
||||
got, err := fx.pinner.Read(fx.ctx, fx.sessionID)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, SandboxPin{ConfigID: "cfg-a", TenantID: 9}, got)
|
||||
require.Empty(t, placer.calls, "reading never places")
|
||||
}
|
||||
|
||||
func TestLookupOnlyPathsNeverPlace(t *testing.T) {
|
||||
fx := newPinFixture(t)
|
||||
placer := &recordingPlacer{answer: func(sandboxalloc.PlaceRequest) allocation.Assignment {
|
||||
return allocation.Assignment{}
|
||||
}}
|
||||
fx.pinner.WithPlacer(placer)
|
||||
_, err := sandboxConfigForExistingSandbox(fx.ctx, fx.pinner, fx.sessionID)
|
||||
require.NoError(t, err)
|
||||
require.Empty(t, placer.calls)
|
||||
}
|
||||
|
||||
type failingSandboxResolver struct{}
|
||||
|
||||
func (failingSandboxResolver) Resolve(context.Context, uint64, string) (sandbox.Manager, error) {
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ import (
|
|||
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/Tencent/WeKnora/internal/utils"
|
||||
|
|
@ -399,6 +400,9 @@ func SanitizeSandboxConfig(
|
|||
if err := validateSkillRollout(merged.SkillRollout); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := validateAllocationPolicy(merged, existing); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return merged, nil
|
||||
}
|
||||
|
||||
|
|
@ -411,6 +415,27 @@ func validateSkillRollout(value string) error {
|
|||
}
|
||||
}
|
||||
|
||||
// validateAllocationPolicy checks allocation_mode and the retention window.
|
||||
// Switching a config TO user needs the deployment gate; a config already
|
||||
// saved as user stays saveable so unrelated edits do not fail after an
|
||||
// operator closes the gate (allocation then falls back per session).
|
||||
func validateAllocationPolicy(merged, existing *types.TenantSandboxConfig) error {
|
||||
mode, err := allocation.ParseMode(merged.AllocationMode)
|
||||
if err != nil {
|
||||
return apperrors.NewBadRequestError("invalid allocation_mode")
|
||||
}
|
||||
alreadyUser := existing != nil && strings.TrimSpace(existing.AllocationMode) == types.SandboxAllocationUser
|
||||
if mode == allocation.ModeUser && !alreadyUser && !allocation.UserModeAvailable() {
|
||||
return apperrors.NewBadRequestError(
|
||||
"allocation_mode=user is not enabled on this deployment " +
|
||||
"(set " + allocation.UserModeEnvVar + "=enabled on every replica; Redis is required)")
|
||||
}
|
||||
if err := allocation.ValidateRetentionDays(merged.UserSandboxEmptyRetentionDays); err != nil {
|
||||
return apperrors.NewBadRequestError(err.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateNamedSandboxBackend(cfg *types.TenantSandboxConfig) error {
|
||||
if cfg == nil || strings.TrimSpace(cfg.SandboxType) == "" {
|
||||
return apperrors.NewBadRequestError("sandbox backend type is required")
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import (
|
|||
|
||||
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
|
@ -2192,6 +2193,42 @@ func (deleteFileService) CopyFile(context.Context, string, uint64, string) (stri
|
|||
return "", nil
|
||||
}
|
||||
|
||||
func allocationTestConfig(mode string, days int) *types.TenantSandboxConfig {
|
||||
cfg := e2bCfg("key-a", "https://api.e2b.app", "e2b.app", "t1", 300)
|
||||
cfg.AllocationMode = mode
|
||||
cfg.UserSandboxEmptyRetentionDays = days
|
||||
return cfg
|
||||
}
|
||||
|
||||
func TestSanitizeSandboxConfigAllocationPolicy(t *testing.T) {
|
||||
// e2bCfg carries an API key. Sanitize refuses secrets when SYSTEM_AES_KEY
|
||||
// is unset, before allocation policy is checked.
|
||||
t.Setenv("SYSTEM_AES_KEY", strings.Repeat("k", 32))
|
||||
t.Cleanup(func() { allocation.SetUserModeAvailable(false) })
|
||||
|
||||
allocation.SetUserModeAvailable(false)
|
||||
_, err := SanitizeSandboxConfig(allocationTestConfig("user", 0), nil)
|
||||
require.ErrorContains(t, err, "allocation_mode=user is not enabled")
|
||||
|
||||
// A config already saved as user stays saveable after the gate closes, so
|
||||
// an unrelated edit does not fail; allocation falls back at decide time.
|
||||
_, err = SanitizeSandboxConfig(allocationTestConfig("user", 0), allocationTestConfig("user", 0))
|
||||
require.NoError(t, err)
|
||||
|
||||
allocation.SetUserModeAvailable(true)
|
||||
out, err := SanitizeSandboxConfig(allocationTestConfig("user", 14), nil)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "user", out.AllocationMode)
|
||||
require.Equal(t, 14, out.UserSandboxEmptyRetentionDays)
|
||||
|
||||
_, err = SanitizeSandboxConfig(allocationTestConfig("machine", 0), nil)
|
||||
require.ErrorContains(t, err, "invalid allocation_mode")
|
||||
_, err = SanitizeSandboxConfig(allocationTestConfig("session", 91), nil)
|
||||
require.ErrorContains(t, err, "between 1 and 90")
|
||||
_, err = SanitizeSandboxConfig(allocationTestConfig("session", -1), nil)
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
||||
var (
|
||||
_ sandbox.ConfigSandboxClient = (*snapshotReleaseClient)(nil)
|
||||
_ sandboxSnapshotReleaser = (*snapshotReleaseClient)(nil)
|
||||
|
|
|
|||
|
|
@ -18,11 +18,13 @@ import (
|
|||
|
||||
"github.com/Tencent/WeKnora/internal/agent/skills"
|
||||
"github.com/Tencent/WeKnora/internal/agent/tools"
|
||||
"github.com/Tencent/WeKnora/internal/application/service/sandboxalloc"
|
||||
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/event"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/models/chat"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
|
|
@ -1675,6 +1677,20 @@ func (s *TenantSkillService) startMaintenanceSession(
|
|||
if sess == nil {
|
||||
return nil, nil, fmt.Errorf("create %s session returned nil", operation)
|
||||
}
|
||||
if s.placer != nil {
|
||||
if _, err := s.placer.Place(types.WithSandboxTenantID(ctx, tenantID), sandboxalloc.PlaceRequest{
|
||||
SessionTenantID: tenantID, SessionID: sess.ID,
|
||||
ConfigTenantID: tenantID, ConfigID: configID, Purpose: allocation.PurposeMaintenance,
|
||||
}); err != nil {
|
||||
// The row was created for this operation only and nothing has run
|
||||
// in it; leaving it would keep a hidden, pinned session around.
|
||||
if delErr := s.sessions.DeleteSession(context.WithoutCancel(ctx), sess.ID); delErr != nil {
|
||||
logger.Warnf(ctx, "[skill] remove %s session %s after a failed placement: %v",
|
||||
operation, sess.ID, delErr)
|
||||
}
|
||||
return nil, nil, fmt.Errorf("record %s sandbox placement: %w", operation, err)
|
||||
}
|
||||
}
|
||||
return sess, mgr, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import (
|
|||
|
||||
"github.com/Tencent/WeKnora/internal/agent/tools"
|
||||
"github.com/Tencent/WeKnora/internal/application/repository"
|
||||
"github.com/Tencent/WeKnora/internal/application/service/sandboxalloc"
|
||||
"github.com/Tencent/WeKnora/internal/event"
|
||||
"github.com/Tencent/WeKnora/internal/models/asr"
|
||||
"github.com/Tencent/WeKnora/internal/models/chat"
|
||||
|
|
@ -29,6 +30,7 @@ import (
|
|||
"github.com/Tencent/WeKnora/internal/models/rerank"
|
||||
"github.com/Tencent/WeKnora/internal/models/vlm"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/google/uuid"
|
||||
|
|
@ -1903,6 +1905,28 @@ func TestStartMaintenanceSessionMarksAndScopesTheSession(t *testing.T) {
|
|||
require.Equal(t, "Skill install", sess.Title)
|
||||
}
|
||||
|
||||
type failingPlacer struct{}
|
||||
|
||||
func (failingPlacer) Place(context.Context, sandboxalloc.PlaceRequest) (allocation.Assignment, error) {
|
||||
return allocation.Assignment{}, errors.New("db unavailable")
|
||||
}
|
||||
|
||||
func (failingPlacer) Lookup(context.Context, string) (allocation.Assignment, bool, error) {
|
||||
return allocation.Assignment{}, false, nil
|
||||
}
|
||||
|
||||
// A placement failure aborts the operation and must not leave the hidden
|
||||
// maintenance session row behind.
|
||||
func TestStartMaintenanceSessionRemovesTheSessionWhenPlacementFails(t *testing.T) {
|
||||
fx := newInstallFixture(t)
|
||||
fx.svc.WithPlacer(failingPlacer{})
|
||||
ctx := context.WithValue(ctxWithTenant(7), types.UserIDContextKey, "admin-1")
|
||||
|
||||
_, _, err := fx.svc.startMaintenanceSession(ctx, 7, "cfg-1", "install")
|
||||
require.ErrorContains(t, err, "record install sandbox placement")
|
||||
require.Equal(t, []string{"CreateSession", "DeleteSession"}, fx.sessionCalls)
|
||||
}
|
||||
|
||||
const installSkillDir = "/opt/weknora/tenant/skills/pdf-tools"
|
||||
|
||||
// The install commands are asserted verbatim: an install runs as root with the
|
||||
|
|
|
|||
|
|
@ -117,6 +117,17 @@ type TenantSkillService struct {
|
|||
// nothing to cancel, and StopSkill then only rewrites the stuck row.
|
||||
runCancelMu sync.Mutex
|
||||
runCancels map[string]*skillRunCancel
|
||||
|
||||
// placer records the maintenance session's sandbox assignment. Nil keeps
|
||||
// today's behaviour: the session row carries the pin and no assignment.
|
||||
placer SessionPlacer
|
||||
}
|
||||
|
||||
// WithPlacer attaches the allocation service. Without one, maintenance
|
||||
// sessions keep today's behaviour.
|
||||
func (s *TenantSkillService) WithPlacer(p SessionPlacer) *TenantSkillService {
|
||||
s.placer = p
|
||||
return s
|
||||
}
|
||||
|
||||
// NewTenantSkillService wires the repositories and runtimes the install and
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ import (
|
|||
"github.com/Tencent/WeKnora/internal/application/service/file"
|
||||
"github.com/Tencent/WeKnora/internal/application/service/memory"
|
||||
"github.com/Tencent/WeKnora/internal/application/service/retriever"
|
||||
"github.com/Tencent/WeKnora/internal/application/service/sandboxalloc"
|
||||
"github.com/Tencent/WeKnora/internal/browserskill"
|
||||
"github.com/Tencent/WeKnora/internal/common"
|
||||
"github.com/Tencent/WeKnora/internal/config"
|
||||
|
|
@ -203,6 +204,7 @@ func BuildContainer(container *dig.Container) *dig.Container {
|
|||
must(container.Provide(repository.NewMemoryRepository))
|
||||
must(container.Provide(repository.NewTaskPendingOpsRepository))
|
||||
must(container.Provide(repository.NewTaskDeadLetterRepository))
|
||||
must(container.Provide(repository.NewSandboxAllocationRepository))
|
||||
|
||||
// MCP manager for managing MCP client connections
|
||||
logger.Debugf(ctx, "[Container] Registering MCP manager...")
|
||||
|
|
@ -546,6 +548,19 @@ func BuildContainer(container *dig.Container) *dig.Container {
|
|||
// it), but Invoke constructs the whole chain. SessionService needs
|
||||
// *chatpipeline.EventManager, which only exists after the pipeline
|
||||
// block above — starting the reaper any earlier panics.
|
||||
must(container.Provide(func(
|
||||
repo interfaces.SandboxAllocationRepository,
|
||||
configs repository.TenantSandboxConfigRepository,
|
||||
sessions interfaces.SessionRepository,
|
||||
) *sandboxalloc.Service {
|
||||
return sandboxalloc.New(repo, configs, sessions, sandboxRedisNamespace())
|
||||
}))
|
||||
must(container.Invoke(func(
|
||||
p *service.SessionSandboxPinner, skills *service.TenantSkillService, a *sandboxalloc.Service,
|
||||
) {
|
||||
p.WithPlacer(a)
|
||||
skills.WithPlacer(a)
|
||||
}))
|
||||
must(container.Invoke(startTenantSkillReaper))
|
||||
logger.Debugf(ctx, "[Container] Tenant skill reaper registered")
|
||||
must(
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import (
|
|||
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
|
@ -27,21 +28,51 @@ func newSandboxManager(
|
|||
return sandbox.NewDisabledManager()
|
||||
}
|
||||
|
||||
func selectSessionBindingStore(
|
||||
redisClient *redis.Client,
|
||||
requireRedis bool,
|
||||
) (sandbox.SessionSandboxBindingStore, string, error) {
|
||||
// sandboxRedisNamespace is the Redis key prefix shared by session bindings
|
||||
// and user-machine digests. An empty env var means the historical default.
|
||||
func sandboxRedisNamespace() string {
|
||||
namespace := strings.TrimSpace(os.Getenv("WEKNORA_REDIS_NAMESPACE"))
|
||||
if namespace == "" {
|
||||
namespace = "weknora"
|
||||
}
|
||||
return namespace
|
||||
}
|
||||
|
||||
// userModeRuntimeReady is false until the session manager places sessions by
|
||||
// their assignment (plan T10). Until then a user-mode assignment would be
|
||||
// recorded for life while the session still runs on a machine of its own in
|
||||
// /workspace, so the gate stays closed whatever the env says.
|
||||
const userModeRuntimeReady = false
|
||||
|
||||
func selectSessionBindingStore(
|
||||
redisClient *redis.Client,
|
||||
requireRedis bool,
|
||||
) (sandbox.SessionSandboxBindingStore, string, error) {
|
||||
namespace := sandboxRedisNamespace()
|
||||
requested := allocation.UserModeRequestedByEnv()
|
||||
if redisClient != nil {
|
||||
store, err := sandbox.NewRedisSessionSandboxBindingStore(redisClient, namespace)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("build redis binding store: %w", err)
|
||||
}
|
||||
allocation.SetUserModeAvailable(requested && userModeRuntimeReady)
|
||||
switch {
|
||||
case requested && !userModeRuntimeReady:
|
||||
logger.Warnf(context.Background(),
|
||||
"[sandbox] %s is set but this build cannot run user-mode sessions yet; "+
|
||||
"user-mode allocation stays disabled",
|
||||
allocation.UserModeEnvVar)
|
||||
case requested:
|
||||
logger.Infof(context.Background(), "[sandbox] user-mode allocation enabled (%s)", allocation.UserModeEnvVar)
|
||||
}
|
||||
return store, "redis", nil
|
||||
}
|
||||
allocation.SetUserModeAvailable(false)
|
||||
if requested {
|
||||
logger.Errorf(context.Background(),
|
||||
"[sandbox] %s is set but Redis is not configured; user-mode allocation stays disabled",
|
||||
allocation.UserModeEnvVar)
|
||||
}
|
||||
_ = requireRedis
|
||||
logger.Warnf(context.Background(),
|
||||
"[sandbox] No Redis configured, using in-memory binding store (single-instance)")
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import (
|
|||
"gorm.io/gorm"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
|
|
@ -40,6 +41,28 @@ func TestSelectSessionBindingStoreFallsBackToMemoryWhenNotRequired(t *testing.T)
|
|||
require.IsType(t, &sandbox.MemorySessionSandboxBindingStore{}, store)
|
||||
}
|
||||
|
||||
func TestSelectSessionBindingStoreUserModeNeedsRedisAndEnv(t *testing.T) {
|
||||
t.Cleanup(func() { allocation.SetUserModeAvailable(false) })
|
||||
|
||||
t.Setenv(allocation.UserModeEnvVar, "enabled")
|
||||
_, _, err := selectSessionBindingStore(nil, true)
|
||||
require.NoError(t, err)
|
||||
require.False(t, allocation.UserModeAvailable(), "env opt-in without Redis must leave the gate closed")
|
||||
|
||||
mini := miniredis.RunT(t)
|
||||
client := redis.NewClient(&redis.Options{Addr: mini.Addr()})
|
||||
t.Cleanup(func() { _ = client.Close() })
|
||||
_, _, err = selectSessionBindingStore(client, true)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, userModeRuntimeReady, allocation.UserModeAvailable(),
|
||||
"env opt-in with Redis opens the gate only once the manager can run user-mode sessions")
|
||||
|
||||
t.Setenv(allocation.UserModeEnvVar, "")
|
||||
_, _, err = selectSessionBindingStore(client, true)
|
||||
require.NoError(t, err)
|
||||
require.False(t, allocation.UserModeAvailable(), "Redis without the env opt-in must leave the gate closed")
|
||||
}
|
||||
|
||||
// fakeSessionLookup implements the narrow sessionExistenceLookup surface the
|
||||
// checker depends on. Keeping the surface narrow means the test double is a
|
||||
// couple of lines instead of the full 16-method SessionRepository stub.
|
||||
|
|
|
|||
|
|
@ -8,6 +8,8 @@ import (
|
|||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
sqlite3migrate "github.com/golang-migrate/migrate/v4/database/sqlite3"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
|
|
@ -43,6 +45,8 @@ var versionedSQLiteTables = []string{
|
|||
"automations",
|
||||
"automation_runs",
|
||||
"notifications",
|
||||
"session_sandbox_assignments",
|
||||
"sandbox_user_machines",
|
||||
}
|
||||
|
||||
// versionedSQLiteColumns maps each existing table to the columns that the
|
||||
|
|
@ -62,15 +66,17 @@ var versionedSQLiteColumns = map[string][]string{
|
|||
"parent_session_id", "forked_from_message_id", "fork_bootstrap", // 000097
|
||||
"sandbox_config_tenant_id", // 000027
|
||||
"host_workspace_dir", // 000029
|
||||
"owner_auth_source", // 000046
|
||||
},
|
||||
"tenant_invitations": {"token", "accepted_count"}, // 000054
|
||||
"embed_channels": {"allow_memory"}, // 000060
|
||||
"im_channels": {"locale"}, // 000030
|
||||
"chunks": {"source_locators"}, // 000033
|
||||
"mcp_oauth_tokens": {"principal_type", "principal_id"}, // 000064
|
||||
"mcp_tool_approvals": {"enabled", "risk", "observed_risk"}, // 000091, 000126
|
||||
"mcp_services": {"usage_instructions"}, // 000092
|
||||
"message_artifacts": {"deleted_at"}, // 000107
|
||||
"session_sandbox_assignments": {"released_at", "cleanup_due_at"}, // 000047
|
||||
"tenant_invitations": {"token", "accepted_count"}, // 000054
|
||||
"embed_channels": {"allow_memory"}, // 000060
|
||||
"im_channels": {"locale"}, // 000030
|
||||
"chunks": {"source_locators"}, // 000033
|
||||
"mcp_oauth_tokens": {"principal_type", "principal_id"}, // 000064
|
||||
"mcp_tool_approvals": {"enabled", "risk", "observed_risk"}, // 000091, 000126
|
||||
"mcp_services": {"usage_instructions"}, // 000092
|
||||
"message_artifacts": {"deleted_at"}, // 000107
|
||||
"tenant_skills": {
|
||||
"envs", "served", "catalog_id", "install_session_id", "install_message_id",
|
||||
}, // 000028
|
||||
|
|
@ -80,7 +86,7 @@ var versionedSQLiteColumns = map[string][]string{
|
|||
}, // 000028
|
||||
}
|
||||
|
||||
const expectedSQLiteMigrationVersion = 45
|
||||
const expectedSQLiteMigrationVersion = 47
|
||||
|
||||
func TestSQLiteMigrationsCreateVersionedSchema(t *testing.T) {
|
||||
repoRoot := sqliteRepoRoot(t)
|
||||
|
|
@ -226,6 +232,55 @@ func TestSQLiteMigrationsUpgradeV16AddsSessionForkColumns(t *testing.T) {
|
|||
require.True(t, sqliteColumnExists(t, db, "messages", "sandbox_checkpoint"))
|
||||
}
|
||||
|
||||
// The allocation migration must come back off cleanly: both tables disappear,
|
||||
// and applying it again restores them including the partial indexes.
|
||||
func TestSQLiteSandboxAllocationMigrationRollsBack(t *testing.T) {
|
||||
repoRoot := sqliteRepoRoot(t)
|
||||
chdirAndRestore(t, repoRoot)
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "down.db")
|
||||
require.NoError(t, RunMigrationsWithOptions("sqlite3://unused", MigrationOptions{SQLiteDBPath: dbPath}))
|
||||
|
||||
db := openSQLiteDB(t, dbPath)
|
||||
version, dirty := sqliteMigrationState(t, db)
|
||||
require.Equal(t, expectedSQLiteMigrationVersion, version)
|
||||
require.False(t, dirty)
|
||||
for _, table := range []string{"session_sandbox_assignments", "sandbox_user_machines"} {
|
||||
require.Truef(t, sqliteTableExists(t, db, table), "up must create %s", table)
|
||||
}
|
||||
require.True(t, sqliteIndexExists(t, db, "idx_session_sandbox_assignments_live_machine"))
|
||||
require.True(t, sqliteIndexExists(t, db, "idx_session_sandbox_assignments_cleanup_due"))
|
||||
require.True(t, sqliteIndexExists(t, db, "idx_sandbox_user_machines_empty"))
|
||||
require.NoError(t, db.Close())
|
||||
|
||||
sqlDB, err := sql.Open("sqlite3", dbPath)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = sqlDB.Close() })
|
||||
driver, err := sqlite3migrate.WithInstance(sqlDB, &sqlite3migrate.Config{})
|
||||
require.NoError(t, err)
|
||||
m, err := migrate.NewWithDatabaseInstance("file://migrations/sqlite", "sqlite3", driver)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _, _ = m.Close() })
|
||||
|
||||
require.NoError(t, m.Steps(-1))
|
||||
version, dirty = sqliteMigrationState(t, sqlDB)
|
||||
require.Equal(t, expectedSQLiteMigrationVersion-1, version)
|
||||
require.False(t, dirty)
|
||||
for _, table := range []string{"session_sandbox_assignments", "sandbox_user_machines"} {
|
||||
require.Falsef(t, sqliteTableExists(t, sqlDB, table), "down must drop %s", table)
|
||||
}
|
||||
require.True(t, sqliteColumnExists(t, sqlDB, "sessions", "owner_auth_source"),
|
||||
"rolling back the allocation tables keeps the owner source of 000034")
|
||||
|
||||
require.NoError(t, m.Steps(1))
|
||||
version, dirty = sqliteMigrationState(t, sqlDB)
|
||||
require.Equal(t, expectedSQLiteMigrationVersion, version)
|
||||
require.False(t, dirty)
|
||||
require.True(t, sqliteTableExists(t, sqlDB, "session_sandbox_assignments"))
|
||||
require.True(t, sqliteTableExists(t, sqlDB, "sandbox_user_machines"))
|
||||
require.True(t, sqliteIndexExists(t, sqlDB, "idx_session_sandbox_assignments_live_machine"))
|
||||
}
|
||||
|
||||
func sqliteRepoRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
repoRoot, err := filepath.Abs(filepath.Join("..", ".."))
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ var DeploymentCapabilityKeys = []string{
|
|||
"settings.sandbox.docker",
|
||||
"settings.sandbox.host",
|
||||
"settings.sandbox.remote",
|
||||
"settings.sandbox.user_allocation",
|
||||
}
|
||||
|
||||
// DeploymentCapability describes whether a deployment exposes a feature route.
|
||||
|
|
@ -41,20 +42,21 @@ type DeploymentCapabilitiesData struct {
|
|||
|
||||
// DeploymentFeatureAvailability mirrors injected backend handlers/services.
|
||||
type DeploymentFeatureAvailability struct {
|
||||
Organizations bool
|
||||
Agents bool
|
||||
IM bool
|
||||
Embed bool
|
||||
API bool
|
||||
MCPServer bool
|
||||
MCP bool
|
||||
WebSearch bool
|
||||
VectorStore bool
|
||||
Storage bool
|
||||
Sandbox bool
|
||||
SandboxDocker bool
|
||||
SandboxHost bool
|
||||
SandboxRemote bool
|
||||
Organizations bool
|
||||
Agents bool
|
||||
IM bool
|
||||
Embed bool
|
||||
API bool
|
||||
MCPServer bool
|
||||
MCP bool
|
||||
WebSearch bool
|
||||
VectorStore bool
|
||||
Storage bool
|
||||
Sandbox bool
|
||||
SandboxDocker bool
|
||||
SandboxHost bool
|
||||
SandboxRemote bool
|
||||
SandboxUserAllocation bool
|
||||
}
|
||||
|
||||
func supportedDeploymentCapability(supported bool) DeploymentCapability {
|
||||
|
|
@ -101,23 +103,34 @@ func BuildDeploymentCapabilities(
|
|||
sandboxRemote.Reason = "route_not_registered"
|
||||
}
|
||||
|
||||
sandboxUserAllocation := DeploymentCapability{
|
||||
Supported: sandboxRemote.Supported && available.SandboxUserAllocation,
|
||||
}
|
||||
if !sandboxUserAllocation.Supported {
|
||||
sandboxUserAllocation.Reason = "deployment_gate_off"
|
||||
if !sandboxRemote.Supported {
|
||||
sandboxUserAllocation.Reason = sandboxRemote.Reason
|
||||
}
|
||||
}
|
||||
|
||||
return DeploymentCapabilitiesData{
|
||||
Edition: edition,
|
||||
Capabilities: map[string]DeploymentCapability{
|
||||
"organizations": organizations,
|
||||
"agents": supportedDeploymentCapability(available.Agents),
|
||||
"integrations.im": supportedDeploymentCapability(available.IM),
|
||||
"integrations.embed": supportedDeploymentCapability(available.Embed),
|
||||
"integrations.api": supportedDeploymentCapability(available.API),
|
||||
"integrations.mcpserver": supportedDeploymentCapability(available.MCPServer),
|
||||
"settings.mcp": supportedDeploymentCapability(available.MCP),
|
||||
"settings.websearch": supportedDeploymentCapability(available.WebSearch),
|
||||
"settings.vectorstore": supportedDeploymentCapability(available.VectorStore),
|
||||
"settings.storage": supportedDeploymentCapability(available.Storage),
|
||||
"settings.sandbox": supportedDeploymentCapability(available.Sandbox),
|
||||
"settings.sandbox.docker": sandboxDocker,
|
||||
"settings.sandbox.host": sandboxHost,
|
||||
"settings.sandbox.remote": sandboxRemote,
|
||||
"organizations": organizations,
|
||||
"agents": supportedDeploymentCapability(available.Agents),
|
||||
"integrations.im": supportedDeploymentCapability(available.IM),
|
||||
"integrations.embed": supportedDeploymentCapability(available.Embed),
|
||||
"integrations.api": supportedDeploymentCapability(available.API),
|
||||
"integrations.mcpserver": supportedDeploymentCapability(available.MCPServer),
|
||||
"settings.mcp": supportedDeploymentCapability(available.MCP),
|
||||
"settings.websearch": supportedDeploymentCapability(available.WebSearch),
|
||||
"settings.vectorstore": supportedDeploymentCapability(available.VectorStore),
|
||||
"settings.storage": supportedDeploymentCapability(available.Storage),
|
||||
"settings.sandbox": supportedDeploymentCapability(available.Sandbox),
|
||||
"settings.sandbox.docker": sandboxDocker,
|
||||
"settings.sandbox.host": sandboxHost,
|
||||
"settings.sandbox.remote": sandboxRemote,
|
||||
"settings.sandbox.user_allocation": sandboxUserAllocation,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -128,3 +128,14 @@ func TestRemoteSandboxCapabilityIsOffOnLiteDesktop(t *testing.T) {
|
|||
none := BuildDeploymentCapabilities("standard", DeploymentFeatureAvailability{})
|
||||
require.Equal(t, "route_not_registered", none.Capabilities["settings.sandbox.remote"].Reason)
|
||||
}
|
||||
|
||||
func TestBuildDeploymentCapabilitiesUserAllocation(t *testing.T) {
|
||||
off := BuildDeploymentCapabilities("standard", DeploymentFeatureAvailability{Sandbox: true, SandboxRemote: true})
|
||||
require.False(t, off.Capabilities["settings.sandbox.user_allocation"].Supported)
|
||||
require.Equal(t, "deployment_gate_off", off.Capabilities["settings.sandbox.user_allocation"].Reason)
|
||||
|
||||
on := BuildDeploymentCapabilities("standard", DeploymentFeatureAvailability{
|
||||
Sandbox: true, SandboxRemote: true, SandboxUserAllocation: true,
|
||||
})
|
||||
require.True(t, on.Capabilities["settings.sandbox.user_allocation"].Supported)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -425,6 +425,7 @@ func (h *EmbedChannelHandler) CreateEmbedSession(c *gin.Context) {
|
|||
logger.Warnf(ctx, "failed to assign embed session owner for %s: %v", created.ID, err)
|
||||
} else {
|
||||
created.UserID = ownerID
|
||||
created.OwnerAuthSource = types.AuthSourceForBoundOwnerID(ownerID)
|
||||
}
|
||||
// Hand back a signed handle bound to this session; the widget must echo it
|
||||
// (X-Embed-Session header) on every subsequent load/chat call.
|
||||
|
|
@ -685,7 +686,9 @@ func (h *EmbedChannelHandler) ensureEmbedSession(c *gin.Context) error {
|
|||
// as shareable, credential-free URLs. Pinning it here covers both the
|
||||
// `?resource_urls=public` query parameter and a deployment-wide
|
||||
// RESOURCE_URL_MODE=public default.
|
||||
ctx = storageurl.WithForcedHandleMode(types.WithPrincipal(ctx, principal))
|
||||
ctx = storageurl.WithForcedHandleMode(types.WithPrincipalAuthSource(
|
||||
types.WithPrincipal(ctx, principal), types.AuthSourceEmbed,
|
||||
))
|
||||
c.Request = c.Request.WithContext(ctx)
|
||||
return nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ func (s *stubSessionServiceForEmbed) SetSessionOwnerID(_ context.Context, tenant
|
|||
return errors.New("session not found")
|
||||
}
|
||||
sess.UserID = ownerID
|
||||
sess.OwnerAuthSource = types.AuthSourceForBoundOwnerID(ownerID)
|
||||
return nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -432,6 +432,7 @@ func withIMIdentity(ctx context.Context, channel *IMChannel, msg *IncomingMessag
|
|||
if msg != nil {
|
||||
principalID := fmt.Sprintf("%d:%s:%s:%s", tenantID, channel.ID, msg.Platform, msg.UserID)
|
||||
ctx = types.WithPrincipal(ctx, types.Principal{Type: types.PrincipalIMUser, ID: principalID})
|
||||
ctx = types.WithPrincipalAuthSource(ctx, types.AuthSourceIM)
|
||||
}
|
||||
ctx = context.WithValue(ctx, types.TenantRoleContextKey, types.TenantRoleViewer)
|
||||
// IM bots have no live client that can complete an in-conversation MCP OAuth
|
||||
|
|
|
|||
|
|
@ -205,6 +205,7 @@ func (s *Server) resolveAskSession(
|
|||
return nil, fmt.Errorf("failed to bind session: %w", err)
|
||||
}
|
||||
created.UserID = ownerID
|
||||
created.OwnerAuthSource = types.AuthSourceForBoundOwnerID(ownerID)
|
||||
return created, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -109,6 +109,7 @@ func attachTenantlessUserContext(c *gin.Context, user *types.User) {
|
|||
User: user,
|
||||
Principal: types.Principal{Type: types.PrincipalWebUser, ID: user.ID},
|
||||
SystemAdmin: user.IsSystemAdmin,
|
||||
AuthSource: types.AuthSourceWeb,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -300,6 +301,7 @@ func authenticateJWTUser(
|
|||
Tenant: tenant,
|
||||
Role: role,
|
||||
SystemAdmin: user.IsSystemAdmin,
|
||||
AuthSource: types.AuthSourceWeb,
|
||||
})
|
||||
return true
|
||||
}
|
||||
|
|
@ -506,6 +508,7 @@ func attachPlatformAPIKeyAuthContext(c *gin.Context, key *types.TenantAPIKey) {
|
|||
FullAccess: false,
|
||||
Capabilities: key.Capabilities,
|
||||
},
|
||||
AuthSource: types.AuthSourceAPIPlatformKey,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -541,12 +544,14 @@ func attachAPIKeyAuthContext(
|
|||
|
||||
var user *types.User
|
||||
var principal types.Principal
|
||||
var authSource string
|
||||
if key != nil && key.IsPlatform() {
|
||||
// A platform key keeps one stable machine identity while selecting the
|
||||
// target workspace through X-Tenant-ID. Tenant API-principal modes and
|
||||
// tenant-owned synthetic users must not rewrite that identity.
|
||||
principal, user = platformAPIKeyIdentity(key)
|
||||
user.TenantID = tenantID
|
||||
authSource = types.AuthSourceAPIPlatformKey
|
||||
} else {
|
||||
user, err = userService.GetUserByTenantID(c.Request.Context(), tenantID)
|
||||
if err != nil || user == nil {
|
||||
|
|
@ -562,7 +567,7 @@ func attachAPIKeyAuthContext(
|
|||
}
|
||||
|
||||
var principalErr error
|
||||
principal, principalErr = resolveAPIPrincipal(c.Request.Context(), t, c.Request.Header)
|
||||
principal, authSource, principalErr = resolveAPIPrincipal(c.Request.Context(), t, c.Request.Header)
|
||||
if principalErr != nil {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": apiPrincipalAuthErrorMessage(principalErr)})
|
||||
c.Abort()
|
||||
|
|
@ -579,11 +584,12 @@ func attachAPIKeyAuthContext(
|
|||
apiKeyTenantRoleContext = types.TenantRoleOwner
|
||||
}
|
||||
session := authSession{
|
||||
User: user,
|
||||
Principal: principal,
|
||||
TenantID: tenantID,
|
||||
Tenant: t,
|
||||
Role: apiKeyTenantRoleContext,
|
||||
User: user,
|
||||
Principal: principal,
|
||||
TenantID: tenantID,
|
||||
Tenant: t,
|
||||
Role: apiKeyTenantRoleContext,
|
||||
AuthSource: authSource,
|
||||
}
|
||||
if key != nil {
|
||||
session.APIKeyScope = &types.TenantAPIKeyScope{
|
||||
|
|
@ -598,53 +604,52 @@ func attachAPIKeyAuthContext(
|
|||
applyAuthSession(c, session)
|
||||
}
|
||||
|
||||
func resolveAPIPrincipal(ctx context.Context, tenant *types.Tenant, header http.Header) (types.Principal, error) {
|
||||
func resolveAPIPrincipal(
|
||||
ctx context.Context, tenant *types.Tenant, header http.Header,
|
||||
) (types.Principal, string, error) {
|
||||
tenantID := uint64(0)
|
||||
if tenant != nil {
|
||||
tenantID = tenant.ID
|
||||
}
|
||||
fallback := types.Principal{
|
||||
Type: types.PrincipalAPITenant,
|
||||
ID: strconv.FormatUint(tenantID, 10),
|
||||
}
|
||||
fallback := types.Principal{Type: types.PrincipalAPITenant, ID: strconv.FormatUint(tenantID, 10)}
|
||||
if tenant == nil || tenantID == 0 {
|
||||
return fallback, nil
|
||||
return fallback, types.AuthSourceAPITenantKey, nil
|
||||
}
|
||||
cfg := tenant.APIPrincipalConfig
|
||||
if cfg == nil || cfg.Mode == "" || cfg.Mode == types.APIPrincipalModeTenant {
|
||||
return fallback, nil
|
||||
return fallback, types.AuthSourceAPITenantKey, nil
|
||||
}
|
||||
switch cfg.Mode {
|
||||
case types.APIPrincipalModeDirect:
|
||||
externalUserID := strings.TrimSpace(header.Get(defaultExternalUserIDHeader))
|
||||
if externalUserID == "" {
|
||||
if cfg.RequireDirectHeader {
|
||||
return types.Principal{}, errMissingDirectHeader
|
||||
return types.Principal{}, "", errMissingDirectHeader
|
||||
}
|
||||
return fallback, nil
|
||||
return fallback, types.AuthSourceAPITenantKey, nil
|
||||
}
|
||||
if err := validateExternalUserID(externalUserID); err != nil {
|
||||
return types.Principal{}, fmt.Errorf("%w: %v", errInvalidExternalUserID, err)
|
||||
return types.Principal{}, "", fmt.Errorf("%w: %v", errInvalidExternalUserID, err)
|
||||
}
|
||||
return types.Principal{
|
||||
Type: types.PrincipalAPIExternalUser,
|
||||
ID: strconv.FormatUint(tenantID, 10) + ":" + externalUserID,
|
||||
}, nil
|
||||
}, types.AuthSourceAPIDirectHeader, nil
|
||||
case types.APIPrincipalModeSignedToken:
|
||||
externalUserID, err := verifyExternalUserJWT(header.Get(defaultExternalUserTokenHeader), tenantID, cfg.HMACSecret)
|
||||
if err != nil || externalUserID == "" {
|
||||
logger.Warnf(ctx, "invalid external user token for tenant=%d: %v", tenantID, err)
|
||||
return types.Principal{}, fmt.Errorf("%w: %w", errInvalidExternalUserToken, err)
|
||||
return types.Principal{}, "", fmt.Errorf("%w: %w", errInvalidExternalUserToken, err)
|
||||
}
|
||||
if err := validateExternalUserID(externalUserID); err != nil {
|
||||
return types.Principal{}, fmt.Errorf("%w: %v", errInvalidExternalUserID, err)
|
||||
return types.Principal{}, "", fmt.Errorf("%w: %v", errInvalidExternalUserID, err)
|
||||
}
|
||||
return types.Principal{
|
||||
Type: types.PrincipalAPIExternalUser,
|
||||
ID: strconv.FormatUint(tenantID, 10) + ":" + externalUserID,
|
||||
}, nil
|
||||
}, types.AuthSourceAPISignedToken, nil
|
||||
default:
|
||||
return fallback, nil
|
||||
return fallback, types.AuthSourceAPITenantKey, nil
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ func signedExternalUserToken(t *testing.T, secret string, claims jwt.MapClaims)
|
|||
}
|
||||
|
||||
func TestResolveAPIPrincipalDefaultsToTenant(t *testing.T) {
|
||||
p, err := resolveAPIPrincipal(context.Background(), &types.Tenant{ID: 7}, http.Header{})
|
||||
p, src, err := resolveAPIPrincipal(context.Background(), &types.Tenant{ID: 7}, http.Header{})
|
||||
if err != nil {
|
||||
t.Fatalf("resolveAPIPrincipal error = %v", err)
|
||||
}
|
||||
|
|
@ -31,13 +31,16 @@ func TestResolveAPIPrincipalDefaultsToTenant(t *testing.T) {
|
|||
if p.Type != types.PrincipalAPITenant || p.ID != "7" {
|
||||
t.Fatalf("principal = %#v", p)
|
||||
}
|
||||
if src != types.AuthSourceAPITenantKey {
|
||||
t.Fatalf("source = %q", src)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveAPIPrincipalDirectHeader(t *testing.T) {
|
||||
header := http.Header{}
|
||||
header.Set("X-External-User-ID", "external-u1")
|
||||
|
||||
p, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
p, src, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeDirect,
|
||||
|
|
@ -50,6 +53,9 @@ func TestResolveAPIPrincipalDirectHeader(t *testing.T) {
|
|||
if p.Type != types.PrincipalAPIExternalUser || p.ID != "7:external-u1" {
|
||||
t.Fatalf("principal = %#v", p)
|
||||
}
|
||||
if src != types.AuthSourceAPIDirectHeader {
|
||||
t.Fatalf("source = %q", src)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveAPIPrincipalSignedToken(t *testing.T) {
|
||||
|
|
@ -62,7 +68,7 @@ func TestResolveAPIPrincipalSignedToken(t *testing.T) {
|
|||
"exp": time.Now().Add(time.Minute).Unix(),
|
||||
}))
|
||||
|
||||
p, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
p, src, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeSignedToken,
|
||||
|
|
@ -76,6 +82,9 @@ func TestResolveAPIPrincipalSignedToken(t *testing.T) {
|
|||
if p.Type != types.PrincipalAPIExternalUser || p.ID != "7:external-u1" {
|
||||
t.Fatalf("principal = %#v", p)
|
||||
}
|
||||
if src != types.AuthSourceAPISignedToken {
|
||||
t.Fatalf("source = %q", src)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveAPIPrincipalSignedTokenRejectsWrongTenant(t *testing.T) {
|
||||
|
|
@ -88,7 +97,7 @@ func TestResolveAPIPrincipalSignedTokenRejectsWrongTenant(t *testing.T) {
|
|||
"exp": time.Now().Add(time.Minute).Unix(),
|
||||
}))
|
||||
|
||||
p, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
_, _, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeSignedToken,
|
||||
|
|
@ -98,7 +107,6 @@ func TestResolveAPIPrincipalSignedTokenRejectsWrongTenant(t *testing.T) {
|
|||
if err == nil {
|
||||
t.Fatalf("resolveAPIPrincipal error = nil, want error")
|
||||
}
|
||||
_ = p
|
||||
}
|
||||
|
||||
func TestResolveAPIPrincipalSignedTokenRejectsExpired(t *testing.T) {
|
||||
|
|
@ -111,7 +119,7 @@ func TestResolveAPIPrincipalSignedTokenRejectsExpired(t *testing.T) {
|
|||
"exp": time.Now().Add(-time.Minute).Unix(),
|
||||
}))
|
||||
|
||||
p, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
_, _, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeSignedToken,
|
||||
|
|
@ -121,11 +129,10 @@ func TestResolveAPIPrincipalSignedTokenRejectsExpired(t *testing.T) {
|
|||
if err == nil {
|
||||
t.Fatalf("resolveAPIPrincipal error = nil, want error")
|
||||
}
|
||||
_ = p
|
||||
}
|
||||
|
||||
func TestResolveAPIPrincipalDirectHeaderRequired(t *testing.T) {
|
||||
_, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
_, _, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeDirect,
|
||||
|
|
@ -141,7 +148,7 @@ func TestResolveAPIPrincipalDirectHeaderRejectsInvalidUserID(t *testing.T) {
|
|||
header := http.Header{}
|
||||
header.Set("X-External-User-ID", strings.Repeat("a", maxExternalUserIDLen+1))
|
||||
|
||||
_, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
_, _, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeDirect,
|
||||
|
|
@ -162,7 +169,7 @@ func TestResolveAPIPrincipalSignedTokenRejectsLongLifetime(t *testing.T) {
|
|||
"exp": time.Now().Add(48 * time.Hour).Unix(),
|
||||
}))
|
||||
|
||||
_, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
_, _, err := resolveAPIPrincipal(context.Background(), &types.Tenant{
|
||||
ID: 7,
|
||||
APIPrincipalConfig: &types.APIPrincipalConfig{
|
||||
Mode: types.APIPrincipalModeSignedToken,
|
||||
|
|
|
|||
|
|
@ -35,6 +35,9 @@ type authSession struct {
|
|||
// Extra carries surface-specific context values (e.g. the authenticated
|
||||
// embed channel) that must be visible on both surfaces like the rest.
|
||||
Extra map[types.ContextKey]any
|
||||
// AuthSource is how Principal was verified (types.AuthSource*). Required
|
||||
// for every principal-bearing session.
|
||||
AuthSource string
|
||||
}
|
||||
|
||||
// applyAuthSession writes the session onto BOTH context surfaces of the
|
||||
|
|
@ -66,6 +69,9 @@ func applyAuthSession(c *gin.Context, s authSession) {
|
|||
}
|
||||
c.Set(types.PrincipalContextKey.String(), s.Principal)
|
||||
ctx = types.WithPrincipal(ctx, s.Principal)
|
||||
if s.AuthSource != "" {
|
||||
set(types.PrincipalAuthSourceContextKey, s.AuthSource)
|
||||
}
|
||||
if s.Role != "" {
|
||||
set(types.TenantRoleContextKey, s.Role)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -166,10 +166,11 @@ func EmbedAuth(
|
|||
Type: types.PrincipalEmbedChannel,
|
||||
ID: fmt.Sprintf("%d:%s", ch.TenantID, ch.ID),
|
||||
},
|
||||
TenantID: ch.TenantID,
|
||||
Tenant: tenant,
|
||||
Role: types.TenantRoleViewer,
|
||||
Extra: map[types.ContextKey]any{types.EmbedChannelContextKey: ch},
|
||||
TenantID: ch.TenantID,
|
||||
Tenant: tenant,
|
||||
Role: types.TenantRoleViewer,
|
||||
Extra: map[types.ContextKey]any{types.EmbedChannelContextKey: ch},
|
||||
AuthSource: types.AuthSourceEmbed,
|
||||
})
|
||||
c.Next()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -73,6 +73,7 @@ func MCPEndpointAuth(svc interfaces.MCPEndpointService, tenantSvc interfaces.Ten
|
|||
Role: types.TenantRoleViewer,
|
||||
APIKeyScope: &scope,
|
||||
Extra: map[types.ContextKey]any{types.MCPEndpointContextKey: ep},
|
||||
AuthSource: types.AuthSourceMCPEndpoint,
|
||||
})
|
||||
c.Next()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -40,10 +40,11 @@ func newMCPAuthTestEngine(svc *fakeMCPEndpointService, capture *map[string]any)
|
|||
tenantID, _ := types.TenantIDFromContext(ctx)
|
||||
principal, _ := types.PrincipalFromContext(ctx)
|
||||
*capture = map[string]any{
|
||||
"endpoint": ep,
|
||||
"scope": scope,
|
||||
"tenant": tenantID,
|
||||
"principal": principal,
|
||||
"endpoint": ep,
|
||||
"scope": scope,
|
||||
"tenant": tenantID,
|
||||
"principal": principal,
|
||||
"authSource": types.PrincipalAuthSourceFromContext(ctx),
|
||||
}
|
||||
c.Status(http.StatusNoContent)
|
||||
})
|
||||
|
|
@ -84,6 +85,9 @@ func TestMCPEndpointAuthAttachesScopeAndPrincipal(t *testing.T) {
|
|||
if principal.Type != types.PrincipalMCPEndpoint || principal.ID != "42:ep-1" {
|
||||
t.Fatalf("principal = %+v", principal)
|
||||
}
|
||||
if got, _ := captured["authSource"].(string); got != types.AuthSourceMCPEndpoint {
|
||||
t.Fatalf("auth source = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMCPEndpointAuthRejectsBadTokens(t *testing.T) {
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ package router
|
|||
import (
|
||||
"github.com/Tencent/WeKnora/internal/handler"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/allocation"
|
||||
)
|
||||
|
||||
func deploymentCapabilitiesFromRouter(params RouterParams) handler.DeploymentCapabilitiesData {
|
||||
|
|
@ -21,11 +22,12 @@ func deploymentCapabilitiesFromRouter(params RouterParams) handler.DeploymentCap
|
|||
WebSearch: params.WebSearchHandler != nil &&
|
||||
params.WebSearchProviderHandler != nil &&
|
||||
params.WebSearchCredentialsHandler != nil,
|
||||
VectorStore: params.VectorStoreHandler != nil,
|
||||
Storage: params.StorageBackendHandler != nil,
|
||||
Sandbox: params.SandboxConfigHandler != nil,
|
||||
SandboxDocker: sandbox.DockerBackendEnabled(),
|
||||
SandboxHost: params.HostSandbox.Manager != nil,
|
||||
SandboxRemote: params.SandboxConfigHandler != nil && !params.HostSandbox.Desktop,
|
||||
VectorStore: params.VectorStoreHandler != nil,
|
||||
Storage: params.StorageBackendHandler != nil,
|
||||
Sandbox: params.SandboxConfigHandler != nil,
|
||||
SandboxDocker: sandbox.DockerBackendEnabled(),
|
||||
SandboxHost: params.HostSandbox.Manager != nil,
|
||||
SandboxRemote: params.SandboxConfigHandler != nil && !params.HostSandbox.Desktop,
|
||||
SandboxUserAllocation: allocation.UserModeAvailable(),
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,262 @@
|
|||
// Package admission tracks operations in flight on a shared machine and on
|
||||
// each session of it, so a rebuild, a reclaim, a rewind or a workspace delete
|
||||
// can stop new work and wait for running work instead of cutting it off.
|
||||
//
|
||||
// Every user-mode operation holds a member in the machine scope and then in
|
||||
// its session scope. A destructive path closes the scope it destroys, then
|
||||
// requires ActiveExcept to report zero, and calls Barrier.Check before every
|
||||
// destructive step. A lease that can no longer be confirmed reports Lost; the
|
||||
// caller cancels the work it guards.
|
||||
package admission
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
// ErrClosed reports that the scope is being rebuilt, reclaimed, or held
|
||||
// exclusively.
|
||||
ErrClosed = errors.New("sandbox: not admitting new operations")
|
||||
// ErrLeaseLost reports a lease or barrier whose ownership can no longer
|
||||
// be confirmed.
|
||||
ErrLeaseLost = errors.New("sandbox: operation lease lost")
|
||||
)
|
||||
|
||||
// DefaultHoldTTL bounds a holder that died without releasing.
|
||||
const DefaultHoldTTL = 2 * time.Minute
|
||||
|
||||
// Scope names one admission domain. Tag is the machine digest, so every
|
||||
// scope of a machine shares one Redis hash tag.
|
||||
type Scope struct {
|
||||
Tag string
|
||||
Name string
|
||||
}
|
||||
|
||||
// MachineScope covers every operation on a shared machine.
|
||||
func MachineScope(digest string) Scope { return Scope{Tag: digest, Name: "machine"} }
|
||||
|
||||
// SessionScope covers one session's operations on its machine.
|
||||
func SessionScope(digest, sessionID string) Scope {
|
||||
return Scope{Tag: digest, Name: "session:" + sessionID}
|
||||
}
|
||||
|
||||
// SessionMember names one operation of sessionID: kind is "turn", "shell",
|
||||
// "file", "exec", "pty", "desktop", "fork" or "cleanup". token is unique per
|
||||
// hold — including turns, which may nest — so releasing one hold never
|
||||
// removes another.
|
||||
func SessionMember(sessionID, kind, token string) string {
|
||||
return "s:" + sessionID + ":" + kind + ":" + token
|
||||
}
|
||||
|
||||
// Signal is the loss side of a lease or a barrier.
|
||||
type Signal interface {
|
||||
// Lost is closed once ownership can no longer be confirmed: the member
|
||||
// is gone, or the local deadline passed without a confirmation.
|
||||
Lost() <-chan struct{}
|
||||
// Check returns ErrLeaseLost once Lost is closed or the deadline passed.
|
||||
// Call it before every destructive step.
|
||||
Check() error
|
||||
}
|
||||
|
||||
// Lease is one registered operation.
|
||||
type Lease interface {
|
||||
Signal
|
||||
Member() string
|
||||
// Release removes the member. Idempotent.
|
||||
Release()
|
||||
}
|
||||
|
||||
// Barrier is an exclusive close of a scope.
|
||||
type Barrier interface {
|
||||
Signal
|
||||
// Release reopens the scope. Idempotent.
|
||||
Release()
|
||||
}
|
||||
|
||||
// Bind returns a context cancelled with ErrLeaseLost as soon as any of
|
||||
// signals is lost, and a function that ends the watch and cancels the
|
||||
// context. Pass the context to every provider call the signals protect, so a
|
||||
// lost lease also stops the call in flight.
|
||||
func Bind(ctx context.Context, signals ...Signal) (context.Context, func()) {
|
||||
bound, cancel := context.WithCancelCause(ctx)
|
||||
done := make(chan struct{})
|
||||
for _, sig := range signals {
|
||||
if sig == nil {
|
||||
continue
|
||||
}
|
||||
if sig.Check() != nil {
|
||||
cancel(ErrLeaseLost)
|
||||
continue
|
||||
}
|
||||
lost := sig.Lost()
|
||||
go func() {
|
||||
select {
|
||||
case <-lost:
|
||||
cancel(ErrLeaseLost)
|
||||
case <-done:
|
||||
case <-bound.Done():
|
||||
}
|
||||
}()
|
||||
}
|
||||
var once sync.Once
|
||||
return bound, func() {
|
||||
once.Do(func() {
|
||||
close(done)
|
||||
cancel(nil)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Registry is the admission contract. Implementations are safe for
|
||||
// concurrent use.
|
||||
type Registry interface {
|
||||
Hold(ctx context.Context, scope Scope, member string, ttl time.Duration) (Lease, error)
|
||||
ActiveExcept(ctx context.Context, scope Scope, except ...string) (int, error)
|
||||
Close(ctx context.Context, scope Scope, ttl time.Duration) (Barrier, error)
|
||||
}
|
||||
|
||||
// signal is the shared Lost/deadline/Release bookkeeping of leases and
|
||||
// barriers. A zero deadline (the memory registry) never expires on its own.
|
||||
type signal struct {
|
||||
lost chan struct{}
|
||||
lostOnce sync.Once
|
||||
releaseOnce sync.Once
|
||||
release func()
|
||||
|
||||
mu sync.Mutex
|
||||
deadline time.Time
|
||||
timer *time.Timer
|
||||
// stopped is set by Release. extend and the timer callback both refuse
|
||||
// to arm another timer once it is set, so a confirmation that arrives
|
||||
// as Release runs cannot resurrect the deadline.
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func newSignal() *signal { return &signal{lost: make(chan struct{})} }
|
||||
|
||||
func (s *signal) markLost() {
|
||||
s.mu.Lock()
|
||||
s.closeLostLocked()
|
||||
s.mu.Unlock()
|
||||
}
|
||||
|
||||
func (s *signal) closeLostLocked() {
|
||||
s.lostOnce.Do(func() { close(s.lost) })
|
||||
}
|
||||
|
||||
func (s *signal) Lost() <-chan struct{} { return s.lost }
|
||||
|
||||
// extend moves the local deadline to sentAt+ttl-margin. sentAt is when the
|
||||
// confirming request was sent: Redis may have applied it any time after, so
|
||||
// counting from the reply would overstate how long the lease is still ours.
|
||||
// A lost or released signal is never extended.
|
||||
func (s *signal) extend(sentAt time.Time, ttl time.Duration) {
|
||||
d := sentAt.Add(ttl - renewMargin(ttl))
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.stopped || s.lostClosed() {
|
||||
return
|
||||
}
|
||||
if !s.deadline.IsZero() && !time.Now().Before(s.deadline) {
|
||||
s.closeLostLocked()
|
||||
return
|
||||
}
|
||||
if !d.After(s.deadline) {
|
||||
return
|
||||
}
|
||||
s.deadline = d
|
||||
s.armLocked(time.Until(d))
|
||||
}
|
||||
|
||||
// armLocked schedules markLost at the deadline. A callback that wakes early,
|
||||
// or one already in flight when the deadline moves, waits again instead of
|
||||
// closing Lost. Go's AfterFunc Reset can run an already-started callback
|
||||
// alongside the rescheduled one; the callback re-reads the deadline under
|
||||
// s.mu so a stale wake cannot drop a lease that was just confirmed.
|
||||
func (s *signal) armLocked(delay time.Duration) {
|
||||
if s.timer != nil {
|
||||
s.timer.Stop()
|
||||
}
|
||||
if delay <= 0 {
|
||||
s.closeLostLocked()
|
||||
return
|
||||
}
|
||||
s.timer = time.AfterFunc(delay, s.onDeadline)
|
||||
}
|
||||
|
||||
func (s *signal) onDeadline() {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.stopped || s.lostClosed() || s.deadline.IsZero() {
|
||||
return
|
||||
}
|
||||
if delay := time.Until(s.deadline); delay > 0 {
|
||||
// This wake is from an earlier deadline. Drop it and any timer it
|
||||
// would race with, and wait out the deadline we hold now.
|
||||
if s.timer != nil {
|
||||
s.timer.Stop()
|
||||
}
|
||||
s.timer = time.AfterFunc(delay, s.onDeadline)
|
||||
return
|
||||
}
|
||||
s.closeLostLocked()
|
||||
}
|
||||
|
||||
func (s *signal) lostClosed() bool {
|
||||
select {
|
||||
case <-s.lost:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func (s *signal) currentDeadline() time.Time {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.deadline
|
||||
}
|
||||
|
||||
func (s *signal) Release() {
|
||||
s.releaseOnce.Do(func() {
|
||||
s.mu.Lock()
|
||||
s.stopped = true
|
||||
if s.timer != nil {
|
||||
s.timer.Stop()
|
||||
s.timer = nil
|
||||
}
|
||||
s.mu.Unlock()
|
||||
if s.release != nil {
|
||||
s.release()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func (s *signal) Check() error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.lostClosed() {
|
||||
return ErrLeaseLost
|
||||
}
|
||||
if d := s.deadline; !d.IsZero() && !time.Now().Before(d) {
|
||||
s.closeLostLocked()
|
||||
return ErrLeaseLost
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type lease struct {
|
||||
*signal
|
||||
member string
|
||||
}
|
||||
|
||||
func (l *lease) Member() string { return l.member }
|
||||
|
||||
type barrier struct{ *signal }
|
||||
|
||||
// renewMargin is how long before expiry an unconfirmed lease is declared
|
||||
// lost: the holder must stop before another replica can see it as gone.
|
||||
func renewMargin(ttl time.Duration) time.Duration { return ttl / 6 }
|
||||
|
|
@ -0,0 +1,353 @@
|
|||
package admission
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/alicebob/miniredis/v2"
|
||||
"github.com/redis/go-redis/v9"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const prefix = "weknora:sandbox:user:"
|
||||
|
||||
func newRedis(t *testing.T) (*RedisRegistry, *miniredis.Miniredis) {
|
||||
t.Helper()
|
||||
mini := miniredis.RunT(t)
|
||||
client := redis.NewClient(&redis.Options{Addr: mini.Addr()})
|
||||
t.Cleanup(func() { _ = client.Close() })
|
||||
return NewRedis(client, prefix), mini
|
||||
}
|
||||
|
||||
func registries(t *testing.T) map[string]Registry {
|
||||
r, _ := newRedis(t)
|
||||
return map[string]Registry{"redis": r, "memory": NewMemory()}
|
||||
}
|
||||
|
||||
func waitClosed(t *testing.T, ch <-chan struct{}) {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-ch:
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("expected the lease to be reported lost")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegistryContract(t *testing.T) {
|
||||
for name, reg := range registries(t) {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
machine := MachineScope("d-1")
|
||||
a := SessionMember("s1", "turn", "t1")
|
||||
b := SessionMember("s1", "pty", "p1")
|
||||
c := SessionMember("s2", "shell", "x1")
|
||||
la, err := reg.Hold(ctx, machine, a, time.Minute)
|
||||
require.NoError(t, err)
|
||||
lb, err := reg.Hold(ctx, machine, b, time.Minute)
|
||||
require.NoError(t, err)
|
||||
lc, err := reg.Hold(ctx, machine, c, time.Minute)
|
||||
require.NoError(t, err)
|
||||
|
||||
n, err := reg.ActiveExcept(ctx, machine)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 3, n)
|
||||
n, err = reg.ActiveExcept(ctx, machine, a)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 2, n, "only the exact members passed are exempt; the same session's PTY still counts")
|
||||
|
||||
n, err = reg.ActiveExcept(ctx, SessionScope("d-1", "s1"))
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, n, "scopes are independent")
|
||||
|
||||
barrier, err := reg.Close(ctx, machine, time.Minute)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, barrier.Check())
|
||||
_, err = reg.Hold(ctx, machine, SessionMember("s3", "file", "f"), time.Minute)
|
||||
require.ErrorIs(t, err, ErrClosed)
|
||||
_, err = reg.Close(ctx, machine, time.Minute)
|
||||
require.ErrorIs(t, err, ErrClosed, "a second closer backs off")
|
||||
|
||||
la.Release()
|
||||
lb.Release()
|
||||
lc.Release()
|
||||
la.Release() // idempotent
|
||||
n, err = reg.ActiveExcept(ctx, machine)
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, n)
|
||||
|
||||
barrier.Release()
|
||||
l, err := reg.Hold(ctx, machine, SessionMember("s3", "file", "f2"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, SessionMember("s3", "file", "f2"), l.Member())
|
||||
l.Release()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisRenewNeverResurrectsAnExpiredMember(t *testing.T) {
|
||||
reg, mini := newRedis(t)
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d-1")
|
||||
lease, err := reg.Hold(ctx, scope, SessionMember("s", "shell", "1"), 300*time.Millisecond)
|
||||
require.NoError(t, err)
|
||||
defer lease.Release()
|
||||
|
||||
// Redis TIME jumps past the lease: the member has expired server-side.
|
||||
mini.SetTime(time.Now().Add(time.Hour))
|
||||
waitClosed(t, lease.Lost())
|
||||
// The renew loop ticks every ttl/3. Wait past one more tick so a loop
|
||||
// that re-added the member after declaring it lost cannot hide.
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
n, err := reg.ActiveExcept(ctx, scope)
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, n, "an expired member must not be re-registered by its renew loop")
|
||||
}
|
||||
|
||||
func TestRedisLeaseIsLostWhenRedisIsUnreachable(t *testing.T) {
|
||||
reg, mini := newRedis(t)
|
||||
lease, err := reg.Hold(
|
||||
context.Background(), MachineScope("d-1"), SessionMember("s", "shell", "1"), 300*time.Millisecond,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
defer lease.Release()
|
||||
mini.Close()
|
||||
waitClosed(t, lease.Lost())
|
||||
}
|
||||
|
||||
// stallHook makes every command hang until its context ends, the way a
|
||||
// Redis behind a partition does.
|
||||
type stallHook struct{ stall *atomic.Bool }
|
||||
|
||||
func (h stallHook) DialHook(next redis.DialHook) redis.DialHook { return next }
|
||||
func (h stallHook) ProcessHook(next redis.ProcessHook) redis.ProcessHook {
|
||||
return func(ctx context.Context, cmd redis.Cmder) error {
|
||||
if h.stall.Load() {
|
||||
<-ctx.Done()
|
||||
return ctx.Err()
|
||||
}
|
||||
return next(ctx, cmd)
|
||||
}
|
||||
}
|
||||
|
||||
func (h stallHook) ProcessPipelineHook(next redis.ProcessPipelineHook) redis.ProcessPipelineHook {
|
||||
return next
|
||||
}
|
||||
|
||||
func newStallingRedis(t *testing.T) (*RedisRegistry, *atomic.Bool) {
|
||||
t.Helper()
|
||||
mini := miniredis.RunT(t)
|
||||
client := redis.NewClient(&redis.Options{Addr: mini.Addr()})
|
||||
t.Cleanup(func() { _ = client.Close() })
|
||||
stall := &atomic.Bool{}
|
||||
client.AddHook(stallHook{stall: stall})
|
||||
return NewRedis(client, prefix), stall
|
||||
}
|
||||
|
||||
func TestRedisLeaseIsLostAtItsLocalDeadlineWhileRenewStalls(t *testing.T) {
|
||||
reg, stall := newStallingRedis(t)
|
||||
ttl := 600 * time.Millisecond
|
||||
lease, err := reg.Hold(context.Background(), MachineScope("d-1"), SessionMember("s", "shell", "1"), ttl)
|
||||
require.NoError(t, err)
|
||||
defer lease.Release()
|
||||
|
||||
stall.Store(true)
|
||||
stalledAt := time.Now()
|
||||
waitClosed(t, lease.Lost())
|
||||
// The last confirmed request was sent before stalledAt, so the deadline
|
||||
// is at most ttl-ttl/6 after it. Losing it only at Redis expiry (ttl) or
|
||||
// after the hung call returns would let another replica act first.
|
||||
require.Less(t, time.Since(stalledAt), ttl)
|
||||
require.ErrorIs(t, lease.Check(), ErrLeaseLost)
|
||||
}
|
||||
|
||||
func TestRedisBarrierIsLostAtItsLocalDeadlineWhileRenewStalls(t *testing.T) {
|
||||
reg, stall := newStallingRedis(t)
|
||||
ttl := 600 * time.Millisecond
|
||||
barrier, err := reg.Close(context.Background(), MachineScope("d-1"), ttl)
|
||||
require.NoError(t, err)
|
||||
defer barrier.Release()
|
||||
|
||||
stall.Store(true)
|
||||
stalledAt := time.Now()
|
||||
waitClosed(t, barrier.Lost())
|
||||
require.Less(t, time.Since(stalledAt), ttl)
|
||||
require.ErrorIs(t, barrier.Check(), ErrLeaseLost)
|
||||
}
|
||||
|
||||
func TestBindCancelsWhenAnySignalIsLost(t *testing.T) {
|
||||
reg := NewMemory()
|
||||
ctx := context.Background()
|
||||
lease, err := reg.Hold(ctx, MachineScope("d"), SessionMember("s", "shell", "1"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
barrier, err := reg.Close(ctx, SessionScope("d", "s"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
|
||||
bound, stop := Bind(ctx, lease, barrier)
|
||||
defer stop()
|
||||
require.NoError(t, bound.Err())
|
||||
reg.LoseBarrier(SessionScope("d", "s"))
|
||||
waitClosed(t, bound.Done())
|
||||
require.ErrorIs(t, context.Cause(bound), ErrLeaseLost)
|
||||
|
||||
again, stopAgain := Bind(ctx, lease)
|
||||
stopAgain()
|
||||
require.ErrorIs(t, again.Err(), context.Canceled, "stop ends the watch and the context")
|
||||
require.NotErrorIs(t, context.Cause(again), ErrLeaseLost, "stop is not a loss")
|
||||
}
|
||||
|
||||
func TestBindOnAnAlreadyLostSignalIsCancelledAtOnce(t *testing.T) {
|
||||
reg := NewMemory()
|
||||
scope := MachineScope("d")
|
||||
member := SessionMember("s", "shell", "1")
|
||||
lease, err := reg.Hold(context.Background(), scope, member, time.Minute)
|
||||
require.NoError(t, err)
|
||||
reg.ExpireMember(scope, member)
|
||||
bound, stop := Bind(context.Background(), lease)
|
||||
defer stop()
|
||||
require.ErrorIs(t, context.Cause(bound), ErrLeaseLost)
|
||||
}
|
||||
|
||||
func TestRedisBarrierRenewsAndReportsLoss(t *testing.T) {
|
||||
reg, mini := newRedis(t)
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d-1")
|
||||
barrier, err := reg.Close(ctx, scope, 300*time.Millisecond)
|
||||
require.NoError(t, err)
|
||||
defer barrier.Release()
|
||||
|
||||
// miniredis ages key TTLs only when FastForward is called, not during a
|
||||
// wall-clock sleep. Leave a short remainder of the initial TTL, wait for
|
||||
// a renew to PEXPIRE the key back to a full TTL, then jump farther than
|
||||
// that remainder. Without a renew the key is gone.
|
||||
mini.FastForward(250 * time.Millisecond)
|
||||
time.Sleep(250 * time.Millisecond) // renew interval is ttl/3 = 100ms
|
||||
mini.FastForward(100 * time.Millisecond)
|
||||
require.True(t, mini.Exists(prefix+"{d-1}:machine:closed"), "the barrier outlives its initial TTL while held")
|
||||
require.NoError(t, barrier.Check())
|
||||
|
||||
mini.Del(prefix + "{d-1}:machine:closed")
|
||||
waitClosed(t, barrier.Lost())
|
||||
require.ErrorIs(t, barrier.Check(), ErrLeaseLost)
|
||||
}
|
||||
|
||||
func TestRedisKeysShareTheMachineHashTag(t *testing.T) {
|
||||
reg, mini := newRedis(t)
|
||||
l, err := reg.Hold(context.Background(), SessionScope("abc", "s1"), SessionMember("s1", "x", "1"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
defer l.Release()
|
||||
require.True(t, mini.Exists(prefix+"{abc}:session:s1:ops"))
|
||||
}
|
||||
|
||||
func TestMemoryHooksSimulateLoss(t *testing.T) {
|
||||
reg := NewMemory()
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d")
|
||||
member := SessionMember("s", "shell", "1")
|
||||
lease, err := reg.Hold(ctx, scope, member, time.Minute)
|
||||
require.NoError(t, err)
|
||||
reg.ExpireMember(scope, member)
|
||||
waitClosed(t, lease.Lost())
|
||||
n, _ := reg.ActiveExcept(ctx, scope)
|
||||
require.Zero(t, n)
|
||||
|
||||
barrier, err := reg.Close(ctx, scope, time.Minute)
|
||||
require.NoError(t, err)
|
||||
reg.LoseBarrier(scope)
|
||||
waitClosed(t, barrier.Lost())
|
||||
require.ErrorIs(t, barrier.Check(), ErrLeaseLost)
|
||||
}
|
||||
|
||||
func TestReleaseIsIdempotentAndIsNotLoss(t *testing.T) {
|
||||
for name, reg := range registries(t) {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d")
|
||||
lease, err := reg.Hold(ctx, scope, SessionMember("s", "shell", "1"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
lease.Release()
|
||||
lease.Release()
|
||||
select {
|
||||
case <-lease.Lost():
|
||||
t.Fatal("release must not report the lease lost")
|
||||
default:
|
||||
}
|
||||
require.NoError(t, lease.Check())
|
||||
n, err := reg.ActiveExcept(ctx, scope)
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, n)
|
||||
|
||||
barrier, err := reg.Close(ctx, scope, time.Minute)
|
||||
require.NoError(t, err)
|
||||
barrier.Release()
|
||||
barrier.Release()
|
||||
select {
|
||||
case <-barrier.Lost():
|
||||
t.Fatal("release must not report the barrier lost")
|
||||
default:
|
||||
}
|
||||
require.NoError(t, barrier.Check())
|
||||
again, err := reg.Hold(ctx, scope, SessionMember("s", "shell", "2"), time.Minute)
|
||||
require.NoError(t, err)
|
||||
again.Release()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestConcurrentHoldAndRelease(t *testing.T) {
|
||||
for name, reg := range registries(t) {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d-conc")
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
member := SessionMember("s", "shell", strconv.Itoa(i))
|
||||
for n := 0; n < 15; n++ {
|
||||
lease, err := reg.Hold(ctx, scope, member, time.Minute)
|
||||
if err != nil {
|
||||
t.Errorf("hold: %v", err)
|
||||
return
|
||||
}
|
||||
if _, err := reg.ActiveExcept(ctx, scope, member); err != nil {
|
||||
t.Errorf("count: %v", err)
|
||||
}
|
||||
lease.Release()
|
||||
lease.Release()
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
n, err := reg.ActiveExcept(ctx, scope)
|
||||
require.NoError(t, err)
|
||||
require.Zero(t, n)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A short hold must not cut the key's lifetime below a longer member's
|
||||
// deadline: once the key expires, ActiveExcept would see an idle machine while
|
||||
// the long operation is still running.
|
||||
func TestRedisShortHoldNeverShortensALongerLease(t *testing.T) {
|
||||
reg, mini := newRedis(t)
|
||||
ctx := context.Background()
|
||||
scope := MachineScope("d-ttl")
|
||||
long, err := reg.Hold(ctx, scope, SessionMember("a", "turn", "1"), 30*time.Minute)
|
||||
require.NoError(t, err)
|
||||
defer long.Release()
|
||||
short, err := reg.Hold(ctx, scope, SessionMember("b", "shell", "1"), 2*time.Minute)
|
||||
require.NoError(t, err)
|
||||
short.Release()
|
||||
|
||||
ops := reg.key(scope, "ops")
|
||||
require.GreaterOrEqual(t, mini.TTL(ops), 30*time.Minute, "the key outlives its longest member")
|
||||
mini.FastForward(5 * time.Minute)
|
||||
n, err := reg.ActiveExcept(ctx, scope)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 1, n, "the long lease is still registered")
|
||||
}
|
||||
|
|
@ -0,0 +1,106 @@
|
|||
package admission
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MemoryRegistry is a process-local Registry for tests and single-process
|
||||
// deployments. User mode is never enabled without Redis, so multi-replica
|
||||
// production never uses it. Leases never expire on their own; tests force
|
||||
// loss with ExpireMember and LoseBarrier.
|
||||
type MemoryRegistry struct {
|
||||
mu sync.Mutex
|
||||
members map[Scope]map[string]*lease
|
||||
barriers map[Scope]*barrier
|
||||
}
|
||||
|
||||
// NewMemory builds an empty registry.
|
||||
func NewMemory() *MemoryRegistry {
|
||||
return &MemoryRegistry{members: map[Scope]map[string]*lease{}, barriers: map[Scope]*barrier{}}
|
||||
}
|
||||
|
||||
// Hold registers member until Release. The memory registry does not expire on its own.
|
||||
func (r *MemoryRegistry) Hold(_ context.Context, scope Scope, member string, _ time.Duration) (Lease, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if _, closed := r.barriers[scope]; closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
set := r.members[scope]
|
||||
if set == nil {
|
||||
set = map[string]*lease{}
|
||||
r.members[scope] = set
|
||||
}
|
||||
l := &lease{signal: newSignal(), member: member}
|
||||
l.release = func() {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.members[scope][member] == l {
|
||||
delete(r.members[scope], member)
|
||||
}
|
||||
}
|
||||
set[member] = l
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// ActiveExcept counts members of scope other than except.
|
||||
func (r *MemoryRegistry) ActiveExcept(_ context.Context, scope Scope, except ...string) (int, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
skip := make(map[string]struct{}, len(except))
|
||||
for _, m := range except {
|
||||
skip[m] = struct{}{}
|
||||
}
|
||||
n := 0
|
||||
for m := range r.members[scope] {
|
||||
if _, ok := skip[m]; !ok {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// Close stops new holds on scope until the barrier is released.
|
||||
func (r *MemoryRegistry) Close(_ context.Context, scope Scope, _ time.Duration) (Barrier, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if _, closed := r.barriers[scope]; closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
b := &barrier{signal: newSignal()}
|
||||
b.release = func() {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.barriers[scope] == b {
|
||||
delete(r.barriers, scope)
|
||||
}
|
||||
}
|
||||
r.barriers[scope] = b
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// ExpireMember simulates a lease that expired server-side.
|
||||
func (r *MemoryRegistry) ExpireMember(scope Scope, member string) {
|
||||
r.mu.Lock()
|
||||
l := r.members[scope][member]
|
||||
delete(r.members[scope], member)
|
||||
r.mu.Unlock()
|
||||
if l != nil {
|
||||
l.markLost()
|
||||
}
|
||||
}
|
||||
|
||||
// LoseBarrier simulates a barrier whose ownership could not be renewed.
|
||||
func (r *MemoryRegistry) LoseBarrier(scope Scope) {
|
||||
r.mu.Lock()
|
||||
b := r.barriers[scope]
|
||||
delete(r.barriers, scope)
|
||||
r.mu.Unlock()
|
||||
if b != nil {
|
||||
b.markLost()
|
||||
}
|
||||
}
|
||||
|
||||
var _ Registry = (*MemoryRegistry)(nil)
|
||||
|
|
@ -0,0 +1,203 @@
|
|||
package admission
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/common/redislock"
|
||||
"github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
const redisNowLua = "local t = redis.call('TIME'); " +
|
||||
"local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)"
|
||||
|
||||
// KEYS[1] ops zset, KEYS[2] closed flag. ARGV[1] member, ARGV[2] ttl ms.
|
||||
var holdScript = redis.NewScript(redisNowLua + `
|
||||
if redis.call('EXISTS', KEYS[2]) == 1 then return redis.error_reply('ERR_SCOPE_CLOSED') end
|
||||
redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', now)
|
||||
redis.call('ZADD', KEYS[1], now + tonumber(ARGV[2]), ARGV[1])
|
||||
-- The zset holds every member's lease; only ever extend its lifetime, or a
|
||||
-- short hold would expire the key under a longer, still-live member.
|
||||
local keep = tonumber(ARGV[2]) * 2
|
||||
if redis.call('PTTL', KEYS[1]) < keep then redis.call('PEXPIRE', KEYS[1], keep) end
|
||||
return 1
|
||||
`)
|
||||
|
||||
// Renewal only extends a member that is still live. An expired member is
|
||||
// removed and reported lost: another replica may already have acted on its
|
||||
// absence, so it must never come back. Renewing while the scope is closed is
|
||||
// allowed — it protects work the closer is waiting for.
|
||||
var renewScript = redis.NewScript(redisNowLua + `
|
||||
local score = redis.call('ZSCORE', KEYS[1], ARGV[1])
|
||||
if (not score) or tonumber(score) <= now then
|
||||
redis.call('ZREM', KEYS[1], ARGV[1])
|
||||
return 0
|
||||
end
|
||||
redis.call('ZADD', KEYS[1], 'XX', now + tonumber(ARGV[2]), ARGV[1])
|
||||
-- The zset holds every member's lease; only ever extend its lifetime, or a
|
||||
-- short hold would expire the key under a longer, still-live member.
|
||||
local keep = tonumber(ARGV[2]) * 2
|
||||
if redis.call('PTTL', KEYS[1]) < keep then redis.call('PEXPIRE', KEYS[1], keep) end
|
||||
return 1
|
||||
`)
|
||||
|
||||
// ARGV: exempt members.
|
||||
var activeExceptScript = redis.NewScript(redisNowLua + `
|
||||
redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', now)
|
||||
local members = redis.call('ZRANGEBYSCORE', KEYS[1], now, '+inf')
|
||||
local skip = {}
|
||||
for i = 1, #ARGV do skip[ARGV[i]] = true end
|
||||
local n = 0
|
||||
for _, m in ipairs(members) do
|
||||
if not skip[m] then n = n + 1 end
|
||||
end
|
||||
return n
|
||||
`)
|
||||
|
||||
// RedisRegistry is the distributed admission registry.
|
||||
type RedisRegistry struct {
|
||||
client redis.UniversalClient
|
||||
prefix string
|
||||
}
|
||||
|
||||
// NewRedis builds a registry whose keys start with keyPrefix, e.g.
|
||||
// "weknora:sandbox:user:", matching the user binding keys.
|
||||
func NewRedis(client redis.UniversalClient, keyPrefix string) *RedisRegistry {
|
||||
return &RedisRegistry{client: client, prefix: keyPrefix}
|
||||
}
|
||||
|
||||
func (r *RedisRegistry) key(scope Scope, suffix string) string {
|
||||
return r.prefix + "{" + scope.Tag + "}:" + scope.Name + ":" + suffix
|
||||
}
|
||||
|
||||
// Hold registers member until Release or until the lease can no longer be renewed.
|
||||
func (r *RedisRegistry) Hold(ctx context.Context, scope Scope, member string, ttl time.Duration) (Lease, error) {
|
||||
if ttl <= 0 {
|
||||
ttl = DefaultHoldTTL
|
||||
}
|
||||
ops, closed := r.key(scope, "ops"), r.key(scope, "closed")
|
||||
sent := time.Now()
|
||||
if err := holdScript.Run(ctx, r.client, []string{ops, closed}, member, ttl.Milliseconds()).Err(); err != nil {
|
||||
if strings.Contains(err.Error(), "ERR_SCOPE_CLOSED") {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
return nil, fmt.Errorf("admission: hold %s: %w", member, err)
|
||||
}
|
||||
l := &lease{signal: newSignal(), member: member}
|
||||
l.extend(sent, ttl)
|
||||
renewCtx, stop := context.WithCancel(context.WithoutCancel(ctx))
|
||||
l.release = func() {
|
||||
stop()
|
||||
relCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
_ = r.client.ZRem(relCtx, ops, member).Err()
|
||||
}
|
||||
go r.renewLease(renewCtx, l, ops, ttl)
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// renewLease confirms the lease every ttl/3. Each request is bounded by the
|
||||
// local deadline; a failed or hung renewal leaves the deadline where it was,
|
||||
// and the deadline timer reports the loss on time.
|
||||
func (r *RedisRegistry) renewLease(ctx context.Context, l *lease, ops string, ttl time.Duration) {
|
||||
ticker := time.NewTicker(renewInterval(ttl))
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-l.Lost():
|
||||
return
|
||||
case <-ticker.C:
|
||||
sent := time.Now()
|
||||
callCtx, cancel := context.WithDeadline(ctx, l.currentDeadline())
|
||||
n, err := renewScript.Run(callCtx, r.client, []string{ops}, l.member, ttl.Milliseconds()).Int()
|
||||
cancel()
|
||||
switch {
|
||||
case err == nil && n == 1:
|
||||
l.extend(sent, ttl)
|
||||
case err == nil:
|
||||
l.markLost()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ActiveExcept counts members of scope other than except.
|
||||
func (r *RedisRegistry) ActiveExcept(ctx context.Context, scope Scope, except ...string) (int, error) {
|
||||
args := make([]any, len(except))
|
||||
for i, m := range except {
|
||||
args[i] = m
|
||||
}
|
||||
n, err := activeExceptScript.Run(ctx, r.client, []string{r.key(scope, "ops")}, args...).Int()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("admission: count %s/%s: %w", scope.Tag, scope.Name, err)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// Close stops new holds on scope until the barrier is released.
|
||||
func (r *RedisRegistry) Close(ctx context.Context, scope Scope, ttl time.Duration) (Barrier, error) {
|
||||
token, err := redislock.NewToken()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
key := r.key(scope, "closed")
|
||||
sent := time.Now()
|
||||
ok, err := redislock.TryAcquire(ctx, r.client, key, token, ttl)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("admission: close %s/%s: %w", scope.Tag, scope.Name, err)
|
||||
}
|
||||
if !ok {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
b := &barrier{signal: newSignal()}
|
||||
b.extend(sent, ttl)
|
||||
renewCtx, stop := context.WithCancel(context.WithoutCancel(ctx))
|
||||
b.release = func() {
|
||||
stop()
|
||||
relCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
_, _ = redislock.Release(relCtx, r.client, key, token)
|
||||
}
|
||||
go func() {
|
||||
ticker := time.NewTicker(renewInterval(ttl))
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-renewCtx.Done():
|
||||
return
|
||||
case <-b.Lost():
|
||||
return
|
||||
case <-ticker.C:
|
||||
sent := time.Now()
|
||||
callCtx, cancel := context.WithDeadline(renewCtx, b.currentDeadline())
|
||||
renewed, err := redislock.Renew(callCtx, r.client, key, token, ttl)
|
||||
cancel()
|
||||
switch {
|
||||
case err == nil && renewed:
|
||||
b.extend(sent, ttl)
|
||||
case err == nil:
|
||||
b.markLost()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// renewInterval is how often a holder reconfirms. It stays inside the
|
||||
// deadline (ttl - ttl/6) so one hung renewal still loses the lease on time.
|
||||
func renewInterval(ttl time.Duration) time.Duration {
|
||||
every := ttl / 3
|
||||
if every <= 0 {
|
||||
return time.Millisecond
|
||||
}
|
||||
return every
|
||||
}
|
||||
|
||||
var _ Registry = (*RedisRegistry)(nil)
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
package allocation
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const sid = "0f8fad5b-d9cb-469f-a165-70867728950e"
|
||||
|
||||
func TestParseMode(t *testing.T) {
|
||||
for in, want := range map[string]Mode{"": ModeSession, "session": ModeSession, " user ": ModeUser} {
|
||||
got, err := ParseMode(in)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, want, got)
|
||||
}
|
||||
_, err := ParseMode("machine")
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
||||
func TestDecideMatrix(t *testing.T) {
|
||||
web := Owner{TenantID: 7, ID: "u-1", Kind: KindWebUser, Source: SourceWeb}
|
||||
ext := Owner{TenantID: 7, ID: "api_external_user:7:alice", Kind: KindAPIExternalUser, Source: SourceAPISignedToken}
|
||||
base := DecideInput{ConfigMode: ModeUser, Purpose: PurposeChat, SessionID: sid, UserModeAvailable: true}
|
||||
cases := []struct {
|
||||
name string
|
||||
mutate func(*DecideInput)
|
||||
mode Mode
|
||||
reason FallbackReason
|
||||
}{
|
||||
{"web user shares", func(in *DecideInput) { in.Owner = web }, ModeUser, ReasonNone},
|
||||
{"signed external user shares", func(in *DecideInput) { in.Owner = ext }, ModeUser, ReasonNone},
|
||||
{"direct external user shares", func(in *DecideInput) {
|
||||
in.Owner = ext
|
||||
in.Owner.Source = SourceAPIDirectHeader
|
||||
}, ModeUser, ReasonNone},
|
||||
{"session config never shares", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.ConfigMode = ModeSession
|
||||
}, ModeSession, ReasonNone},
|
||||
{"maintenance first", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.Purpose = PurposeMaintenance
|
||||
}, ModeSession, ReasonMaintenanceDedicated},
|
||||
{"gate off", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.UserModeAvailable = false
|
||||
}, ModeSession, ReasonUserModeUnavailable},
|
||||
{"tenant api key", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "api_tenant_key:7:3", Kind: KindAPITenantKey, Source: SourceAPITenantKey}
|
||||
}, ModeSession, ReasonSharedAPIKey},
|
||||
{"platform api key", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "api_platform:3", Kind: KindAPIPlatformKey, Source: SourceAPIPlatformKey}
|
||||
}, ModeSession, ReasonSharedAPIKey},
|
||||
{"mcp endpoint", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "mcp_endpoint:7:e", Kind: KindMCPEndpoint, Source: SourceMCPEndpoint}
|
||||
}, ModeSession, ReasonSharedMCPEndpoint},
|
||||
{"embed", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "embed_session:7:c:s", Kind: KindEmbed}
|
||||
}, ModeSession, ReasonChannelIdentity},
|
||||
{"im", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "x", Kind: KindIM}
|
||||
}, ModeSession, ReasonChannelIdentity},
|
||||
{"empty owner", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7}
|
||||
}, ModeSession, ReasonOwnerIdentityUnverified},
|
||||
{"web without web source", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.Owner.Source = SourceInternal
|
||||
}, ModeSession, ReasonOwnerIdentityUnverified},
|
||||
{"external with unknown source", func(in *DecideInput) {
|
||||
in.Owner = ext
|
||||
in.Owner.Source = SourceUnknown
|
||||
}, ModeSession, ReasonOwnerIdentityUnverified},
|
||||
{"unknown kind", func(in *DecideInput) {
|
||||
in.Owner = Owner{TenantID: 7, ID: "u", Source: SourceWeb}
|
||||
}, ModeSession, ReasonOwnerIdentityUnverified},
|
||||
{"owner with braces", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.Owner.ID = "u{1}"
|
||||
}, ModeSession, ReasonOwnerIdentityInvalid},
|
||||
{"owner too long", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.Owner.ID = strings.Repeat("a", 513)
|
||||
}, ModeSession, ReasonOwnerIdentityInvalid},
|
||||
{"session id not uuid", func(in *DecideInput) {
|
||||
in.Owner = web
|
||||
in.SessionID = "abc"
|
||||
}, ModeSession, ReasonSessionIDNotUUID},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
in := base
|
||||
tc.mutate(&in)
|
||||
got := Decide(in)
|
||||
require.Equal(t, tc.mode, got.Mode)
|
||||
require.Equal(t, tc.reason, got.Reason)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEligibleForUser(t *testing.T) {
|
||||
require.True(t, EligibleForUser(KindWebUser, SourceWeb))
|
||||
require.True(t, EligibleForUser(KindAPIExternalUser, SourceAPISignedToken))
|
||||
require.True(t, EligibleForUser(KindAPIExternalUser, SourceAPIDirectHeader))
|
||||
require.False(t, EligibleForUser(KindWebUser, SourceInternal))
|
||||
require.False(t, EligibleForUser(KindAPITenantKey, SourceAPITenantKey))
|
||||
require.False(t, EligibleForUser(KindUnknown, SourceWeb))
|
||||
}
|
||||
|
||||
func TestUserMachineDigestIsStableAndUnambiguous(t *testing.T) {
|
||||
m := UserMachine{OwnerTenantID: 7, OwnerID: "u-1", ConfigTenantID: 9, ConfigID: "cfg"}
|
||||
require.NoError(t, m.Validate())
|
||||
d := m.Digest("weknora")
|
||||
require.Len(t, d, 64)
|
||||
require.Equal(t, d, m.Digest("weknora"))
|
||||
require.NotEqual(t, d, m.Digest("other"))
|
||||
// Length-prefixing keeps field boundaries: these two must not collide.
|
||||
a := UserMachine{OwnerTenantID: 7, OwnerID: "ab", ConfigTenantID: 9, ConfigID: "c"}
|
||||
b := UserMachine{OwnerTenantID: 7, OwnerID: "a", ConfigTenantID: 9, ConfigID: "bc"}
|
||||
require.NotEqual(t, a.Digest("n"), b.Digest("n"))
|
||||
|
||||
require.Error(t, UserMachine{OwnerID: "u", ConfigTenantID: 9, ConfigID: "c"}.Validate())
|
||||
require.Error(t, UserMachine{OwnerTenantID: 7, ConfigTenantID: 9, ConfigID: "c"}.Validate())
|
||||
require.Error(t, UserMachine{OwnerTenantID: 7, OwnerID: "u", ConfigID: "c"}.Validate())
|
||||
require.Error(t, UserMachine{OwnerTenantID: 7, OwnerID: "u", ConfigTenantID: 9}.Validate())
|
||||
}
|
||||
|
||||
func TestAssignmentLayout(t *testing.T) {
|
||||
s := Assignment{SessionID: sid, ConfigTenantID: 9, ConfigID: "cfg"}
|
||||
require.Equal(t, ModeSession, s.Mode())
|
||||
l, err := s.Layout()
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/workspace", l.Root)
|
||||
_, ok := s.UserMachine()
|
||||
require.False(t, ok)
|
||||
|
||||
want := UserMachine{OwnerTenantID: 7, OwnerID: "u-1", ConfigTenantID: 9, ConfigID: "cfg"}
|
||||
u := Assignment{SessionID: sid, ConfigTenantID: 9, ConfigID: "cfg", Machine: want}
|
||||
require.Equal(t, ModeUser, u.Mode())
|
||||
l, err = u.Layout()
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/workspace/sessions/"+sid, l.Root)
|
||||
m, ok := u.UserMachine()
|
||||
require.True(t, ok)
|
||||
require.Equal(t, want, m)
|
||||
}
|
||||
|
||||
func TestRetention(t *testing.T) {
|
||||
require.Equal(t, 7, EffectiveRetentionDays(0))
|
||||
require.Equal(t, 30, EffectiveRetentionDays(30))
|
||||
require.NoError(t, ValidateRetentionDays(0))
|
||||
require.NoError(t, ValidateRetentionDays(90))
|
||||
require.Error(t, ValidateRetentionDays(-1))
|
||||
require.Error(t, ValidateRetentionDays(91))
|
||||
}
|
||||
|
||||
func TestGate(t *testing.T) {
|
||||
t.Cleanup(func() { SetUserModeAvailable(false) })
|
||||
require.False(t, UserModeAvailable())
|
||||
SetUserModeAvailable(true)
|
||||
require.True(t, UserModeAvailable())
|
||||
t.Setenv(UserModeEnvVar, "enabled")
|
||||
require.True(t, UserModeRequestedByEnv())
|
||||
t.Setenv(UserModeEnvVar, "nope")
|
||||
require.False(t, UserModeRequestedByEnv())
|
||||
}
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
package allocation
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
)
|
||||
|
||||
// Assignment is the persisted placement of one session. Only the config
|
||||
// address, the shared machine (if any) and the release time are stored; the
|
||||
// mode and the workspace layout follow from whether there is a machine, and
|
||||
// the session keeps both for life whatever the config switch says later.
|
||||
type Assignment struct {
|
||||
SessionID string
|
||||
ConfigTenantID uint64
|
||||
ConfigID string
|
||||
// Machine is the shared machine of a user-mode session; zero for a
|
||||
// session with a machine of its own.
|
||||
Machine UserMachine
|
||||
ReleasedAt *time.Time
|
||||
}
|
||||
|
||||
// Released reports a session whose workspace has been given back.
|
||||
func (a Assignment) Released() bool { return a.ReleasedAt != nil }
|
||||
|
||||
// Mode is ModeUser for a session on a shared machine, ModeSession otherwise.
|
||||
func (a Assignment) Mode() Mode {
|
||||
if a.Machine != (UserMachine{}) {
|
||||
return ModeUser
|
||||
}
|
||||
return ModeSession
|
||||
}
|
||||
|
||||
// IsUser reports a session on a shared machine.
|
||||
func (a Assignment) IsUser() bool { return a.Mode() == ModeUser }
|
||||
|
||||
// UserMachine returns the shared machine a user-mode session lives on.
|
||||
func (a Assignment) UserMachine() (UserMachine, bool) {
|
||||
return a.Machine, a.IsUser()
|
||||
}
|
||||
|
||||
// Layout is the workspace the session was assigned, independent of the
|
||||
// config's current switch.
|
||||
func (a Assignment) Layout() (workspace.Layout, error) {
|
||||
if a.IsUser() {
|
||||
return workspace.UserSession(a.SessionID)
|
||||
}
|
||||
return workspace.Remote(), nil
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
package allocation
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
// UserModeEnvVar opts a deployment into user-mode allocation. It must be set
|
||||
// on every replica before any config is switched to "user".
|
||||
const UserModeEnvVar = "WEKNORA_SANDBOX_USER_ALLOCATION"
|
||||
|
||||
var userModeAvailable atomic.Bool
|
||||
|
||||
// UserModeRequestedByEnv reports whether the operator asked for user mode.
|
||||
func UserModeRequestedByEnv() bool {
|
||||
switch strings.ToLower(strings.TrimSpace(os.Getenv(UserModeEnvVar))) {
|
||||
case "1", "true", "on", "enabled":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// SetUserModeAvailable is called once by the container after it knows whether
|
||||
// the deployment can coordinate shared machines (env opt-in and Redis).
|
||||
func SetUserModeAvailable(available bool) { userModeAvailable.Store(available) }
|
||||
|
||||
// UserModeAvailable reports the deployment gate.
|
||||
func UserModeAvailable() bool { return userModeAvailable.Load() }
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
package allocation
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/binary"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// UserMachine is the logical identity of one shared machine: one owner, one
|
||||
// sandbox config. It is comparable so it can key maps. ConfigID must already
|
||||
// be normalized (sandbox.NormalizeConfigID) and ConfigTenantID resolved.
|
||||
type UserMachine struct {
|
||||
OwnerTenantID uint64
|
||||
OwnerID string
|
||||
ConfigTenantID uint64
|
||||
ConfigID string
|
||||
}
|
||||
|
||||
// Validate rejects identities that cannot address a machine.
|
||||
func (m UserMachine) Validate() error {
|
||||
if m.OwnerTenantID == 0 {
|
||||
return errors.New("allocation: user machine requires an owner tenant")
|
||||
}
|
||||
if err := ValidateOwnerID(m.OwnerID); err != nil {
|
||||
return err
|
||||
}
|
||||
if m.ConfigTenantID == 0 {
|
||||
return errors.New("allocation: user machine requires a resolved config tenant")
|
||||
}
|
||||
if strings.TrimSpace(m.ConfigID) == "" {
|
||||
return errors.New("allocation: user machine requires a config id")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Digest is the hex SHA-256 of a length-prefixed canonical encoding, so no
|
||||
// two identities share a digest by field-boundary ambiguity. The namespace is
|
||||
// part of it: two deployments sharing one Redis never address each other's
|
||||
// machines. It is the Redis hash tag, the DB key and the provider metadata.
|
||||
func (m UserMachine) Digest(namespace string) string {
|
||||
h := sha256.New()
|
||||
write := func(s string) {
|
||||
var n [8]byte
|
||||
binary.BigEndian.PutUint64(n[:], uint64(len(s)))
|
||||
h.Write(n[:])
|
||||
h.Write([]byte(s))
|
||||
}
|
||||
write("weknora-user-sandbox/v1")
|
||||
write(namespace)
|
||||
write(strconv.FormatUint(m.OwnerTenantID, 10))
|
||||
write(m.OwnerID)
|
||||
write(strconv.FormatUint(m.ConfigTenantID, 10))
|
||||
write(m.ConfigID)
|
||||
return hex.EncodeToString(h.Sum(nil))
|
||||
}
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
// Package allocation is the domain model for placing a session on a remote
|
||||
// sandbox: which mode it uses, whether its owner may share a machine, and the
|
||||
// identity of a shared machine. It performs no I/O.
|
||||
package allocation
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Mode is how a session is placed on remote machines.
|
||||
type Mode string
|
||||
|
||||
const (
|
||||
// ModeSession gives the session a machine of its own (historical behaviour).
|
||||
ModeSession Mode = "session"
|
||||
// ModeUser places the session in a directory on its owner's shared machine.
|
||||
ModeUser Mode = "user"
|
||||
)
|
||||
|
||||
// ParseMode accepts the stored config value. Empty means session.
|
||||
func ParseMode(raw string) (Mode, error) {
|
||||
switch strings.TrimSpace(raw) {
|
||||
case "", string(ModeSession):
|
||||
return ModeSession, nil
|
||||
case string(ModeUser):
|
||||
return ModeUser, nil
|
||||
default:
|
||||
return "", fmt.Errorf("allocation: unknown mode %q", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// Purpose is why a session uses a sandbox.
|
||||
type Purpose string
|
||||
|
||||
// Placement purposes. PurposeChat is an ordinary session.
|
||||
// PurposeMaintenance is a skill-install session and stays on its own machine.
|
||||
const (
|
||||
PurposeChat Purpose = "chat"
|
||||
PurposeMaintenance Purpose = "maintenance"
|
||||
)
|
||||
|
||||
// FallbackReason explains why a session did not get the config's mode. The
|
||||
// values are stable, non-sensitive tokens safe to show an admin.
|
||||
type FallbackReason string
|
||||
|
||||
// Reasons a placement stayed on a per-session machine.
|
||||
const (
|
||||
ReasonNone FallbackReason = ""
|
||||
ReasonMaintenanceDedicated FallbackReason = "MAINTENANCE_DEDICATED"
|
||||
ReasonUserModeUnavailable FallbackReason = "USER_MODE_UNAVAILABLE"
|
||||
ReasonOwnerIdentityInvalid FallbackReason = "OWNER_IDENTITY_INVALID"
|
||||
ReasonSharedAPIKey FallbackReason = "SHARED_API_KEY"
|
||||
ReasonSharedMCPEndpoint FallbackReason = "SHARED_MCP_ENDPOINT"
|
||||
ReasonChannelIdentity FallbackReason = "CHANNEL_IDENTITY_NOT_SUPPORTED"
|
||||
ReasonOwnerIdentityUnverified FallbackReason = "OWNER_IDENTITY_UNVERIFIED"
|
||||
ReasonSessionIDNotUUID FallbackReason = "SESSION_ID_NOT_UUID"
|
||||
ReasonLegacySessionSandbox FallbackReason = "LEGACY_SESSION_SANDBOX"
|
||||
)
|
||||
|
|
@ -0,0 +1,142 @@
|
|||
package allocation
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
)
|
||||
|
||||
// OwnerKind classifies who sessions.user_id names.
|
||||
type OwnerKind string
|
||||
|
||||
// Kinds derived from owner_auth_source and the shape of user_id.
|
||||
const (
|
||||
KindUnknown OwnerKind = ""
|
||||
KindWebUser OwnerKind = "web_user"
|
||||
KindAPIExternalUser OwnerKind = "api_external_user"
|
||||
KindAPITenantKey OwnerKind = "api_tenant_key"
|
||||
KindAPIPlatformKey OwnerKind = "api_platform_key"
|
||||
KindMCPEndpoint OwnerKind = "mcp_endpoint"
|
||||
KindEmbed OwnerKind = "embed"
|
||||
KindIM OwnerKind = "im"
|
||||
)
|
||||
|
||||
// AuthSource is how the server authenticated the owner when the session was
|
||||
// created or its owner was first bound. It is recorded server-side only.
|
||||
type AuthSource string
|
||||
|
||||
// Sources stamped by the server. SourceUnknown is an empty or legacy row.
|
||||
const (
|
||||
SourceUnknown AuthSource = "unknown"
|
||||
SourceWeb AuthSource = "web"
|
||||
SourceAPIDirectHeader AuthSource = "api_direct_header"
|
||||
SourceAPISignedToken AuthSource = "api_signed_token"
|
||||
SourceAPITenantKey AuthSource = "api_tenant_key"
|
||||
SourceAPIPlatformKey AuthSource = "api_platform_key"
|
||||
SourceMCPEndpoint AuthSource = "mcp_endpoint"
|
||||
SourceEmbed AuthSource = "embed"
|
||||
SourceIM AuthSource = "im"
|
||||
SourceInternal AuthSource = "internal"
|
||||
)
|
||||
|
||||
// MaxOwnerIDLength matches sessions.user_id.
|
||||
const MaxOwnerIDLength = 512
|
||||
|
||||
// Owner is the persisted owner of a session as the allocator sees it.
|
||||
type Owner struct {
|
||||
TenantID uint64
|
||||
ID string
|
||||
Kind OwnerKind
|
||||
Source AuthSource
|
||||
}
|
||||
|
||||
// ValidateOwnerID rejects ids that cannot be part of a machine identity.
|
||||
func ValidateOwnerID(id string) error {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return errors.New("allocation: owner id is empty")
|
||||
}
|
||||
if len(id) > MaxOwnerIDLength {
|
||||
return errors.New("allocation: owner id is too long")
|
||||
}
|
||||
if strings.ContainsAny(id, "{}") {
|
||||
return errors.New("allocation: owner id must not contain braces")
|
||||
}
|
||||
for _, r := range id {
|
||||
if unicode.IsControl(r) {
|
||||
return errors.New("allocation: owner id must not contain control characters")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DecideInput is everything Decide needs; callers load it, Decide is pure.
|
||||
type DecideInput struct {
|
||||
ConfigMode Mode
|
||||
Purpose Purpose
|
||||
Owner Owner
|
||||
SessionID string
|
||||
UserModeAvailable bool
|
||||
}
|
||||
|
||||
// Decision is the mode a session gets and why it differs from the config.
|
||||
type Decision struct {
|
||||
Mode Mode
|
||||
Reason FallbackReason
|
||||
}
|
||||
|
||||
// Decide applies the user-mode admission whitelist. Only a server-verified web
|
||||
// account or API external user may share a machine; every other owner keeps a
|
||||
// machine per session. Checks run in a fixed order so the recorded reason is
|
||||
// deterministic: maintenance, deployment gate, invalid owner, shared entry or
|
||||
// channel, unverified identity, session id shape.
|
||||
func Decide(in DecideInput) Decision {
|
||||
session := func(reason FallbackReason) Decision { return Decision{Mode: ModeSession, Reason: reason} }
|
||||
if in.Purpose == PurposeMaintenance {
|
||||
return session(ReasonMaintenanceDedicated)
|
||||
}
|
||||
if in.ConfigMode != ModeUser {
|
||||
return session(ReasonNone)
|
||||
}
|
||||
if !in.UserModeAvailable {
|
||||
return session(ReasonUserModeUnavailable)
|
||||
}
|
||||
if strings.TrimSpace(in.Owner.ID) == "" {
|
||||
return session(ReasonOwnerIdentityUnverified)
|
||||
}
|
||||
if ValidateOwnerID(in.Owner.ID) != nil {
|
||||
return session(ReasonOwnerIdentityInvalid)
|
||||
}
|
||||
if EligibleForUser(in.Owner.Kind, in.Owner.Source) {
|
||||
if !workspace.IsCanonicalUUID(in.SessionID) {
|
||||
return session(ReasonSessionIDNotUUID)
|
||||
}
|
||||
return Decision{Mode: ModeUser}
|
||||
}
|
||||
switch in.Owner.Kind {
|
||||
case KindAPITenantKey, KindAPIPlatformKey:
|
||||
return session(ReasonSharedAPIKey)
|
||||
case KindMCPEndpoint:
|
||||
return session(ReasonSharedMCPEndpoint)
|
||||
case KindEmbed, KindIM:
|
||||
return session(ReasonChannelIdentity)
|
||||
default:
|
||||
return session(ReasonOwnerIdentityUnverified)
|
||||
}
|
||||
}
|
||||
|
||||
// EligibleForUser is the whitelist itself: a server-verified web account, or
|
||||
// an API external user whose subject came from the direct header or a signed
|
||||
// token. Decide applies it at first use; the manager re-applies it to stored
|
||||
// assignments so a row whose identity is no longer acceptable stops running.
|
||||
func EligibleForUser(kind OwnerKind, source AuthSource) bool {
|
||||
switch kind {
|
||||
case KindWebUser:
|
||||
return source == SourceWeb
|
||||
case KindAPIExternalUser:
|
||||
return source == SourceAPIDirectHeader || source == SourceAPISignedToken
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
package allocation
|
||||
|
||||
import "fmt"
|
||||
|
||||
const (
|
||||
// DefaultEmptyRetentionDays is how long a user machine with no live
|
||||
// session is kept when the config leaves the value unset.
|
||||
DefaultEmptyRetentionDays = 7
|
||||
// MaxEmptyRetentionDays bounds the setting; there is no "keep forever".
|
||||
MaxEmptyRetentionDays = 90
|
||||
)
|
||||
|
||||
// EffectiveRetentionDays maps the stored value (0 = unset) onto days.
|
||||
func EffectiveRetentionDays(stored int) int {
|
||||
if stored <= 0 {
|
||||
return DefaultEmptyRetentionDays
|
||||
}
|
||||
return stored
|
||||
}
|
||||
|
||||
// ValidateRetentionDays accepts 0 (use the default) through 90.
|
||||
func ValidateRetentionDays(stored int) error {
|
||||
if stored < 0 || stored > MaxEmptyRetentionDays {
|
||||
return fmt.Errorf("user_sandbox_empty_retention_days must be between 1 and %d, or 0 for the default of %d",
|
||||
MaxEmptyRetentionDays, DefaultEmptyRetentionDays)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
|
@ -34,18 +34,19 @@ import (
|
|||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// SessionInputRoot is reserved for durable user attachments restored from
|
||||
// file storage. Generated artifacts must remain under SessionOutputRoot.
|
||||
const SessionInputRoot = "/workspace/input"
|
||||
const SessionInputRoot = workspace.InputDir
|
||||
|
||||
// SessionOutputRoot is where skill scripts write artifacts for collection.
|
||||
// The skills manager injects this path via skillOutputEnvVar; Execute
|
||||
// materialises the directory via envd before the script runs so scripts
|
||||
// do not depend on the template user being able to mkdir under /workspace.
|
||||
const SessionOutputRoot = "/workspace/output"
|
||||
const SessionOutputRoot = workspace.OutputDir
|
||||
|
||||
// skillOutputEnvVar matches the skills manager's WEKNORA_SKILL_OUTPUT_DIR.
|
||||
const skillOutputEnvVar = "WEKNORA_SKILL_OUTPUT_DIR"
|
||||
|
|
@ -58,7 +59,7 @@ const sessionInputEnvVar = "WEKNORA_SESSION_INPUT_DIR"
|
|||
|
||||
// SessionWorkspaceRoot is the writable workspace root inside remote sandboxes.
|
||||
// shell_exec work_dir must stay underneath this path.
|
||||
const SessionWorkspaceRoot = "/workspace"
|
||||
const SessionWorkspaceRoot = workspace.Root
|
||||
|
||||
// SessionGitDir is the git metadata directory for per-turn workspace
|
||||
// checkpoints. It lives on the sandbox root filesystem so a fork snapshot
|
||||
|
|
@ -70,7 +71,7 @@ const SessionWorkspaceRoot = "/workspace"
|
|||
// provisioned before this constant existed still hold theirs there. The
|
||||
// shared git preamble adopts that repository on first use so SHAs recorded
|
||||
// before the move keep resolving; see gitWorkspaceAdoptLegacyRepo.
|
||||
const SessionGitDir = "/var/lib/weknora/workspace.git"
|
||||
const SessionGitDir = workspace.GitDir
|
||||
|
||||
// sessionArtifactDirBootstrapTimeout bounds directory creation and access
|
||||
// checks, performed with the execution identity.
|
||||
|
|
@ -1503,58 +1504,20 @@ func (m *SessionBoundManager) requireRemoteBackend() error {
|
|||
}
|
||||
|
||||
func cleanSessionInputPath(filePath string) (string, error) {
|
||||
clean := path.Clean(strings.TrimSpace(filePath))
|
||||
if clean == SessionInputRoot || strings.HasPrefix(clean, SessionInputRoot+"/") {
|
||||
return clean, nil
|
||||
}
|
||||
return "", fmt.Errorf(
|
||||
"sandbox: session input path %q is outside %s",
|
||||
filePath, SessionInputRoot,
|
||||
)
|
||||
return workspace.CleanInputPathIn(RemoteWorkspaceLayout(), filePath)
|
||||
}
|
||||
|
||||
// cleanSessionWorkspaceWritePath normalizes model-authored sandbox writes and
|
||||
// protects staged attachments. The remote session binding isolates the files.
|
||||
func cleanSessionWorkspaceWritePath(filePath string) (string, error) {
|
||||
clean := ResolveWorkspacePathIn(RemoteWorkspaceLayout(), filePath)
|
||||
if !path.IsAbs(clean) || clean == "." || clean == "/" {
|
||||
return "", fmt.Errorf("sandbox: workspace write path %q must be an absolute file path", filePath)
|
||||
}
|
||||
if clean == SessionWorkspaceRoot || clean == SessionOutputRoot || clean == SessionInputRoot {
|
||||
return "", fmt.Errorf("sandbox: workspace write path %q is a directory, not a file", filePath)
|
||||
}
|
||||
if strings.HasPrefix(clean, SessionInputRoot+"/") {
|
||||
return "", fmt.Errorf("sandbox: session input %s is read-only", SessionInputRoot)
|
||||
}
|
||||
return clean, nil
|
||||
return workspace.CleanWriteFileIn(RemoteWorkspaceLayout(), filePath)
|
||||
}
|
||||
|
||||
// cleanSessionWorkDir requires an absolute sandbox-local working directory.
|
||||
// Ordinary calls may work anywhere; the install/maintenance option retains its
|
||||
// explicit workspace/skills scope. Neither check changes filesystem privileges.
|
||||
func cleanSessionWorkDir(workDir string, allowSkillsRoot bool) (string, error) {
|
||||
clean := path.Clean(strings.TrimSpace(workDir))
|
||||
if !path.IsAbs(clean) {
|
||||
return "", fmt.Errorf("sandbox: work dir %q must be absolute", workDir)
|
||||
}
|
||||
if !allowSkillsRoot {
|
||||
return clean, nil
|
||||
}
|
||||
if clean == SessionWorkspaceRoot || strings.HasPrefix(clean, SessionWorkspaceRoot+"/") {
|
||||
return clean, nil
|
||||
}
|
||||
if allowSkillsRoot &&
|
||||
(clean == SkillsImageRoot || strings.HasPrefix(clean, SkillsImageRoot+"/")) {
|
||||
return clean, nil
|
||||
}
|
||||
allowed := SessionWorkspaceRoot
|
||||
if allowSkillsRoot {
|
||||
allowed = SessionWorkspaceRoot + ", " + SkillsImageRoot
|
||||
}
|
||||
return "", fmt.Errorf(
|
||||
"sandbox: work dir %q is outside allowed roots (%s)",
|
||||
workDir, allowed,
|
||||
)
|
||||
return workspace.CleanWorkDirIn(RemoteWorkspaceLayout(), workDir, allowSkillsRoot)
|
||||
}
|
||||
|
||||
// buildSessionCreateRequest projects Config into a provider-neutral remote
|
||||
|
|
|
|||
|
|
@ -5,13 +5,15 @@ import (
|
|||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
)
|
||||
|
||||
const (
|
||||
// SkillsImageRoot is where installed skills live inside the snapshot image.
|
||||
// SkillsImageRoot is where installed skills live inside the sandbox image.
|
||||
// It is outside /workspace on purpose: /workspace is per-session scratch
|
||||
// and is wiped before every snapshot.
|
||||
SkillsImageRoot = "/opt/weknora/tenant/skills"
|
||||
SkillsImageRoot = workspace.SkillsImageRoot
|
||||
|
||||
// SkillsManifestPath lists what the image claims to contain. It is a
|
||||
// troubleshooting aid, never the source of truth for execution.
|
||||
|
|
@ -95,20 +97,14 @@ func SkillRequirementsPathIn(skillDir string) string {
|
|||
|
||||
// RunnableWorkspaceScript reports whether scriptPath is a session-writable
|
||||
// file under /workspace that may be executed with an installed skill's
|
||||
// interpreter. /workspace/input is reserved for attachments and is excluded;
|
||||
// so are the workspace directory roots themselves.
|
||||
// interpreter. See RunnableWorkspaceScriptIn for shared machines.
|
||||
func RunnableWorkspaceScript(scriptPath string) (string, bool) {
|
||||
clean := path.Clean(strings.TrimSpace(scriptPath))
|
||||
if clean == SessionWorkspaceRoot || clean == SessionOutputRoot || clean == SessionInputRoot {
|
||||
return "", false
|
||||
}
|
||||
if !strings.HasPrefix(clean, SessionWorkspaceRoot+"/") {
|
||||
return "", false
|
||||
}
|
||||
if strings.HasPrefix(clean, SessionInputRoot+"/") {
|
||||
return "", false
|
||||
}
|
||||
return clean, true
|
||||
return workspace.RunnableScriptIn(RemoteWorkspaceLayout(), scriptPath)
|
||||
}
|
||||
|
||||
// RunnableWorkspaceScriptIn is RunnableWorkspaceScript for an explicit layout.
|
||||
func RunnableWorkspaceScriptIn(l WorkspaceLayout, scriptPath string) (string, bool) {
|
||||
return workspace.RunnableScriptIn(l, scriptPath)
|
||||
}
|
||||
|
||||
// ValidatedSessionOutputDir normalises a configured artifact directory and
|
||||
|
|
@ -126,11 +122,12 @@ func RunnableWorkspaceScript(scriptPath string) (string, bool) {
|
|||
// included — and callers that compare the two (artifact collection) read
|
||||
// that as "this backend collects nothing", silently dropping every artifact.
|
||||
func ValidatedSessionOutputDir(dir string) (string, bool) {
|
||||
clean := path.Clean(strings.TrimSpace(dir))
|
||||
if !strings.HasPrefix(clean, SessionWorkspaceRoot+"/") {
|
||||
return "", false
|
||||
}
|
||||
return clean, true
|
||||
return workspace.ValidOutputDirIn(RemoteWorkspaceLayout(), dir)
|
||||
}
|
||||
|
||||
// ValidatedOutputDirIn is ValidatedSessionOutputDir for an explicit layout.
|
||||
func ValidatedOutputDirIn(l WorkspaceLayout, dir string) (string, bool) {
|
||||
return workspace.ValidOutputDirIn(l, dir)
|
||||
}
|
||||
|
||||
// ValidatedImageSkillDir reports whether skillDir is exactly one installed
|
||||
|
|
|
|||
|
|
@ -0,0 +1,214 @@
|
|||
// Package workspace describes where one session's files live inside a remote
|
||||
// sandbox and the lexical path rules the manager and the agent tools share.
|
||||
//
|
||||
// It is a leaf package. internal/sandbox re-exports Layout as
|
||||
// sandbox.WorkspaceLayout, so this package must never import it back.
|
||||
package workspace
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Fixed paths inside every remote sandbox image.
|
||||
const (
|
||||
Root = "/workspace"
|
||||
InputDir = "/workspace/input"
|
||||
OutputDir = "/workspace/output"
|
||||
GitDir = "/var/lib/weknora/workspace.git"
|
||||
SkillsImageRoot = "/opt/weknora/tenant/skills"
|
||||
TmpRoot = "/tmp"
|
||||
|
||||
// UserSessionsRoot holds one directory per session on a machine that
|
||||
// several sessions of one owner share; UserGitRoot holds their checkpoint
|
||||
// repositories, outside the work trees for the same reason GitDir is.
|
||||
UserSessionsRoot = Root + "/sessions"
|
||||
UserGitRoot = "/var/lib/weknora/sessions"
|
||||
)
|
||||
|
||||
// Origin says what kind of workspace a layout describes. Isolation policy
|
||||
// (work_dir clamping, inspect enforcement, prompt copy) follows Origin, never
|
||||
// the Root string: a host project can itself be /workspace.
|
||||
type Origin int
|
||||
|
||||
const (
|
||||
// OriginUnspecified is the zero value. Tools treat it as remote.
|
||||
OriginUnspecified Origin = iota
|
||||
// OriginRemote is a disposable container one session owns outright.
|
||||
OriginRemote
|
||||
// OriginHost is a directory on the user's machine.
|
||||
OriginHost
|
||||
// OriginRemoteShared is a remote machine shared by several sessions of
|
||||
// one owner. Tools must stay inside the layout's roots.
|
||||
OriginRemoteShared
|
||||
)
|
||||
|
||||
// Layout describes the shape of one session's workspace. It is pure data, so
|
||||
// remote managers and local adapters can both produce it and the tools can
|
||||
// consume it for prompts and tool-layer scope.
|
||||
//
|
||||
// It is not a symlink-safe privilege boundary. A host SessionFileStore must
|
||||
// still enforce the OS-sandbox PathGuard on every read and write. Empty
|
||||
// InputDir / OutputDir is the host shape: there is no separate attachment or
|
||||
// collect tree, and scanning Root would upload the user's project.
|
||||
type Layout struct {
|
||||
// Origin selects remote vs host vs shared-remote policy.
|
||||
Origin Origin
|
||||
// Root anchors relative paths and is the shell's working directory.
|
||||
Root string
|
||||
// WriteRoots are the directories write/edit tools may create files under.
|
||||
// A root itself is a directory, not a writable file path.
|
||||
WriteRoots []string
|
||||
// ReadRoots are the directories read/list tools may inspect, ordered
|
||||
// most-specific-first so the reported root names the narrowest match.
|
||||
ReadRoots []string
|
||||
// InputDir holds read-only user attachments. Empty when the backend has none.
|
||||
InputDir string
|
||||
// OutputDir is what artifact collection sweeps. Empty when it collects none.
|
||||
OutputDir string
|
||||
// Hint is the wording tools put in descriptions and scope errors.
|
||||
Hint string
|
||||
// GitDir is the checkpoint repository for Root. Empty when the backend
|
||||
// does not version the workspace (host).
|
||||
GitDir string
|
||||
// WorkDirRoots limits shell work_dir on a shared machine. Empty means
|
||||
// unrestricted for remote layouts; host layouts clamp to WriteRoots.
|
||||
WorkDirRoots []string
|
||||
}
|
||||
|
||||
// IsHost reports a local OS workspace. Host adapters must set Origin; a
|
||||
// non-empty Root that is not /workspace is not enough (Gitpod uses /workspace).
|
||||
func (l Layout) IsHost() bool { return l.Origin == OriginHost }
|
||||
|
||||
// IsSharedRemote reports a remote machine shared by several sessions.
|
||||
func (l Layout) IsSharedRemote() bool { return l.Origin == OriginRemoteShared }
|
||||
|
||||
// HasRoot reports a usable workspace root.
|
||||
func (l Layout) HasRoot() bool { return strings.TrimSpace(l.Root) != "" }
|
||||
|
||||
// Normalized cleans every path in the layout and drops empty roots.
|
||||
//
|
||||
// Scope checks compare cleaned paths against these entries verbatim, so an
|
||||
// adapter that hands back a trailing slash or an uncleaned path would silently
|
||||
// deny every write and work_dir inside its own workspace.
|
||||
func (l Layout) Normalized() Layout {
|
||||
l.Root = cleanLayoutPath(l.Root)
|
||||
l.InputDir = cleanLayoutPath(l.InputDir)
|
||||
l.OutputDir = cleanLayoutPath(l.OutputDir)
|
||||
l.GitDir = cleanLayoutPath(l.GitDir)
|
||||
l.WriteRoots = cleanLayoutRoots(l.WriteRoots)
|
||||
l.ReadRoots = cleanLayoutRoots(l.ReadRoots)
|
||||
l.WorkDirRoots = cleanLayoutRoots(l.WorkDirRoots)
|
||||
l.Hint = strings.TrimSpace(l.Hint)
|
||||
return l
|
||||
}
|
||||
|
||||
// Remote is the /workspace layout of a machine one session owns.
|
||||
func Remote() Layout {
|
||||
return Layout{
|
||||
Origin: OriginRemote,
|
||||
Root: Root,
|
||||
WriteRoots: []string{Root, OutputDir},
|
||||
ReadRoots: []string{OutputDir, InputDir, Root},
|
||||
InputDir: InputDir,
|
||||
OutputDir: OutputDir,
|
||||
Hint: Root,
|
||||
GitDir: GitDir,
|
||||
}
|
||||
}
|
||||
|
||||
// UserSession is the layout of sessionID on a machine its owner's sessions
|
||||
// share. Only canonical lower-case UUIDs are accepted, so the id can never
|
||||
// carry a path segment of its own.
|
||||
func UserSession(sessionID string) (Layout, error) {
|
||||
if !IsCanonicalUUID(sessionID) {
|
||||
return Layout{}, fmt.Errorf("workspace: session id %q is not a canonical UUID", sessionID)
|
||||
}
|
||||
root := path.Join(UserSessionsRoot, sessionID)
|
||||
input := path.Join(root, "input")
|
||||
output := path.Join(root, "output")
|
||||
return Layout{
|
||||
Origin: OriginRemoteShared,
|
||||
Root: root,
|
||||
WriteRoots: []string{root, output},
|
||||
ReadRoots: []string{output, input, root, SkillsImageRoot},
|
||||
InputDir: input,
|
||||
OutputDir: output,
|
||||
Hint: root,
|
||||
GitDir: path.Join(UserGitRoot, sessionID+".git"),
|
||||
WorkDirRoots: []string{root, TmpRoot, SkillsImageRoot},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// FailedHost is what tools and prompts use when a layout provider exists but
|
||||
// errors: do not fall back to /workspace.
|
||||
func FailedHost() Layout { return Layout{Origin: OriginHost} }
|
||||
|
||||
// IsCanonicalUUID reports the 36-character lower-case form uuid.New().String()
|
||||
// produces.
|
||||
func IsCanonicalUUID(s string) bool {
|
||||
if len(s) != 36 {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < len(s); i++ {
|
||||
c := s[i]
|
||||
switch i {
|
||||
case 8, 13, 18, 23:
|
||||
if c != '-' {
|
||||
return false
|
||||
}
|
||||
default:
|
||||
if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// PromptSafePath returns p when it can be embedded verbatim in prompt text,
|
||||
// and "" when it cannot. Callers fall back to generic wording instead of
|
||||
// sanitizing, so a real path is never shown wrong.
|
||||
func PromptSafePath(p string) string {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
return ""
|
||||
}
|
||||
for _, r := range p {
|
||||
if r < 0x20 || r == 0x7f || r == '<' || r == '>' || r == '&' {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func cleanLayoutPath(p string) string {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
return ""
|
||||
}
|
||||
return path.Clean(p)
|
||||
}
|
||||
|
||||
// cleanLayoutRoots preserves order (ReadRoots is most-specific-first) and
|
||||
// drops duplicates that only differed before cleaning.
|
||||
func cleanLayoutRoots(roots []string) []string {
|
||||
if len(roots) == 0 {
|
||||
return roots
|
||||
}
|
||||
out := make([]string, 0, len(roots))
|
||||
seen := make(map[string]struct{}, len(roots))
|
||||
for _, root := range roots {
|
||||
clean := cleanLayoutPath(root)
|
||||
if clean == "" {
|
||||
continue
|
||||
}
|
||||
if _, dup := seen[clean]; dup {
|
||||
continue
|
||||
}
|
||||
seen[clean] = struct{}{}
|
||||
out = append(out, clean)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
package workspace
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const testSessionID = "0f8fad5b-d9cb-469f-a165-70867728950e"
|
||||
|
||||
func TestRemoteLayoutKeepsTheHistoricalContract(t *testing.T) {
|
||||
l := Remote()
|
||||
require.Equal(t, OriginRemote, l.Origin)
|
||||
require.Equal(t, "/workspace", l.Root)
|
||||
require.Equal(t, []string{"/workspace", "/workspace/output"}, l.WriteRoots)
|
||||
require.Equal(t, []string{"/workspace/output", "/workspace/input", "/workspace"}, l.ReadRoots)
|
||||
require.Equal(t, "/workspace/input", l.InputDir)
|
||||
require.Equal(t, "/workspace/output", l.OutputDir)
|
||||
require.Equal(t, "/workspace", l.Hint)
|
||||
require.Equal(t, "/var/lib/weknora/workspace.git", l.GitDir)
|
||||
require.Empty(t, l.WorkDirRoots, "a session-owned machine keeps unrestricted work_dir")
|
||||
require.False(t, l.IsSharedRemote())
|
||||
}
|
||||
|
||||
func TestUserSessionLayoutScopesEveryPathToTheSession(t *testing.T) {
|
||||
l, err := UserSession(testSessionID)
|
||||
require.NoError(t, err)
|
||||
root := "/workspace/sessions/" + testSessionID
|
||||
require.Equal(t, OriginRemoteShared, l.Origin)
|
||||
require.True(t, l.IsSharedRemote())
|
||||
require.False(t, l.IsHost())
|
||||
require.Equal(t, root, l.Root)
|
||||
require.Equal(t, root+"/input", l.InputDir)
|
||||
require.Equal(t, root+"/output", l.OutputDir)
|
||||
require.Equal(t, []string{root, root + "/output"}, l.WriteRoots)
|
||||
require.Equal(t, []string{root + "/output", root + "/input", root, SkillsImageRoot}, l.ReadRoots)
|
||||
require.Equal(t, []string{root, TmpRoot, SkillsImageRoot}, l.WorkDirRoots)
|
||||
require.Equal(t, "/var/lib/weknora/sessions/"+testSessionID+".git", l.GitDir)
|
||||
require.Equal(t, root, l.Hint)
|
||||
}
|
||||
|
||||
func TestUserSessionRejectsNonCanonicalIDs(t *testing.T) {
|
||||
for _, id := range []string{
|
||||
"", "abc", "../etc", testSessionID + "x",
|
||||
"0F8FAD5B-D9CB-469F-A165-70867728950E", // upper case is not what uuid.New().String() emits
|
||||
"0f8fad5b_d9cb_469f_a165_70867728950e",
|
||||
} {
|
||||
_, err := UserSession(id)
|
||||
require.Error(t, err, id)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizedCleansNewFields(t *testing.T) {
|
||||
l := Layout{GitDir: " /var/lib/x.git/ ", WorkDirRoots: []string{"/tmp/", "/tmp", ""}}.Normalized()
|
||||
require.Equal(t, "/var/lib/x.git", l.GitDir)
|
||||
require.Equal(t, []string{"/tmp"}, l.WorkDirRoots)
|
||||
}
|
||||
|
|
@ -0,0 +1,123 @@
|
|||
package workspace
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ResolvePathIn anchors a relative path at the layout root.
|
||||
func ResolvePathIn(l Layout, value string) string {
|
||||
value = strings.TrimSpace(value)
|
||||
if !path.IsAbs(value) {
|
||||
root := l.Root
|
||||
if root == "" {
|
||||
root = Root
|
||||
}
|
||||
return path.Join(root, value)
|
||||
}
|
||||
return path.Clean(value)
|
||||
}
|
||||
|
||||
// IsUnder reports whether p is root or lies beneath it. Both must be clean.
|
||||
func IsUnder(p, root string) bool {
|
||||
if root == "/" {
|
||||
return strings.HasPrefix(p, "/")
|
||||
}
|
||||
return p == root || strings.HasPrefix(p, root+"/")
|
||||
}
|
||||
|
||||
func underAny(p string, roots []string) bool {
|
||||
for _, root := range roots {
|
||||
if IsUnder(p, root) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// CleanWriteFileIn normalizes a model-authored sandbox write and protects the
|
||||
// attachment tree. A session-owned machine keeps today's rule (tools enforce
|
||||
// WriteRoots); a shared machine additionally refuses anything outside the
|
||||
// session's WriteRoots, so one session cannot write into a sibling's tree.
|
||||
func CleanWriteFileIn(l Layout, filePath string) (string, error) {
|
||||
clean := ResolvePathIn(l, filePath)
|
||||
if !path.IsAbs(clean) || clean == "." || clean == "/" {
|
||||
return "", fmt.Errorf("sandbox: workspace write path %q must be an absolute file path", filePath)
|
||||
}
|
||||
if clean == l.Root || clean == l.OutputDir || clean == l.InputDir {
|
||||
return "", fmt.Errorf("sandbox: workspace write path %q is a directory, not a file", filePath)
|
||||
}
|
||||
if l.InputDir != "" && strings.HasPrefix(clean, l.InputDir+"/") {
|
||||
return "", fmt.Errorf("sandbox: session input %s is read-only", l.InputDir)
|
||||
}
|
||||
if l.IsSharedRemote() && !underAny(clean, l.WriteRoots) {
|
||||
return "", fmt.Errorf("sandbox: workspace write path %q is outside the session workspace %s", filePath, l.Root)
|
||||
}
|
||||
return clean, nil
|
||||
}
|
||||
|
||||
// CleanInputPathIn requires a path inside the layout's attachment tree.
|
||||
func CleanInputPathIn(l Layout, filePath string) (string, error) {
|
||||
clean := path.Clean(strings.TrimSpace(filePath))
|
||||
if l.InputDir != "" && IsUnder(clean, l.InputDir) {
|
||||
return clean, nil
|
||||
}
|
||||
return "", fmt.Errorf("sandbox: session input path %q is outside %s", filePath, l.InputDir)
|
||||
}
|
||||
|
||||
// CleanWorkDirIn requires an absolute working directory. Install calls keep
|
||||
// the workspace/skills scope; a shared machine limits ordinary calls to
|
||||
// WorkDirRoots; a session-owned machine allows any directory. None of these
|
||||
// checks changes filesystem privileges.
|
||||
func CleanWorkDirIn(l Layout, workDir string, allowSkillsRoot bool) (string, error) {
|
||||
clean := path.Clean(strings.TrimSpace(workDir))
|
||||
if !path.IsAbs(clean) {
|
||||
return "", fmt.Errorf("sandbox: work dir %q must be absolute", workDir)
|
||||
}
|
||||
var allowed []string
|
||||
switch {
|
||||
case allowSkillsRoot:
|
||||
allowed = []string{l.Root, SkillsImageRoot}
|
||||
case l.IsSharedRemote():
|
||||
allowed = l.WorkDirRoots
|
||||
default:
|
||||
return clean, nil
|
||||
}
|
||||
if underAny(clean, allowed) {
|
||||
return clean, nil
|
||||
}
|
||||
return "", fmt.Errorf("sandbox: work dir %q is outside allowed roots (%s)", workDir, strings.Join(allowed, ", "))
|
||||
}
|
||||
|
||||
// ValidOutputDirIn reports whether dir may be used as the artifact directory.
|
||||
// The workspace root itself is refused: it is the whole workspace, drafts
|
||||
// included, and artifact collection reads that as "collects nothing".
|
||||
func ValidOutputDirIn(l Layout, dir string) (string, bool) {
|
||||
root := l.Root
|
||||
if root == "" {
|
||||
root = Root
|
||||
}
|
||||
clean := path.Clean(strings.TrimSpace(dir))
|
||||
if !strings.HasPrefix(clean, root+"/") {
|
||||
return "", false
|
||||
}
|
||||
return clean, true
|
||||
}
|
||||
|
||||
// RunnableScriptIn reports whether scriptPath is a session-writable file that
|
||||
// may be executed with an installed skill's interpreter. The attachment tree
|
||||
// and the directory roots themselves are excluded.
|
||||
func RunnableScriptIn(l Layout, scriptPath string) (string, bool) {
|
||||
clean := path.Clean(strings.TrimSpace(scriptPath))
|
||||
if clean == l.Root || clean == l.OutputDir || clean == l.InputDir {
|
||||
return "", false
|
||||
}
|
||||
if !strings.HasPrefix(clean, l.Root+"/") {
|
||||
return "", false
|
||||
}
|
||||
if l.InputDir != "" && strings.HasPrefix(clean, l.InputDir+"/") {
|
||||
return "", false
|
||||
}
|
||||
return clean, true
|
||||
}
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
package workspace
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func userLayout(t *testing.T) Layout {
|
||||
t.Helper()
|
||||
l, err := UserSession(testSessionID)
|
||||
require.NoError(t, err)
|
||||
return l
|
||||
}
|
||||
|
||||
func TestCleanWriteFileInRemoteMatchesHistoricalErrors(t *testing.T) {
|
||||
l := Remote()
|
||||
got, err := CleanWriteFileIn(l, "notes/a.md")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/workspace/notes/a.md", got)
|
||||
|
||||
got, err = CleanWriteFileIn(l, "/opt/elsewhere.txt")
|
||||
require.NoError(t, err, "a session-owned machine still lets the manager write anywhere; tools enforce WriteRoots")
|
||||
require.Equal(t, "/opt/elsewhere.txt", got)
|
||||
|
||||
_, err = CleanWriteFileIn(l, "/workspace/output")
|
||||
require.EqualError(t, err, `sandbox: workspace write path "/workspace/output" is a directory, not a file`)
|
||||
_, err = CleanWriteFileIn(l, "/workspace/input/a.pdf")
|
||||
require.EqualError(t, err, "sandbox: session input /workspace/input is read-only")
|
||||
_, err = CleanWriteFileIn(l, "/")
|
||||
require.EqualError(t, err, `sandbox: workspace write path "/" must be an absolute file path`)
|
||||
}
|
||||
|
||||
func TestCleanWriteFileInSharedStaysInsideTheSession(t *testing.T) {
|
||||
l := userLayout(t)
|
||||
got, err := CleanWriteFileIn(l, "report.md")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, l.Root+"/report.md", got)
|
||||
|
||||
_, err = CleanWriteFileIn(l, "/workspace/sessions/11111111-2222-4333-8444-555555555555/a.md")
|
||||
require.ErrorContains(t, err, "outside the session workspace")
|
||||
_, err = CleanWriteFileIn(l, "../11111111-2222-4333-8444-555555555555/a.md")
|
||||
require.ErrorContains(t, err, "outside the session workspace")
|
||||
_, err = CleanWriteFileIn(l, l.InputDir+"/x")
|
||||
require.ErrorContains(t, err, "read-only")
|
||||
}
|
||||
|
||||
func TestCleanWorkDirIn(t *testing.T) {
|
||||
remote := Remote()
|
||||
got, err := CleanWorkDirIn(remote, "/opt/build", false)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/opt/build", got)
|
||||
_, err = CleanWorkDirIn(remote, "/opt/build", true)
|
||||
require.EqualError(t, err,
|
||||
`sandbox: work dir "/opt/build" is outside allowed roots (/workspace, /opt/weknora/tenant/skills)`)
|
||||
_, err = CleanWorkDirIn(remote, "rel", false)
|
||||
require.EqualError(t, err, `sandbox: work dir "rel" must be absolute`)
|
||||
|
||||
shared := userLayout(t)
|
||||
for _, ok := range []string{shared.Root, shared.Root + "/src", "/tmp", "/tmp/x", SkillsImageRoot + "/pdf"} {
|
||||
_, err := CleanWorkDirIn(shared, ok, false)
|
||||
require.NoError(t, err, ok)
|
||||
}
|
||||
for _, bad := range []string{"/workspace", "/workspace/sessions", "/opt", "/"} {
|
||||
_, err := CleanWorkDirIn(shared, bad, false)
|
||||
require.Error(t, err, bad)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanInputPathIn(t *testing.T) {
|
||||
shared := userLayout(t)
|
||||
got, err := CleanInputPathIn(shared, shared.InputDir+"/a.pdf")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, shared.InputDir+"/a.pdf", got)
|
||||
_, err = CleanInputPathIn(shared, "/workspace/input/a.pdf")
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
||||
func TestValidOutputDirAndRunnableScript(t *testing.T) {
|
||||
remote := Remote()
|
||||
got, ok := ValidOutputDirIn(remote, "/workspace/out2/")
|
||||
require.True(t, ok)
|
||||
require.Equal(t, "/workspace/out2", got)
|
||||
_, ok = ValidOutputDirIn(remote, "/workspace")
|
||||
require.False(t, ok)
|
||||
|
||||
shared := userLayout(t)
|
||||
_, ok = ValidOutputDirIn(shared, "/workspace/output")
|
||||
require.False(t, ok, "another session's tree is not a delivery directory")
|
||||
_, ok = RunnableScriptIn(shared, shared.Root+"/run.py")
|
||||
require.True(t, ok)
|
||||
_, ok = RunnableScriptIn(shared, shared.InputDir+"/run.py")
|
||||
require.False(t, ok)
|
||||
_, ok = RunnableScriptIn(shared, "/workspace/run.py")
|
||||
require.False(t, ok)
|
||||
}
|
||||
|
|
@ -1,144 +1,34 @@
|
|||
package sandbox
|
||||
|
||||
import (
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
import "github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
|
||||
// WorkspaceOrigin says whether a layout is a disposable remote sandbox or a
|
||||
// directory on the host. Isolation policy (work_dir clamping, inspect
|
||||
// enforcement, prompt copy) follows Origin, never the Root string: a host
|
||||
// project can itself be /workspace.
|
||||
type WorkspaceOrigin int
|
||||
// WorkspaceOrigin is where a workspace path is rooted. The type lives in the
|
||||
// workspace subpackage; this alias keeps existing callers compiling.
|
||||
type WorkspaceOrigin = workspace.Origin
|
||||
|
||||
// WorkspaceLayout is the set of roots a session may read and write. The type
|
||||
// lives in the workspace subpackage; this alias keeps existing callers compiling.
|
||||
type WorkspaceLayout = workspace.Layout
|
||||
|
||||
// Workspace origins re-exported for callers that do not import the subpackage.
|
||||
const (
|
||||
// WorkspaceOriginUnspecified is the zero value. Tools treat it as remote.
|
||||
WorkspaceOriginUnspecified WorkspaceOrigin = iota
|
||||
// WorkspaceOriginRemote is a disposable container that owns /workspace.
|
||||
WorkspaceOriginRemote
|
||||
// WorkspaceOriginHost is a directory on the user's machine.
|
||||
WorkspaceOriginHost
|
||||
WorkspaceOriginUnspecified = workspace.OriginUnspecified
|
||||
WorkspaceOriginRemote = workspace.OriginRemote
|
||||
WorkspaceOriginHost = workspace.OriginHost
|
||||
WorkspaceOriginRemoteShared = workspace.OriginRemoteShared
|
||||
)
|
||||
|
||||
// WorkspaceLayout describes the shape of one session's workspace. It is pure
|
||||
// data with no behaviour, so both the remote managers and a local adapter can
|
||||
// produce it and the tools can consume it for prompts and tool-layer scope.
|
||||
//
|
||||
// It is not a symlink-safe privilege boundary. A host SessionFileStore must
|
||||
// still enforce the OS-sandbox PathGuard (EvalSymlinks, ProtectGit, deny-read)
|
||||
// on every read and write. Empty InputDir / OutputDir is the host shape:
|
||||
// there is no separate attachment or collect tree, and scanning Root would
|
||||
// upload the user's project.
|
||||
type WorkspaceLayout struct {
|
||||
// Origin selects remote vs host policy. RemoteWorkspaceLayout sets
|
||||
// WorkspaceOriginRemote; host adapters must set WorkspaceOriginHost.
|
||||
Origin WorkspaceOrigin
|
||||
// Root anchors relative paths and is the shell's working directory.
|
||||
Root string
|
||||
// WriteRoots are the directories write/edit tools may create files under.
|
||||
// A root itself is a directory, not a writable file path.
|
||||
WriteRoots []string
|
||||
// ReadRoots are the directories read/list tools may inspect, ordered
|
||||
// most-specific-first so the reported root names the narrowest match.
|
||||
ReadRoots []string
|
||||
// InputDir holds read-only user attachments. Empty when the backend has none.
|
||||
InputDir string
|
||||
// OutputDir is what artifact collection sweeps. Empty when it collects none.
|
||||
OutputDir string
|
||||
// Hint is the wording tools put in descriptions and scope errors.
|
||||
Hint string
|
||||
}
|
||||
// RemoteWorkspaceLayout is the /workspace layout of a session-owned machine.
|
||||
func RemoteWorkspaceLayout() WorkspaceLayout { return workspace.Remote() }
|
||||
|
||||
// IsHost reports a local OS workspace. Host adapters must set Origin; a
|
||||
// non-empty Root that is not /workspace is not enough (Gitpod uses /workspace).
|
||||
func (l WorkspaceLayout) IsHost() bool {
|
||||
return l.Origin == WorkspaceOriginHost
|
||||
}
|
||||
|
||||
// RemoteWorkspaceLayout is the /workspace layout every remote provider shares.
|
||||
func RemoteWorkspaceLayout() WorkspaceLayout {
|
||||
return WorkspaceLayout{
|
||||
Origin: WorkspaceOriginRemote,
|
||||
Root: SessionWorkspaceRoot,
|
||||
WriteRoots: []string{SessionWorkspaceRoot, SessionOutputRoot},
|
||||
ReadRoots: []string{SessionOutputRoot, SessionInputRoot, SessionWorkspaceRoot},
|
||||
InputDir: SessionInputRoot,
|
||||
OutputDir: SessionOutputRoot,
|
||||
Hint: SessionWorkspaceRoot,
|
||||
}
|
||||
// UserWorkspaceLayout is sessionID's layout on a machine its owner shares.
|
||||
func UserWorkspaceLayout(sessionID string) (WorkspaceLayout, error) {
|
||||
return workspace.UserSession(sessionID)
|
||||
}
|
||||
|
||||
// FailedHostWorkspaceLayout is what tools and prompts use when a layout
|
||||
// provider exists but errors: do not fall back to /workspace.
|
||||
func FailedHostWorkspaceLayout() WorkspaceLayout {
|
||||
return WorkspaceLayout{Origin: WorkspaceOriginHost}
|
||||
}
|
||||
func FailedHostWorkspaceLayout() WorkspaceLayout { return workspace.FailedHost() }
|
||||
|
||||
// HasRoot reports a usable workspace root.
|
||||
func (l WorkspaceLayout) HasRoot() bool {
|
||||
return strings.TrimSpace(l.Root) != ""
|
||||
}
|
||||
|
||||
// Normalized cleans every path in the layout and drops empty roots.
|
||||
//
|
||||
// Scope checks compare cleaned paths against these entries verbatim, so an
|
||||
// adapter that hands back a trailing slash or an uncleaned path would
|
||||
// silently deny every write and work_dir inside its own workspace. Consumers
|
||||
// normalize on receipt rather than trusting the producer.
|
||||
func (l WorkspaceLayout) Normalized() WorkspaceLayout {
|
||||
l.Root = cleanLayoutPath(l.Root)
|
||||
l.InputDir = cleanLayoutPath(l.InputDir)
|
||||
l.OutputDir = cleanLayoutPath(l.OutputDir)
|
||||
l.WriteRoots = cleanLayoutRoots(l.WriteRoots)
|
||||
l.ReadRoots = cleanLayoutRoots(l.ReadRoots)
|
||||
l.Hint = strings.TrimSpace(l.Hint)
|
||||
return l
|
||||
}
|
||||
|
||||
func cleanLayoutPath(p string) string {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
return ""
|
||||
}
|
||||
return path.Clean(p)
|
||||
}
|
||||
|
||||
// cleanLayoutRoots preserves order (ReadRoots is most-specific-first) and
|
||||
// drops duplicates that only differed before cleaning.
|
||||
func cleanLayoutRoots(roots []string) []string {
|
||||
if len(roots) == 0 {
|
||||
return roots
|
||||
}
|
||||
out := make([]string, 0, len(roots))
|
||||
seen := make(map[string]struct{}, len(roots))
|
||||
for _, root := range roots {
|
||||
clean := cleanLayoutPath(root)
|
||||
if clean == "" {
|
||||
continue
|
||||
}
|
||||
if _, dup := seen[clean]; dup {
|
||||
continue
|
||||
}
|
||||
seen[clean] = struct{}{}
|
||||
out = append(out, clean)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// PromptSafePath returns p when it can be embedded verbatim in prompt text,
|
||||
// and "" when it cannot. On a host layout the workspace root is a directory
|
||||
// the user chose, so a name carrying newlines or markup would otherwise reach
|
||||
// the model as instructions rather than as a path. Callers fall back to
|
||||
// generic wording instead of sanitizing, so a real path is never shown wrong.
|
||||
func PromptSafePath(p string) string {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
return ""
|
||||
}
|
||||
for _, r := range p {
|
||||
if r < 0x20 || r == 0x7f || r == '<' || r == '>' || r == '&' {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
return p
|
||||
}
|
||||
// PromptSafePath returns p when it can be embedded verbatim in prompt text.
|
||||
func PromptSafePath(p string) string { return workspace.PromptSafePath(p) }
|
||||
|
|
|
|||
|
|
@ -1,21 +1,8 @@
|
|||
package sandbox
|
||||
|
||||
import (
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
import "github.com/Tencent/WeKnora/internal/sandbox/workspace"
|
||||
|
||||
// ResolveWorkspacePathIn gives file tools the same relative-path semantics as
|
||||
// commands, anchored at the session's workspace root. This normalizes only;
|
||||
// callers still enforce their read/write roots.
|
||||
// ResolveWorkspacePathIn anchors a relative path at the layout root.
|
||||
func ResolveWorkspacePathIn(l WorkspaceLayout, value string) string {
|
||||
value = strings.TrimSpace(value)
|
||||
if !path.IsAbs(value) {
|
||||
root := l.Root
|
||||
if root == "" {
|
||||
root = RemoteWorkspaceLayout().Root
|
||||
}
|
||||
return path.Join(root, value)
|
||||
}
|
||||
return path.Clean(value)
|
||||
return workspace.ResolvePathIn(l, value)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -396,6 +396,16 @@ func MergeSandboxConfigForUpdate(incoming, existing *TenantSandboxConfig) *Tenan
|
|||
if strings.TrimSpace(out.SkillRollout) == "" {
|
||||
out.SkillRollout = existing.SkillRollout
|
||||
}
|
||||
// The runtime form may omit the allocation policy. Only an explicit
|
||||
// "session" switches a user config back.
|
||||
if strings.TrimSpace(out.AllocationMode) == "" {
|
||||
out.AllocationMode = existing.AllocationMode
|
||||
}
|
||||
// 0 is indistinguishable from an omitted field here, so it keeps the
|
||||
// stored value. Returning to the default means sending 7 explicitly.
|
||||
if out.UserSandboxEmptyRetentionDays == 0 {
|
||||
out.UserSandboxEmptyRetentionDays = existing.UserSandboxEmptyRetentionDays
|
||||
}
|
||||
}
|
||||
|
||||
return &out
|
||||
|
|
|
|||
|
|
@ -358,3 +358,16 @@ func TestMergeSandboxConfigForUpdateDoesNotMutateInputs(t *testing.T) {
|
|||
require.Equal(t, "old-e2b", existing.E2B.APIKey,
|
||||
"merge must not mutate the stored config")
|
||||
}
|
||||
|
||||
func TestMergeSandboxConfigForUpdateKeepsAllocationPolicyWhenOmitted(t *testing.T) {
|
||||
existing := &TenantSandboxConfig{AllocationMode: SandboxAllocationUser, UserSandboxEmptyRetentionDays: 30}
|
||||
out := MergeSandboxConfigForUpdate(&TenantSandboxConfig{}, existing)
|
||||
require.Equal(t, SandboxAllocationUser, out.AllocationMode)
|
||||
require.Equal(t, 30, out.UserSandboxEmptyRetentionDays)
|
||||
}
|
||||
|
||||
func TestMergeSandboxConfigForUpdateHonoursExplicitSessionMode(t *testing.T) {
|
||||
existing := &TenantSandboxConfig{AllocationMode: SandboxAllocationUser}
|
||||
out := MergeSandboxConfigForUpdate(&TenantSandboxConfig{AllocationMode: SandboxAllocationSession}, existing)
|
||||
require.Equal(t, SandboxAllocationSession, out.AllocationMode)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -26,6 +26,9 @@ const (
|
|||
UserIDContextKey ContextKey = "UserID"
|
||||
// PrincipalContextKey is the context key for the terminal caller principal.
|
||||
PrincipalContextKey ContextKey = "Principal"
|
||||
// PrincipalAuthSourceContextKey records how the auth middleware verified
|
||||
// the principal (web session, API key mode, embed, IM, MCP endpoint).
|
||||
PrincipalAuthSourceContextKey ContextKey = "PrincipalAuthSource"
|
||||
// TenantAPIKeyScopeContextKey carries per-API-key operation and KB scopes.
|
||||
TenantAPIKeyScopeContextKey ContextKey = "TenantAPIKeyScope"
|
||||
// AuditAPIKeyContextKey carries the API key id/name for activity
|
||||
|
|
|
|||
|
|
@ -34,7 +34,10 @@ var contextCloneAcrossDetach = map[ContextKey]bool{
|
|||
UserContextKey: true,
|
||||
UserIDContextKey: true,
|
||||
PrincipalContextKey: true,
|
||||
SystemAdminContextKey: true,
|
||||
// How the principal was authenticated. Background work that creates a
|
||||
// session (SSE clone) must stamp the same source the request had.
|
||||
PrincipalAuthSourceContextKey: true,
|
||||
SystemAdminContextKey: true,
|
||||
// TenantRoleContextKey: the caller's resolved role in the active tenant
|
||||
// (PR 2 #1303). Must survive for the same reason as TenantIDContextKey —
|
||||
// any handler that does `ctx := logger.CloneContext(c.Request.Context())`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,93 @@
|
|||
package interfaces
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// ErrPlacementConflict is returned when the session's pin changed between the
|
||||
// caller's read and the placement transaction. Re-read the session and decide again.
|
||||
var ErrPlacementConflict = errors.New("sandbox placement conflict")
|
||||
|
||||
// ErrSessionRetired is returned when the session is being deleted (or its
|
||||
// owner was). It may never get a sandbox placement again.
|
||||
var ErrSessionRetired = errors.New("sandbox: session is retired")
|
||||
|
||||
// RetiredSession is one session's outcome of RetireSessions. Assignment is
|
||||
// nil when the session was never placed (it got a tombstone). A released
|
||||
// user-mode assignment has CleanupDueAt set: its directory still has to go.
|
||||
type RetiredSession struct {
|
||||
SessionID string
|
||||
Assignment *types.SessionSandboxAssignment
|
||||
MachineEmpty bool
|
||||
}
|
||||
|
||||
// EmptyMachineCursor is the keyset position of ListEmptyUserMachines. The
|
||||
// zero value starts from the beginning.
|
||||
type EmptyMachineCursor struct {
|
||||
EmptySince time.Time
|
||||
Digest string
|
||||
}
|
||||
|
||||
// PlacementPin is the (sandbox_config_id, sandbox_config_tenant_id) pair on
|
||||
// the sessions row. A zero ConfigID means the session has no pin.
|
||||
type PlacementPin struct {
|
||||
ConfigID string
|
||||
TenantID uint64
|
||||
}
|
||||
|
||||
// SandboxAllocationRepository persists session placements and shared machines.
|
||||
type SandboxAllocationRepository interface {
|
||||
// GetAssignment returns the session's row, tombstones included.
|
||||
GetAssignment(ctx context.Context, sessionID string) (*types.SessionSandboxAssignment, error)
|
||||
// ClaimPlacement records a's placement and projects it onto the session's
|
||||
// pin in one transaction that locks the session row. machine is the
|
||||
// shared machine a user-mode placement joins (nil for a machine of its
|
||||
// own); it is registered, and its empty clock cleared, with the row. An
|
||||
// existing assignment wins and is returned unchanged. expected is the pin
|
||||
// the caller decided from; if the row holds anything else, it returns
|
||||
// ErrPlacementConflict. A soft-deleted or missing session is
|
||||
// gorm.ErrRecordNotFound; a released one (tombstones included) is
|
||||
// ErrSessionRetired.
|
||||
ClaimPlacement(ctx context.Context, a *types.SessionSandboxAssignment, machine *types.SandboxUserMachine,
|
||||
expected PlacementPin) (*types.SessionSandboxAssignment, error)
|
||||
// RetireSessions is the first step of deleting sessions. In one
|
||||
// transaction it locks each session row — the lock ClaimPlacement takes —
|
||||
// and releases its live assignment, scheduling the workspace cleanup of a
|
||||
// user-mode one. A session without an assignment gets a tombstone, so a
|
||||
// placement racing the deletion cannot land after it. Idempotent; results
|
||||
// are in sessionIDs order.
|
||||
RetireSessions(ctx context.Context, sessionIDs []string, at time.Time) ([]RetiredSession, error)
|
||||
HasSandboxCheckpoint(ctx context.Context, sessionID string) (bool, error)
|
||||
|
||||
ListUserMachinesByConfig(
|
||||
ctx context.Context, configTenantID uint64, configID string,
|
||||
) ([]*types.SandboxUserMachine, error)
|
||||
ListUserMachinesByOwner(
|
||||
ctx context.Context, ownerTenantID uint64, ownerID string,
|
||||
) ([]*types.SandboxUserMachine, error)
|
||||
// ListEmptyUserMachines pages the machines empty since before emptyBefore
|
||||
// in (empty_since, machine_digest) order, strictly after the cursor, so a
|
||||
// candidate that is skipped never hides the ones behind it.
|
||||
ListEmptyUserMachines(
|
||||
ctx context.Context, emptyBefore time.Time, after EmptyMachineCursor, limit int,
|
||||
) ([]*types.SandboxUserMachine, error)
|
||||
GetUserMachine(ctx context.Context, digest string) (*types.SandboxUserMachine, error)
|
||||
CountLiveAssignments(ctx context.Context, digest string) (int64, error)
|
||||
// DeleteUserMachineIfEmptySince drops the registry row only while it
|
||||
// still has no live assignment and its empty_since is exactly
|
||||
// emptySince — the value the reclaimer verified — so a machine that was
|
||||
// joined and emptied again meanwhile keeps its row and its new clock.
|
||||
DeleteUserMachineIfEmptySince(ctx context.Context, digest string, emptySince time.Time) (bool, error)
|
||||
DeleteUserMachinesByConfig(ctx context.Context, configTenantID uint64, configID string) error
|
||||
|
||||
// ListDueWorkspaceCleanups returns released assignments whose directory
|
||||
// cleanup is due at now, oldest first.
|
||||
ListDueWorkspaceCleanups(ctx context.Context, now time.Time, limit int) ([]*types.SessionSandboxAssignment, error)
|
||||
CompleteWorkspaceCleanup(ctx context.Context, sessionID string) error
|
||||
// RetryWorkspaceCleanup moves the session's next cleanup attempt to next.
|
||||
RetryWorkspaceCleanup(ctx context.Context, sessionID string, next time.Time) error
|
||||
}
|
||||
|
|
@ -214,3 +214,73 @@ func SessionOwnerIDFromContext(ctx context.Context) string {
|
|||
userID, _ := UserIDFromContext(ctx)
|
||||
return userID
|
||||
}
|
||||
|
||||
// Authentication sources recorded on sessions.owner_auth_source. The values
|
||||
// match internal/sandbox/allocation.AuthSource.
|
||||
const (
|
||||
AuthSourceWeb = "web"
|
||||
AuthSourceAPITenantKey = "api_tenant_key"
|
||||
AuthSourceAPIPlatformKey = "api_platform_key"
|
||||
AuthSourceAPIDirectHeader = "api_direct_header"
|
||||
AuthSourceAPISignedToken = "api_signed_token"
|
||||
AuthSourceEmbed = "embed"
|
||||
AuthSourceIM = "im"
|
||||
AuthSourceMCPEndpoint = "mcp_endpoint"
|
||||
AuthSourceInternal = "internal"
|
||||
AuthSourceUnknown = "unknown"
|
||||
)
|
||||
|
||||
// WithPrincipalAuthSource records how the request's principal was verified.
|
||||
func WithPrincipalAuthSource(ctx context.Context, source string) context.Context {
|
||||
source = strings.TrimSpace(source)
|
||||
if source == "" {
|
||||
return ctx
|
||||
}
|
||||
return context.WithValue(ctx, PrincipalAuthSourceContextKey, source)
|
||||
}
|
||||
|
||||
// PrincipalAuthSourceFromContext returns "" when no auth middleware ran.
|
||||
func PrincipalAuthSourceFromContext(ctx context.Context) string {
|
||||
if ctx == nil {
|
||||
return ""
|
||||
}
|
||||
if v, ok := ctx.Value(PrincipalAuthSourceContextKey).(string); ok {
|
||||
return strings.TrimSpace(v)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// StampSessionOwnerSource overwrites the owner's authentication source from
|
||||
// the verified request context. The caller's credentials only vouch for a row
|
||||
// whose owner is the caller's own owner scope; anything else (internal
|
||||
// services, a row created on someone's behalf) is recorded as internal. The
|
||||
// owner's kind is not stored: it follows from the source and user_id.
|
||||
func StampSessionOwnerSource(ctx context.Context, s *Session) {
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
if strings.TrimSpace(s.UserID) == "" {
|
||||
s.OwnerAuthSource = AuthSourceUnknown
|
||||
return
|
||||
}
|
||||
source := PrincipalAuthSourceFromContext(ctx)
|
||||
if _, ok := PrincipalFromContext(ctx); !ok || source == "" || SessionOwnerIDFromContext(ctx) != s.UserID {
|
||||
s.OwnerAuthSource = AuthSourceInternal
|
||||
return
|
||||
}
|
||||
s.OwnerAuthSource = source
|
||||
}
|
||||
|
||||
// AuthSourceForBoundOwnerID describes an owner bound after creation (MCP ask,
|
||||
// embed backfill). Those binders only ever bind shared-entry or channel
|
||||
// owners, so the result can never qualify a session for a shared machine.
|
||||
func AuthSourceForBoundOwnerID(ownerID string) string {
|
||||
switch {
|
||||
case strings.HasPrefix(ownerID, PrincipalMCPEndpoint+":"):
|
||||
return AuthSourceMCPEndpoint
|
||||
case strings.HasPrefix(ownerID, PrincipalEmbedSession+":"):
|
||||
return AuthSourceEmbed
|
||||
default:
|
||||
return AuthSourceInternal
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,6 +3,8 @@ package types
|
|||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestPrincipalFromContextFallsBackToWebUser(t *testing.T) {
|
||||
|
|
@ -82,3 +84,42 @@ func TestMCPOAuthPrincipalMapsEmbedSessionToVisitor(t *testing.T) {
|
|||
t.Fatalf("MCPOAuthPrincipalFromContext() = %q, want %q", got.StorageID(), want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrincipalAuthSourceRoundTrip(t *testing.T) {
|
||||
ctx := WithPrincipalAuthSource(context.Background(), AuthSourceWeb)
|
||||
require.Equal(t, AuthSourceWeb, PrincipalAuthSourceFromContext(ctx))
|
||||
require.Equal(t, "", PrincipalAuthSourceFromContext(context.Background()))
|
||||
}
|
||||
|
||||
func TestStampSessionOwnerSource(t *testing.T) {
|
||||
web := WithPrincipalAuthSource(
|
||||
WithPrincipal(context.WithValue(context.Background(), UserIDContextKey, "u-1"),
|
||||
Principal{Type: PrincipalWebUser, ID: "u-1"}),
|
||||
AuthSourceWeb)
|
||||
|
||||
s := &Session{UserID: "u-1", OwnerAuthSource: "forged"}
|
||||
StampSessionOwnerSource(web, s)
|
||||
require.Equal(t, AuthSourceWeb, s.OwnerAuthSource)
|
||||
|
||||
// A row created for someone else (internal service, admin tooling) is not
|
||||
// vouched for by the caller's credentials.
|
||||
other := &Session{UserID: "u-2"}
|
||||
StampSessionOwnerSource(web, other)
|
||||
require.Equal(t, AuthSourceInternal, other.OwnerAuthSource)
|
||||
|
||||
// No middleware source: background or internal context.
|
||||
bare := &Session{UserID: "u-1"}
|
||||
StampSessionOwnerSource(context.WithValue(context.Background(), UserIDContextKey, "u-1"), bare)
|
||||
require.Equal(t, AuthSourceInternal, bare.OwnerAuthSource)
|
||||
|
||||
empty := &Session{}
|
||||
StampSessionOwnerSource(web, empty)
|
||||
require.Equal(t, AuthSourceUnknown, empty.OwnerAuthSource)
|
||||
}
|
||||
|
||||
func TestAuthSourceForBoundOwnerID(t *testing.T) {
|
||||
require.Equal(t, AuthSourceMCPEndpoint, AuthSourceForBoundOwnerID(MCPEndpointPrincipal(7, "e").StorageID()))
|
||||
require.Equal(t, AuthSourceEmbed, AuthSourceForBoundOwnerID(EmbedSessionPrincipal(7, "c", "s").StorageID()))
|
||||
require.Equal(t, AuthSourceInternal, AuthSourceForBoundOwnerID("u-1"),
|
||||
"binding never elevates an owner to a verified person")
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,48 @@
|
|||
package types
|
||||
|
||||
import "time"
|
||||
|
||||
// SessionSandboxAssignment is the persisted placement of one session on remote
|
||||
// sandboxes. Written once on first use by sandboxalloc; never rewritten except
|
||||
// to release the session.
|
||||
//
|
||||
// Everything else about the placement is derived: the mode and workspace
|
||||
// layout from MachineDigest (empty = a machine of the session's own), the owner
|
||||
// of a shared machine from its sandbox_user_machines row, and the owner's
|
||||
// eligibility from the session row.
|
||||
type SessionSandboxAssignment struct {
|
||||
SessionID string `gorm:"column:session_id;type:varchar(36);primaryKey"`
|
||||
ConfigTenantID uint64 `gorm:"column:config_tenant_id;not null;default:0"`
|
||||
ConfigID string `gorm:"column:config_id;type:varchar(64);not null;default:''"`
|
||||
MachineDigest string `gorm:"column:machine_digest;type:varchar(64);not null;default:''"`
|
||||
// ReleasedAt marks a deleted (retired) session. A row that has it may
|
||||
// never be placed again; a session retired before its first placement
|
||||
// gets such a row with an empty ConfigID (a tombstone).
|
||||
ReleasedAt *time.Time `gorm:"column:released_at"`
|
||||
// CleanupDueAt is set while the session's directory on a shared machine
|
||||
// still has to be removed, and is the time of the next attempt.
|
||||
CleanupDueAt *time.Time `gorm:"column:cleanup_due_at"`
|
||||
}
|
||||
|
||||
// TableName is the assignments table.
|
||||
func (SessionSandboxAssignment) TableName() string { return "session_sandbox_assignments" }
|
||||
|
||||
// IsTombstone reports a row written only to retire a session that was never
|
||||
// placed. Real placements always carry a normalized, non-empty config id.
|
||||
func (a *SessionSandboxAssignment) IsTombstone() bool {
|
||||
return a != nil && a.ConfigID == "" && a.ReleasedAt != nil
|
||||
}
|
||||
|
||||
// SandboxUserMachine is the logical registry row of one shared machine: the
|
||||
// identity behind its digest, and the empty-retention clock.
|
||||
type SandboxUserMachine struct {
|
||||
MachineDigest string `gorm:"column:machine_digest;type:varchar(64);primaryKey"`
|
||||
OwnerTenantID uint64 `gorm:"column:owner_tenant_id;not null"`
|
||||
OwnerID string `gorm:"column:owner_id;type:varchar(512);not null"`
|
||||
ConfigTenantID uint64 `gorm:"column:config_tenant_id;not null"`
|
||||
ConfigID string `gorm:"column:config_id;type:varchar(64);not null"`
|
||||
EmptySince *time.Time `gorm:"column:empty_since"`
|
||||
}
|
||||
|
||||
// TableName is the shared-machine registry table.
|
||||
func (SandboxUserMachine) TableName() string { return "sandbox_user_machines" }
|
||||
|
|
@ -101,6 +101,12 @@ type Session struct {
|
|||
// UserID is the owner scope for this session. WeKnora user UUIDs, API
|
||||
// external-user principals, and embed visitor principals all use this column.
|
||||
UserID string `json:"user_id,omitempty" gorm:"type:varchar(512);index"`
|
||||
// OwnerAuthSource records, server-side, how UserID was authenticated when
|
||||
// the session was created or its owner first bound (AuthSource*).
|
||||
// Per-user sandbox allocation reads it; client input never sets it
|
||||
// (CreateSession overwrites). Empty on rows created before the column
|
||||
// existed, which means "unknown".
|
||||
OwnerAuthSource string `json:"owner_auth_source,omitempty" gorm:"type:varchar(32);not null;default:''"`
|
||||
// IsPinned indicates whether the session is pinned in the list.
|
||||
IsPinned bool `json:"is_pinned" gorm:"default:false" extensions:"x-agent,x-agent-response"`
|
||||
// PinnedAt records when the session was pinned; nil when not pinned.
|
||||
|
|
|
|||
|
|
@ -689,6 +689,18 @@ type TenantSandboxConfig struct {
|
|||
// afterwards boot the new snapshot.
|
||||
SkillRollout string `json:"skill_rollout,omitempty"`
|
||||
|
||||
// AllocationMode decides how a session that has not yet used a sandbox
|
||||
// of this config is placed: "session" (default) gives it a machine of its
|
||||
// own, "user" lets one verified owner's sessions share a machine. A
|
||||
// session never changes mode after its first use; see
|
||||
// internal/sandbox/allocation.
|
||||
AllocationMode string `json:"allocation_mode,omitempty"`
|
||||
|
||||
// UserSandboxEmptyRetentionDays is how long a user machine with no live
|
||||
// session is kept before the reclaim job deletes it. 0 means the default
|
||||
// (7); values above 90 are rejected.
|
||||
UserSandboxEmptyRetentionDays int `json:"user_sandbox_empty_retention_days,omitempty"`
|
||||
|
||||
// Network is the outbound/inbound network policy applied to every sandbox
|
||||
// created from this config — chat sessions, skill installs and deep
|
||||
// connectivity probes alike. nil and the zero value mean the same thing:
|
||||
|
|
@ -840,6 +852,14 @@ const (
|
|||
SkillRolloutNewSession = "new_session"
|
||||
)
|
||||
|
||||
const (
|
||||
// SandboxAllocationSession is one sandbox per session.
|
||||
SandboxAllocationSession = "session"
|
||||
// SandboxAllocationUser shares one sandbox across a user's sessions.
|
||||
// allocation.ParseMode reads the same strings.
|
||||
SandboxAllocationUser = "user"
|
||||
)
|
||||
|
||||
// RebuildsExistingOnSkillChange reports whether an install or removal should
|
||||
// mark already-bound sandboxes stale. Unknown values fail toward rebuilding
|
||||
// so a corrupted row cannot pin every session on a retired image.
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE sessions DROP COLUMN owner_auth_source;
|
||||
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE sessions ADD COLUMN owner_auth_source VARCHAR(32) NOT NULL DEFAULT '';
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
DROP TABLE IF EXISTS sandbox_user_machines;
|
||||
DROP TABLE IF EXISTS session_sandbox_assignments;
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
CREATE TABLE IF NOT EXISTS session_sandbox_assignments (
|
||||
session_id VARCHAR(36) PRIMARY KEY,
|
||||
config_tenant_id INTEGER NOT NULL DEFAULT 0,
|
||||
config_id VARCHAR(64) NOT NULL DEFAULT '',
|
||||
machine_digest VARCHAR(64) NOT NULL DEFAULT '',
|
||||
released_at DATETIME NULL,
|
||||
cleanup_due_at DATETIME NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_sandbox_assignments_live_machine
|
||||
ON session_sandbox_assignments (machine_digest)
|
||||
WHERE released_at IS NULL AND machine_digest <> '';
|
||||
CREATE INDEX IF NOT EXISTS idx_session_sandbox_assignments_cleanup_due
|
||||
ON session_sandbox_assignments (cleanup_due_at)
|
||||
WHERE cleanup_due_at IS NOT NULL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sandbox_user_machines (
|
||||
machine_digest VARCHAR(64) PRIMARY KEY,
|
||||
owner_tenant_id INTEGER NOT NULL,
|
||||
owner_id VARCHAR(512) NOT NULL,
|
||||
config_tenant_id INTEGER NOT NULL,
|
||||
config_id VARCHAR(64) NOT NULL,
|
||||
empty_since DATETIME NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_config ON sandbox_user_machines (config_tenant_id, config_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_owner ON sandbox_user_machines (owner_id, owner_tenant_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_empty ON sandbox_user_machines (empty_since) WHERE empty_since IS NOT NULL;
|
||||
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE sessions DROP COLUMN IF EXISTS owner_auth_source;
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
-- Migration: 000129_session_owner_auth_source
|
||||
-- sessions.user_id names the owner scope but not how the server authenticated
|
||||
-- it. Per-user sandbox allocation may only share a machine between sessions
|
||||
-- of a verified person (a web account or an API external user), so the
|
||||
-- authentication source is recorded server-side when the session is created
|
||||
-- or its owner is first bound; the owner's kind follows from the source and
|
||||
-- the shape of user_id. Existing rows stay '' and are treated as unknown:
|
||||
-- they keep a sandbox per session forever.
|
||||
DO $$ BEGIN RAISE NOTICE '[Migration 000129] Adding sessions.owner_auth_source'; END $$;
|
||||
|
||||
ALTER TABLE sessions ADD COLUMN IF NOT EXISTS owner_auth_source VARCHAR(32) NOT NULL DEFAULT '';
|
||||
|
||||
COMMENT ON COLUMN sessions.owner_auth_source IS 'Server-verified authentication source of user_id; empty = unknown';
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
DROP TABLE IF EXISTS sandbox_user_machines;
|
||||
DROP TABLE IF EXISTS session_sandbox_assignments;
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
-- Migration: 000130_sandbox_user_allocation
|
||||
-- Per-user sandbox allocation. A session's placement (its own machine, or a
|
||||
-- directory on its owner's shared machine) is chosen once, on first use, and
|
||||
-- recorded here; the config switch only affects sessions without a row.
|
||||
DO $$ BEGIN RAISE NOTICE '[Migration 000130] Creating sandbox allocation tables'; END $$;
|
||||
|
||||
-- One row per placed session. machine_digest is '' for a machine of its own
|
||||
-- and names the shared machine otherwise. released_at marks a deleted (or
|
||||
-- retired) session; a row with it set may never be placed again, which is
|
||||
-- also how a session retired before its first placement is recorded.
|
||||
-- cleanup_due_at is set while the session's directory on a shared machine
|
||||
-- still has to be removed.
|
||||
CREATE TABLE IF NOT EXISTS session_sandbox_assignments (
|
||||
session_id VARCHAR(36) PRIMARY KEY,
|
||||
config_tenant_id BIGINT NOT NULL DEFAULT 0,
|
||||
config_id VARCHAR(64) NOT NULL DEFAULT '',
|
||||
machine_digest VARCHAR(64) NOT NULL DEFAULT '',
|
||||
released_at TIMESTAMP WITH TIME ZONE NULL,
|
||||
cleanup_due_at TIMESTAMP WITH TIME ZONE NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_sandbox_assignments_live_machine
|
||||
ON session_sandbox_assignments (machine_digest)
|
||||
WHERE released_at IS NULL AND machine_digest <> '';
|
||||
CREATE INDEX IF NOT EXISTS idx_session_sandbox_assignments_cleanup_due
|
||||
ON session_sandbox_assignments (cleanup_due_at)
|
||||
WHERE cleanup_due_at IS NOT NULL;
|
||||
|
||||
-- Shared machines. The physical sandbox id stays in the Redis binding; this
|
||||
-- table holds the identity behind each digest, answers "which machines does
|
||||
-- this config / owner have", carries the empty-retention clock, and is the
|
||||
-- row joins, releases and reclaims lock.
|
||||
CREATE TABLE IF NOT EXISTS sandbox_user_machines (
|
||||
machine_digest VARCHAR(64) PRIMARY KEY,
|
||||
owner_tenant_id BIGINT NOT NULL,
|
||||
owner_id VARCHAR(512) NOT NULL,
|
||||
config_tenant_id BIGINT NOT NULL,
|
||||
config_id VARCHAR(64) NOT NULL,
|
||||
empty_since TIMESTAMP WITH TIME ZONE NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_config ON sandbox_user_machines (config_tenant_id, config_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_owner ON sandbox_user_machines (owner_id, owner_tenant_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sandbox_user_machines_empty ON sandbox_user_machines (empty_since) WHERE empty_since IS NOT NULL;
|
||||
Loading…
Reference in New Issue