2025-08-05 07:08:07 +00:00
|
|
|
|
package handler
|
|
|
|
|
|
|
|
|
|
|
|
import (
|
2026-01-30 08:28:01 +00:00
|
|
|
|
"context"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"encoding/json"
|
|
|
|
|
|
"fmt"
|
|
|
|
|
|
"io"
|
2026-03-16 10:01:07 +00:00
|
|
|
|
"mime"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"net/http"
|
2025-08-08 09:19:06 +00:00
|
|
|
|
"strconv"
|
2025-12-31 10:18:22 +00:00
|
|
|
|
"strings"
|
2026-03-09 05:14:42 +00:00
|
|
|
|
"time"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-02-10 04:15:21 +00:00
|
|
|
|
goerrors "errors"
|
|
|
|
|
|
|
2026-04-23 09:22:18 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/agent/tools"
|
2026-03-09 05:14:42 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/application/repository"
|
2026-02-04 12:12:29 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/application/service"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/errors"
|
|
|
|
|
|
"github.com/Tencent/WeKnora/internal/logger"
|
feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests
(chat / search / eval), so file uploads and every downstream asynq task
(document parse, chunk embedding, OCR/VLM, summary / question gen, wiki
ingest, datasource sync, etc.) produced either disconnected shallow
traces or no observation at all.
This change threads one trace end-to-end:
- tracer: add SPAN observation type and StartSpan; add ResumeTrace so a
worker can attach to an upstream trace without emitting a duplicate
trace-create; StartGeneration now auto-picks parentObservationId from
ctx so nested trace -> span -> generation trees render correctly.
- types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq
payloads so trace_id / parent_obs_id / user_id / session_id serialise
into every job.
- langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal
so the HTTP-layer trace survives the Redis hop.
- langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either
resumes the upstream trace or opens a standalone asynq.<type> trace
for scheduled jobs, and wraps the handler in a SPAN with task metadata
(id / queue / retry / payload_bytes) plus ERROR level on failure.
- GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation /
FAQ / wiki / datasource endpoints so the root trace actually starts.
- Tests: tracer_test.go covers span nesting, error status, and
ResumeTrace no-trace-create guarantee; asynq_test.go covers
InjectTracing round-trip, middleware resume path, and standalone
trace fallback.
- Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types
and documents the cross-process propagation model.
No behavioural change when Langfuse is disabled (all new code paths are
no-ops and carriers serialise to empty strings with omitempty).
2026-04-24 04:23:30 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/types"
|
|
|
|
|
|
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
2026-03-09 05:14:42 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/utils"
|
2025-11-25 14:19:41 +00:00
|
|
|
|
secutils "github.com/Tencent/WeKnora/internal/utils"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/gin-gonic/gin"
|
2026-03-09 05:14:42 +00:00
|
|
|
|
"github.com/hibiken/asynq"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
// KnowledgeHandler processes HTTP requests related to knowledge resources
|
|
|
|
|
|
type KnowledgeHandler struct {
|
2026-02-10 04:15:21 +00:00
|
|
|
|
kgService interfaces.KnowledgeService
|
|
|
|
|
|
kbService interfaces.KnowledgeBaseService
|
|
|
|
|
|
kbShareService interfaces.KBShareService
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentShareService interfaces.AgentShareService
|
2026-03-09 05:14:42 +00:00
|
|
|
|
asynqClient interfaces.TaskEnqueuer
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
spanRepo repository.KnowledgeSpanRepository
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// NewKnowledgeHandler creates a new knowledge handler instance
|
|
|
|
|
|
func NewKnowledgeHandler(
|
|
|
|
|
|
kgService interfaces.KnowledgeService,
|
|
|
|
|
|
kbService interfaces.KnowledgeBaseService,
|
2026-01-30 08:28:01 +00:00
|
|
|
|
kbShareService interfaces.KBShareService,
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentShareService interfaces.AgentShareService,
|
2026-03-09 05:14:42 +00:00
|
|
|
|
asynqClient interfaces.TaskEnqueuer,
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
spanRepo repository.KnowledgeSpanRepository,
|
2025-08-05 07:08:07 +00:00
|
|
|
|
) *KnowledgeHandler {
|
2026-01-30 08:28:01 +00:00
|
|
|
|
return &KnowledgeHandler{
|
2026-02-10 04:15:21 +00:00
|
|
|
|
kgService: kgService,
|
|
|
|
|
|
kbService: kbService,
|
|
|
|
|
|
kbShareService: kbShareService,
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentShareService: agentShareService,
|
2026-03-09 05:14:42 +00:00
|
|
|
|
asynqClient: asynqClient,
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
spanRepo: spanRepo,
|
2026-01-30 08:28:01 +00:00
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// validateKnowledgeBaseAccess validates access permissions to a knowledge base
|
2026-02-28 10:11:37 +00:00
|
|
|
|
// using the ":id" URL path parameter. It delegates to validateKnowledgeBaseAccessWithKBID.
|
2026-01-30 08:28:01 +00:00
|
|
|
|
func (h *KnowledgeHandler) validateKnowledgeBaseAccess(c *gin.Context) (*types.KnowledgeBase, string, uint64, types.OrgMemberRole, error) {
|
2025-11-26 02:50:53 +00:00
|
|
|
|
kbID := secutils.SanitizeForLog(c.Param("id"))
|
2026-02-28 10:11:37 +00:00
|
|
|
|
return h.validateKnowledgeBaseAccessWithKBID(c, kbID)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// validateKnowledgeBaseAccessWithKBID validates access to the given knowledge base ID (e.g. from query or body).
|
|
|
|
|
|
// Returns the knowledge base, kbID, effective tenant ID, permission, and error.
|
|
|
|
|
|
func (h *KnowledgeHandler) validateKnowledgeBaseAccessWithKBID(c *gin.Context, kbID string) (*types.KnowledgeBase, string, uint64, types.OrgMemberRole, error) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
tenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if tenantID == 0 {
|
|
|
|
|
|
logger.Error(ctx, "Failed to get tenant ID")
|
|
|
|
|
|
return nil, "", 0, "", errors.NewUnauthorizedError("Unauthorized")
|
|
|
|
|
|
}
|
|
|
|
|
|
userID, userExists := c.Get(types.UserIDContextKey.String())
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
callerTenantRole := types.TenantRoleFromContext(ctx)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
kbID = secutils.SanitizeForLog(kbID)
|
|
|
|
|
|
if kbID == "" {
|
|
|
|
|
|
return nil, "", 0, "", errors.NewBadRequestError("Knowledge base ID cannot be empty")
|
|
|
|
|
|
}
|
|
|
|
|
|
kb, err := h.kbService.GetKnowledgeBaseByID(ctx, kbID)
|
|
|
|
|
|
if err != nil {
|
2026-05-14 12:10:04 +00:00
|
|
|
|
// Same not-found-vs-real-error split as knowledgebase.go's
|
|
|
|
|
|
// validateAndGetKnowledgeBase: ErrKnowledgeBaseNotFound is the
|
|
|
|
|
|
// expected outcome for a probed/stale kb id and must surface as
|
|
|
|
|
|
// 404, not the generic 500 the original code produced.
|
|
|
|
|
|
if goerrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
return nil, kbID, 0, "", errors.NewNotFoundError("knowledge base not found")
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
return nil, kbID, 0, "", errors.NewInternalServerError(err.Error())
|
|
|
|
|
|
}
|
|
|
|
|
|
if kb.TenantID == tenantID {
|
|
|
|
|
|
return kb, kbID, tenantID, types.OrgRoleAdmin, nil
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
if h.kbShareService != nil {
|
|
|
|
|
|
permission, isShared, permErr := h.kbShareService.CheckTenantKBPermission(ctx, kbID, tenantID, callerTenantRole)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
if permErr == nil && isShared {
|
|
|
|
|
|
sourceTenantID, srcErr := h.kbShareService.GetKBSourceTenant(ctx, kbID)
|
|
|
|
|
|
if srcErr == nil {
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
logger.Infof(ctx, "Tenant %d accessing shared KB %s with permission %s, source tenant: %d",
|
|
|
|
|
|
tenantID, kbID, permission, sourceTenantID)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
return kb, kbID, sourceTenantID, permission, nil
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
if h.agentShareService != nil {
|
|
|
|
|
|
can, err := h.agentShareService.TenantCanAccessKBViaSomeSharedAgent(ctx, tenantID, callerTenantRole, kb)
|
2026-02-08 06:32:18 +00:00
|
|
|
|
if err == nil && can {
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
logger.Infof(ctx, "Tenant %d accessing KB %s via some shared agent", tenantID, kbID)
|
2026-02-08 06:32:18 +00:00
|
|
|
|
return kb, kbID, kb.TenantID, types.OrgRoleViewer, nil
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
_ = userID
|
|
|
|
|
|
_ = userExists
|
2026-01-30 08:28:01 +00:00
|
|
|
|
logger.Warnf(ctx, "Permission denied to access KB %s, tenant ID: %d, KB tenant: %d", kbID, tenantID, kb.TenantID)
|
|
|
|
|
|
return nil, kbID, 0, "", errors.NewForbiddenError("Permission denied to access this knowledge base")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// resolveKnowledgeAndValidateKBAccess resolves knowledge by ID and validates KB access (owner or shared with required permission).
|
|
|
|
|
|
// Returns the knowledge, context with effectiveTenantID set for downstream service calls, and error.
|
|
|
|
|
|
func (h *KnowledgeHandler) resolveKnowledgeAndValidateKBAccess(c *gin.Context, knowledgeID string, requiredPermission types.OrgMemberRole) (*types.Knowledge, context.Context, error) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
tenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if tenantID == 0 {
|
|
|
|
|
|
return nil, ctx, errors.NewUnauthorizedError("Unauthorized")
|
|
|
|
|
|
}
|
|
|
|
|
|
userID, userExists := c.Get(types.UserIDContextKey.String())
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
callerTenantRole := types.TenantRoleFromContext(ctx)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
knowledge, err := h.kgService.GetKnowledgeByIDOnly(ctx, knowledgeID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
return nil, ctx, errors.NewNotFoundError("Knowledge not found")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Owner: knowledge belongs to caller's tenant
|
|
|
|
|
|
if knowledge.TenantID == tenantID {
|
|
|
|
|
|
return knowledge, context.WithValue(ctx, types.TenantIDContextKey, tenantID), nil
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Shared KB: check organization permission
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
if h.kbShareService != nil {
|
|
|
|
|
|
permission, isShared, permErr := h.kbShareService.CheckTenantKBPermission(ctx, knowledge.KnowledgeBaseID, tenantID, callerTenantRole)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if permErr == nil && isShared && permission.HasPermission(requiredPermission) {
|
|
|
|
|
|
effectiveTenantID := knowledge.TenantID
|
|
|
|
|
|
return knowledge, context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID), nil
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
}
|
2026-02-08 06:32:18 +00:00
|
|
|
|
// Shared agent: request passes agent_id, or user has any shared agent that can access this KB
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
if h.agentShareService != nil && requiredPermission == types.OrgRoleViewer {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentID := c.Query("agent_id")
|
|
|
|
|
|
if agentID != "" {
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
agent, err := h.agentShareService.GetSharedAgentForTenant(ctx, tenantID, callerTenantRole, agentID)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if err == nil && agent != nil {
|
|
|
|
|
|
if knowledge.TenantID != agent.TenantID {
|
|
|
|
|
|
return nil, ctx, errors.NewForbiddenError("Permission denied to access this knowledge")
|
|
|
|
|
|
}
|
|
|
|
|
|
mode := agent.Config.KBSelectionMode
|
|
|
|
|
|
if mode == "none" {
|
|
|
|
|
|
return nil, ctx, errors.NewForbiddenError("Permission denied to access this knowledge")
|
|
|
|
|
|
}
|
|
|
|
|
|
if mode == "all" {
|
|
|
|
|
|
return knowledge, context.WithValue(ctx, types.TenantIDContextKey, knowledge.TenantID), nil
|
|
|
|
|
|
}
|
|
|
|
|
|
if mode == "selected" {
|
|
|
|
|
|
for _, kbID := range agent.Config.KnowledgeBases {
|
|
|
|
|
|
if kbID == knowledge.KnowledgeBaseID {
|
|
|
|
|
|
return knowledge, context.WithValue(ctx, types.TenantIDContextKey, knowledge.TenantID), nil
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return nil, ctx, errors.NewForbiddenError("Permission denied to access this knowledge")
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
2026-02-08 06:32:18 +00:00
|
|
|
|
} else {
|
|
|
|
|
|
kbRef := &types.KnowledgeBase{ID: knowledge.KnowledgeBaseID, TenantID: knowledge.TenantID}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
can, err := h.agentShareService.TenantCanAccessKBViaSomeSharedAgent(ctx, tenantID, callerTenantRole, kbRef)
|
2026-02-08 06:32:18 +00:00
|
|
|
|
if err == nil && can {
|
|
|
|
|
|
return knowledge, context.WithValue(ctx, types.TenantIDContextKey, knowledge.TenantID), nil
|
|
|
|
|
|
}
|
2026-02-04 12:12:29 +00:00
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
_ = userID
|
|
|
|
|
|
_ = userExists
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return nil, ctx, errors.NewForbiddenError("Permission denied to access this knowledge")
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// handleDuplicateKnowledgeError handles cases where duplicate knowledge is detected
|
|
|
|
|
|
// Returns true if the error was a duplicate error and was handled, false otherwise
|
|
|
|
|
|
func (h *KnowledgeHandler) handleDuplicateKnowledgeError(c *gin.Context,
|
|
|
|
|
|
err error, knowledge *types.Knowledge, duplicateType string,
|
|
|
|
|
|
) bool {
|
|
|
|
|
|
if dupErr, ok := err.(*types.DuplicateKnowledgeError); ok {
|
|
|
|
|
|
ctx := c.Request.Context()
|
2025-11-25 15:33:34 +00:00
|
|
|
|
logger.Warnf(ctx, "Detected duplicate %s: %s", duplicateType, secutils.SanitizeForLog(dupErr.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusConflict, gin.H{
|
|
|
|
|
|
"success": false,
|
|
|
|
|
|
"message": dupErr.Error(),
|
|
|
|
|
|
"data": knowledge, // knowledge contains the existing document
|
|
|
|
|
|
"code": fmt.Sprintf("duplicate_%s", duplicateType),
|
|
|
|
|
|
})
|
|
|
|
|
|
return true
|
|
|
|
|
|
}
|
|
|
|
|
|
return false
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-29 03:47:59 +00:00
|
|
|
|
// enqueueKnowledgeListDelete enqueues an async batch-delete task for the
|
|
|
|
|
|
// given knowledge IDs and returns the asynq task ID.
|
|
|
|
|
|
func (h *KnowledgeHandler) enqueueKnowledgeListDelete(
|
|
|
|
|
|
ctx context.Context, tenantID uint64, ids []string,
|
|
|
|
|
|
) (string, error) {
|
|
|
|
|
|
payload := types.KnowledgeListDeletePayload{
|
|
|
|
|
|
TenantID: tenantID,
|
|
|
|
|
|
KnowledgeIDs: ids,
|
|
|
|
|
|
}
|
|
|
|
|
|
langfuse.InjectTracing(ctx, &payload)
|
|
|
|
|
|
payloadBytes, err := json.Marshal(payload)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
return "", fmt.Errorf("marshal payload: %w", err)
|
|
|
|
|
|
}
|
|
|
|
|
|
task := asynq.NewTask(types.TypeKnowledgeListDelete, payloadBytes,
|
|
|
|
|
|
asynq.Queue("low"), asynq.MaxRetry(3))
|
|
|
|
|
|
info, err := h.asynqClient.Enqueue(task)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
return "", fmt.Errorf("enqueue task: %w", err)
|
|
|
|
|
|
}
|
|
|
|
|
|
return info.ID, nil
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// CreateKnowledgeFromFile godoc
|
|
|
|
|
|
// @Summary 从文件创建知识
|
|
|
|
|
|
// @Description 上传文件并创建知识条目
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept multipart/form-data
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param file formData file true "上传的文件"
|
|
|
|
|
|
// @Param fileName formData string false "自定义文件名"
|
|
|
|
|
|
// @Param metadata formData string false "元数据JSON"
|
|
|
|
|
|
// @Param enable_multimodel formData bool false "启用多模态处理"
|
2026-06-09 09:49:39 +00:00
|
|
|
|
// @Param process_config formData string false "处理配置JSON(KnowledgeProcessOverrides)"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "创建的知识"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 409 {object} map[string]interface{} "文件重复"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/knowledge/file [post]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) CreateKnowledgeFromFile(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start creating knowledge from file")
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Validate access to the knowledge base (only owner or admin/editor can create)
|
|
|
|
|
|
_, kbID, effectiveTenantID, permission, err := h.validateKnowledgeBaseAccess(c)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
// Check write permission
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to create knowledge"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Get the uploaded file
|
|
|
|
|
|
file, err := c.FormFile("file")
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Error(ctx, "File upload failed", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("File upload failed").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(system): consolidate system admin and settings into one Settings panel
Replace the standalone /platform/system/* routes with a single
"系统设置" section inside the canonical Settings modal. The previous
SystemLayout.vue / SystemAdmins.vue surfaces are removed and their
functionality (system admin roster, global settings) is hosted directly
in SystemSettings.vue under the standard `.section-header` /
`.settings-group` skeleton. Legacy URLs redirect to the modal section
so external bookmarks don't 404.
Backend:
- SystemSettingService.Reset + repo.Delete: drop the DB override for a
key so the 3-tier resolver falls back to ENV / built-in default.
Idempotent (resetting a never-persisted key returns nil); emits an
audit row only on real deletions, invalidates the local cache, and
publishes to peers via the existing pubsub channel.
- TenantService.BulkSetStorageQuota: overwrite every tenant's
storage_quota in one statement. Powers POST /system/admin/tenants/
apply-default-storage-quota; bypasses the per-tenant whitelist on
PUT /tenants/:id which intentionally forbids storage_quota edits.
- AuditAction{SystemAdminPromoted,SystemAdminRevoked} constants and
emitAdminAudit() in SystemHandler — promote / revoke now leave an
audit trail with TenantID=0 and {target_email,target_username,
idempotent|changed} in details.
- SystemSetting.LastModifiedByName: derived per-request display label
(username, email fallback) so the UI shows "wizardchen" instead of
a UUID prefix without storing a denormalised column.
Frontend:
- SystemSettings.vue rewritten against the Settings modal skeleton with
auto-persisting controls (switch/select @change, input/inputnumber
@blur, tag-input + per-delta popconfirm for SSRF whitelist and admin
roster). auth.registration_mode change goes through an inline
popconfirm; cancel rolls back. Reset / bulk-apply also inline
popconfirms — no dialog modal for these per-row affordances.
- Priority hint panel surfaces the DB > ENV > default resolver order
so operators can reason about "I set the env but it doesn't show up".
- Router: /platform/system, /platform/system/settings, /platform/system/
admins are now compatibility redirects to /platform/settings?section=
system-global.
- Settings modal sized 900x700 → 1080x780 and content-wrapper 600 → 760
so the wider tables (members, system settings) breathe; <1100px
viewport still flexes to the screen.
- i18n: system.globalSettings.* keys for title / description / loading /
empty / badges / priority hint / per-key labels / reset / bulkApply /
admins (label, placeholder, save messages, popconfirm copy) across
zh-CN, en-US, ko-KR, ru-RU.
Misc:
- internal/utils/filesize.go doc: clarify MAX_FILE_SIZE_MB is a
deploy-time-only knob (nginx + docreader + frontend each cache the
env at startup); a SystemAdmin UI override would mislead operators
because nginx would still 413. Until all four layers can hot-reload
the limit in lockstep, this stays env-only.
- internal/utils/security.go: SSRF whitelist parser/runtime now drives
off SystemSettingService for live updates; ENV remains the fallback
for never-overridden deployments.
2026-05-26 12:03:35 +00:00
|
|
|
|
// Validate file size — read MAX_FILE_SIZE_MB env (50MB default).
|
|
|
|
|
|
// Deliberately not a runtime system_setting; see filesize.go for the
|
|
|
|
|
|
// rationale (nginx / docreader / browser bundle all cache this at
|
|
|
|
|
|
// container startup, so a UI knob would silently mismatch).
|
|
|
|
|
|
maxSizeMB := utils.GetMaxFileSizeMB()
|
2026-05-24 13:38:05 +00:00
|
|
|
|
maxSize := maxSizeMB * 1024 * 1024
|
2025-12-30 03:08:05 +00:00
|
|
|
|
if file.Size > maxSize {
|
|
|
|
|
|
logger.Error(ctx, "File size too large")
|
2026-05-24 13:38:05 +00:00
|
|
|
|
c.Error(errors.NewBadRequestError(fmt.Sprintf("文件大小不能超过%dMB", maxSizeMB)))
|
2025-12-30 03:08:05 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-23 09:27:30 +00:00
|
|
|
|
// Get custom filename if provided (for folder uploads with path)
|
|
|
|
|
|
customFileName := c.PostForm("fileName")
|
2025-11-26 02:50:53 +00:00
|
|
|
|
customFileName = secutils.SanitizeForLog(customFileName)
|
2025-11-23 09:27:30 +00:00
|
|
|
|
displayFileName := file.Filename
|
2025-11-26 02:50:53 +00:00
|
|
|
|
displayFileName = secutils.SanitizeForLog(displayFileName)
|
2025-11-23 09:27:30 +00:00
|
|
|
|
if customFileName != "" {
|
|
|
|
|
|
displayFileName = customFileName
|
2025-11-26 02:50:53 +00:00
|
|
|
|
logger.Infof(ctx, "Using custom filename: %s (original: %s)", customFileName, displayFileName)
|
2025-11-23 09:27:30 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 02:50:53 +00:00
|
|
|
|
logger.Infof(ctx, "File upload successful, filename: %s, size: %.2f KB", displayFileName, float64(file.Size)/1024)
|
|
|
|
|
|
logger.Infof(ctx, "Creating knowledge, knowledge base ID: %s, filename: %s", kbID, displayFileName)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Parse metadata if provided
|
|
|
|
|
|
var metadata map[string]string
|
|
|
|
|
|
metadataStr := c.PostForm("metadata")
|
|
|
|
|
|
if metadataStr != "" {
|
|
|
|
|
|
if err := json.Unmarshal([]byte(metadataStr), &metadata); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse metadata", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid metadata format").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-11-25 15:33:34 +00:00
|
|
|
|
logger.Infof(ctx, "Received file metadata: %s", secutils.SanitizeForLog(fmt.Sprintf("%v", metadata)))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-08 09:19:06 +00:00
|
|
|
|
enableMultimodelForm := c.PostForm("enable_multimodel")
|
|
|
|
|
|
var enableMultimodel *bool
|
|
|
|
|
|
if enableMultimodelForm != "" {
|
|
|
|
|
|
parseBool, err := strconv.ParseBool(enableMultimodelForm)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse enable_multimodel", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid enable_multimodel format").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
enableMultimodel = &parseBool
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-06-09 09:49:39 +00:00
|
|
|
|
var processOverrides *types.KnowledgeProcessOverrides
|
|
|
|
|
|
if raw := c.PostForm("process_config"); raw != "" {
|
|
|
|
|
|
processOverrides = &types.KnowledgeProcessOverrides{}
|
|
|
|
|
|
if err := json.Unmarshal([]byte(raw), processOverrides); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse process_config", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid process_config format").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if enableMultimodel != nil && (processOverrides == nil || processOverrides.EnableMultimodel == nil) {
|
|
|
|
|
|
if processOverrides == nil {
|
|
|
|
|
|
processOverrides = &types.KnowledgeProcessOverrides{EnableMultimodel: enableMultimodel}
|
|
|
|
|
|
} else {
|
|
|
|
|
|
processOverrides.EnableMultimodel = enableMultimodel
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-14 10:51:12 +00:00
|
|
|
|
// 获取分类ID(如果提供),用于知识分类管理
|
|
|
|
|
|
tagID := c.PostForm("tag_id")
|
|
|
|
|
|
// 过滤特殊值,空字符串或 "__untagged__" 表示未分类
|
|
|
|
|
|
if tagID == "__untagged__" || tagID == "" {
|
|
|
|
|
|
tagID = ""
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-24 11:50:13 +00:00
|
|
|
|
channel := c.PostForm("channel")
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Create knowledge entry from the file
|
2026-06-09 09:49:39 +00:00
|
|
|
|
knowledge, err := h.kgService.CreateKnowledgeFromFile(ctx, kbID, file, metadata, enableMultimodel, customFileName, tagID, channel, processOverrides)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Check for duplicate knowledge error
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if h.handleDuplicateKnowledgeError(c, err, knowledge, "file") {
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-10 09:04:39 +00:00
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-28 08:17:52 +00:00
|
|
|
|
logger.Infof(
|
|
|
|
|
|
ctx,
|
|
|
|
|
|
"Knowledge created successfully, ID: %s, title: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(knowledge.ID),
|
|
|
|
|
|
secutils.SanitizeForLog(knowledge.Title),
|
|
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledge,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// CreateKnowledgeFromURL godoc
|
|
|
|
|
|
// @Summary 从URL创建知识
|
2026-02-26 07:26:03 +00:00
|
|
|
|
// @Description 从指定URL抓取内容并创建知识条目。当提供 file_name/file_type 或 URL 路径含已知文件扩展名时,自动切换为文件下载模式
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
2026-02-26 07:26:03 +00:00
|
|
|
|
// @Param request body object{url=string,file_name=string,file_type=string,enable_multimodel=bool,title=string,tag_id=string} true "URL请求"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 201 {object} map[string]interface{} "创建的知识"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 409 {object} map[string]interface{} "URL重复"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/knowledge/url [post]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) CreateKnowledgeFromURL(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start creating knowledge from URL")
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Validate access to the knowledge base (only owner or admin/editor can create)
|
|
|
|
|
|
_, kbID, effectiveTenantID, permission, err := h.validateKnowledgeBaseAccess(c)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
// Check write permission
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to create knowledge"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Parse URL from request body
|
|
|
|
|
|
var req struct {
|
2026-06-09 09:49:39 +00:00
|
|
|
|
URL string `json:"url" binding:"required"`
|
|
|
|
|
|
FileName string `json:"file_name"`
|
|
|
|
|
|
FileType string `json:"file_type"`
|
|
|
|
|
|
EnableMultimodel *bool `json:"enable_multimodel"`
|
|
|
|
|
|
Title string `json:"title"`
|
|
|
|
|
|
TagID string `json:"tag_id"`
|
|
|
|
|
|
Channel string `json:"channel"`
|
|
|
|
|
|
ProcessConfig *types.KnowledgeProcessOverrides `json:"process_config"`
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse URL request", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-26 07:26:03 +00:00
|
|
|
|
logger.Infof(ctx, "Received URL request: %s, file_name: %s, file_type: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(req.URL),
|
|
|
|
|
|
secutils.SanitizeForLog(req.FileName),
|
|
|
|
|
|
secutils.SanitizeForLog(req.FileType),
|
|
|
|
|
|
)
|
2026-03-17 07:33:19 +00:00
|
|
|
|
|
|
|
|
|
|
// SSRF validation for user-supplied URL
|
|
|
|
|
|
if err := secutils.ValidateURLForSSRF(req.URL); err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "SSRF validation failed for knowledge URL: %v", err)
|
fix(initialization): surface upstream and SSRF errors verbatim in test-connection responses
Test-connection's "连接失败" was nearly useless for debugging because
two layers conspired to swallow the actual upstream error:
Backend swallow
---------------
checkChatModelConnection and CheckASRModel pattern-matched on the
upstream error string (401 / 404 / timeout / ...) and replaced it with
a fixed Chinese hint. The original SDK message (request URL, response
body, provider's error code description) was discarded.
Refactor: extract classifyConnectionError(errMsg) — returns just the
short Chinese hint. Callers combine "<hint>:<raw err>" so the hint
indicates "where to start looking" and the raw error shows "what
actually happened". The status-400 special-case (endpoint reachable,
parameter mismatch only) still returns success.
Frontend swallow
----------------
ModelEditorDialog.testConnection ignored result.message on success
paths (fine), but also overwrote it with a generic i18n string on
failure paths and 4xx rejections — even though axios's interceptor
already lifts the backend's error.message to error.message. Failure
branch now uses error?.message || result.message as the toast content;
i18n fallback only when neither is present.
SSRF allowlist hint
-------------------
SSRF validation failures previously returned
"Base URL 未通过安全校验: <reason>" with no mention of how to recover
(the allowlist is server-side, not configurable in the UI). Operators
hit it and were stuck.
New utils.FormatSSRFError(label, rawURL, err) wraps the error with
guidance to add the host to SSRF_WHITELIST_EXTRA (the operator-facing
append-only escape hatch — SSRF_WHITELIST itself is the project's
baseline). The hint echoes the failed host so it can be copy-pasted
straight into the env var:
Base URL 未通过安全校验:DNS resolution failed for hostname
test.local.com ... 如该地址确实可信,请联系运维在服务端环境变量
SSRF_WHITELIST_EXTRA 中加入该主机(支持精确域名 / *.example.com
通配 / IP / CIDR),示例:SSRF_WHITELIST_EXTRA=test.local.com,
*.example.com,10.0.0.0/8
All SSRF call sites switch to the helper:
initialization (CheckRemoteModel / TestEmbedding / CheckRerank /
CheckASR / TestMultimodal / batch validation), model CRUD,
mcp_service CRUD, knowledge URL ingest, system ReconnectDocReader.
2026-05-17 07:10:42 +00:00
|
|
|
|
c.Error(errors.NewBadRequestError(secutils.FormatSSRFError("URL", req.URL, err)))
|
2026-03-17 07:33:19 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-26 07:26:03 +00:00
|
|
|
|
logger.Infof(ctx,
|
2025-11-28 08:17:52 +00:00
|
|
|
|
"Creating knowledge from URL, knowledge base ID: %s, URL: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(kbID),
|
|
|
|
|
|
secutils.SanitizeForLog(req.URL),
|
|
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Create knowledge entry from the URL
|
2026-06-09 09:49:39 +00:00
|
|
|
|
knowledge, err := h.kgService.CreateKnowledgeFromURL(
|
|
|
|
|
|
ctx, kbID, req.URL, req.FileName, req.FileType, req.EnableMultimodel, req.Title, req.TagID, req.Channel, req.ProcessConfig,
|
|
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Check for duplicate knowledge error
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if h.handleDuplicateKnowledgeError(c, err, knowledge, "url") {
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-02-26 06:28:14 +00:00
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-26 07:26:03 +00:00
|
|
|
|
logger.Infof(
|
|
|
|
|
|
ctx,
|
|
|
|
|
|
"Knowledge created successfully from URL, ID: %s, title: %s",
|
2026-02-26 06:28:14 +00:00
|
|
|
|
secutils.SanitizeForLog(knowledge.ID),
|
|
|
|
|
|
secutils.SanitizeForLog(knowledge.Title),
|
|
|
|
|
|
)
|
|
|
|
|
|
c.JSON(http.StatusCreated, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledge,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// CreateManualKnowledge godoc
|
|
|
|
|
|
// @Summary 手工创建知识
|
|
|
|
|
|
// @Description 手工录入Markdown格式的知识内容
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param request body types.ManualKnowledgePayload true "手工知识内容"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "创建的知识"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/knowledge/manual [post]
|
2025-11-12 13:27:56 +00:00
|
|
|
|
func (h *KnowledgeHandler) CreateManualKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start creating manual knowledge")
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Validate access to the knowledge base (only owner or admin/editor can create)
|
|
|
|
|
|
_, kbID, effectiveTenantID, permission, err := h.validateKnowledgeBaseAccess(c)
|
2025-11-12 13:27:56 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
// Check write permission
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to create knowledge"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-11-12 13:27:56 +00:00
|
|
|
|
|
|
|
|
|
|
var req types.ManualKnowledgePayload
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse manual knowledge request", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-24 11:50:13 +00:00
|
|
|
|
knowledge, err := h.kgService.CreateKnowledgeFromManual(ctx, kbID, &req, req.Channel)
|
2025-11-12 13:27:56 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, map[string]interface{}{
|
|
|
|
|
|
"kb_id": kbID,
|
|
|
|
|
|
})
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Manual knowledge created successfully, knowledge ID: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(knowledge.ID))
|
2025-11-12 13:27:56 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledge,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// GetKnowledge godoc
|
|
|
|
|
|
// @Summary 获取知识详情
|
|
|
|
|
|
// @Description 根据ID获取知识条目详情
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "知识详情"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "知识不存在"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/{id} [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) GetKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start retrieving knowledge")
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Resolve knowledge and validate KB access (at least viewer)
|
|
|
|
|
|
knowledge, _, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleViewer)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
2026-01-30 08:28:01 +00:00
|
|
|
|
c.Error(err)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge retrieved successfully, ID: %s, title: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(knowledge.ID), secutils.SanitizeForLog(knowledge.Title))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledge,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// GetKnowledgeSpans godoc
|
|
|
|
|
|
// @Summary 获取知识文档解析的 Span 树(含历史尝试)
|
|
|
|
|
|
// @Description 返回该知识在解析流水线的 trace tree(root → stage → subspan):每段状态、耗时、input/output、错误码、langfuse_trace_id。支持 ?attempt=N 查看历史尝试;不传则返回最新尝试。前端用于渲染时间线 + 多模态/embedding 子节点 + 一键跳转 Langfuse。
|
2026-05-27 05:31:06 +00:00
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Param attempt query int false "指定尝试号;省略=最新"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{}
|
|
|
|
|
|
// @Router /api/v1/knowledge/{id}/spans [get]
|
2026-05-27 05:31:06 +00:00
|
|
|
|
//
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// Always returns the canonical 5-stage timeline; missing stage rows are
|
2026-05-27 05:31:06 +00:00
|
|
|
|
// synthesized as "pending" so the frontend timeline always renders five
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// segments. Subspans (multimodal.image[i], generation.*) ride along under
|
|
|
|
|
|
// each stage as children when present.
|
|
|
|
|
|
func (h *KnowledgeHandler) GetKnowledgeSpans(c *gin.Context) {
|
2026-05-27 05:31:06 +00:00
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
knowledge, _, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleViewer)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// Pick attempt: explicit ?attempt=N wins; otherwise pull the
|
|
|
|
|
|
// latest attempt from the spans table. Lite-mode / fresh installs
|
|
|
|
|
|
// with zero rows fall through to attempt=0, in which case we
|
|
|
|
|
|
// return a placeholder tree (5 pending stages, no root, no
|
|
|
|
|
|
// children) so the UI still renders.
|
|
|
|
|
|
requestedAttempt := 0
|
|
|
|
|
|
if v := strings.TrimSpace(c.Query("attempt")); v != "" {
|
|
|
|
|
|
if n, perr := strconv.Atoi(v); perr == nil && n > 0 {
|
|
|
|
|
|
requestedAttempt = n
|
2026-05-27 05:31:06 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
rows := []types.KnowledgeProcessingSpan{}
|
|
|
|
|
|
currentAttempt := 0
|
|
|
|
|
|
if h.spanRepo != nil {
|
|
|
|
|
|
if requestedAttempt == 0 {
|
|
|
|
|
|
latest, lerr := h.spanRepo.LatestAttempt(ctx, knowledge.ID)
|
|
|
|
|
|
if lerr != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "spans LatestAttempt failed for %s: %v", knowledge.ID, lerr)
|
|
|
|
|
|
} else {
|
|
|
|
|
|
currentAttempt = latest
|
2026-05-27 05:31:06 +00:00
|
|
|
|
}
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
} else {
|
|
|
|
|
|
currentAttempt = requestedAttempt
|
|
|
|
|
|
}
|
|
|
|
|
|
if currentAttempt > 0 {
|
|
|
|
|
|
rows, err = h.spanRepo.ListByAttempt(ctx, knowledge.ID, currentAttempt)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "spans ListByAttempt failed kid=%s attempt=%d: %v",
|
|
|
|
|
|
knowledge.ID, currentAttempt, err)
|
|
|
|
|
|
rows = nil
|
2026-05-27 05:31:06 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// Build tree: index by SpanID, then attach to parents. Stages
|
|
|
|
|
|
// missing from the DB are synthesized as "pending" placeholders
|
|
|
|
|
|
// under a synthetic (or real, if present) root so the timeline
|
2026-05-27 09:35:58 +00:00
|
|
|
|
// always renders five segments. parse_status threads through so
|
|
|
|
|
|
// pre-tracker historical knowledge (no rows but parse_status is
|
|
|
|
|
|
// already terminal) renders as done/failed instead of pending —
|
|
|
|
|
|
// otherwise legacy completed documents would forever look like
|
|
|
|
|
|
// they're still waiting in the queue.
|
|
|
|
|
|
tree, currentStageName, lastErr := buildSpanTree(knowledge.ID, currentAttempt, rows, knowledge.ParseStatus)
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
|
2026-05-27 05:31:06 +00:00
|
|
|
|
resp := gin.H{
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
"knowledge_id": knowledge.ID,
|
|
|
|
|
|
"parse_status": knowledge.ParseStatus,
|
|
|
|
|
|
"current_attempt": currentAttempt,
|
|
|
|
|
|
"current_stage": currentStageName,
|
|
|
|
|
|
"trace": tree,
|
2026-05-27 05:31:06 +00:00
|
|
|
|
}
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
if lastErr != nil {
|
2026-05-27 05:31:06 +00:00
|
|
|
|
resp["last_error"] = gin.H{
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
"stage": lastErr.Name,
|
|
|
|
|
|
"code": lastErr.ErrorCode,
|
|
|
|
|
|
"message": lastErr.ErrorMessage,
|
|
|
|
|
|
"finished_at": lastErr.FinishedAt,
|
2026-05-27 05:31:06 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": resp,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// buildSpanTree assembles a flat list of span rows into a parent-child
|
|
|
|
|
|
// tree rooted at the (knowledge, attempt)'s root span. Missing canonical
|
|
|
|
|
|
// stages are filled in with pending placeholders so the UI always renders
|
|
|
|
|
|
// the five timeline segments. Returns the root, the current_stage name
|
|
|
|
|
|
// (the running stage if any), and the most recent failed span if one
|
|
|
|
|
|
// exists.
|
2026-05-27 09:35:58 +00:00
|
|
|
|
//
|
|
|
|
|
|
// parseStatus is the knowledge.parse_status string. When the spans table
|
|
|
|
|
|
// has zero rows for this attempt (legacy data parsed before tracking, or
|
|
|
|
|
|
// a fresh knowledge before the pipeline starts), the placeholder status
|
|
|
|
|
|
// is inferred from parseStatus: completed → done, failed → failed,
|
|
|
|
|
|
// otherwise pending. Without this, every historical knowledge would
|
|
|
|
|
|
// render as "all 5 stages pending" forever despite having actually
|
|
|
|
|
|
// completed parsing.
|
|
|
|
|
|
func buildSpanTree(knowledgeID string, attempt int, rows []types.KnowledgeProcessingSpan, parseStatus string) (
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
root *types.SpanTreeNode, currentStage string, lastFailure *types.KnowledgeProcessingSpan,
|
|
|
|
|
|
) {
|
|
|
|
|
|
now := time.Now()
|
|
|
|
|
|
// Build node lookup, identify root.
|
|
|
|
|
|
nodes := make(map[string]*types.SpanTreeNode, len(rows))
|
|
|
|
|
|
var rootRow *types.KnowledgeProcessingSpan
|
|
|
|
|
|
stageRowByName := map[string]*types.KnowledgeProcessingSpan{}
|
|
|
|
|
|
for i := range rows {
|
|
|
|
|
|
r := rows[i]
|
|
|
|
|
|
nodes[r.SpanID] = &types.SpanTreeNode{KnowledgeProcessingSpan: r}
|
|
|
|
|
|
if r.Kind == types.SpanKindRoot && rootRow == nil {
|
|
|
|
|
|
cp := r
|
|
|
|
|
|
rootRow = &cp
|
|
|
|
|
|
}
|
|
|
|
|
|
if r.Kind == types.SpanKindStage {
|
|
|
|
|
|
cp := r
|
|
|
|
|
|
stageRowByName[r.Name] = &cp
|
|
|
|
|
|
}
|
|
|
|
|
|
if r.Status == types.SpanStatusRunning && r.Kind == types.SpanKindStage && currentStage == "" {
|
|
|
|
|
|
currentStage = r.Name
|
|
|
|
|
|
}
|
|
|
|
|
|
if r.Status == types.SpanStatusFailed {
|
|
|
|
|
|
cp := r
|
|
|
|
|
|
lastFailure = &cp
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-27 09:35:58 +00:00
|
|
|
|
// Pick the synthesized stage status from parse_status. Without this,
|
|
|
|
|
|
// historical knowledge that completed before span tracking was wired
|
|
|
|
|
|
// would render as "5 pending stages" forever — the rows simply
|
|
|
|
|
|
// weren't recorded, but parse_status correctly reads "completed".
|
|
|
|
|
|
// The synthesized stages don't carry duration/timing data; they
|
|
|
|
|
|
// just communicate the inferred terminal state.
|
|
|
|
|
|
syntheticStatus := types.SpanStatusPending
|
|
|
|
|
|
switch parseStatus {
|
|
|
|
|
|
case types.ParseStatusCompleted:
|
|
|
|
|
|
syntheticStatus = types.SpanStatusDone
|
|
|
|
|
|
case types.ParseStatusFailed:
|
|
|
|
|
|
syntheticStatus = types.SpanStatusFailed
|
|
|
|
|
|
}
|
|
|
|
|
|
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
// Synthesize root if no rows came back so the API contract stays
|
|
|
|
|
|
// stable (frontend always expects a `trace` object).
|
|
|
|
|
|
if rootRow == nil {
|
|
|
|
|
|
root = &types.SpanTreeNode{KnowledgeProcessingSpan: types.KnowledgeProcessingSpan{
|
|
|
|
|
|
KnowledgeID: knowledgeID,
|
|
|
|
|
|
Attempt: attempt,
|
|
|
|
|
|
SpanID: "",
|
|
|
|
|
|
Name: "knowledge_processing",
|
|
|
|
|
|
Kind: types.SpanKindRoot,
|
2026-05-27 09:35:58 +00:00
|
|
|
|
Status: syntheticStatus,
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
CreatedAt: now,
|
|
|
|
|
|
UpdatedAt: now,
|
|
|
|
|
|
}}
|
|
|
|
|
|
} else {
|
|
|
|
|
|
root = nodes[rootRow.SpanID]
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-27 17:24:49 +00:00
|
|
|
|
// Link real children to their parents. Walk `rows` (not the
|
|
|
|
|
|
// `nodes` map!) so the append order matches the repo's stable
|
|
|
|
|
|
// `ORDER BY id ASC`. Iterating the map directly would give
|
|
|
|
|
|
// callers a different child ordering on every request — Go map
|
|
|
|
|
|
// iteration is intentionally randomised — and the UI would
|
|
|
|
|
|
// flicker subspans into a different order on each refresh.
|
|
|
|
|
|
for i := range rows {
|
|
|
|
|
|
r := &rows[i]
|
|
|
|
|
|
n := nodes[r.SpanID]
|
|
|
|
|
|
if n == nil || n == root {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if r.ParentSpanID == "" {
|
|
|
|
|
|
// Real top-level row with no parent and not the root
|
|
|
|
|
|
// itself — attach to root so it doesn't dangle.
|
|
|
|
|
|
root.Children = append(root.Children, n)
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
parent, ok := nodes[r.ParentSpanID]
|
|
|
|
|
|
if !ok {
|
|
|
|
|
|
// Unknown parent (orphan); attach to root.
|
|
|
|
|
|
root.Children = append(root.Children, n)
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
parent.Children = append(parent.Children, n)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-27 09:35:58 +00:00
|
|
|
|
// Synthesize missing stage rows as children of root so the timeline
|
|
|
|
|
|
// always shows 5 segments. Status mirrors the synthesized root —
|
|
|
|
|
|
// pending while the pipeline is still running, done/failed for
|
|
|
|
|
|
// historical knowledge whose terminal state we know but whose
|
2026-05-27 17:24:49 +00:00
|
|
|
|
// per-stage timing was never recorded. Appended in AllStages order
|
|
|
|
|
|
// so the canonical stage layout is deterministic regardless of
|
|
|
|
|
|
// which rows are missing.
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
for _, name := range types.AllStages {
|
|
|
|
|
|
if _, ok := stageRowByName[name]; ok {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
placeholder := types.KnowledgeProcessingSpan{
|
|
|
|
|
|
KnowledgeID: knowledgeID,
|
|
|
|
|
|
Attempt: attempt,
|
|
|
|
|
|
Name: name,
|
|
|
|
|
|
Kind: types.SpanKindStage,
|
2026-05-27 09:35:58 +00:00
|
|
|
|
Status: syntheticStatus,
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
CreatedAt: now,
|
|
|
|
|
|
UpdatedAt: now,
|
|
|
|
|
|
}
|
2026-05-27 17:24:49 +00:00
|
|
|
|
root.Children = append(root.Children, &types.SpanTreeNode{KnowledgeProcessingSpan: placeholder})
|
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
2026-05-27 06:05:47 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
return root, currentStage, lastFailure
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// ListKnowledge godoc
|
|
|
|
|
|
// @Summary 获取知识列表
|
|
|
|
|
|
// @Description 获取知识库下的知识列表,支持分页和筛选
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param page query int false "页码"
|
|
|
|
|
|
// @Param page_size query int false "每页数量"
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
// @Param tag_id query string false "标签ID筛选"
|
|
|
|
|
|
// @Param keyword query string false "关键词搜索"
|
|
|
|
|
|
// @Param file_type query string false "文件类型筛选"
|
|
|
|
|
|
// @Param parse_status query string false "解析状态筛选 (pending/processing/completed/failed)"
|
|
|
|
|
|
// @Param source query string false "来源/渠道筛选 (web/api/feishu/notion/yuque/wechat/...,或 manual/url 按 type 过滤)"
|
|
|
|
|
|
// @Param start_time query string false "更新时间起点,RFC3339 格式"
|
|
|
|
|
|
// @Param end_time query string false "更新时间终点,RFC3339 格式"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "知识列表"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/knowledge [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) ListKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start retrieving knowledge list")
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Validate access to the knowledge base (read access - any permission level)
|
|
|
|
|
|
_, kbID, effectiveTenantID, _, err := h.validateKnowledgeBaseAccess(c)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Update context with effective tenant ID for shared KB access
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Parse pagination parameters from query string
|
|
|
|
|
|
var pagination types.Pagination
|
|
|
|
|
|
if err := c.ShouldBindQuery(&pagination); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse pagination parameters", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
filter := types.KnowledgeListFilter{
|
|
|
|
|
|
TagID: c.Query("tag_id"),
|
|
|
|
|
|
Keyword: c.Query("keyword"),
|
|
|
|
|
|
FileType: c.Query("file_type"),
|
|
|
|
|
|
ParseStatus: c.Query("parse_status"),
|
|
|
|
|
|
Source: c.Query("source"),
|
|
|
|
|
|
}
|
|
|
|
|
|
if raw := c.Query("start_time"); raw != "" {
|
|
|
|
|
|
t, err := parseFilterTime(raw)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("invalid start_time: " + err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
filter.UpdatedFrom = t
|
|
|
|
|
|
}
|
|
|
|
|
|
if raw := c.Query("end_time"); raw != "" {
|
|
|
|
|
|
t, err := parseFilterTime(raw)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("invalid end_time: " + err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
filter.UpdatedTo = t
|
|
|
|
|
|
}
|
2025-11-17 10:15:02 +00:00
|
|
|
|
|
2025-11-28 08:17:52 +00:00
|
|
|
|
logger.Infof(
|
|
|
|
|
|
ctx,
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
"Retrieving knowledge list under knowledge base, kb_id=%s tag_id=%s keyword=%s file_type=%s parse_status=%s source=%s start_time=%s end_time=%s page=%d page_size=%d effectiveTenantID=%d",
|
2025-11-28 08:17:52 +00:00
|
|
|
|
secutils.SanitizeForLog(kbID),
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
secutils.SanitizeForLog(filter.TagID),
|
|
|
|
|
|
secutils.SanitizeForLog(filter.Keyword),
|
|
|
|
|
|
secutils.SanitizeForLog(filter.FileType),
|
|
|
|
|
|
secutils.SanitizeForLog(filter.ParseStatus),
|
|
|
|
|
|
secutils.SanitizeForLog(filter.Source),
|
|
|
|
|
|
secutils.SanitizeForLog(c.Query("start_time")),
|
|
|
|
|
|
secutils.SanitizeForLog(c.Query("end_time")),
|
2025-11-28 08:17:52 +00:00
|
|
|
|
pagination.Page,
|
|
|
|
|
|
pagination.PageSize,
|
2026-01-30 08:28:01 +00:00
|
|
|
|
effectiveTenantID,
|
2025-11-28 08:17:52 +00:00
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Retrieve paginated knowledge entries
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
result, err := h.kgService.ListPagedKnowledgeByKnowledgeBaseID(ctx, kbID, &pagination, filter)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-28 08:17:52 +00:00
|
|
|
|
logger.Infof(
|
|
|
|
|
|
ctx,
|
|
|
|
|
|
"Knowledge list retrieved successfully, knowledge base ID: %s, total: %d",
|
|
|
|
|
|
secutils.SanitizeForLog(kbID),
|
|
|
|
|
|
result.Total,
|
|
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": result.Data,
|
|
|
|
|
|
"total": result.Total,
|
|
|
|
|
|
"page": result.Page,
|
|
|
|
|
|
"page_size": result.PageSize,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// DeleteKnowledge godoc
|
|
|
|
|
|
// @Summary 删除知识
|
2026-05-25 08:54:26 +00:00
|
|
|
|
// @Description 根据ID异步删除知识条目。请求会被入队到与批量删除相同的异步管道(asynq);
|
|
|
|
|
|
// @Description 接口返回 200 仅表示任务已提交(响应 data.task_id 为任务 ID),实际删除由后台 worker 完成。
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
2026-05-25 08:54:26 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "任务已提交,返回 task_id"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/{id} [delete]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) DeleteKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start deleting knowledge")
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-05-25 08:54:26 +00:00
|
|
|
|
|
|
|
|
|
|
// Reuse the batch async pipeline so single-item delete shares the same
|
|
|
|
|
|
// hardening (asynq retries, business-aware queue routing, marking-as-deleting
|
|
|
|
|
|
// inside the worker) as BatchDeleteKnowledge / ClearKnowledgeBaseContents.
|
|
|
|
|
|
effectiveTenantID, _ := effCtx.Value(types.TenantIDContextKey).(uint64)
|
|
|
|
|
|
if effectiveTenantID == 0 {
|
|
|
|
|
|
logger.Error(ctx, "Effective tenant ID missing after access validation")
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("tenant context unavailable"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Enqueuing knowledge delete, ID: %s", secutils.SanitizeForLog(id))
|
|
|
|
|
|
taskID, err := h.enqueueKnowledgeListDelete(effCtx, effectiveTenantID, []string{id})
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
2026-05-25 08:54:26 +00:00
|
|
|
|
logger.Errorf(ctx, "Failed to enqueue knowledge delete task: %v", err)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to enqueue delete task"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-25 08:54:26 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge delete task enqueued: %s, knowledge_id: %s", taskID, secutils.SanitizeForLog(id))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
2026-05-25 08:54:26 +00:00
|
|
|
|
"message": "Delete task submitted",
|
|
|
|
|
|
"data": gin.H{
|
|
|
|
|
|
"task_id": taskID,
|
|
|
|
|
|
},
|
2025-08-05 07:08:07 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(knowledge-base): add document batch delete and list view
Add a list-view alongside the existing card grid (issue #957) and
multi-select batch delete (issue #1045) for documents inside a knowledge
base. The two views are toggled from the toolbar and the preference is
persisted in localStorage. Selection state is shared between views with
shift-range support and a sticky batch-action bar.
Backend
- POST /api/v1/knowledge/batch-delete: validates KB scope and editor
permission, then delegates to the existing DeleteKnowledgeList service
which already cascades vector/file/graph cleanup. Caps batch size at
200 and uses a single GetKnowledgeBatch call for membership validation.
Frontend
- DocumentListView: table layout with sticky header, checkbox column
(with indeterminate select-all), file-type icons, status badges,
per-row action menu.
- DocumentBatchBar: floating pill bar with selected count and delete
action; appears when items are selected.
- KnowledgeBase.vue: view-mode toggle, selection state, batch-delete
confirmation dialog, hover/active checkbox overlay on existing cards.
- Extracted formatFileSize and getFileIcon to utils/files.ts to share
with the new list view and avoid further drift from the existing
inline copies.
- i18n keys added for zh-CN, en-US, ko-KR, ru-RU.
Closes #1045
Closes #957
2026-04-28 19:24:35 +00:00
|
|
|
|
// BatchDeleteKnowledgeRequest is the body schema for POST /knowledge/batch-delete.
|
|
|
|
|
|
type BatchDeleteKnowledgeRequest struct {
|
|
|
|
|
|
KBID string `json:"kb_id" binding:"required"`
|
|
|
|
|
|
IDs []string `json:"ids" binding:"required"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// BatchDeleteKnowledge godoc
|
|
|
|
|
|
// @Summary 批量删除知识
|
|
|
|
|
|
// @Description 按 ID 列表批量删除单个知识库下的多个知识条目
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param request body BatchDeleteKnowledgeRequest true "批量删除请求"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "删除成功"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 403 {object} errors.AppError "权限不足"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/batch-delete [post]
|
|
|
|
|
|
func (h *KnowledgeHandler) BatchDeleteKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
var req BatchDeleteKnowledgeRequest
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid request parameters: " + err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Deduplicate and drop empty IDs.
|
|
|
|
|
|
seen := make(map[string]struct{}, len(req.IDs))
|
|
|
|
|
|
ids := make([]string, 0, len(req.IDs))
|
|
|
|
|
|
for _, raw := range req.IDs {
|
|
|
|
|
|
id := strings.TrimSpace(raw)
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if _, ok := seen[id]; ok {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
seen[id] = struct{}{}
|
|
|
|
|
|
ids = append(ids, id)
|
|
|
|
|
|
}
|
|
|
|
|
|
if len(ids) == 0 {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("ids cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
const maxBatch = 200
|
|
|
|
|
|
if len(ids) > maxBatch {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(fmt.Sprintf("too many ids (max %d per batch)", maxBatch)))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate KB access (editor or admin) using the kb_id from body.
|
|
|
|
|
|
_, kbID, effectiveTenantID, permission, err := h.validateKnowledgeBaseAccessWithKBID(c, req.KBID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to delete knowledge"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
// Single batch fetch to validate that every id exists and belongs to the
|
|
|
|
|
|
// requested KB. The service-layer DeleteKnowledgeList only enforces tenant
|
|
|
|
|
|
// scope, not KB scope, so the handler must guard against cross-KB deletion.
|
|
|
|
|
|
knowledgeList, err := h.kgService.GetKnowledgeBatch(ctx, effectiveTenantID, ids)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if len(knowledgeList) != len(ids) {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("One or more knowledge entries not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
for _, k := range knowledgeList {
|
|
|
|
|
|
if k.KnowledgeBaseID != kbID {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(
|
|
|
|
|
|
fmt.Sprintf("Knowledge %s does not belong to knowledge base %s",
|
|
|
|
|
|
secutils.SanitizeForLog(k.ID), secutils.SanitizeForLog(kbID))))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-29 03:47:59 +00:00
|
|
|
|
taskID, err := h.enqueueKnowledgeListDelete(ctx, effectiveTenantID, ids)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to enqueue batch knowledge delete task: %v", err)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to enqueue batch delete task"))
|
feat(knowledge-base): add document batch delete and list view
Add a list-view alongside the existing card grid (issue #957) and
multi-select batch delete (issue #1045) for documents inside a knowledge
base. The two views are toggled from the toolbar and the preference is
persisted in localStorage. Selection state is shared between views with
shift-range support and a sticky batch-action bar.
Backend
- POST /api/v1/knowledge/batch-delete: validates KB scope and editor
permission, then delegates to the existing DeleteKnowledgeList service
which already cascades vector/file/graph cleanup. Caps batch size at
200 and uses a single GetKnowledgeBatch call for membership validation.
Frontend
- DocumentListView: table layout with sticky header, checkbox column
(with indeterminate select-all), file-type icons, status badges,
per-row action menu.
- DocumentBatchBar: floating pill bar with selected count and delete
action; appears when items are selected.
- KnowledgeBase.vue: view-mode toggle, selection state, batch-delete
confirmation dialog, hover/active checkbox overlay on existing cards.
- Extracted formatFileSize and getFileIcon to utils/files.ts to share
with the new list view and avoid further drift from the existing
inline copies.
- i18n keys added for zh-CN, en-US, ko-KR, ru-RU.
Closes #1045
Closes #957
2026-04-28 19:24:35 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-29 03:47:59 +00:00
|
|
|
|
logger.Infof(ctx, "Batch knowledge delete task enqueued: %s, kb_id: %s, count: %d",
|
|
|
|
|
|
taskID, secutils.SanitizeForLog(kbID), len(ids))
|
|
|
|
|
|
|
feat(knowledge-base): add document batch delete and list view
Add a list-view alongside the existing card grid (issue #957) and
multi-select batch delete (issue #1045) for documents inside a knowledge
base. The two views are toggled from the toolbar and the preference is
persisted in localStorage. Selection state is shared between views with
shift-range support and a sticky batch-action bar.
Backend
- POST /api/v1/knowledge/batch-delete: validates KB scope and editor
permission, then delegates to the existing DeleteKnowledgeList service
which already cascades vector/file/graph cleanup. Caps batch size at
200 and uses a single GetKnowledgeBatch call for membership validation.
Frontend
- DocumentListView: table layout with sticky header, checkbox column
(with indeterminate select-all), file-type icons, status badges,
per-row action menu.
- DocumentBatchBar: floating pill bar with selected count and delete
action; appears when items are selected.
- KnowledgeBase.vue: view-mode toggle, selection state, batch-delete
confirmation dialog, hover/active checkbox overlay on existing cards.
- Extracted formatFileSize and getFileIcon to utils/files.ts to share
with the new list view and avoid further drift from the existing
inline copies.
- i18n keys added for zh-CN, en-US, ko-KR, ru-RU.
Closes #1045
Closes #957
2026-04-28 19:24:35 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
2026-04-29 03:47:59 +00:00
|
|
|
|
"message": "Batch delete task submitted",
|
|
|
|
|
|
"data": gin.H{
|
|
|
|
|
|
"task_id": taskID,
|
|
|
|
|
|
"deleted_count": len(ids),
|
|
|
|
|
|
},
|
feat(knowledge-base): add document batch delete and list view
Add a list-view alongside the existing card grid (issue #957) and
multi-select batch delete (issue #1045) for documents inside a knowledge
base. The two views are toggled from the toolbar and the preference is
persisted in localStorage. Selection state is shared between views with
shift-range support and a sticky batch-action bar.
Backend
- POST /api/v1/knowledge/batch-delete: validates KB scope and editor
permission, then delegates to the existing DeleteKnowledgeList service
which already cascades vector/file/graph cleanup. Caps batch size at
200 and uses a single GetKnowledgeBatch call for membership validation.
Frontend
- DocumentListView: table layout with sticky header, checkbox column
(with indeterminate select-all), file-type icons, status badges,
per-row action menu.
- DocumentBatchBar: floating pill bar with selected count and delete
action; appears when items are selected.
- KnowledgeBase.vue: view-mode toggle, selection state, batch-delete
confirmation dialog, hover/active checkbox overlay on existing cards.
- Extracted formatFileSize and getFileIcon to utils/files.ts to share
with the new list view and avoid further drift from the existing
inline copies.
- i18n keys added for zh-CN, en-US, ko-KR, ru-RU.
Closes #1045
Closes #957
2026-04-28 19:24:35 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-24 08:29:55 +00:00
|
|
|
|
// ClearKnowledgeBaseContents godoc
|
|
|
|
|
|
// @Summary 清空知识库内容
|
|
|
|
|
|
// @Description 删除知识库下的所有知识条目(异步任务)。知识库本身保留,仅清空其中的内容
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "清空任务已提交"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 403 {object} errors.AppError "权限不足"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge-bases/{id}/knowledge [delete]
|
|
|
|
|
|
func (h *KnowledgeHandler) ClearKnowledgeBaseContents(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start clearing knowledge base contents")
|
|
|
|
|
|
|
|
|
|
|
|
kb, kbID, effectiveTenantID, permission, err := h.validateKnowledgeBaseAccess(c)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Only owner (admin with matching tenant) can clear knowledge base contents
|
|
|
|
|
|
tenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if kb.TenantID != tenantID || permission != types.OrgRoleAdmin {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("Only knowledge base owner can clear contents"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
knowledgeList, err := h.kgService.ListKnowledgeByKnowledgeBaseID(ctx, kbID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to list knowledge entries").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
if len(knowledgeList) == 0 {
|
|
|
|
|
|
logger.Infof(ctx, "Knowledge base %s is already empty", secutils.SanitizeForLog(kbID))
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge base is already empty",
|
|
|
|
|
|
"data": gin.H{"deleted_count": 0},
|
|
|
|
|
|
})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
knowledgeIDs := make([]string, 0, len(knowledgeList))
|
|
|
|
|
|
for _, knowledge := range knowledgeList {
|
|
|
|
|
|
knowledgeIDs = append(knowledgeIDs, knowledge.ID)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-29 03:47:59 +00:00
|
|
|
|
taskID, err := h.enqueueKnowledgeListDelete(ctx, effectiveTenantID, knowledgeIDs)
|
2026-03-24 08:29:55 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to enqueue knowledge list delete task: %v", err)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to enqueue cleanup task"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Knowledge base contents clear task enqueued: %s, kb_id: %s, count: %d",
|
2026-04-29 03:47:59 +00:00
|
|
|
|
taskID, secutils.SanitizeForLog(kbID), len(knowledgeIDs))
|
2026-03-24 08:29:55 +00:00
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge base contents clear task submitted",
|
|
|
|
|
|
"data": gin.H{"deleted_count": len(knowledgeIDs)},
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// DownloadKnowledgeFile godoc
|
|
|
|
|
|
// @Summary 下载知识文件
|
|
|
|
|
|
// @Description 下载知识条目关联的原始文件
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce application/octet-stream
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Success 200 {file} file "文件内容"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/{id}/download [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) DownloadKnowledgeFile(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start downloading knowledge file")
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleViewer)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-11-25 15:33:34 +00:00
|
|
|
|
logger.Infof(ctx, "Retrieving knowledge file, ID: %s", secutils.SanitizeForLog(id))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
file, filename, err := h.kgService.GetKnowledgeFile(effCtx, id)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to retrieve file").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
defer file.Close()
|
|
|
|
|
|
|
2025-11-28 08:17:52 +00:00
|
|
|
|
logger.Infof(
|
|
|
|
|
|
ctx,
|
|
|
|
|
|
"Knowledge file retrieved successfully, ID: %s, filename: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(id),
|
|
|
|
|
|
secutils.SanitizeForLog(filename),
|
|
|
|
|
|
)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Set response headers for file download
|
|
|
|
|
|
c.Header("Content-Description", "File Transfer")
|
|
|
|
|
|
c.Header("Content-Transfer-Encoding", "binary")
|
2026-03-16 10:01:07 +00:00
|
|
|
|
cd := mime.FormatMediaType("attachment", map[string]string{"filename": filename})
|
|
|
|
|
|
c.Header("Content-Disposition", cd)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.Header("Content-Type", "application/octet-stream")
|
|
|
|
|
|
c.Header("Expires", "0")
|
|
|
|
|
|
c.Header("Cache-Control", "must-revalidate")
|
|
|
|
|
|
c.Header("Pragma", "public")
|
|
|
|
|
|
|
|
|
|
|
|
// Stream file content to response
|
|
|
|
|
|
c.Stream(func(w io.Writer) bool {
|
|
|
|
|
|
if _, err := io.Copy(w, file); err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to send file: %v", err)
|
|
|
|
|
|
return false
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.Debug(ctx, "File sending completed")
|
|
|
|
|
|
return false
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-28 05:22:07 +00:00
|
|
|
|
// mimeTypeByExt returns the MIME type for a given file extension.
|
|
|
|
|
|
func mimeTypeByExt(filename string) string {
|
|
|
|
|
|
ext := strings.ToLower(filename)
|
|
|
|
|
|
if idx := strings.LastIndex(ext, "."); idx >= 0 {
|
|
|
|
|
|
ext = ext[idx:]
|
|
|
|
|
|
} else {
|
|
|
|
|
|
ext = ""
|
|
|
|
|
|
}
|
|
|
|
|
|
m := map[string]string{
|
|
|
|
|
|
".pdf": "application/pdf",
|
|
|
|
|
|
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
|
|
|
|
|
|
".doc": "application/msword",
|
|
|
|
|
|
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
|
|
|
|
|
|
".ppt": "application/vnd.ms-powerpoint",
|
|
|
|
|
|
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
|
|
|
|
|
".xls": "application/vnd.ms-excel",
|
|
|
|
|
|
".csv": "text/csv",
|
|
|
|
|
|
".jpg": "image/jpeg",
|
|
|
|
|
|
".jpeg": "image/jpeg",
|
|
|
|
|
|
".png": "image/png",
|
|
|
|
|
|
".gif": "image/gif",
|
|
|
|
|
|
".bmp": "image/bmp",
|
|
|
|
|
|
".webp": "image/webp",
|
|
|
|
|
|
".svg": "image/svg+xml",
|
|
|
|
|
|
".tiff": "image/tiff",
|
|
|
|
|
|
".txt": "text/plain; charset=utf-8",
|
|
|
|
|
|
".md": "text/markdown; charset=utf-8",
|
|
|
|
|
|
".markdown": "text/markdown; charset=utf-8",
|
|
|
|
|
|
".json": "application/json; charset=utf-8",
|
|
|
|
|
|
".xml": "application/xml; charset=utf-8",
|
|
|
|
|
|
".html": "text/html; charset=utf-8",
|
|
|
|
|
|
".css": "text/css; charset=utf-8",
|
|
|
|
|
|
".js": "text/javascript; charset=utf-8",
|
|
|
|
|
|
".ts": "text/typescript; charset=utf-8",
|
|
|
|
|
|
".py": "text/x-python; charset=utf-8",
|
|
|
|
|
|
".go": "text/x-go; charset=utf-8",
|
|
|
|
|
|
".java": "text/x-java; charset=utf-8",
|
|
|
|
|
|
".yaml": "text/yaml; charset=utf-8",
|
|
|
|
|
|
".yml": "text/yaml; charset=utf-8",
|
|
|
|
|
|
".sh": "text/x-shellscript; charset=utf-8",
|
|
|
|
|
|
}
|
|
|
|
|
|
if ct, ok := m[ext]; ok {
|
|
|
|
|
|
return ct
|
|
|
|
|
|
}
|
|
|
|
|
|
return "application/octet-stream"
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// PreviewKnowledgeFile godoc
|
|
|
|
|
|
// @Summary 预览知识文件
|
|
|
|
|
|
// @Description 返回知识条目关联的原始文件,Content-Type 根据文件类型设置,用于浏览器内嵌预览
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce application/pdf,image/jpeg,image/png,text/plain
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Success 200 {file} file "文件内容"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/{id}/preview [get]
|
|
|
|
|
|
func (h *KnowledgeHandler) PreviewKnowledgeFile(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleViewer)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
file, filename, err := h.kgService.GetKnowledgeFile(effCtx, id)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to retrieve file").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
defer file.Close()
|
|
|
|
|
|
|
|
|
|
|
|
contentType := mimeTypeByExt(filename)
|
|
|
|
|
|
c.Header("Content-Type", contentType)
|
2026-03-16 10:01:07 +00:00
|
|
|
|
c.Header("Content-Disposition", mime.FormatMediaType("inline", map[string]string{"filename": filename}))
|
2026-02-28 05:22:07 +00:00
|
|
|
|
c.Header("Cache-Control", "private, max-age=3600")
|
|
|
|
|
|
|
|
|
|
|
|
c.Stream(func(w io.Writer) bool {
|
|
|
|
|
|
if _, err := io.Copy(w, file); err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to stream preview: %v", err)
|
|
|
|
|
|
return false
|
|
|
|
|
|
}
|
|
|
|
|
|
return false
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// GetKnowledgeBatchRequest defines parameters for batch knowledge retrieval
|
|
|
|
|
|
type GetKnowledgeBatchRequest struct {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
IDs []string `form:"ids" binding:"required"` // List of knowledge IDs
|
|
|
|
|
|
KBID string `form:"kb_id"` // Optional: scope to this KB (validates access and uses effective tenant for shared KB)
|
|
|
|
|
|
AgentID string `form:"agent_id"` // Optional: when using a shared agent, use agent's tenant for retrieval (validates shared agent access)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// GetKnowledgeBatch godoc
|
|
|
|
|
|
// @Summary 批量获取知识
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Description 根据ID列表批量获取知识条目。可选 kb_id:指定时按该知识库校验权限并用于共享知识库的租户解析;可选 agent_id:使用共享智能体时传此参数,后端按智能体所属租户查询(用于刷新后恢复共享知识库下的文件)
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Param ids query []string true "知识ID列表"
|
|
|
|
|
|
// @Param kb_id query string false "可选,知识库ID(用于共享知识库时指定范围)"
|
|
|
|
|
|
// @Param agent_id query string false "可选,共享智能体ID(用于按智能体租户批量拉取文件详情)"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "知识列表"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/batch [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) GetKnowledgeBatch(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
tenantID, ok := c.Get(types.TenantIDContextKey.String())
|
|
|
|
|
|
if !ok {
|
|
|
|
|
|
logger.Error(ctx, "Failed to get tenant ID")
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
effectiveTenantID := tenantID.(uint64)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
var req GetKnowledgeBatchRequest
|
|
|
|
|
|
if err := c.ShouldBindQuery(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid request parameters").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-01 13:45:35 +00:00
|
|
|
|
// agentAllowedKBIDs restricts results to the agent's configured KB scope.
|
|
|
|
|
|
// nil = no agent restriction; empty slice = agent has no KB access (none mode).
|
|
|
|
|
|
var agentAllowedKBIDs []string
|
|
|
|
|
|
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// Optional agent_id: when using shared agent, resolve agent and use its tenant for batch retrieval (so shared KB files can be loaded after refresh)
|
|
|
|
|
|
if agentID := secutils.SanitizeForLog(req.AgentID); agentID != "" && h.agentShareService != nil {
|
|
|
|
|
|
userIDVal, ok := c.Get(types.UserIDContextKey.String())
|
|
|
|
|
|
if !ok {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
userID, _ := userIDVal.(string)
|
|
|
|
|
|
currentTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if currentTenantID == 0 {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
callerTenantRole := types.TenantRoleFromContext(ctx)
|
|
|
|
|
|
agent, err := h.agentShareService.GetSharedAgentForTenant(ctx, currentTenantID, callerTenantRole, agentID)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if err != nil || agent == nil {
|
|
|
|
|
|
logger.Warnf(ctx, "GetKnowledgeBatch: invalid or inaccessible shared agent %s: %v", agentID, err)
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("Invalid or inaccessible shared agent").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
_ = userID
|
2026-02-04 12:12:29 +00:00
|
|
|
|
effectiveTenantID = agent.TenantID
|
2026-04-01 13:45:35 +00:00
|
|
|
|
agentAllowedKBIDs = resolveAgentAllowedKBIDs(agent)
|
|
|
|
|
|
|
|
|
|
|
|
if agentAllowedKBIDs != nil && len(agentAllowedKBIDs) == 0 {
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{"success": true, "data": []*types.Knowledge{}})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.Infof(ctx, "Batch retrieving knowledge with agent_id, effective tenant ID: %d, IDs count: %d, allowed KBs: %v",
|
|
|
|
|
|
effectiveTenantID, len(req.IDs), agentAllowedKBIDs)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
var knowledges []*types.Knowledge
|
|
|
|
|
|
var err error
|
|
|
|
|
|
|
2026-04-01 13:45:35 +00:00
|
|
|
|
// scopeKBID tracks the single KB the results must belong to (set by explicit kb_id).
|
|
|
|
|
|
var scopeKBID string
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Optional kb_id: validate KB access and use effective tenant for shared KB
|
|
|
|
|
|
if kbID := secutils.SanitizeForLog(req.KBID); kbID != "" {
|
|
|
|
|
|
_, _, effID, _, err := h.validateKnowledgeBaseAccessWithKBID(c, kbID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
if agentAllowedKBIDs != nil && !sliceContains(agentAllowedKBIDs, kbID) {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("Knowledge base not accessible through this agent"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
scopeKBID = kbID
|
2026-01-30 08:28:01 +00:00
|
|
|
|
effectiveTenantID = effID
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effectiveTenantID)
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Batch retrieving knowledge with kb_id, effective tenant ID: %d, IDs count: %d",
|
|
|
|
|
|
effectiveTenantID, len(req.IDs))
|
|
|
|
|
|
|
|
|
|
|
|
knowledges, err = h.kgService.GetKnowledgeBatch(ctx, effectiveTenantID, req.IDs)
|
|
|
|
|
|
} else {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// No kb_id: use GetKnowledgeBatchWithSharedAccess (or effectiveTenantID may already be set by agent_id for shared agent)
|
|
|
|
|
|
logger.Infof(ctx, "Batch retrieving knowledge without kb_id, effective tenant ID: %d, IDs count: %d",
|
2026-01-30 08:28:01 +00:00
|
|
|
|
effectiveTenantID, len(req.IDs))
|
|
|
|
|
|
|
|
|
|
|
|
knowledges, err = h.kgService.GetKnowledgeBatchWithSharedAccess(ctx, effectiveTenantID, req.IDs)
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-04-01 13:45:35 +00:00
|
|
|
|
// Build the effective allowed-KB set from both scopeKBID and agentAllowedKBIDs.
|
|
|
|
|
|
// scopeKBID (from explicit kb_id) restricts to a single KB;
|
|
|
|
|
|
// agentAllowedKBIDs (from shared agent) restricts to the agent's configured KBs.
|
|
|
|
|
|
var allowedKBSet map[string]bool
|
|
|
|
|
|
if scopeKBID != "" {
|
|
|
|
|
|
allowedKBSet = map[string]bool{scopeKBID: true}
|
|
|
|
|
|
} else if agentAllowedKBIDs != nil {
|
|
|
|
|
|
allowedKBSet = make(map[string]bool, len(agentAllowedKBIDs))
|
|
|
|
|
|
for _, id := range agentAllowedKBIDs {
|
|
|
|
|
|
allowedKBSet[id] = true
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if allowedKBSet != nil && len(knowledges) > 0 {
|
|
|
|
|
|
filtered := make([]*types.Knowledge, 0, len(knowledges))
|
|
|
|
|
|
for _, k := range knowledges {
|
|
|
|
|
|
if allowedKBSet[k.KnowledgeBaseID] {
|
|
|
|
|
|
filtered = append(filtered, k)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
knowledges = filtered
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to retrieve knowledge list").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
logger.Infof(ctx, "Batch knowledge retrieval successful, requested count: %d, returned count: %d",
|
|
|
|
|
|
len(req.IDs), len(knowledges))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledges,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// UpdateKnowledge godoc
|
|
|
|
|
|
// @Summary 更新知识
|
|
|
|
|
|
// @Description 更新知识条目信息
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Param request body types.Knowledge true "知识信息"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新成功"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/{id} [put]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) UpdateKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
2026-01-30 08:28:01 +00:00
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
var knowledge types.Knowledge
|
|
|
|
|
|
if err := c.ShouldBindJSON(&knowledge); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-01-30 08:28:01 +00:00
|
|
|
|
knowledge.ID = id
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
if err := h.kgService.UpdateKnowledge(effCtx, &knowledge); err != nil {
|
2025-08-05 07:08:07 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge updated successfully, knowledge ID: %s", id)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge chunk updated successfully",
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// UpdateManualKnowledge godoc
|
|
|
|
|
|
// @Summary 更新手工知识
|
|
|
|
|
|
// @Description 更新手工录入的Markdown知识内容
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Param request body types.ManualKnowledgePayload true "手工知识内容"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新后的知识"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/manual/{id} [put]
|
2025-11-12 13:27:56 +00:00
|
|
|
|
func (h *KnowledgeHandler) UpdateManualKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start updating manual knowledge")
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
|
|
|
|
|
if id == "" {
|
2025-11-12 13:27:56 +00:00
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-12 13:27:56 +00:00
|
|
|
|
var req types.ManualKnowledgePayload
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse manual knowledge update request", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
knowledge, err := h.kgService.UpdateManualKnowledge(effCtx, id, &req)
|
2025-11-12 13:27:56 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, map[string]interface{}{
|
2025-11-25 16:19:26 +00:00
|
|
|
|
"knowledge_id": id,
|
2025-11-12 13:27:56 +00:00
|
|
|
|
})
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
logger.Infof(ctx, "Manual knowledge updated successfully, knowledge ID: %s", id)
|
2025-11-12 13:27:56 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledge,
|
2026-02-10 04:15:21 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// ReparseKnowledge godoc
|
|
|
|
|
|
// @Summary 重新解析知识
|
|
|
|
|
|
// @Description 删除知识中现有的文档内容并重新解析,使用异步任务方式处理
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
2026-06-09 12:34:16 +00:00
|
|
|
|
// @Param body body object false "可选的处理配置覆盖:{\"process_config\": KnowledgeProcessOverrides}"
|
2026-02-10 04:15:21 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "重新解析任务已提交"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 403 {object} errors.AppError "权限不足"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/{id}/reparse [post]
|
|
|
|
|
|
func (h *KnowledgeHandler) ReparseKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start re-parsing knowledge")
|
|
|
|
|
|
|
|
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate KB access with editor permission (reparse requires write access)
|
|
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-06-09 12:34:16 +00:00
|
|
|
|
// Optional per-reparse parse config override. Empty body keeps the
|
|
|
|
|
|
// overrides stored at upload time.
|
|
|
|
|
|
var processOverrides *types.KnowledgeProcessOverrides
|
|
|
|
|
|
if c.Request.ContentLength != 0 {
|
|
|
|
|
|
var req struct {
|
|
|
|
|
|
ProcessConfig *types.KnowledgeProcessOverrides `json:"process_config"`
|
|
|
|
|
|
}
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse reparse request body", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid reparse request body").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
processOverrides = req.ProcessConfig
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-10 04:15:21 +00:00
|
|
|
|
// Call service to reparse knowledge
|
2026-06-09 12:34:16 +00:00
|
|
|
|
knowledge, err := h.kgService.ReparseKnowledge(effCtx, id, processOverrides)
|
2026-02-10 04:15:21 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, map[string]interface{}{
|
|
|
|
|
|
"knowledge_id": id,
|
|
|
|
|
|
})
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Knowledge reparse task submitted successfully, knowledge ID: %s", id)
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge reparse task submitted",
|
|
|
|
|
|
"data": knowledge,
|
2025-11-12 13:27:56 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat: add knowledge parse cancellation with finalizing post-process state
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.
State machine
pending -> processing -> finalizing -> completed
|
+-> cancelled (any of the three
in-flight states)
+-> failed
+-> deleting
`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.
Backend
- New ParseStatusFinalizing + pending_subtasks_count column with
migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
count up front, flip to finalizing (or completed when no
enrichment is enabled), and only then fan out subtasks. Subtask
handlers (summary, question, graph extract) defer-decrement on
terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
pending / processing / finalizing. Marks the row cancelled,
zeroes the counter, best-effort dequeues asynq tasks via a new
TaskInspector abstraction (asynq-mode walks pending/scheduled/
retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
for the attempt via a new repo.CancelAllOpenSpans helper so the
trace viewer's striped bars all flip to cancelled, even leaf
generations whose parent stage already EndSpan'd (multimodal
fan-out pattern). knowledge_post_process closes its postSpan
via SkipSpan on the cancel/deleting entry guard so a worker
that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
expensive enrichment (graph extract) bails immediately when the
parent knowledge is aborted.
Frontend
- New cancelKnowledgeParse API client + "Stop parsing" entry in
both list view and card view more menus, gated on
pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
that recognises `finalizing` (previously the doc list silently
stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
new isHardTerminal short-circuits LIVE for cancelled/failed/
completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
("增强中") from completed and shows the previous "生成摘要中"
copy when summary_status is still pending under finalizing.
Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
across zh-CN, en-US, ko-KR, ru-RU.
Docs / SDK
- docs/api/knowledge.md: documents the new finalizing state,
cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
the cancellable statuses.
2026-05-28 10:26:56 +00:00
|
|
|
|
// CancelKnowledgeParse godoc
|
|
|
|
|
|
// @Summary 取消知识解析
|
|
|
|
|
|
// @Description 取消进行中的知识解析任务。当前已写入的 chunk / 索引保留,可通过 reparse 接口重新触发解析。已完成 / 已失败 / 删除中的知识不支持取消。
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "取消已提交"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "状态不支持取消"
|
|
|
|
|
|
// @Failure 403 {object} errors.AppError "权限不足"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "知识不存在"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/{id}/cancel-parse [post]
|
|
|
|
|
|
func (h *KnowledgeHandler) CancelKnowledgeParse(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start cancelling knowledge parse")
|
|
|
|
|
|
|
|
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Editor permission — same gate as ReparseKnowledge / DeleteKnowledge.
|
|
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
knowledge, err := h.kgService.CancelKnowledgeParse(effCtx, id)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if appErr, ok := errors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, map[string]interface{}{
|
|
|
|
|
|
"knowledge_id": id,
|
|
|
|
|
|
})
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Knowledge parse cancelled successfully, knowledge ID: %s", id)
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge parse cancelled",
|
|
|
|
|
|
"data": knowledge,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-18 03:48:37 +00:00
|
|
|
|
type knowledgeTagBatchRequest struct {
|
|
|
|
|
|
Updates map[string]*string `json:"updates" binding:"required,min=1"`
|
2026-01-30 08:28:01 +00:00
|
|
|
|
KBID string `json:"kb_id"` // Optional: scope to this KB (validates editor access and uses effective tenant for shared KB)
|
2025-11-18 03:48:37 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// UpdateKnowledgeTagBatch godoc
|
|
|
|
|
|
// @Summary 批量更新知识标签
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// @Description 批量更新知识条目的标签。可选 kb_id:指定时按该知识库校验编辑权限并用于共享知识库的租户解析
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// @Param request body object true "标签更新请求(updates 必填,kb_id 可选)"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新成功"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/tags [put]
|
2025-11-18 03:48:37 +00:00
|
|
|
|
func (h *KnowledgeHandler) UpdateKnowledgeTagBatch(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
2026-01-30 08:28:01 +00:00
|
|
|
|
|
|
|
|
|
|
// Ensure tenant ID is in context (service reads it; may be missing if request context was not set by auth)
|
|
|
|
|
|
tenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if tenantID == 0 {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, tenantID)
|
|
|
|
|
|
|
2025-11-18 03:48:37 +00:00
|
|
|
|
var req knowledgeTagBatchRequest
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse knowledge tag batch request", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("请求参数不合法").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
// Resolve effective tenant and the authorized KB scope.
|
|
|
|
|
|
var authorizedKBID string
|
2026-01-30 08:28:01 +00:00
|
|
|
|
if kbID := secutils.SanitizeForLog(req.KBID); kbID != "" {
|
|
|
|
|
|
_, _, effID, permission, err := h.validateKnowledgeBaseAccessWithKBID(c, kbID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to update knowledge tags"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
authorizedKBID = kbID
|
2026-01-30 08:28:01 +00:00
|
|
|
|
ctx = context.WithValue(ctx, types.TenantIDContextKey, effID)
|
|
|
|
|
|
} else if len(req.Updates) > 0 {
|
|
|
|
|
|
// No kb_id: infer from first knowledge ID so shared-KB updates work without client sending kb_id
|
|
|
|
|
|
var firstKnowledgeID string
|
|
|
|
|
|
for id := range req.Updates {
|
|
|
|
|
|
firstKnowledgeID = id
|
|
|
|
|
|
break
|
|
|
|
|
|
}
|
|
|
|
|
|
if firstKnowledgeID != "" {
|
2026-04-01 13:45:35 +00:00
|
|
|
|
knowledge, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, firstKnowledgeID, types.OrgRoleEditor)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
authorizedKBID = knowledge.KnowledgeBaseID
|
2026-01-30 08:28:01 +00:00
|
|
|
|
ctx = effCtx
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
if err := h.kgService.UpdateKnowledgeTagBatch(ctx, authorizedKBID, req.Updates); err != nil {
|
2025-11-18 03:48:37 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// UpdateImageInfo godoc
|
|
|
|
|
|
// @Summary 更新图像信息
|
|
|
|
|
|
// @Description 更新知识分块的图像信息
|
|
|
|
|
|
// @Tags 知识管理
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识ID"
|
|
|
|
|
|
// @Param chunk_id path string true "分块ID"
|
|
|
|
|
|
// @Param request body object{image_info=string} true "图像信息"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新成功"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge/image/{id}/{chunk_id} [put]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeHandler) UpdateImageInfo(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start updating image info")
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if id == "" {
|
|
|
|
|
|
logger.Error(ctx, "Knowledge ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-11-25 16:19:26 +00:00
|
|
|
|
chunkID := secutils.SanitizeForLog(c.Param("chunk_id"))
|
|
|
|
|
|
if chunkID == "" {
|
2025-08-05 07:08:07 +00:00
|
|
|
|
logger.Error(ctx, "Chunk ID is empty")
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Chunk ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
var request struct {
|
|
|
|
|
|
ImageInfo string `json:"image_info"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
if err := c.ShouldBindJSON(&request); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
logger.Infof(ctx, "Updating knowledge chunk, knowledge ID: %s, chunk ID: %s", id, chunkID)
|
2026-01-30 08:28:01 +00:00
|
|
|
|
err = h.kgService.UpdateImageInfo(effCtx, id, chunkID, secutils.SanitizeForLog(request.ImageInfo))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-25 16:19:26 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge chunk updated successfully, knowledge ID: %s, chunk ID: %s", id, chunkID)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge chunk image updated successfully",
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
2025-12-19 14:17:56 +00:00
|
|
|
|
|
|
|
|
|
|
// SearchKnowledge godoc
|
|
|
|
|
|
// @Summary Search knowledge
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Description Search knowledge files by keyword. When agent_id is set (shared agent), scope is the agent's configured knowledge bases.
|
2025-12-19 14:17:56 +00:00
|
|
|
|
// @Tags Knowledge
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Param keyword query string false "Keyword to search"
|
|
|
|
|
|
// @Param offset query int false "Offset for pagination"
|
|
|
|
|
|
// @Param limit query int false "Limit for pagination (default 20)"
|
|
|
|
|
|
// @Param file_types query string false "Comma-separated file extensions to filter (e.g., csv,xlsx)"
|
|
|
|
|
|
// @Param agent_id query string false "Shared agent ID (search within agent's KB scope)"
|
2025-12-31 10:18:22 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "Search results"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "Invalid request"
|
2025-12-19 14:17:56 +00:00
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-19 14:17:56 +00:00
|
|
|
|
// @Router /knowledge/search [get]
|
|
|
|
|
|
func (h *KnowledgeHandler) SearchKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
2026-01-30 08:28:01 +00:00
|
|
|
|
if userID, ok := c.Get(types.UserIDContextKey.String()); ok {
|
|
|
|
|
|
ctx = context.WithValue(ctx, types.UserIDContextKey, userID)
|
|
|
|
|
|
}
|
2026-06-05 00:01:02 +00:00
|
|
|
|
// Accept both ?keyword= (legacy / upstream name) and ?query= (what most
|
|
|
|
|
|
// MCP / agent integrations send). Falling silently back to an unsorted
|
|
|
|
|
|
// listing when both are empty caused the "all queries return the same 5
|
|
|
|
|
|
// newest cards" footgun — return 400 instead so the caller gets a clear
|
|
|
|
|
|
// signal that the search wasn't query-driven.
|
2025-12-19 14:17:56 +00:00
|
|
|
|
keyword := c.Query("keyword")
|
2026-06-05 00:01:02 +00:00
|
|
|
|
if keyword == "" {
|
|
|
|
|
|
keyword = c.Query("query")
|
|
|
|
|
|
}
|
|
|
|
|
|
if strings.TrimSpace(keyword) == "" {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("missing search keyword: pass ?keyword=... or ?query=..."))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-12-19 14:17:56 +00:00
|
|
|
|
offset, _ := strconv.Atoi(c.DefaultQuery("offset", "0"))
|
|
|
|
|
|
limit, _ := strconv.Atoi(c.DefaultQuery("limit", "20"))
|
|
|
|
|
|
|
2025-12-31 10:18:22 +00:00
|
|
|
|
var fileTypes []string
|
|
|
|
|
|
if fileTypesStr := c.Query("file_types"); fileTypesStr != "" {
|
|
|
|
|
|
for _, ft := range strings.Split(fileTypesStr, ",") {
|
|
|
|
|
|
ft = strings.TrimSpace(ft)
|
|
|
|
|
|
if ft != "" {
|
|
|
|
|
|
fileTypes = append(fileTypes, ft)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentID := c.Query("agent_id")
|
|
|
|
|
|
if agentID != "" {
|
|
|
|
|
|
userIDVal, ok := c.Get(types.UserIDContextKey.String())
|
|
|
|
|
|
if !ok {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("user ID not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
_ = userIDVal
|
2026-02-04 12:12:29 +00:00
|
|
|
|
currentTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
if currentTenantID == 0 {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("tenant ID not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"
This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.
Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:
effective = min(share.Permission, tenant_org_role, tenant_role_cap)
where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.
Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.
Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.
Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-15 13:49:54 +00:00
|
|
|
|
callerTenantRole := types.TenantRoleFromContext(ctx)
|
|
|
|
|
|
agent, err := h.agentShareService.GetSharedAgentForTenant(ctx, currentTenantID, callerTenantRole, agentID)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
if goerrors.Is(err, service.ErrAgentShareNotFound) || goerrors.Is(err, service.ErrAgentSharePermission) || goerrors.Is(err, service.ErrAgentNotFoundForShare) {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("no permission for this shared agent"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to verify shared agent access").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
sourceTenantID := agent.TenantID
|
|
|
|
|
|
mode := agent.Config.KBSelectionMode
|
|
|
|
|
|
if mode == "none" {
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": []interface{}{},
|
|
|
|
|
|
"has_more": false,
|
|
|
|
|
|
})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
var scopes []types.KnowledgeSearchScope
|
|
|
|
|
|
if mode == "selected" && len(agent.Config.KnowledgeBases) > 0 {
|
|
|
|
|
|
for _, kbID := range agent.Config.KnowledgeBases {
|
|
|
|
|
|
if kbID != "" {
|
|
|
|
|
|
scopes = append(scopes, types.KnowledgeSearchScope{TenantID: sourceTenantID, KBID: kbID})
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if len(scopes) == 0 {
|
|
|
|
|
|
kbs, err := h.kbService.ListKnowledgeBasesByTenantID(ctx, sourceTenantID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to list knowledge bases").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-23 09:22:18 +00:00
|
|
|
|
// `all` mode: authoritative server-side capability filter. Mirrors the
|
|
|
|
|
|
// logic in ListKnowledgeBases so @file search, KB listing, and runtime
|
2026-05-12 07:41:47 +00:00
|
|
|
|
// all agree on what "mode=all" actually means for this agent. The
|
|
|
|
|
|
// filter is agent-mode aware so quick-answer (RAG-only) skips
|
|
|
|
|
|
// wiki-only KBs even though it has no `allowed_tools`.
|
|
|
|
|
|
filter := tools.DeriveKBFilterForAgent(agent.Config.AgentMode, agent.Config.AllowedTools)
|
2026-04-23 09:22:18 +00:00
|
|
|
|
removed := 0
|
2026-02-04 12:12:29 +00:00
|
|
|
|
for _, kb := range kbs {
|
2026-04-23 09:22:18 +00:00
|
|
|
|
if kb == nil || kb.Type != types.KnowledgeBaseTypeDocument {
|
|
|
|
|
|
continue
|
2026-02-04 12:12:29 +00:00
|
|
|
|
}
|
2026-05-12 07:41:47 +00:00
|
|
|
|
if !filter.IsEmpty() && !tools.KBSatisfiesAgentRequirements(kb.Capabilities(), agent.Config.AgentMode, agent.Config.AllowedTools) {
|
2026-04-23 09:22:18 +00:00
|
|
|
|
removed++
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
scopes = append(scopes, types.KnowledgeSearchScope{TenantID: sourceTenantID, KBID: kb.ID})
|
|
|
|
|
|
}
|
|
|
|
|
|
if removed > 0 {
|
|
|
|
|
|
logger.Infof(ctx,
|
2026-05-12 07:41:47 +00:00
|
|
|
|
"SearchKnowledge(agent=%s, mode=all): capability filter removed %d KBs",
|
2026-04-23 09:22:18 +00:00
|
|
|
|
agentID, removed)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
knowledges, hasMore, err := h.kgService.SearchKnowledgeForScopes(ctx, scopes, keyword, offset, limit, fileTypes)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to search knowledge").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledges,
|
|
|
|
|
|
"has_more": hasMore,
|
|
|
|
|
|
})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Default: own + shared KBs
|
2025-12-31 10:18:22 +00:00
|
|
|
|
knowledges, hasMore, err := h.kgService.SearchKnowledge(ctx, keyword, offset, limit, fileTypes)
|
2025-12-19 14:17:56 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to search knowledge").WithDetails(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": knowledges,
|
|
|
|
|
|
"has_more": hasMore,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
2026-03-09 05:14:42 +00:00
|
|
|
|
|
|
|
|
|
|
// MoveKnowledgeRequest defines the request for moving knowledge items
|
|
|
|
|
|
type MoveKnowledgeRequest struct {
|
|
|
|
|
|
KnowledgeIDs []string `json:"knowledge_ids" binding:"required,min=1"`
|
|
|
|
|
|
SourceKBID string `json:"source_kb_id" binding:"required"`
|
|
|
|
|
|
TargetKBID string `json:"target_kb_id" binding:"required"`
|
|
|
|
|
|
Mode string `json:"mode" binding:"required,oneof=reuse_vectors reparse"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MoveKnowledgeResponse defines the response for move knowledge
|
|
|
|
|
|
type MoveKnowledgeResponse struct {
|
|
|
|
|
|
TaskID string `json:"task_id"`
|
|
|
|
|
|
SourceKBID string `json:"source_kb_id"`
|
|
|
|
|
|
TargetKBID string `json:"target_kb_id"`
|
|
|
|
|
|
KnowledgeCount int `json:"knowledge_count"`
|
|
|
|
|
|
Message string `json:"message"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MoveKnowledge moves knowledge items from one knowledge base to another (async task).
|
2026-05-11 09:37:14 +00:00
|
|
|
|
//
|
|
|
|
|
|
// MoveKnowledge godoc
|
|
|
|
|
|
// @Summary 移动知识到其他知识库
|
|
|
|
|
|
// @Description 将一条或多条知识从源知识库移动到目标知识库(异步),返回任务 ID 用于查询进度
|
|
|
|
|
|
// @Tags 知识
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param request body handler.MoveKnowledgeRequest true "{source_kb_id, target_kb_id, knowledge_ids}"
|
|
|
|
|
|
// @Success 200 {object} handler.MoveKnowledgeResponse "任务信息"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/move [post]
|
2026-03-09 05:14:42 +00:00
|
|
|
|
func (h *KnowledgeHandler) MoveKnowledge(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
var req MoveKnowledgeRequest
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "MoveKnowledge: failed to parse request", err)
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Invalid request parameters: " + err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate source != target
|
|
|
|
|
|
if req.SourceKBID == req.TargetKBID {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Source and target knowledge base cannot be the same"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
tenantID, exists := c.Get(types.TenantIDContextKey.String())
|
|
|
|
|
|
if !exists {
|
|
|
|
|
|
c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate source KB
|
|
|
|
|
|
sourceKB, err := h.kbService.GetKnowledgeBaseByID(ctx, req.SourceKBID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if goerrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(errors.NewNotFoundError("Source knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if sourceKB.TenantID != tenantID.(uint64) {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to access source knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate target KB
|
|
|
|
|
|
targetKB, err := h.kbService.GetKnowledgeBaseByID(ctx, req.TargetKBID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if goerrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(errors.NewNotFoundError("Target knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if targetKB.TenantID != tenantID.(uint64) {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to access target knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate type match
|
|
|
|
|
|
if sourceKB.Type != targetKB.Type {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Source and target knowledge bases must be the same type"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate embedding model match
|
|
|
|
|
|
if sourceKB.EmbeddingModelID != targetKB.EmbeddingModelID {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Source and target must use the same embedding model"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-06-05 01:13:45 +00:00
|
|
|
|
// reuse_vectors copies index entries directly between KBs, which only works
|
|
|
|
|
|
// inside the same VectorStore backend. A cross-store reuse_vectors move would
|
|
|
|
|
|
// route CopyIndices through the SOURCE store and then delete the source
|
|
|
|
|
|
// indices, corrupting the vector data. Reject it and point the caller at
|
|
|
|
|
|
// reparse mode, which re-indexes into the target store safely.
|
|
|
|
|
|
if req.Mode == "reuse_vectors" && !sourceKB.SharesStoreWith(targetKB) {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(
|
|
|
|
|
|
"reuse_vectors move across different vector stores is not supported; " +
|
|
|
|
|
|
"use reparse mode to move into a different store"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-09 05:14:42 +00:00
|
|
|
|
// Validate all knowledge IDs belong to source KB and are in completed status
|
|
|
|
|
|
for _, kID := range req.KnowledgeIDs {
|
|
|
|
|
|
knowledge, err := h.kgService.GetKnowledgeByID(ctx, kID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(fmt.Sprintf("Knowledge item %s not found", kID)))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if knowledge.KnowledgeBaseID != req.SourceKBID {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(fmt.Sprintf("Knowledge item %s does not belong to the source knowledge base", kID)))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if knowledge.ParseStatus != types.ParseStatusCompleted {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError(fmt.Sprintf("Knowledge item %s is not in completed status (current: %s)", kID, knowledge.ParseStatus)))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Generate task ID
|
|
|
|
|
|
taskID := utils.GenerateTaskID("kg_move", tenantID.(uint64), req.SourceKBID)
|
|
|
|
|
|
|
|
|
|
|
|
// Create move payload
|
|
|
|
|
|
payload := types.KnowledgeMovePayload{
|
|
|
|
|
|
TenantID: tenantID.(uint64),
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
KnowledgeIDs: req.KnowledgeIDs,
|
|
|
|
|
|
SourceKBID: req.SourceKBID,
|
|
|
|
|
|
TargetKBID: req.TargetKBID,
|
|
|
|
|
|
Mode: req.Mode,
|
|
|
|
|
|
}
|
feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests
(chat / search / eval), so file uploads and every downstream asynq task
(document parse, chunk embedding, OCR/VLM, summary / question gen, wiki
ingest, datasource sync, etc.) produced either disconnected shallow
traces or no observation at all.
This change threads one trace end-to-end:
- tracer: add SPAN observation type and StartSpan; add ResumeTrace so a
worker can attach to an upstream trace without emitting a duplicate
trace-create; StartGeneration now auto-picks parentObservationId from
ctx so nested trace -> span -> generation trees render correctly.
- types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq
payloads so trace_id / parent_obs_id / user_id / session_id serialise
into every job.
- langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal
so the HTTP-layer trace survives the Redis hop.
- langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either
resumes the upstream trace or opens a standalone asynq.<type> trace
for scheduled jobs, and wraps the handler in a SPAN with task metadata
(id / queue / retry / payload_bytes) plus ERROR level on failure.
- GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation /
FAQ / wiki / datasource endpoints so the root trace actually starts.
- Tests: tracer_test.go covers span nesting, error status, and
ResumeTrace no-trace-create guarantee; asynq_test.go covers
InjectTracing round-trip, middleware resume path, and standalone
trace fallback.
- Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types
and documents the cross-process propagation model.
No behavioural change when Langfuse is disabled (all new code paths are
no-ops and carriers serialise to empty strings with omitempty).
2026-04-24 04:23:30 +00:00
|
|
|
|
langfuse.InjectTracing(ctx, &payload)
|
2026-03-09 05:14:42 +00:00
|
|
|
|
|
|
|
|
|
|
payloadBytes, err := json.Marshal(payload)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "MoveKnowledge: failed to marshal payload: %v", err)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to create task"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Enqueue move task
|
|
|
|
|
|
task := asynq.NewTask(types.TypeKnowledgeMove, payloadBytes,
|
|
|
|
|
|
asynq.TaskID(taskID), asynq.Queue("default"), asynq.MaxRetry(3))
|
|
|
|
|
|
info, err := h.asynqClient.Enqueue(task)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "MoveKnowledge: failed to enqueue task: %v", err)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError("Failed to enqueue task"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "MoveKnowledge: task enqueued: %s, asynq_id: %s, source: %s, target: %s, count: %d",
|
|
|
|
|
|
taskID, info.ID, secutils.SanitizeForLog(req.SourceKBID), secutils.SanitizeForLog(req.TargetKBID), len(req.KnowledgeIDs))
|
|
|
|
|
|
|
|
|
|
|
|
// Save initial progress
|
|
|
|
|
|
initialProgress := &types.KnowledgeMoveProgress{
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
SourceKBID: req.SourceKBID,
|
|
|
|
|
|
TargetKBID: req.TargetKBID,
|
|
|
|
|
|
Status: types.KBCloneStatusPending,
|
|
|
|
|
|
Total: len(req.KnowledgeIDs),
|
|
|
|
|
|
Progress: 0,
|
|
|
|
|
|
Message: "Task queued, waiting to start...",
|
|
|
|
|
|
CreatedAt: time.Now().Unix(),
|
|
|
|
|
|
UpdatedAt: time.Now().Unix(),
|
|
|
|
|
|
}
|
|
|
|
|
|
if err := h.kgService.SaveKnowledgeMoveProgress(ctx, initialProgress); err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "MoveKnowledge: failed to save initial progress: %v", err)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": MoveKnowledgeResponse{
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
SourceKBID: req.SourceKBID,
|
|
|
|
|
|
TargetKBID: req.TargetKBID,
|
|
|
|
|
|
KnowledgeCount: len(req.KnowledgeIDs),
|
|
|
|
|
|
Message: "Knowledge move task started",
|
|
|
|
|
|
},
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// GetKnowledgeMoveProgress retrieves the progress of a knowledge move task.
|
2026-05-11 09:37:14 +00:00
|
|
|
|
//
|
|
|
|
|
|
// GetKnowledgeMoveProgress godoc
|
|
|
|
|
|
// @Summary 获取知识移动进度
|
|
|
|
|
|
// @Description 按任务 ID 查询移动进度
|
|
|
|
|
|
// @Tags 知识
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param task_id path string true "移动任务 ID"
|
|
|
|
|
|
// @Success 200 {object} types.KnowledgeMoveProgress "进度信息"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "任务不存在"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge/move/progress/{task_id} [get]
|
2026-03-09 05:14:42 +00:00
|
|
|
|
func (h *KnowledgeHandler) GetKnowledgeMoveProgress(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
taskID := c.Param("task_id")
|
|
|
|
|
|
if taskID == "" {
|
|
|
|
|
|
c.Error(errors.NewBadRequestError("Task ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
progress, err := h.kgService.GetKnowledgeMoveProgress(ctx, taskID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": progress,
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
2026-04-01 13:45:35 +00:00
|
|
|
|
|
|
|
|
|
|
// resolveAgentAllowedKBIDs returns the set of knowledge base IDs that the
|
|
|
|
|
|
// shared agent is allowed to access based on its KBSelectionMode config.
|
|
|
|
|
|
// Returns nil when no restriction applies ("all" mode), or a concrete slice
|
|
|
|
|
|
// (possibly empty for "none" mode) when the results must be filtered.
|
|
|
|
|
|
func resolveAgentAllowedKBIDs(agent *types.CustomAgent) []string {
|
|
|
|
|
|
switch agent.Config.KBSelectionMode {
|
|
|
|
|
|
case "all":
|
|
|
|
|
|
return nil
|
|
|
|
|
|
case "none":
|
|
|
|
|
|
return []string{}
|
|
|
|
|
|
case "selected":
|
|
|
|
|
|
return agent.Config.KnowledgeBases
|
|
|
|
|
|
default:
|
|
|
|
|
|
if len(agent.Config.KnowledgeBases) > 0 {
|
|
|
|
|
|
return agent.Config.KnowledgeBases
|
|
|
|
|
|
}
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 10:16:34 +00:00
|
|
|
|
// parseFilterTime parses a query-string timestamp accepted by knowledge list
|
|
|
|
|
|
// filters. It supports RFC3339, RFC3339 with milliseconds, and the date-only
|
|
|
|
|
|
// "2006-01-02" form (interpreted at start of day in the local timezone).
|
|
|
|
|
|
func parseFilterTime(raw string) (time.Time, error) {
|
|
|
|
|
|
raw = strings.TrimSpace(raw)
|
|
|
|
|
|
if raw == "" {
|
|
|
|
|
|
return time.Time{}, nil
|
|
|
|
|
|
}
|
|
|
|
|
|
layouts := []string{time.RFC3339Nano, time.RFC3339, "2006-01-02 15:04:05", "2006-01-02"}
|
|
|
|
|
|
var lastErr error
|
|
|
|
|
|
for _, layout := range layouts {
|
|
|
|
|
|
if t, err := time.ParseInLocation(layout, raw, time.Local); err == nil {
|
|
|
|
|
|
return t, nil
|
|
|
|
|
|
} else {
|
|
|
|
|
|
lastErr = err
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return time.Time{}, lastErr
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-04-01 13:45:35 +00:00
|
|
|
|
func sliceContains(ss []string, target string) bool {
|
|
|
|
|
|
for _, s := range ss {
|
|
|
|
|
|
if s == target {
|
|
|
|
|
|
return true
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return false
|
|
|
|
|
|
}
|