2025-08-05 07:08:07 +00:00
|
|
|
|
package handler
|
|
|
|
|
|
|
|
|
|
|
|
import (
|
2026-05-18 01:10:01 +00:00
|
|
|
|
"context"
|
2025-12-16 09:38:28 +00:00
|
|
|
|
"encoding/json"
|
2026-02-06 06:45:37 +00:00
|
|
|
|
stderrors "errors"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"net/http"
|
2025-11-25 08:44:44 +00:00
|
|
|
|
"strconv"
|
2026-04-29 10:58:09 +00:00
|
|
|
|
"strings"
|
2025-12-16 09:38:28 +00:00
|
|
|
|
"time"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-09-08 07:36:33 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/application/access"
|
2026-02-06 06:45:37 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/application/repository"
|
2026-09-08 07:36:33 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/config"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/errors"
|
2026-02-04 12:12:29 +00:00
|
|
|
|
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/logger"
|
2026-09-08 07:36:33 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/middleware"
|
2026-08-26 09:31:05 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/storageurl"
|
feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests
(chat / search / eval), so file uploads and every downstream asynq task
(document parse, chunk embedding, OCR/VLM, summary / question gen, wiki
ingest, datasource sync, etc.) produced either disconnected shallow
traces or no observation at all.
This change threads one trace end-to-end:
- tracer: add SPAN observation type and StartSpan; add ResumeTrace so a
worker can attach to an upstream trace without emitting a duplicate
trace-create; StartGeneration now auto-picks parentObservationId from
ctx so nested trace -> span -> generation trees render correctly.
- types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq
payloads so trace_id / parent_obs_id / user_id / session_id serialise
into every job.
- langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal
so the HTTP-layer trace survives the Redis hop.
- langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either
resumes the upstream trace or opens a standalone asynq.<type> trace
for scheduled jobs, and wraps the handler in a SPAN with task metadata
(id / queue / retry / payload_bytes) plus ERROR level on failure.
- GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation /
FAQ / wiki / datasource endpoints so the root trace actually starts.
- Tests: tracer_test.go covers span nesting, error status, and
ResumeTrace no-trace-create guarantee; asynq_test.go covers
InjectTracing round-trip, middleware resume path, and standalone
trace fallback.
- Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types
and documents the cross-process propagation model.
No behavioural change when Langfuse is disabled (all new code paths are
no-ops and carriers serialise to empty strings with omitempty).
2026-04-24 04:23:30 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/types"
|
|
|
|
|
|
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
2026-01-07 07:18:18 +00:00
|
|
|
|
"github.com/Tencent/WeKnora/internal/utils"
|
2025-11-26 03:27:48 +00:00
|
|
|
|
secutils "github.com/Tencent/WeKnora/internal/utils"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"github.com/gin-gonic/gin"
|
2026-09-08 07:36:33 +00:00
|
|
|
|
"github.com/google/uuid"
|
2025-12-16 09:38:28 +00:00
|
|
|
|
"github.com/hibiken/asynq"
|
2025-08-05 07:08:07 +00:00
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
// KnowledgeBaseHandler defines the HTTP handler for knowledge base operations
|
|
|
|
|
|
type KnowledgeBaseHandler struct {
|
2026-09-08 07:36:33 +00:00
|
|
|
|
cfg *config.Config
|
2026-05-18 01:10:01 +00:00
|
|
|
|
service interfaces.KnowledgeBaseService
|
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 14:21:55 +00:00
|
|
|
|
profileService interfaces.KnowledgeBaseProfileService
|
2026-05-18 01:10:01 +00:00
|
|
|
|
knowledgeService interfaces.KnowledgeService
|
|
|
|
|
|
kbShareService interfaces.KBShareService
|
|
|
|
|
|
agentShareService interfaces.AgentShareService
|
|
|
|
|
|
asynqClient interfaces.TaskEnqueuer
|
|
|
|
|
|
vectorStoreService interfaces.VectorStoreService // enriches KB responses with bound store display
|
2026-05-17 16:28:16 +00:00
|
|
|
|
// userService 仅在 list 类接口里用于批量回填 creator_name;
|
|
|
|
|
|
// 真正的鉴权由 RBAC 中间件 + Lookup 完成,这里不参与决策。
|
|
|
|
|
|
userService interfaces.UserService
|
2026-08-26 09:31:05 +00:00
|
|
|
|
// fileService and storageResolver back the optional `resource_urls=public`
|
|
|
|
|
|
// mode on hybrid-search. Both may be nil in tests, in which case only the
|
|
|
|
|
|
// default handle mode is available.
|
|
|
|
|
|
fileService interfaces.FileService
|
|
|
|
|
|
storageResolver interfaces.StorageBackendResolver
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// NewKnowledgeBaseHandler creates a new knowledge base handler instance
|
|
|
|
|
|
func NewKnowledgeBaseHandler(
|
2026-09-08 07:36:33 +00:00
|
|
|
|
cfg *config.Config,
|
2025-08-05 07:08:07 +00:00
|
|
|
|
service interfaces.KnowledgeBaseService,
|
|
|
|
|
|
knowledgeService interfaces.KnowledgeService,
|
2026-01-30 08:28:01 +00:00
|
|
|
|
kbShareService interfaces.KBShareService,
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentShareService interfaces.AgentShareService,
|
2026-02-28 05:22:07 +00:00
|
|
|
|
asynqClient interfaces.TaskEnqueuer,
|
2026-05-18 01:10:01 +00:00
|
|
|
|
vectorStoreService interfaces.VectorStoreService,
|
2026-05-17 16:28:16 +00:00
|
|
|
|
userService interfaces.UserService,
|
2026-08-26 09:31:05 +00:00
|
|
|
|
fileService interfaces.FileService,
|
|
|
|
|
|
storageResolver interfaces.StorageBackendResolver,
|
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 14:21:55 +00:00
|
|
|
|
profileService interfaces.KnowledgeBaseProfileService,
|
2025-08-05 07:08:07 +00:00
|
|
|
|
) *KnowledgeBaseHandler {
|
2025-12-16 09:38:28 +00:00
|
|
|
|
return &KnowledgeBaseHandler{
|
2026-09-08 07:36:33 +00:00
|
|
|
|
cfg: cfg,
|
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 14:21:55 +00:00
|
|
|
|
profileService: profileService,
|
2026-05-18 01:10:01 +00:00
|
|
|
|
service: service,
|
|
|
|
|
|
knowledgeService: knowledgeService,
|
|
|
|
|
|
kbShareService: kbShareService,
|
|
|
|
|
|
agentShareService: agentShareService,
|
|
|
|
|
|
asynqClient: asynqClient,
|
|
|
|
|
|
vectorStoreService: vectorStoreService,
|
2026-05-17 16:28:16 +00:00
|
|
|
|
userService: userService,
|
2026-08-26 09:31:05 +00:00
|
|
|
|
fileService: fileService,
|
|
|
|
|
|
storageResolver: storageResolver,
|
2025-12-16 09:38:28 +00:00
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2026-08-26 09:31:05 +00:00
|
|
|
|
// resolveResourceRewriter builds the storage-reference rewriter for one response
|
|
|
|
|
|
// from the request's `resource_urls` parameter, falling back to the deployment
|
|
|
|
|
|
// default. The returned error is already an AppError the caller can hand to
|
|
|
|
|
|
// c.Error: a rejected scope is a 403, a typo in the parameter is a 400.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) resolveResourceRewriter(c *gin.Context) (*storageurl.Rewriter, error) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
mode, err := storageurl.ResolveMode(ctx, c.Query(storageurl.QueryParam))
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if stderrors.Is(err, storageurl.ErrPublicModeForbidden) {
|
|
|
|
|
|
return nil, apperrors.NewForbiddenError(err.Error())
|
|
|
|
|
|
}
|
|
|
|
|
|
return nil, apperrors.NewBadRequestError(err.Error())
|
|
|
|
|
|
}
|
|
|
|
|
|
return storageurl.NewRequestRewriter(ctx, mode, h.fileService, h.storageResolver), nil
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-18 01:10:01 +00:00
|
|
|
|
// buildKBResponse turns a knowledge base into a JSON-ready response shape,
|
|
|
|
|
|
// merging the bound vector store's display metadata and any caller-supplied
|
|
|
|
|
|
// extras (e.g., my_permission for shared KBs). Returns the kb pointer
|
|
|
|
|
|
// unchanged on serialization failure so the request still succeeds.
|
|
|
|
|
|
//
|
|
|
|
|
|
// The map-merge approach (rather than a wrapper struct embedding the kb)
|
|
|
|
|
|
// is deliberate: KnowledgeBase has a custom MarshalJSON, and embedding
|
|
|
|
|
|
// would promote it to any wrapper struct and silently swallow the extra
|
|
|
|
|
|
// fields. The same pattern is already used by GetKnowledgeBase to add the
|
|
|
|
|
|
// my_permission field for shared knowledge bases.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Shared-KB suppression: when storeView.Source == StoreSourceShared (the
|
|
|
|
|
|
// caller is not the KB owner), the raw vector_store_id UUID is stripped
|
|
|
|
|
|
// from the response so the owner-tenant's store inventory cannot be
|
|
|
|
|
|
// correlated across multiple shared KBs. Name / engine type / status are
|
|
|
|
|
|
// already empty in the SharedStoreDisplay payload; suppressing the UUID
|
|
|
|
|
|
// completes the cross-tenant metadata hiding.
|
|
|
|
|
|
func buildKBResponse(
|
|
|
|
|
|
kb *types.KnowledgeBase,
|
|
|
|
|
|
storeView types.StoreDisplay,
|
|
|
|
|
|
extras map[string]interface{},
|
|
|
|
|
|
) interface{} {
|
|
|
|
|
|
b, err := json.Marshal(kb)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
return kb
|
|
|
|
|
|
}
|
|
|
|
|
|
var m map[string]interface{}
|
|
|
|
|
|
if err := json.Unmarshal(b, &m); err != nil || m == nil {
|
|
|
|
|
|
return kb
|
|
|
|
|
|
}
|
|
|
|
|
|
if storeView.Source == types.StoreSourceShared {
|
|
|
|
|
|
delete(m, "vector_store_id")
|
|
|
|
|
|
}
|
|
|
|
|
|
if storeView.Name != "" {
|
|
|
|
|
|
m["vector_store_name"] = storeView.Name
|
|
|
|
|
|
}
|
|
|
|
|
|
if storeView.Source != "" {
|
|
|
|
|
|
m["vector_store_source"] = storeView.Source
|
|
|
|
|
|
}
|
|
|
|
|
|
if storeView.EngineType != "" {
|
|
|
|
|
|
m["vector_store_engine_type"] = storeView.EngineType
|
|
|
|
|
|
}
|
|
|
|
|
|
if storeView.Status != "" {
|
|
|
|
|
|
m["vector_store_status"] = storeView.Status
|
|
|
|
|
|
}
|
|
|
|
|
|
for k, v := range extras {
|
|
|
|
|
|
m[k] = v
|
|
|
|
|
|
}
|
|
|
|
|
|
return m
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
// buildKBListResponse turns a slice of knowledge bases into a JSON-ready
|
|
|
|
|
|
// slice that mirrors the single-KB enrichment in buildKBResponse. Store
|
|
|
|
|
|
// views are batch-resolved once via BatchResolveStoreView to keep the
|
|
|
|
|
|
// list endpoint O(1) in vector-store service calls — the per-KB
|
|
|
|
|
|
// resolveKBStoreView would otherwise be N+1.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Caller-vs-owner semantics match the single-KB path:
|
|
|
|
|
|
// - KB has no binding → DefaultStoreDisplay()
|
|
|
|
|
|
// - KB is owned by another tenant (cross-tenant shared) → SharedStoreDisplay()
|
|
|
|
|
|
// - KB is own-tenant bound → look up in the batch result; misses
|
2026-06-02 11:25:38 +00:00
|
|
|
|
// fall back to UnavailableStoreDisplay()
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
//
|
|
|
|
|
|
// Resolver failures degrade gracefully: every own-tenant bound KB renders
|
|
|
|
|
|
// as unavailable instead of breaking the list response.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) buildKBListResponse(
|
|
|
|
|
|
ctx context.Context, kbs []*types.KnowledgeBase, callerTenantID uint64,
|
|
|
|
|
|
) []interface{} {
|
|
|
|
|
|
defaultView := h.envDefaultStoreView(ctx)
|
|
|
|
|
|
storeViews := h.batchResolveKBStoreViews(ctx, kbs, callerTenantID)
|
|
|
|
|
|
out := make([]interface{}, 0, len(kbs))
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
var view types.StoreDisplay
|
|
|
|
|
|
switch {
|
|
|
|
|
|
case !kb.HasVectorStore():
|
|
|
|
|
|
view = defaultView
|
|
|
|
|
|
case kb.TenantID != callerTenantID:
|
|
|
|
|
|
view = types.SharedStoreDisplay()
|
|
|
|
|
|
default:
|
|
|
|
|
|
v, ok := storeViews[*kb.VectorStoreID]
|
|
|
|
|
|
if !ok || v.Source == "" {
|
|
|
|
|
|
view = types.UnavailableStoreDisplay()
|
|
|
|
|
|
} else {
|
|
|
|
|
|
view = v
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
out = append(out, buildKBResponse(kb, view, nil))
|
|
|
|
|
|
}
|
|
|
|
|
|
return out
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// sharedKBRow projects a SharedKnowledgeBaseInfo into a response payload
|
|
|
|
|
|
// that respects the cross-tenant strip rule: the embedded KnowledgeBase
|
|
|
|
|
|
// row runs through buildKBResponse with SharedStoreDisplay() so its
|
|
|
|
|
|
// vector_store_id and any owner-tenant store metadata never reach the
|
|
|
|
|
|
// wire. The share-record fields (share_id, organization_id, etc.) are
|
|
|
|
|
|
// kept intact alongside the stripped KB. Callers can pass extras to
|
|
|
|
|
|
// merge view-specific keys such as is_mine or source_from_agent.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Always uses SharedStoreDisplay() regardless of whether the caller is
|
|
|
|
|
|
// the owner; the cross-tenant share endpoints serve mixed audiences and
|
|
|
|
|
|
// the owner's "rich" view of their own bindings is already served by
|
|
|
|
|
|
// ListKnowledgeBases / single-KB GET on the standard knowledge-base
|
|
|
|
|
|
// routes. Trying to enrich own-row entries here would either require
|
|
|
|
|
|
// threading the vector-store service through the organization handler
|
|
|
|
|
|
// or duplicating the lookup logic — both larger than the security fix
|
|
|
|
|
|
// warrants and easy to follow up on once needed.
|
|
|
|
|
|
func sharedKBRow(
|
|
|
|
|
|
info *types.SharedKnowledgeBaseInfo, extras map[string]interface{},
|
|
|
|
|
|
) map[string]interface{} {
|
|
|
|
|
|
kbView := buildKBResponse(info.KnowledgeBase, types.SharedStoreDisplay(), nil)
|
|
|
|
|
|
row := map[string]interface{}{
|
|
|
|
|
|
"knowledge_base": kbView,
|
|
|
|
|
|
"share_id": info.ShareID,
|
|
|
|
|
|
"organization_id": info.OrganizationID,
|
|
|
|
|
|
"org_name": info.OrgName,
|
|
|
|
|
|
"permission": info.Permission,
|
|
|
|
|
|
"source_tenant_id": info.SourceTenantID,
|
|
|
|
|
|
"shared_at": info.SharedAt,
|
|
|
|
|
|
}
|
|
|
|
|
|
for k, v := range extras {
|
|
|
|
|
|
row[k] = v
|
|
|
|
|
|
}
|
|
|
|
|
|
return row
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// envDefaultStoreView returns the env-fallback store display enriched with
|
|
|
|
|
|
// the configured env-store engine type when the service is available. The
|
|
|
|
|
|
// service path populates EngineType so the caller can show "postgres" or
|
|
|
|
|
|
// "qdrant" on the env-default badge instead of leaving it blank. A nil
|
|
|
|
|
|
// service (e.g. in narrow unit-test setups) falls back to the bare default
|
|
|
|
|
|
// display rather than failing the list response.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) envDefaultStoreView(ctx context.Context) types.StoreDisplay {
|
|
|
|
|
|
if h.vectorStoreService == nil {
|
|
|
|
|
|
return types.DefaultStoreDisplay()
|
|
|
|
|
|
}
|
|
|
|
|
|
return h.vectorStoreService.EnvDefaultStoreView(ctx)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// batchResolveKBStoreViews collects the unique own-tenant store IDs across
|
|
|
|
|
|
// the KB slice and resolves them in one BatchResolveStoreView call.
|
|
|
|
|
|
// Cross-tenant shared KBs never enter the batch — they always render
|
|
|
|
|
|
// via SharedStoreDisplay, which deliberately suppresses the owner
|
|
|
|
|
|
// tenant's store name and engine type so cross-tenant viewers cannot
|
|
|
|
|
|
// correlate the owner's store inventory from KB responses alone.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) batchResolveKBStoreViews(
|
|
|
|
|
|
ctx context.Context, kbs []*types.KnowledgeBase, callerTenantID uint64,
|
|
|
|
|
|
) map[string]types.StoreDisplay {
|
|
|
|
|
|
if h.vectorStoreService == nil {
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
storeIDs := make([]string, 0, len(kbs))
|
|
|
|
|
|
seen := make(map[string]bool, len(kbs))
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if !kb.HasVectorStore() || kb.TenantID != callerTenantID {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
sid := *kb.VectorStoreID
|
|
|
|
|
|
if !seen[sid] {
|
|
|
|
|
|
seen[sid] = true
|
|
|
|
|
|
storeIDs = append(storeIDs, sid)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if len(storeIDs) == 0 {
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
views, err := h.vectorStoreService.BatchResolveStoreView(ctx, callerTenantID, storeIDs)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.WarnWithFields(ctx, logger.Fields{
|
|
|
|
|
|
"tenant_id": callerTenantID,
|
|
|
|
|
|
"store_count": len(storeIDs),
|
|
|
|
|
|
}, "[kb.list] batch store view resolve failed; rendering bound KBs as unavailable")
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
return views
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-18 01:10:01 +00:00
|
|
|
|
// resolveKBStoreView returns the store display payload to embed in the KB
|
|
|
|
|
|
// response. It applies two policies on top of the service-level resolver:
|
|
|
|
|
|
//
|
|
|
|
|
|
// - When the KB does not have a DB-managed vector store binding,
|
|
|
|
|
|
// the env-fallback display is returned without touching the service.
|
|
|
|
|
|
// - When the caller is not the KB owner (shared access), the underlying
|
|
|
|
|
|
// store's name and engine are suppressed so operator-chosen names do
|
|
|
|
|
|
// not leak across tenants. The Source value is set to "shared".
|
|
|
|
|
|
//
|
|
|
|
|
|
// On resolution error, an unavailable display is returned and the failure
|
|
|
|
|
|
// is logged for ops; the request itself still succeeds.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) resolveKBStoreView(
|
|
|
|
|
|
ctx context.Context, kb *types.KnowledgeBase, callerTenantID uint64,
|
|
|
|
|
|
) types.StoreDisplay {
|
|
|
|
|
|
if !kb.HasVectorStore() {
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
return h.envDefaultStoreView(ctx)
|
2026-05-18 01:10:01 +00:00
|
|
|
|
}
|
|
|
|
|
|
if kb.TenantID != callerTenantID {
|
|
|
|
|
|
return types.SharedStoreDisplay()
|
|
|
|
|
|
}
|
|
|
|
|
|
if h.vectorStoreService == nil {
|
|
|
|
|
|
return types.UnavailableStoreDisplay()
|
|
|
|
|
|
}
|
|
|
|
|
|
view, err := h.vectorStoreService.ResolveStoreView(ctx, kb.TenantID, *kb.VectorStoreID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.WarnWithFields(ctx, logger.Fields{
|
|
|
|
|
|
"kb_id": secutils.SanitizeForLog(kb.ID),
|
|
|
|
|
|
"tenant_id": kb.TenantID,
|
|
|
|
|
|
}, "[kb.view] vector store resolve failed; returning unavailable")
|
|
|
|
|
|
return types.UnavailableStoreDisplay()
|
|
|
|
|
|
}
|
|
|
|
|
|
return view
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// HybridSearch godoc
|
|
|
|
|
|
// @Summary 混合搜索
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
// @Description 底层召回:向量+关键词混合检索,默认不 rerank(可用 rerank 字段开启);一般检索请用 /knowledge-search。推荐 POST,GET 带 JSON 体仅兼容旧客户端。
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-08-26 09:31:05 +00:00
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param request body types.SearchParams true "搜索参数"
|
|
|
|
|
|
// @Param resource_urls query string false "文件引用形式,public 返回可加载直链" Enums(handle, public) default(handle)
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "搜索结果"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2026-06-22 09:20:59 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/hybrid-search [post]
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id}/hybrid-search [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) HybridSearch(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start hybrid search")
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Validate and check permission for knowledge base access
|
|
|
|
|
|
_, id, effectiveTenantID, _, err := h.validateAndGetKnowledgeBase(c)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-09 12:23:03 +00:00
|
|
|
|
// Parse request body
|
|
|
|
|
|
var req types.SearchParams
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewBadRequestError("Invalid request parameters").WithDetails(err.Error()))
|
2025-08-09 12:23:03 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-07-22 22:32:30 +00:00
|
|
|
|
precomputedVectorOnly := len(req.QueryEmbedding) > 0 && req.DisableKeywordsMatch && !req.DisableVectorMatch
|
|
|
|
|
|
if strings.TrimSpace(req.QueryText) == "" && !precomputedVectorOnly {
|
|
|
|
|
|
_ = c.Error(apperrors.NewBadRequestError("query_text is required"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
if err := req.Rerank.Validate(); err != nil {
|
|
|
|
|
|
_ = c.Error(apperrors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if req.Rerank.IsEnabled() && strings.TrimSpace(req.QueryText) == "" {
|
|
|
|
|
|
// The rerank model scores passages against the query text.
|
|
|
|
|
|
_ = c.Error(apperrors.NewBadRequestError("query_text is required when rerank is enabled"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-09 12:23:03 +00:00
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
logger.Infof(ctx, "Executing hybrid search, knowledge base ID: %s, query: %s, effectiveTenantID: %d",
|
|
|
|
|
|
secutils.SanitizeForLog(id), secutils.SanitizeForLog(req.QueryText), effectiveTenantID)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2026-08-26 09:31:05 +00:00
|
|
|
|
// Resolve before retrieving so a typo or a rejected scope costs nothing.
|
|
|
|
|
|
rewriter, err := h.resolveResourceRewriter(c)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "Rejected resource URL mode: %v", err)
|
|
|
|
|
|
_ = c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
// Execute hybrid search. Without a rerank object this is the raw recall
|
|
|
|
|
|
// primitive it has always been; with one, the response carries meta.
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Note: For shared KBs, the service uses effectiveTenantID internally via context
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
var retrieval *types.RetrievalResult
|
|
|
|
|
|
if req.Rerank != nil {
|
|
|
|
|
|
retrieval, err = h.service.HybridSearchWithRerank(c.Request.Context(), id, req)
|
|
|
|
|
|
} else {
|
|
|
|
|
|
var results []*types.SearchResult
|
|
|
|
|
|
results, err = h.service.HybridSearch(c.Request.Context(), id, req)
|
|
|
|
|
|
retrieval = &types.RetrievalResult{Results: results}
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
feat(search): fan-out KB retrieval across bound vector stores
Multi-KB hybrid search now groups KBs by their bound VectorStore (partition
key (storeID, owner_tenant_id)), retrieves in parallel via errgroup with a
SetLimit(4) cap and a per-group timeout (MULTI_STORE_RETRIEVE_TIMEOUT_SEC,
default 30s), and merges results. When the collected results span more than
one engine type, an EngineAwareNormalizer rescales vector scores to [0, 1];
keyword (BM25) scores pass through to the existing RRF fusion. Single-group
calls take the fast path with zero fan-out overhead, preserving today's
behavior for deployments where every KB has vector_store_id = NULL.
Embedding-model consistency is now enforced explicitly via
ResolveEmbeddingModelKeys. Multi-KB searches across KBs whose resolved
model identities differ return BadRequest instead of silently producing
incomparable scores. Cross-tenant Organization-shared KBs are preserved by
partitioning on KB.TenantID so the factory's ownership lookup runs against
the source tenant. Foreign-tenant KB UUIDs injected via the request body
are rejected via kbShareService.HasTenantKBPermission (Plan 3 of #1303,
3-D capped) before any retrieval; rejected scopes surface as 404 to avoid
leaking foreign KB existence.
Service-layer typed AppErrors (ErrVectorStoreBindingInvalid 2200 /
ErrVectorStoreUnavailable 2201) are mapped from PR2 sentinel hierarchy and
preserved end-to-end: the iterative FAQ path returns them rather than
swallowing, and the HybridSearch handler routes typed AppErrors to the
client unchanged instead of downgrading to 500.
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 4 (Multi-store fan-out search).
Depends on #994, #1310, #1372.
2026-05-18 09:38:57 +00:00
|
|
|
|
// Service-layer typed AppErrors (e.g. ErrVectorStoreBindingInvalid,
|
|
|
|
|
|
// ErrVectorStoreUnavailable, BadRequest from multi-store fan-out)
|
|
|
|
|
|
// must reach the client with their original code rather than be
|
|
|
|
|
|
// downgraded to InternalServerError. Mirrors the pattern used in
|
|
|
|
|
|
// CreateKnowledgeBase.
|
|
|
|
|
|
if appErr, ok := apperrors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Hybrid search completed, knowledge base ID: %s, result count: %d",
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
secutils.SanitizeForLog(id), len(retrieval.Results))
|
|
|
|
|
|
response := gin.H{
|
2025-08-05 07:08:07 +00:00
|
|
|
|
"success": true,
|
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 12:41:49 +00:00
|
|
|
|
"data": rewriter.CopyReferences(ctx, retrieval.Results),
|
|
|
|
|
|
}
|
|
|
|
|
|
if retrieval.Meta.Rerank != nil {
|
|
|
|
|
|
response["meta"] = retrieval.Meta
|
|
|
|
|
|
}
|
|
|
|
|
|
c.JSON(http.StatusOK, response)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// CreateKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 创建知识库
|
|
|
|
|
|
// @Description 创建新的知识库
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param request body types.KnowledgeBase true "知识库信息"
|
|
|
|
|
|
// @Success 201 {object} map[string]interface{} "创建的知识库"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases [post]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) CreateKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
logger.Info(ctx, "Start creating knowledge base")
|
|
|
|
|
|
|
|
|
|
|
|
// Parse request body
|
|
|
|
|
|
var req types.KnowledgeBase
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewBadRequestError("Invalid request parameters").WithDetails(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-11-25 08:44:44 +00:00
|
|
|
|
if err := validateExtractConfig(req.ExtractConfig); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Invalid extract configuration", err)
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-07-13 04:14:36 +00:00
|
|
|
|
types.NormalizeKnowledgeBasePromptInstructions(&req)
|
2026-07-13 03:57:30 +00:00
|
|
|
|
if err := validateKnowledgeBasePromptInstructions(&req); err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-04-29 10:58:09 +00:00
|
|
|
|
provider := strings.ToLower(strings.TrimSpace(req.GetStorageProvider()))
|
|
|
|
|
|
if provider != "" && !isStorageProviderAllowed(provider) {
|
|
|
|
|
|
c.Error(apperrors.NewBadRequestError("Storage provider is not allowed by STORAGE_ALLOW_LIST"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Creating knowledge base, name: %s", secutils.SanitizeForLog(req.Name))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Create knowledge base using the service
|
|
|
|
|
|
kb, err := h.service.CreateKnowledgeBase(ctx, &req)
|
|
|
|
|
|
if err != nil {
|
2026-05-18 01:10:01 +00:00
|
|
|
|
// Surface typed AppErrors (notably the 400-class codes
|
|
|
|
|
|
// ErrVectorStoreBindingInvalid and ErrVectorStoreUnavailable
|
|
|
|
|
|
// returned by validateVectorStoreBinding) instead of wrapping them
|
|
|
|
|
|
// into a generic 500. The middleware renders the original code and
|
|
|
|
|
|
// HTTP status verbatim. Falls through to 500 only for raw infra
|
|
|
|
|
|
// errors that the service did not classify.
|
|
|
|
|
|
if appErr, ok := apperrors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge base created successfully, ID: %s, name: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(kb.ID), secutils.SanitizeForLog(kb.Name))
|
2026-05-18 01:10:01 +00:00
|
|
|
|
callerTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusCreated, gin.H{
|
|
|
|
|
|
"success": true,
|
2026-05-18 01:10:01 +00:00
|
|
|
|
"data": buildKBResponse(kb, h.resolveKBStoreView(ctx, kb, callerTenantID), nil),
|
2025-08-05 07:08:07 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-05 02:55:00 +00:00
|
|
|
|
// validateAndGetKnowledgeBase validates request parameters and retrieves the knowledge base.
|
|
|
|
|
|
// Enforces per-API-key KB scope before tenant/share/agent resolution.
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Returns the knowledge base, knowledge base ID, effective tenant ID for embedding, permission level, and any errors encountered
|
|
|
|
|
|
// For owned KBs, effectiveTenantID is the caller's tenant ID
|
|
|
|
|
|
// For shared KBs, effectiveTenantID is the source tenant ID (owner's tenant)
|
2026-09-08 07:36:33 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) validateAndGetKnowledgeBase(
|
|
|
|
|
|
c *gin.Context,
|
|
|
|
|
|
) (*types.KnowledgeBase, string, uint64, types.OrgMemberRole, error) {
|
2025-11-26 03:27:48 +00:00
|
|
|
|
id := secutils.SanitizeForLog(c.Param("id"))
|
2026-09-08 07:36:33 +00:00
|
|
|
|
grant, err := resolveHandlerKBAccess(c, id, h.service, h.kbShareService, h.agentShareService)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
2026-09-08 07:36:33 +00:00
|
|
|
|
return nil, id, 0, "", err
|
2026-02-04 12:12:29 +00:00
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
return grant.KnowledgeBase, id, grant.EffectiveTenantID, grant.Permission, nil
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// GetKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 获取知识库详情
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Description 根据ID获取知识库详情。当使用共享智能体时,可传 agent_id 以校验该智能体是否有权访问该知识库。
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param agent_id query string false "共享智能体 ID(用于校验智能体是否有权访问该知识库)"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "知识库详情"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "知识库不存在"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id} [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) GetKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
// Validate and get the knowledge base
|
2026-02-08 06:32:18 +00:00
|
|
|
|
kb, _, _, permission, err := h.validateAndGetKnowledgeBase(c)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// Fill counts (knowledge_count, chunk_count, is_processing) so hover/detail shows correct numbers
|
|
|
|
|
|
if fillErr := h.service.FillKnowledgeBaseCounts(c.Request.Context(), kb); fillErr != nil {
|
|
|
|
|
|
logger.Warnf(c.Request.Context(), "Failed to fill KB counts for %s: %v", kb.ID, fillErr)
|
|
|
|
|
|
}
|
2026-02-08 06:32:18 +00:00
|
|
|
|
tenantID := c.GetUint64(types.TenantIDContextKey.String())
|
2026-05-18 01:10:01 +00:00
|
|
|
|
storeView := h.resolveKBStoreView(c.Request.Context(), kb, tenantID)
|
|
|
|
|
|
var extras map[string]interface{}
|
2026-02-08 06:32:18 +00:00
|
|
|
|
if kb.TenantID != tenantID && permission != "" {
|
|
|
|
|
|
// Include my_permission in data so frontend can show role (e.g. "只读") instead of "--" for agent-visible KBs
|
2026-05-18 01:10:01 +00:00
|
|
|
|
extras = map[string]interface{}{"my_permission": permission}
|
2026-02-08 06:32:18 +00:00
|
|
|
|
}
|
2026-05-18 01:10:01 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{"success": true, "data": buildKBResponse(kb, storeView, extras)})
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// ListKnowledgeBases godoc
|
|
|
|
|
|
// @Summary 获取知识库列表
|
2026-07-14 08:13:49 +00:00
|
|
|
|
// @Description 获取当前空间的所有知识库;或当传入 agent_id(共享智能体)时,校验权限后返回该智能体配置的知识库范围(用于 @ 提及)
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-02-04 12:12:29 +00:00
|
|
|
|
// @Param agent_id query string false "共享智能体 ID(传入时返回该智能体可用的知识库)"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Success 200 {object} map[string]interface{} "知识库列表"
|
|
|
|
|
|
// @Failure 500 {object} errors.AppError "服务器错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases [get]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) ListKnowledgeBases(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
2026-02-04 12:12:29 +00:00
|
|
|
|
agentID := c.Query("agent_id")
|
|
|
|
|
|
if agentID != "" {
|
2026-09-08 07:36:33 +00:00
|
|
|
|
agent, err := resolveSharedAgentForRequest(c, agentID, h.agentShareService)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if err != nil {
|
2026-09-08 07:36:33 +00:00
|
|
|
|
_ = c.Error(err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
currentTenantID := middleware.KBAccessRequest(c).Caller.TenantID
|
|
|
|
|
|
scope := types.NewSharedAgentKBScope(agent)
|
|
|
|
|
|
if scope.IsEmpty() {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{"success": true, "data": []interface{}{}})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
kbs, err := h.service.ListKnowledgeBasesByTenantID(ctx, agent.TenantID)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
kbs = filterKnowledgeBasesForSharedAgent(kbs, agent)
|
2026-07-05 02:25:37 +00:00
|
|
|
|
kbs = filterKnowledgeBasesForAPIKeyScope(ctx, kbs)
|
2026-04-23 09:22:18 +00:00
|
|
|
|
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
"data": h.buildKBListResponse(ctx, kbs, currentTenantID),
|
2026-02-04 12:12:29 +00:00
|
|
|
|
})
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Get all knowledge bases for this tenant
|
|
|
|
|
|
kbs, err := h.service.ListKnowledgeBases(ctx)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-16 15:04:03 +00:00
|
|
|
|
// Optional creator filter — drives the [All | Mine | Others] segmented
|
|
|
|
|
|
// control on the list page. We filter in-process rather than pushing
|
|
|
|
|
|
// down into SQL because the tenant-bounded KB list is small (typically
|
|
|
|
|
|
// <100 rows) and adding a creator predicate to ListKnowledgeBases would
|
|
|
|
|
|
// ripple through every other caller (chat pipeline, agent editor, …).
|
|
|
|
|
|
// Rows with empty CreatorID predate the RBAC migration (PR 5); we treat
|
|
|
|
|
|
// them as "not anyone in particular" so they never appear under "mine"
|
|
|
|
|
|
// or "others" — they fall out of both filters cleanly.
|
|
|
|
|
|
creatorFilter := strings.ToLower(strings.TrimSpace(c.Query("creator")))
|
|
|
|
|
|
if creatorFilter == "mine" || creatorFilter == "others" {
|
|
|
|
|
|
callerUserID, _ := c.Get(types.UserIDContextKey.String())
|
|
|
|
|
|
callerUserIDStr, _ := callerUserID.(string)
|
|
|
|
|
|
filtered := make([]*types.KnowledgeBase, 0, len(kbs))
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if kb.CreatorID == "" {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if creatorFilter == "mine" && kb.CreatorID == callerUserIDStr {
|
|
|
|
|
|
filtered = append(filtered, kb)
|
|
|
|
|
|
} else if creatorFilter == "others" && kb.CreatorID != callerUserIDStr {
|
|
|
|
|
|
filtered = append(filtered, kb)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
kbs = filtered
|
|
|
|
|
|
}
|
2026-07-05 02:25:37 +00:00
|
|
|
|
kbs = filterKnowledgeBasesForAPIKeyScope(ctx, kbs)
|
2026-05-16 15:04:03 +00:00
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Get share counts for all knowledge bases
|
|
|
|
|
|
if len(kbs) > 0 && h.kbShareService != nil {
|
|
|
|
|
|
kbIDs := make([]string, len(kbs))
|
|
|
|
|
|
for i, kb := range kbs {
|
|
|
|
|
|
kbIDs[i] = kb.ID
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
shareCounts, err := h.kbShareService.CountSharesByKnowledgeBaseIDs(ctx, kbIDs)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "Failed to get share counts: %v", err)
|
|
|
|
|
|
} else {
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if count, ok := shareCounts[kb.ID]; ok {
|
|
|
|
|
|
kb.ShareCount = count
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-14 08:13:49 +00:00
|
|
|
|
// 批量回填 creator_name,让前端列表能区分「我创建」与「同空间其他成员创建」。
|
2026-05-17 16:28:16 +00:00
|
|
|
|
// 仅在 list 接口里回填,详情 / 编辑场景不依赖这个字段;解析失败(用户已删除、
|
|
|
|
|
|
// CreatorID 为空的老数据)就让字段为空,前端按 fallback 渲染。
|
|
|
|
|
|
enrichKBCreatorNames(ctx, h.userService, kbs)
|
|
|
|
|
|
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
callerTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-18 14:50:09 +00:00
|
|
|
|
"data": h.buildKBListResponse(ctx, kbs, callerTenantID),
|
2025-08-05 07:08:07 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-05 02:25:37 +00:00
|
|
|
|
func filterKnowledgeBasesForAPIKeyScope(ctx context.Context, kbs []*types.KnowledgeBase) []*types.KnowledgeBase {
|
|
|
|
|
|
scope, ok := types.TenantAPIKeyScopeFromContext(ctx)
|
|
|
|
|
|
if !ok || len(scope.KnowledgeBaseIDs) == 0 {
|
|
|
|
|
|
return kbs
|
|
|
|
|
|
}
|
|
|
|
|
|
filtered := make([]*types.KnowledgeBase, 0, len(kbs))
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if kb != nil && scope.AllowsKnowledgeBase(kb.ID) {
|
|
|
|
|
|
filtered = append(filtered, kb)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return filtered
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-17 16:28:16 +00:00
|
|
|
|
// enrichKBCreatorNames 把 KB 列表里的 CreatorID 批量解析成展示名(username
|
|
|
|
|
|
// 优先,退化到 email)。任意一步失败都吞掉错误:creator_name 缺失只会影响
|
|
|
|
|
|
// 卡片右下角的徽章展示,不该影响列表本身可用。
|
|
|
|
|
|
func enrichKBCreatorNames(ctx context.Context, userSvc interfaces.UserService, kbs []*types.KnowledgeBase) {
|
|
|
|
|
|
if userSvc == nil || len(kbs) == 0 {
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
idSet := make(map[string]struct{}, len(kbs))
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if kb.CreatorID != "" {
|
|
|
|
|
|
idSet[kb.CreatorID] = struct{}{}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if len(idSet) == 0 {
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
ids := make([]string, 0, len(idSet))
|
|
|
|
|
|
for id := range idSet {
|
|
|
|
|
|
ids = append(ids, id)
|
|
|
|
|
|
}
|
|
|
|
|
|
users, err := userSvc.GetUsersByIDs(ctx, ids)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "Failed to resolve KB creator names: %v", err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
for _, kb := range kbs {
|
|
|
|
|
|
if kb.CreatorID == "" {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
u, ok := users[kb.CreatorID]
|
|
|
|
|
|
if !ok || u == nil {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
kb.CreatorName = pickUserDisplayName(u)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// pickUserDisplayName picks the field most users will recognise: Username
|
|
|
|
|
|
// if present (it's required at registration), Email as a fallback. Used by
|
|
|
|
|
|
// both KB and Agent list enrichment so the badge text stays consistent.
|
|
|
|
|
|
func pickUserDisplayName(u *types.User) string {
|
|
|
|
|
|
if u == nil {
|
|
|
|
|
|
return ""
|
|
|
|
|
|
}
|
|
|
|
|
|
if u.Username != "" {
|
|
|
|
|
|
return u.Username
|
|
|
|
|
|
}
|
|
|
|
|
|
return u.Email
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-05 04:02:22 +00:00
|
|
|
|
// TogglePinKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 置顶/取消置顶知识库
|
|
|
|
|
|
// @Description 切换知识库的置顶状态
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新后的知识库"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "知识库不存在"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge-bases/{id}/pin [put]
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) TogglePinKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
id := c.Param("id")
|
|
|
|
|
|
if id == "" {
|
|
|
|
|
|
c.Error(apperrors.NewBadRequestError("knowledge base ID is required"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
kb, err := h.service.TogglePinKnowledgeBase(ctx, id)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if stderrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(apperrors.NewNotFoundError("knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-05-18 01:10:01 +00:00
|
|
|
|
callerTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
2026-03-05 04:02:22 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
2026-05-18 01:10:01 +00:00
|
|
|
|
"data": buildKBResponse(kb, h.resolveKBStoreView(ctx, kb, callerTenantID), nil),
|
2026-03-05 04:02:22 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// UpdateKnowledgeBaseRequest defines the request body structure for updating a knowledge base
|
|
|
|
|
|
type UpdateKnowledgeBaseRequest struct {
|
2025-11-28 08:17:52 +00:00
|
|
|
|
Name string `json:"name" binding:"required"`
|
2025-08-05 07:08:07 +00:00
|
|
|
|
Description string `json:"description"`
|
2026-03-12 06:47:56 +00:00
|
|
|
|
Config *types.KnowledgeBaseConfig `json:"config"`
|
2025-08-05 07:08:07 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// UpdateKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 更新知识库
|
|
|
|
|
|
// @Description 更新知识库的名称、描述和配置
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Param request body UpdateKnowledgeBaseRequest true "更新请求"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "更新后的知识库"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id} [put]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) UpdateKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start updating knowledge base")
|
|
|
|
|
|
|
|
|
|
|
|
// Validate and get the knowledge base
|
2026-01-30 08:28:01 +00:00
|
|
|
|
_, id, _, permission, err := h.validateAndGetKnowledgeBase(c)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Only admin/editor can update knowledge base
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewForbiddenError("No permission to update knowledge base"))
|
2026-01-30 08:28:01 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
// Parse request body
|
|
|
|
|
|
var req UpdateKnowledgeBaseRequest
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewBadRequestError("Invalid request parameters").WithDetails(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-07-13 03:57:30 +00:00
|
|
|
|
if req.Config != nil {
|
|
|
|
|
|
probe := &types.KnowledgeBase{
|
feat(gallery): add an image gallery tab with schema-driven filters (#3694)
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 04:33:59 +00:00
|
|
|
|
ChunkingConfig: req.Config.ChunkingConfig,
|
|
|
|
|
|
WikiConfig: req.Config.WikiConfig,
|
|
|
|
|
|
ProfileConfig: req.Config.ProfileConfig,
|
|
|
|
|
|
}
|
|
|
|
|
|
if req.Config.ImageProcessingConfig != nil {
|
|
|
|
|
|
probe.ImageProcessingConfig = *req.Config.ImageProcessingConfig
|
2026-07-13 03:57:30 +00:00
|
|
|
|
}
|
|
|
|
|
|
if err := validateKnowledgeBasePromptInstructions(probe); err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Updating knowledge base, ID: %s, name: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(id), secutils.SanitizeForLog(req.Name))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Update the knowledge base
|
|
|
|
|
|
kb, err := h.service.UpdateKnowledgeBase(ctx, id, req.Name, req.Description, req.Config)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge base updated successfully, ID: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(id))
|
2026-05-18 01:10:01 +00:00
|
|
|
|
callerTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
2026-05-18 01:10:01 +00:00
|
|
|
|
"data": buildKBResponse(kb, h.resolveKBStoreView(ctx, kb, callerTenantID), nil),
|
2025-08-05 07:08:07 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 14:21:55 +00:00
|
|
|
|
// GenerateKnowledgeBaseProfile godoc
|
|
|
|
|
|
// @Summary 生成知识库描述
|
|
|
|
|
|
// @Description 基于文档画像聚合,立即重新生成知识库的 AI 描述(不覆盖手写描述)
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "生成的知识库画像"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "知识库类型不支持或未配置模型"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge-bases/{id}/profile/generate [post]
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) GenerateKnowledgeBaseProfile(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
kb, id, _, permission, err := h.validateAndGetKnowledgeBase(c)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
_ = c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if permission != types.OrgRoleAdmin && permission != types.OrgRoleEditor {
|
|
|
|
|
|
_ = c.Error(apperrors.NewForbiddenError("No permission to update knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if h.profileService == nil {
|
|
|
|
|
|
_ = c.Error(apperrors.NewInternalServerError("knowledge base profile service unavailable"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
genCtx, cancel := context.WithTimeout(ctx, knowledgeBaseProfileRequestTimeout)
|
|
|
|
|
|
defer cancel()
|
|
|
|
|
|
profile, err := h.profileService.GenerateKnowledgeBaseProfile(genCtx, kb, true)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, map[string]interface{}{"knowledge_base_id": id})
|
|
|
|
|
|
switch {
|
|
|
|
|
|
case stderrors.Is(err, types.ErrKnowledgeBaseProfileUnsupported),
|
|
|
|
|
|
stderrors.Is(err, types.ErrKnowledgeBaseProfileModelNotConfigured):
|
|
|
|
|
|
_ = c.Error(apperrors.NewBadRequestError(err.Error()))
|
|
|
|
|
|
default:
|
|
|
|
|
|
_ = c.Error(apperrors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
}
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{"success": true, "data": profile})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// knowledgeBaseProfileRequestTimeout bounds the synchronous regeneration a
|
|
|
|
|
|
// user triggers from the settings dialog: one aggregation plus one small
|
|
|
|
|
|
// model call.
|
|
|
|
|
|
const knowledgeBaseProfileRequestTimeout = 2 * time.Minute
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// DeleteKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 删除知识库
|
|
|
|
|
|
// @Description 删除指定的知识库及其所有内容
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "知识库ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "删除成功"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/{id} [delete]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) DeleteKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
logger.Info(ctx, "Start deleting knowledge base")
|
|
|
|
|
|
|
|
|
|
|
|
// Validate and get the knowledge base
|
2026-01-30 08:28:01 +00:00
|
|
|
|
kb, id, _, permission, err := h.validateAndGetKnowledgeBase(c)
|
2025-08-05 07:08:07 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-01-30 08:28:01 +00:00
|
|
|
|
// Only owner (admin with matching tenant) can delete knowledge base
|
|
|
|
|
|
tenantID, _ := c.Get(types.TenantIDContextKey.String())
|
|
|
|
|
|
if kb.TenantID != tenantID.(uint64) || permission != types.OrgRoleAdmin {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewForbiddenError("Only knowledge base owner can delete"))
|
2026-01-30 08:28:01 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Deleting knowledge base, ID: %s, name: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(id), secutils.SanitizeForLog(kb.Name))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
|
|
|
|
|
|
// Delete the knowledge base
|
|
|
|
|
|
if err := h.service.DeleteKnowledgeBase(ctx, id); err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-11-26 11:52:07 +00:00
|
|
|
|
logger.Infof(ctx, "Knowledge base deleted successfully, ID: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(id))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"message": "Knowledge base deleted successfully",
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
type CopyKnowledgeBaseRequest struct {
|
2026-01-05 09:03:50 +00:00
|
|
|
|
TaskID string `json:"task_id"`
|
2025-08-05 07:08:07 +00:00
|
|
|
|
SourceID string `json:"source_id" binding:"required"`
|
|
|
|
|
|
TargetID string `json:"target_id"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 09:38:28 +00:00
|
|
|
|
// CopyKnowledgeBaseResponse defines the response for copy knowledge base
|
|
|
|
|
|
type CopyKnowledgeBaseResponse struct {
|
|
|
|
|
|
TaskID string `json:"task_id"`
|
|
|
|
|
|
SourceID string `json:"source_id"`
|
|
|
|
|
|
TargetID string `json:"target_id"`
|
|
|
|
|
|
Message string `json:"message"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-08 12:43:12 +00:00
|
|
|
|
type DuplicateKnowledgeBaseResponse struct {
|
|
|
|
|
|
SourceID string `json:"source_id"`
|
|
|
|
|
|
TargetID string `json:"target_id"`
|
|
|
|
|
|
Message string `json:"message"`
|
|
|
|
|
|
KnowledgeBase interface{} `json:"knowledge_base"`
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// CopyKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 复制知识库
|
2025-12-16 09:38:28 +00:00
|
|
|
|
// @Description 将一个知识库的内容复制到另一个知识库(异步任务)
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2025-12-16 09:38:28 +00:00
|
|
|
|
// @Param request body CopyKnowledgeBaseRequest true "复制请求"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "任务ID"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 08:40:13 +00:00
|
|
|
|
// @Router /knowledge-bases/copy [post]
|
2025-08-05 07:08:07 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) CopyKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
var req CopyKnowledgeBaseRequest
|
|
|
|
|
|
if err := c.ShouldBindJSON(&req); err != nil {
|
|
|
|
|
|
logger.Error(ctx, "Failed to parse request parameters", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewBadRequestError("Invalid request parameters").WithDetails(err.Error()))
|
2025-08-05 07:08:07 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-09-08 07:36:33 +00:00
|
|
|
|
caller := types.CallerFromContext(ctx)
|
|
|
|
|
|
if caller.TenantID == 0 {
|
|
|
|
|
|
_ = c.Error(errors.NewUnauthorizedError("Unauthorized"))
|
2025-12-16 09:38:28 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
tenantID := caller.TenantID
|
|
|
|
|
|
sourceGrant, err := resolveHandlerKBAccessFor(c, req.SourceID, h.service, nil, nil, types.OrgRoleViewer)
|
|
|
|
|
|
if err != nil {
|
2026-07-05 02:55:00 +00:00
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
sourceKB := sourceGrant.KnowledgeBase
|
|
|
|
|
|
if sourceKB.TenantID != caller.TenantID {
|
|
|
|
|
|
_ = c.Error(errors.NewForbiddenError("No permission to copy this knowledge base"))
|
2026-02-06 06:45:37 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
taskID := req.TaskID
|
|
|
|
|
|
if taskID == "" {
|
|
|
|
|
|
taskID = utils.GenerateTaskID("kb_clone", caller.TenantID, req.SourceID)
|
|
|
|
|
|
} else if err := requireTaskProgressTenant(ctx, taskID); err != nil {
|
|
|
|
|
|
_ = c.Error(err)
|
2026-02-06 06:45:37 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
create := req.TargetID == ""
|
|
|
|
|
|
targetKB := &types.KnowledgeBase{ID: uuid.NewString(), TenantID: caller.TenantID}
|
|
|
|
|
|
if create {
|
|
|
|
|
|
if !types.IsSyntheticUserID(caller.UserID) {
|
|
|
|
|
|
targetKB.CreatorID = caller.UserID
|
|
|
|
|
|
}
|
|
|
|
|
|
} else {
|
|
|
|
|
|
targetGrant, err := resolveHandlerKBAccessFor(c, req.TargetID, h.service, nil, nil, types.OrgRoleEditor)
|
2026-02-06 06:45:37 +00:00
|
|
|
|
if err != nil {
|
2026-09-08 07:36:33 +00:00
|
|
|
|
_ = c.Error(err)
|
2026-02-06 06:45:37 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
targetKB = targetGrant.KnowledgeBase
|
|
|
|
|
|
if targetKB.TenantID != caller.TenantID {
|
2026-02-06 06:45:37 +00:00
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to copy to this knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
if err := middleware.EvaluateOwnershipOrRole(c.Request.Context(),
|
|
|
|
|
|
h.cfg,
|
|
|
|
|
|
types.TenantRoleAdmin,
|
|
|
|
|
|
func() (string,
|
|
|
|
|
|
error,
|
|
|
|
|
|
) {
|
|
|
|
|
|
return targetKB.CreatorID,
|
|
|
|
|
|
nil
|
|
|
|
|
|
}); err != nil {
|
|
|
|
|
|
_ = c.Error(errors.NewForbiddenError("No permission to replace this knowledge base's contents"))
|
2026-05-18 01:10:01 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
tenant, _ := ctx.Value(types.TenantInfoContextKey).(*types.Tenant)
|
|
|
|
|
|
if err := access.ValidateKBTransferCompatibility(sourceKB,
|
|
|
|
|
|
targetKB,
|
|
|
|
|
|
access.KBTransferClone,
|
|
|
|
|
|
"",
|
|
|
|
|
|
tenant); err != nil {
|
|
|
|
|
|
_ = c.Error(errors.NewBadRequestError(err.Error()))
|
2026-05-18 01:10:01 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-02-06 06:45:37 +00:00
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
ctx, err = access.WithKBTransfer(c.Request.Context(), sourceKB, targetKB, access.KBTransferClone, taskID, create)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
_ = c.Error(kbAccessHTTPError(err))
|
|
|
|
|
|
return
|
2026-01-05 09:03:50 +00:00
|
|
|
|
}
|
2026-09-08 07:36:33 +00:00
|
|
|
|
// Reserve the destination in the payload; enqueue failures do not leave
|
|
|
|
|
|
// empty KBs and every worker delivery creates/resumes the same destination.
|
|
|
|
|
|
req.TargetID = targetKB.ID
|
2025-12-16 09:38:28 +00:00
|
|
|
|
|
|
|
|
|
|
// Create KB clone payload
|
|
|
|
|
|
payload := types.KBClonePayload{
|
2026-09-08 07:36:33 +00:00
|
|
|
|
TenantID: tenantID,
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
SourceID: req.SourceID,
|
|
|
|
|
|
TargetID: req.TargetID,
|
|
|
|
|
|
CreateTarget: create,
|
|
|
|
|
|
CreatorID: targetKB.CreatorID,
|
|
|
|
|
|
Initiator: types.TaskInitiatorFromContext(ctx),
|
2025-12-16 09:38:28 +00:00
|
|
|
|
}
|
feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests
(chat / search / eval), so file uploads and every downstream asynq task
(document parse, chunk embedding, OCR/VLM, summary / question gen, wiki
ingest, datasource sync, etc.) produced either disconnected shallow
traces or no observation at all.
This change threads one trace end-to-end:
- tracer: add SPAN observation type and StartSpan; add ResumeTrace so a
worker can attach to an upstream trace without emitting a duplicate
trace-create; StartGeneration now auto-picks parentObservationId from
ctx so nested trace -> span -> generation trees render correctly.
- types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq
payloads so trace_id / parent_obs_id / user_id / session_id serialise
into every job.
- langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal
so the HTTP-layer trace survives the Redis hop.
- langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either
resumes the upstream trace or opens a standalone asynq.<type> trace
for scheduled jobs, and wraps the handler in a SPAN with task metadata
(id / queue / retry / payload_bytes) plus ERROR level on failure.
- GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation /
FAQ / wiki / datasource endpoints so the root trace actually starts.
- Tests: tracer_test.go covers span nesting, error status, and
ResumeTrace no-trace-create guarantee; asynq_test.go covers
InjectTracing round-trip, middleware resume path, and standalone
trace fallback.
- Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types
and documents the cross-process propagation model.
No behavioural change when Langfuse is disabled (all new code paths are
no-ops and carriers serialise to empty strings with omitempty).
2026-04-24 04:23:30 +00:00
|
|
|
|
langfuse.InjectTracing(ctx, &payload)
|
2025-12-16 09:38:28 +00:00
|
|
|
|
|
|
|
|
|
|
payloadBytes, err := json.Marshal(payload)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to marshal KB clone payload: %v", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError("Failed to create task"))
|
2025-12-16 09:38:28 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Enqueue KB clone task to Asynq
|
2026-01-07 07:18:18 +00:00
|
|
|
|
task := asynq.NewTask(types.TypeKBClone, payloadBytes,
|
2026-07-10 15:19:41 +00:00
|
|
|
|
asynq.TaskID(taskID), asynq.Queue(types.QueueMaintenance),
|
|
|
|
|
|
asynq.MaxRetry(3), asynq.Timeout(2*time.Hour))
|
2025-12-16 09:38:28 +00:00
|
|
|
|
info, err := h.asynqClient.Enqueue(task)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.Errorf(ctx, "Failed to enqueue KB clone task: %v", err)
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewInternalServerError("Failed to enqueue task"))
|
2025-12-16 09:38:28 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "KB clone task enqueued: %s, asynq task ID: %s, source: %s, target: %s",
|
|
|
|
|
|
taskID, info.ID, secutils.SanitizeForLog(req.SourceID), secutils.SanitizeForLog(req.TargetID))
|
|
|
|
|
|
|
|
|
|
|
|
// Save initial progress to Redis so frontend can query immediately
|
|
|
|
|
|
initialProgress := &types.KBCloneProgress{
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
SourceID: req.SourceID,
|
|
|
|
|
|
TargetID: req.TargetID,
|
|
|
|
|
|
Status: types.KBCloneStatusPending,
|
|
|
|
|
|
Progress: 0,
|
|
|
|
|
|
Message: "Task queued, waiting to start...",
|
|
|
|
|
|
CreatedAt: time.Now().Unix(),
|
|
|
|
|
|
UpdatedAt: time.Now().Unix(),
|
|
|
|
|
|
}
|
|
|
|
|
|
if err := h.knowledgeService.SaveKBCloneProgress(ctx, initialProgress); err != nil {
|
|
|
|
|
|
logger.Warnf(ctx, "Failed to save initial KB clone progress: %v", err)
|
|
|
|
|
|
// Don't fail the request, task is already enqueued
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": CopyKnowledgeBaseResponse{
|
|
|
|
|
|
TaskID: taskID,
|
|
|
|
|
|
SourceID: req.SourceID,
|
|
|
|
|
|
TargetID: req.TargetID,
|
|
|
|
|
|
Message: "Knowledge base copy task started",
|
|
|
|
|
|
},
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-08 12:43:12 +00:00
|
|
|
|
// DuplicateKnowledgeBase godoc
|
|
|
|
|
|
// @Summary 创建知识库副本
|
|
|
|
|
|
// @Description 创建一个只包含设置的新知识库副本,不复制知识、FAQ 内容、分块、索引、Wiki 页面、分享或置顶状态
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
2026-07-08 12:53:59 +00:00
|
|
|
|
// @Param id path string true "源知识库 ID"
|
|
|
|
|
|
// @Success 201 {object} map[string]interface{} "创建后的知识库副本"
|
2026-07-08 12:43:12 +00:00
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Router /knowledge-bases/{id}/duplicate [post]
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) DuplicateKnowledgeBase(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
sourceID := c.Param("id")
|
|
|
|
|
|
if sourceID == "" {
|
|
|
|
|
|
c.Error(apperrors.NewBadRequestError("Knowledge base ID cannot be empty"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
callerTenantID := c.GetUint64(types.TenantIDContextKey.String())
|
|
|
|
|
|
sourceKB, err := h.service.GetKnowledgeBaseByID(ctx, sourceID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if stderrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(errors.NewNotFoundError("Source knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if sourceKB.TenantID != callerTenantID {
|
|
|
|
|
|
logger.Warnf(ctx,
|
|
|
|
|
|
"Knowledge base duplicate rejected: source belongs to another tenant, source_id: %s, caller_tenant: %d, kb_tenant: %d",
|
|
|
|
|
|
secutils.SanitizeForLog(sourceID), callerTenantID, sourceKB.TenantID)
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to duplicate this knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-07-08 12:53:59 +00:00
|
|
|
|
targetKB, err := h.service.DuplicateKnowledgeBase(ctx, sourceID)
|
2026-07-08 12:43:12 +00:00
|
|
|
|
if err != nil {
|
|
|
|
|
|
if appErr, ok := apperrors.IsAppError(err); ok {
|
|
|
|
|
|
c.Error(appErr)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if stderrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(errors.NewNotFoundError("Source knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
logger.Infof(ctx, "Knowledge base duplicate created, source: %s, target: %s",
|
|
|
|
|
|
secutils.SanitizeForLog(sourceID), secutils.SanitizeForLog(targetKB.ID))
|
|
|
|
|
|
c.JSON(http.StatusCreated, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": DuplicateKnowledgeBaseResponse{
|
|
|
|
|
|
SourceID: sourceID,
|
|
|
|
|
|
TargetID: targetKB.ID,
|
|
|
|
|
|
Message: "Knowledge base duplicate created",
|
|
|
|
|
|
KnowledgeBase: buildKBResponse(targetKB, h.resolveKBStoreView(ctx, targetKB, callerTenantID), nil),
|
|
|
|
|
|
},
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2025-12-16 09:38:28 +00:00
|
|
|
|
// GetKBCloneProgress godoc
|
|
|
|
|
|
// @Summary 获取知识库复制进度
|
|
|
|
|
|
// @Description 获取知识库复制任务的进度
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Accept json
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param task_id path string true "任务ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "进度信息"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "任务不存在"
|
|
|
|
|
|
// @Security Bearer
|
2025-12-22 06:36:29 +00:00
|
|
|
|
// @Security ApiKeyAuth
|
2025-12-16 09:38:28 +00:00
|
|
|
|
// @Router /knowledge-bases/copy/progress/{task_id} [get]
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) GetKBCloneProgress(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
taskID := c.Param("task_id")
|
|
|
|
|
|
if taskID == "" {
|
|
|
|
|
|
logger.Error(ctx, "Task ID is empty")
|
2026-02-04 12:12:29 +00:00
|
|
|
|
c.Error(apperrors.NewBadRequestError("Task ID cannot be empty"))
|
2025-12-16 09:38:28 +00:00
|
|
|
|
return
|
|
|
|
|
|
}
|
2026-07-02 03:59:50 +00:00
|
|
|
|
if err := requireTaskProgressTenant(ctx, taskID); err != nil {
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-12-16 09:38:28 +00:00
|
|
|
|
|
|
|
|
|
|
progress, err := h.knowledgeService.GetKBCloneProgress(ctx, taskID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
logger.ErrorWithFields(ctx, err, nil)
|
|
|
|
|
|
c.Error(err)
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
2025-08-29 06:49:48 +00:00
|
|
|
|
|
2025-08-05 07:08:07 +00:00
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
2025-12-16 09:38:28 +00:00
|
|
|
|
"data": progress,
|
2025-08-05 07:08:07 +00:00
|
|
|
|
})
|
|
|
|
|
|
}
|
2025-11-25 08:44:44 +00:00
|
|
|
|
|
|
|
|
|
|
// validateExtractConfig validates the graph configuration parameters
|
|
|
|
|
|
func validateExtractConfig(config *types.ExtractConfig) error {
|
|
|
|
|
|
if config == nil {
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
if !config.Enabled {
|
2026-07-13 04:14:36 +00:00
|
|
|
|
config.Enabled = false
|
2025-11-25 08:44:44 +00:00
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
// Validate text field
|
|
|
|
|
|
if config.Text == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("text cannot be empty")
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate tags field
|
|
|
|
|
|
if len(config.Tags) == 0 {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("tags cannot be empty")
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
for i, tag := range config.Tags {
|
|
|
|
|
|
if tag == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("tag cannot be empty at index " + strconv.Itoa(i))
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Validate nodes
|
|
|
|
|
|
if len(config.Nodes) == 0 {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("nodes cannot be empty")
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
nodeNames := make(map[string]bool)
|
|
|
|
|
|
for i, node := range config.Nodes {
|
|
|
|
|
|
if node.Name == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("node name cannot be empty at index " + strconv.Itoa(i))
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
// Check for duplicate node names
|
|
|
|
|
|
if nodeNames[node.Name] {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("duplicate node name: " + node.Name)
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
nodeNames[node.Name] = true
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
if len(config.Relations) == 0 {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relations cannot be empty")
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
// Validate relations
|
|
|
|
|
|
for i, relation := range config.Relations {
|
|
|
|
|
|
if relation.Node1 == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relation node1 cannot be empty at index " + strconv.Itoa(i))
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
if relation.Node2 == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relation node2 cannot be empty at index " + strconv.Itoa(i))
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
if relation.Type == "" {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relation type cannot be empty at index " + strconv.Itoa(i))
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
// Check if referenced nodes exist
|
|
|
|
|
|
if !nodeNames[relation.Node1] {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relation references non-existent node1: " + relation.Node1)
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
if !nodeNames[relation.Node2] {
|
2026-02-04 12:12:29 +00:00
|
|
|
|
return apperrors.NewBadRequestError("relation references non-existent node2: " + relation.Node2)
|
2025-11-25 08:44:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
2026-03-09 05:14:42 +00:00
|
|
|
|
|
2026-07-13 03:57:30 +00:00
|
|
|
|
func validateKnowledgeBasePromptInstructions(kb *types.KnowledgeBase) error {
|
2026-07-13 04:14:36 +00:00
|
|
|
|
if err := types.ValidateKnowledgeBasePromptInstructions(kb); err != nil {
|
|
|
|
|
|
return apperrors.NewBadRequestError(err.Error())
|
2026-07-13 03:57:30 +00:00
|
|
|
|
}
|
|
|
|
|
|
return nil
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-03-09 05:14:42 +00:00
|
|
|
|
// ListMoveTargets returns knowledge bases eligible as move targets for the given source KB.
|
|
|
|
|
|
// Filters: same Type, same EmbeddingModelID, different ID, not temporary.
|
2026-05-11 09:37:14 +00:00
|
|
|
|
//
|
|
|
|
|
|
// ListMoveTargets godoc
|
|
|
|
|
|
// @Summary 获取可移动目标知识库列表
|
|
|
|
|
|
// @Description 返回与源知识库 Type 一致、EmbeddingModelID 一致、非临时且不是自身的目标知识库列表
|
|
|
|
|
|
// @Tags 知识库
|
|
|
|
|
|
// @Produce json
|
|
|
|
|
|
// @Param id path string true "源知识库 ID"
|
|
|
|
|
|
// @Success 200 {object} map[string]interface{} "可移动目标列表"
|
|
|
|
|
|
// @Failure 400 {object} errors.AppError "请求参数错误"
|
|
|
|
|
|
// @Failure 404 {object} errors.AppError "知识库不存在"
|
|
|
|
|
|
// @Security Bearer
|
|
|
|
|
|
// @Security ApiKeyAuth
|
|
|
|
|
|
// @Router /knowledge-bases/{id}/move-targets [get]
|
2026-03-09 05:14:42 +00:00
|
|
|
|
func (h *KnowledgeBaseHandler) ListMoveTargets(c *gin.Context) {
|
|
|
|
|
|
ctx := c.Request.Context()
|
|
|
|
|
|
|
|
|
|
|
|
sourceKBID := c.Param("id")
|
|
|
|
|
|
if sourceKBID == "" {
|
|
|
|
|
|
c.Error(apperrors.NewBadRequestError("Knowledge base ID is required"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
tenantID, exists := c.Get(types.TenantIDContextKey.String())
|
|
|
|
|
|
if !exists {
|
|
|
|
|
|
c.Error(apperrors.NewUnauthorizedError("Unauthorized"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Get source knowledge base
|
|
|
|
|
|
sourceKB, err := h.service.GetKnowledgeBaseByID(ctx, sourceKBID)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
if stderrors.Is(err, repository.ErrKnowledgeBaseNotFound) {
|
|
|
|
|
|
c.Error(errors.NewNotFoundError("Source knowledge base not found"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
if sourceKB.TenantID != tenantID.(uint64) {
|
|
|
|
|
|
c.Error(errors.NewForbiddenError("No permission to access this knowledge base"))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Get all knowledge bases
|
|
|
|
|
|
allKBs, err := h.service.ListKnowledgeBases(ctx)
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
|
c.Error(errors.NewInternalServerError(err.Error()))
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Filter eligible targets
|
|
|
|
|
|
targets := make([]*types.KnowledgeBase, 0)
|
|
|
|
|
|
for _, kb := range allKBs {
|
|
|
|
|
|
if kb.ID == sourceKBID {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if kb.IsTemporary {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if kb.Type != sourceKB.Type {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
if kb.EmbeddingModelID != sourceKB.EmbeddingModelID {
|
|
|
|
|
|
continue
|
|
|
|
|
|
}
|
|
|
|
|
|
targets = append(targets, kb)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": targets,
|
|
|
|
|
|
})
|
2026-04-22 12:24:12 +00:00
|
|
|
|
}
|
feat(gallery): add an image gallery tab with schema-driven filters (#3694)
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 04:33:59 +00:00
|
|
|
|
|
|
|
|
|
|
// GetImageAttrsSchema returns the canonical image-attribute registry for this
|
|
|
|
|
|
// release. It is the single source of truth that drives the frontend attribute
|
|
|
|
|
|
// panel — both the attributes and their display text (label, description, the
|
|
|
|
|
|
// meaning of each value) — so adding an attribute later is a backend-only
|
|
|
|
|
|
// change (one registry row) and the UI follows automatically. Read-only; the
|
|
|
|
|
|
// registry is global, not per-KB, so it carries no KB id and only the Viewer
|
|
|
|
|
|
// role is required.
|
|
|
|
|
|
func (h *KnowledgeBaseHandler) GetImageAttrsSchema(c *gin.Context) {
|
|
|
|
|
|
c.JSON(http.StatusOK, gin.H{
|
|
|
|
|
|
"success": true,
|
|
|
|
|
|
"data": gin.H{
|
|
|
|
|
|
"version": types.ImageAttrSchemaVersion,
|
|
|
|
|
|
"prompt": types.ImageAttrPromptVersion,
|
|
|
|
|
|
"attributes": types.ImageAttrRegistry,
|
|
|
|
|
|
"default_actions": types.DefaultImageActions(),
|
|
|
|
|
|
},
|
|
|
|
|
|
})
|
|
|
|
|
|
}
|