Commit Graph

18 Commits (5fea7a1695febf50942e5e8fac5b71d4e41e75f6)

Author SHA1 Message Date
lyingbug fa67dfd44b feat(knowledge): store folder path for folder uploads and expose folder tree API
Folder uploads used to encode their relative directory inside file_name,
which made the document list render the whole path as the title and left
no way to browse or query a single folder. The directory now lives in a
dedicated folder_path column (with a backfill for existing rows), the
document list accepts folder_path / folder_recursive filters, and a new
GET /knowledge-bases/:id/knowledge/folders endpoint returns the folder
tree with per-folder document counts.
2026-08-04 20:45:29 +08:00
wizardchen 8dd997f0f4 refactor(knowledge): make the supported import extension set the single gate
The xlsx URL-import fix left the accepted extension set reachable through two
indirections and widened ApplyKnowledgeProcessOverrides to gate file types for
callers that are not file imports (web URL fetch, connector sync, reparse).

Collapse the set into one map that both isValidFileType and the extension check
read, move the shared validation next to the other file-type predicates, and
return ApplyKnowledgeProcessOverrides to its override-only scope so
resolveFileImportProcessConfig is the only import gate.

Also drop the unreachable branch in createKnowledgeFromFileURL: getFileType
never yields an empty string, so the resolvedFileType fallback and its else arm
could not run. Normalize the resolved type before it reaches the record and the
task payload, and add an invariant test asserting direct upload and URL import
accept the same extensions so #2447 cannot drift back.
2026-07-31 13:46:56 +08:00
wizardchen 43afa4c1f8 fix(knowledge): align URL file import validation with direct upload
Extract shared file-import checks (type, VLM/ASR, video) into
resolveKnowledgeFileImportConfig and reuse them for file URL import.
Also enqueue DataTableSummaryTask for spreadsheet URL imports and reparse.
2026-07-31 13:46:56 +08:00
wizardchen 48756d13d6 fix(knowledge): allow xlsx and other file types in URL import (#2447)
Align URL-based file import validation with direct file upload by
delegating to isValidFileType instead of a narrower whitelist.
2026-07-31 13:46:56 +08:00
wizardchen bf68db4653 fix(storage): allow image uploads to use file service fallback 2026-07-29 11:36:24 +08:00
koishi514-Z 18b95ebb3a fix(knowledge): distinguish duplicate files by type 2026-07-29 10:36:17 +08:00
wizardchen a3a0308b62 feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
wizardchen f8b3e5f148 feat(storage): add multi-instance storage backends per workspace
Introduce tenant-scoped storage backend registry so workspaces can register
multiple storage instances and bind each knowledge base to a specific backend,
with migration from legacy tenant-level storage config and new settings UI.
2026-07-15 14:54:39 +08:00
wizardchen d496d0fc73 feat(task-queue): split upstream workers into isolated core/enrichment/maintenance pools
Route summary/sync/maintenance tasks to dedicated queues and servers so parsing, enrichment, and batch work no longer share one concurrency budget, while extending the runtime dashboard and settings validation to reflect the new topology.
2026-07-10 23:51:29 +08:00
wizardchen 2f29054110 fix(knowledge): harden multi-tag data integrity and UX follow-ups
Clear tag relations on move/delete, validate tags on create, batch-load tags,
restore KB-scoped reference counts, dedupe SetKnowledgeTags, fix filter
double-fetch and ResizeObserver leaks, and regenerate Swagger for tag_ids.
2026-06-26 10:52:52 +08:00
Anky c007a897cf
feat: support multi-tag for document knowledge (#1790)
* feat: support multi-tag for document knowledge

- Add knowledge_tag_relations join table for many-to-many tag-knowledge mapping
- Migrate existing single tag_id data to the new table
- Update Knowledge struct, filter, payload types (TagID -> Tags/TagIDs)
- Rewrite repository layer: SetKnowledgeTags, GetKnowledgeTags, ListIDsByTagIDs
- Update tag count references to query knowledge_tag_relations
- Update service/handler layers for multi-tag create/list/batch-update
- Frontend: multi-select tag sidebar, multi-tag chip display, tag toggle in popup
- FAQ tags remain single-tag model (unchanged)

* feat(knowledge): tag edit dialog with dynamic overflow and inline creation

- Add TagEditDialog component with search, multi-select, and confirm
- Replace inline t-popup tag selector with dialog-based editing
- Tags overflow with +N badge, visible count adapts to container width via ResizeObserver
- Date/card layout: fixed date width, tags flex to fill remaining space
- Add persistent new-tag input bar in dialog (Enter to create, auto-select)
- Tag creation emits callback to refresh parent tag list immediately
- Apply same overflow + dialog trigger to both card view and list view
- Add i18n keys: tagEditDialogTitle, tagEditSearch, tagSelectedCount, tagOverflowTip, tagNewPlaceholder

* fix: exclude soft-deleted knowledge from tag reference counts

CountReferences, BatchCountReferences, and DeleteUnusedTags now
JOIN with knowledges table to filter out soft-deleted records
when counting knowledge_tag_relations. Previously they queried
knowledge_tag_relations directly without the deleted_at check,
causing stale tag counts after knowledge deletion.

* fix: refresh tag counts after async knowledge deletion completes

Add a second delayed tag list refresh (2.5s) after tag deletion to capture
updated reference counts for remaining tags. The original immediate refresh
handles removing the deleted tag from the list, while this delayed refresh
ensures other tags' counts are correct after the async deletion finishes.

---------

Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-06-26 00:18:48 +08:00
wizardchen ed9e780c8c feat(client): enhance knowledge file upload with process configuration support
- Updated the `CreateKnowledgeFromFile` function to accept an optional `processConfig` parameter for per-upload parse configuration overrides.
- Introduced the `KnowledgeProcessOverrides` type to define various parsing settings, including parser engine rules and chunking configurations.
- Modified the API client and related components to handle the new process configuration, improving flexibility in knowledge file uploads.
- Updated documentation and API specifications to reflect the new `process_config` parameter in the upload process.
- Added a new `UploadConfirmHost` component to facilitate user confirmation for uploads, integrating with the updated upload logic.
- Enhanced user experience by providing detailed configuration options during the upload process.
2026-06-09 20:38:24 +08:00
wolfkill 2a54bb2d45 fix: avoid stale knowledge records on upload failure 2026-05-29 16:49:07 +08:00
HelloWeit edfabf3135 bugfix 2026-05-28 20:22:55 +08:00
wizardchen 31471f5efa refactor(knowledgeService): update checkStorageEngineConfigured method for improved clarity and functionality
- Refactored checkStorageEngineConfigured to be a method of knowledgeService, enhancing encapsulation and readability.
- Updated logic to allow fallback to global file service when no storage provider is configured at the KB or tenant level, improving error handling.
- Added detailed comments to clarify the method's behavior and internal logic, ensuring better understanding for future maintenance.
2026-05-25 19:16:08 +08:00
wizardchen d4c30126a9 refactor(knowledge): split 9.8k-line service file by responsibility
internal/application/service/knowledge.go had grown to 9883 lines /
149 functions spanning CRUD, document processing, summary/question
generation, clone/move, FAQ (CRUD+import+index+export), wiki cleanup,
multimodal image, and file utilities. Navigating it, or diffing
changes in it, was increasingly painful.

Purely mechanical split into 8 files in the same package. No
signatures, behaviors, or types were modified. knowledgeService
struct, NewKnowledgeService, and package-level errors remain in
knowledge.go unchanged. Per-file imports were trimmed by goimports.

  knowledge.go              497  struct + constructor + errs + CRUD reads + Search
  knowledge_create.go      1144  CreateFrom{File,URL,Passage,Manual} + helpers
  knowledge_delete.go       686  Delete + wiki cleanup + ProcessKnowledgeListDelete
  knowledge_process.go     2407  ProcessDocument/Summary/Question + Reparse + UpdateImageInfo
  knowledge_clone_move.go  1049  Clone* + Move* + progress persistence
  knowledge_faq.go         1932  FAQ CRUD + Search + Export
  knowledge_faq_import.go  1939  FAQ import/validate/index + runningFAQImportInfo
  knowledge_util.go         352  file-type/url/hash + VLM+Storage config + resolveFileService

Verification: go build, go vet, gofmt on the 8 files, and
golangci-lint all clean (22 issues post-split matches 22 pre-split
in knowledge.go — no new issues introduced). The pre-existing
unused warnings on getVLMConfig / buildStorageConfig were left as-is
rather than removing dead code beyond the scope of this task.
2026-05-06 17:40:10 +08:00
lyingbug 0d8540c93c Revert "refactor(knowledge): split 9.8k-line service file by responsibility"
This reverts commit f67d0b53ce.
2026-05-06 17:09:24 +08:00
wizardchen f67d0b53ce refactor(knowledge): split 9.8k-line service file by responsibility
internal/application/service/knowledge.go had grown to 9883 lines /
149 functions spanning CRUD, document processing, summary/question
generation, clone/move, FAQ (CRUD+import+index+export), wiki cleanup,
multimodal image, and file utilities. Navigating it, or diffing
changes in it, was increasingly painful.

Purely mechanical split into 8 files in the same package. No
signatures, behaviors, or types were modified. knowledgeService
struct, NewKnowledgeService, and package-level errors remain in
knowledge.go unchanged. Per-file imports were trimmed by goimports.

  knowledge.go              497  struct + constructor + errs + CRUD reads + Search
  knowledge_create.go      1144  CreateFrom{File,URL,Passage,Manual} + helpers
  knowledge_delete.go       686  Delete + wiki cleanup + ProcessKnowledgeListDelete
  knowledge_process.go     2407  ProcessDocument/Summary/Question + Reparse + UpdateImageInfo
  knowledge_clone_move.go  1049  Clone* + Move* + progress persistence
  knowledge_faq.go         1932  FAQ CRUD + Search + Export
  knowledge_faq_import.go  1939  FAQ import/validate/index + runningFAQImportInfo
  knowledge_util.go         352  file-type/url/hash + VLM+Storage config + resolveFileService

Verification: go build, go vet, gofmt on the 8 files, and
golangci-lint all clean (22 issues post-split matches 22 pre-split
in knowledge.go — no new issues introduced). The pre-existing
unused warnings on getVLMConfig / buildStorageConfig were left as-is
rather than removing dead code beyond the scope of this task.
2026-05-06 17:06:05 +08:00