* perf(chunk): speed up chunk ingestion and batch updates
Chunk writes were dominated by avoidable work in the repository layer:
- UpdateChunks built one CASE WHEN id = ? chain per column, so a batch of
N rows evaluated 5N branches per row (O(N²)). On PostgreSQL it now joins
against a VALUES list keyed by the primary key (O(N), batched by 1000);
other dialects run one small UPDATE per row inside a single transaction.
- CreateChunks copied Content into SourceContent for every new row, doubling
insert volume. SourceContent is only needed once a chunk is edited and the
edit path already backfills it lazily, so it is left empty on create.
Batch size raised from 100 to 500 and the insert (including the SQLite
seq_id pre-assignment) now runs inside one transaction.
- New UpdateChunkFieldsByIDs applies uniform field values with a single
UPDATE ... WHERE id IN per batch. FAQ import uses it to flip status after
indexing instead of shipping every row's content back, and resolves tag
names per batch instead of per entry.
- Drop three chunks indexes that only slow down writes: chunk_type (low
cardinality), content_hash (never used as a filter) and
(tenant_id, knowledge_id) (covered by idx_chunks_knowledge_enabled).
Migrations 000100 (PostgreSQL) and 000020 (SQLite) with down scripts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(chunk): rewrite chunk batch writes and FAQ import status updates
Go changes belonging to the chunk write-path optimization. The previous
commit on this branch only carried the migrations because the repo's
pre-commit hook re-staged files from the main checkout when run inside a
git worktree, silently dropping the staged Go changes.
- UpdateChunks: O(N²) CASE chains replaced by a VALUES join on PostgreSQL
(batched by 1000) and per-row updates in one transaction elsewhere.
- CreateChunks: stop copying Content into SourceContent, batch 500 rows,
run the insert and the SQLite seq_id pre-assignment in one transaction.
- New UpdateChunkFieldsByIDs for uniform, tenant-scoped field updates.
- FAQ import flips status with UpdateChunkFieldsByIDs and resolves tag
names once per batch instead of once per entry.
- Drop the gorm index tag on ContentHash to match migration 000100;
SQLite schema test expects migration version 20.
- SQLite tests for the new and rewritten repository methods.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>