WeKnora/migrations/sqlite/000023_message_artifacts_ta...

60 lines
2.7 KiB
MySQL
Raw Normal View History

feat(artifacts): artifacts library page backed by a message_artifacts table (#3384) * feat(artifacts): add an artifacts library with its own message_artifacts table Skill-generated files were only reachable from inside the conversation that produced them. This adds a sidebar entry and a page that lists them across every conversation the user can see. Storage: artifacts move out of the messages.artifacts JSONB column into a new message_artifacts table (PostgreSQL 000103, SQLite 000023), backfilled on upgrade. The table is now the only source of truth; the old column is kept and the down migration writes the table back into it. The message repository loads artifacts for every message it returns and writes them on create, update (when non-nil) and fork, so callers still use Message.Artifacts. mod_time is stored as RFC 3339 text because the collector matches files on nanosecond mtimes, which PostgreSQL timestamps would truncate. API: GET /api/v1/artifacts lists the latest version of each file (grouped by session + source path, with version_count), scoped like the session list, with keyword / file type filters and paging. Downloads keep using the per-session endpoint. Frontend: "产物/Artifacts" sidebar entry (hidden without sandbox support), a library page with type tabs, search, date groups, infinite loading, preview drawer, download and jump-to-conversation; strings in all five locales. The sidebar's top-menu paths are now one shared list. * fix(artifacts): keep the restore hash guard and count versions by URL RecordRestoredArtifactMtime again only stamps rows whose content hash matches the restored content (and ignores empty hashes), and never rewrites the hash, matching MessageArtifacts.WithRestoredMtime. Without the guard a restore of v1's content also moved v2's mtime and relabelled v2 with v1's hash. The library's version_count now counts distinct URLs per (session, source path). A later answer that references an earlier file stores another row with the same URL, which COUNT(*) reported as an extra version. * fix(artifacts): scope the library to web chats and clear the legacy column The library used the session list's default scope, which counts sessions without an owner as legacy tenant-level rows. IM sessions are created without an owner, so every member saw the file names and titles of every IM chat (downloads then 404 for non-admins). The library now applies the same "web" bucket predicate as the home sidebar, extracted from QueryPaged so the two cannot drift: no IM mapping (including cleared ones), no embed marker, no API-key owner. Migrations 000103 / SQLite 000023 now clear messages.artifacts after the backfill. Message loads still SELECT *, so the stale copy was read on every history load. Only rows that carried artifacts are rewritten; the column stays for instances on the old binary during a rolling upgrade, and the down migration still rebuilds it from the table. * fix(migrations): keep the 000103 backfill alive on malformed artifact JSON A single file_size that is not an integer ("12.5", "") or a created_at that does not parse ("garbage", month 13) raised a cast error and failed the whole migration, blocking startup. Casts now go through pg_temp helpers that return NULL on bad input, falling back to 0 and the message's created_at; valid rows are unchanged. SQLite casts never raise; its migration test now covers the same malformed values. * docs(artifacts): describe the web-only library scope and the cleared legacy column
2026-09-18 04:10:19 +00:00
-- Mirrors versioned migration 000103_message_artifacts_table:
-- skill-generated files move out of messages.artifacts into their own table,
-- which becomes the only source of truth. After the backfill the legacy column
-- is cleared (message loads still SELECT *) but kept; the down migration
-- rebuilds it from the table.
CREATE TABLE IF NOT EXISTS message_artifacts (
id VARCHAR(36) PRIMARY KEY,
session_id VARCHAR(36) NOT NULL,
message_id VARCHAR(36) NOT NULL,
position INTEGER NOT NULL,
url TEXT NOT NULL DEFAULT '',
file_name TEXT NOT NULL DEFAULT '',
file_type VARCHAR(32) NOT NULL DEFAULT '',
file_size INTEGER NOT NULL DEFAULT 0,
content_hash VARCHAR(64) NOT NULL DEFAULT '',
source_path TEXT NOT NULL DEFAULT '',
mod_time VARCHAR(64) NOT NULL DEFAULT '',
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_message_artifacts_message_position
ON message_artifacts (message_id, position);
CREATE INDEX IF NOT EXISTS idx_message_artifacts_session_created
ON message_artifacts (session_id, created_at);
-- mod_time is kept as the RFC 3339 text Go wrote (nanosecond precision; the
-- collector matches files on it). created_at is an ORDER BY key compared as
-- text, so it is normalised to UTC in the layout the repository writes.
INSERT OR IGNORE INTO message_artifacts (
id, session_id, message_id, position, url, file_name, file_type, file_size,
content_hash, source_path, mod_time, created_at
)
SELECT
lower(hex(randomblob(4)) || '-' || hex(randomblob(2)) || '-' || hex(randomblob(2)) || '-' ||
hex(randomblob(2)) || '-' || hex(randomblob(6))),
m.session_id,
m.id,
CAST(a.key AS INTEGER),
COALESCE(json_extract(a.value, '$.url'), ''),
COALESCE(json_extract(a.value, '$.file_name'), ''),
substr(COALESCE(json_extract(a.value, '$.file_type'), ''), 1, 32),
COALESCE(CAST(json_extract(a.value, '$.file_size') AS INTEGER), 0),
substr(COALESCE(json_extract(a.value, '$.content_hash'), ''), 1, 64),
COALESCE(json_extract(a.value, '$.source_path'), ''),
substr(COALESCE(json_extract(a.value, '$.mod_time'), ''), 1, 64),
COALESCE(
strftime('%Y-%m-%d %H:%M:%f+00:00', NULLIF(json_extract(a.value, '$.created_at'), '')),
strftime('%Y-%m-%d %H:%M:%f+00:00', m.created_at),
strftime('%Y-%m-%d %H:%M:%f+00:00', 'now')
)
FROM messages m, json_each(
CASE WHEN json_valid(m.artifacts) AND json_type(m.artifacts) = 'array' THEN m.artifacts ELSE '[]' END
) a
WHERE a.type = 'object';
-- Only rows that carried artifacts; '[]' rows are left as they are.
UPDATE messages SET artifacts = NULL
WHERE artifacts IS NOT NULL AND artifacts <> '[]';