WeKnora/internal/handler/dto/datasource_test.go

113 lines
3.4 KiB
Go
Raw Normal View History

refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
package dto
import (
"encoding/json"
"testing"
"github.com/Tencent/WeKnora/internal/types"
"github.com/stretchr/testify/assert"
)
func TestDataSourceResponse_OmitsCredentials(t *testing.T) {
cfg := types.DataSourceConfig{
Type: "github",
Credentials: map[string]interface{}{
"token": "ghp-secret-do-not-leak",
},
ResourceIDs: []string{"repo-1"},
Settings: map[string]interface{}{"branch": "main"},
}
blob, _ := cfg.ToJSON()
ds := &types.DataSource{
ID: "ds-1",
Name: "github-prod",
Type: "github",
Config: blob,
}
body, err := json.Marshal(NewDataSourceResponse(ds))
assert.NoError(t, err)
s := string(body)
assert.NotContains(t, s, "ghp-secret-do-not-leak")
// The inner config object must not carry the credentials map (the
// DataSourceConfigDTO type omits it structurally).
var raw map[string]json.RawMessage
assert.NoError(t, json.Unmarshal(body, &raw))
if cfgRaw, ok := raw["config"]; ok {
var inner map[string]json.RawMessage
assert.NoError(t, json.Unmarshal(cfgRaw, &inner))
_, hasCredsInConfig := inner["credentials"]
assert.False(t, hasCredsInConfig,
"credentials map must not appear inside the config DTO")
}
// Top-level credentials map is just the "configured?" indicator,
// replaces the removed GET /credentials endpoint.
assert.Contains(t, s, `"credentials":{"credentials":{"configured":true}}`)
// Non-secret config fields pass through.
assert.Contains(t, s, "repo-1")
assert.Contains(t, s, "branch")
assert.Contains(t, s, "main")
}
func TestDataSourceResponse_NilSafe(t *testing.T) {
assert.Nil(t, NewDataSourceResponse(nil))
assert.Equal(t, []*DataSourceResponse{}, NewDataSourceResponses(nil))
}
func TestDataSourceResponse_RSSFeedURLsFromCredentials(t *testing.T) {
cfg := types.DataSourceConfig{
Type: types.ConnectorTypeRSS,
Credentials: map[string]interface{}{
"feed_urls": "https://example.com/a.xml\nhttps://example.com/b.xml",
},
ResourceIDs: []string{"https://example.com/a.xml"},
}
blob, _ := cfg.ToJSON()
ds := &types.DataSource{
ID: "ds-rss",
Name: "my-rss",
Type: types.ConnectorTypeRSS,
Config: blob,
}
resp := NewDataSourceResponse(ds)
assert.NotNil(t, resp.Config)
assert.Equal(t, "https://example.com/a.xml\nhttps://example.com/b.xml", resp.Config.Settings["feed_urls"])
assert.False(t, resp.Credentials["credentials"].Configured)
body, err := json.Marshal(resp)
assert.NoError(t, err)
s := string(body)
assert.Contains(t, s, "https://example.com/a.xml")
assert.NotContains(t, s, "Bearer secret")
}
func TestDataSourceResponse_RSSAuthHeadersConfigured(t *testing.T) {
cfg := types.DataSourceConfig{
Type: types.ConnectorTypeRSS,
Credentials: map[string]interface{}{
"auth_headers": "Authorization: Bearer secret",
},
Settings: map[string]interface{}{
"feed_urls": "https://example.com/feed.xml",
},
}
blob, _ := cfg.ToJSON()
ds := &types.DataSource{
ID: "ds-rss-auth",
Name: "my-rss",
Type: types.ConnectorTypeRSS,
Config: blob,
}
resp := NewDataSourceResponse(ds)
assert.True(t, resp.Credentials["credentials"].Configured)
body, err := json.Marshal(resp)
assert.NoError(t, err)
assert.NotContains(t, string(body), "Bearer secret")
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
func TestDataSourceResponse_NoConfig(t *testing.T) {
ds := &types.DataSource{ID: "x", Name: "x"}
body, err := json.Marshal(NewDataSourceResponse(ds))
assert.NoError(t, err)
// No config jsonb stored → no config object in the response.
assert.NotContains(t, string(body), `"config":`)
}