WeKnora/internal/handler/model.go

802 lines
26 KiB
Go
Raw Normal View History

2025-08-05 07:08:07 +00:00
package handler
import (
"encoding/json"
"fmt"
"io"
2025-08-05 07:08:07 +00:00
"net/http"
"strings"
"time"
2025-08-05 07:08:07 +00:00
"github.com/Tencent/WeKnora/internal/application/service"
"github.com/Tencent/WeKnora/internal/errors"
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
"github.com/Tencent/WeKnora/internal/handler/dto"
2025-08-05 07:08:07 +00:00
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/models/chat"
"github.com/Tencent/WeKnora/internal/models/provider"
2025-08-05 07:08:07 +00:00
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
secutils "github.com/Tencent/WeKnora/internal/utils"
2025-08-05 07:08:07 +00:00
"github.com/gin-gonic/gin"
)
// ModelHandler handles HTTP requests for model-related operations
// It implements the necessary methods to create, retrieve, update, and delete models
type ModelHandler struct {
service interfaces.ModelService
}
// NewModelHandler creates a new instance of ModelHandler
// It requires a model service implementation that handles business logic
// Parameters:
// - service: An implementation of the ModelService interface
//
// Returns a pointer to the newly created ModelHandler
func NewModelHandler(service interfaces.ModelService) *ModelHandler {
return &ModelHandler{service: service}
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
// Per-response redaction/stripping for Model now lives in
// dto.NewModelResponse — handlers must use it for every body that contains a
// model. The previous hideSensitiveInfo helper has been removed.
2025-08-05 07:08:07 +00:00
// CreateModelRequest defines the structure for model creation requests
// Contains all fields required to create a new model in the system
type CreateModelRequest struct {
Name string `json:"name" binding:"required"`
2026-05-28 10:36:38 +00:00
DisplayName string `json:"display_name"`
Type types.ModelType `json:"type" binding:"required"`
Source types.ModelSource `json:"source" binding:"required"`
2025-08-05 07:08:07 +00:00
Description string `json:"description"`
Parameters types.ModelParameters `json:"parameters" binding:"required"`
2025-08-05 07:08:07 +00:00
}
// CreateModel godoc
// @Summary 创建模型
// @Description 创建新的模型配置
// @Tags 模型管理
// @Accept json
// @Produce json
// @Param request body CreateModelRequest true "模型信息"
// @Success 201 {object} map[string]interface{} "创建的模型"
// @Failure 400 {object} errors.AppError "请求参数错误"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models [post]
2025-08-05 07:08:07 +00:00
func (h *ModelHandler) CreateModel(c *gin.Context) {
ctx := c.Request.Context()
logger.Info(ctx, "Start creating model")
var req CreateModelRequest
if err := c.ShouldBindJSON(&req); err != nil {
logger.Error(ctx, "Failed to parse request parameters", err)
c.Error(errors.NewBadRequestError(err.Error()))
return
}
tenantID := c.GetUint64(types.TenantIDContextKey.String())
2025-08-05 07:08:07 +00:00
if tenantID == 0 {
logger.Error(ctx, "Tenant ID is empty")
c.Error(errors.NewBadRequestError("Workspace ID cannot be empty"))
2025-08-05 07:08:07 +00:00
return
}
logger.Infof(ctx, "Creating model, Tenant ID: %d, Model name: %s, Model type: %s",
tenantID, secutils.SanitizeForLog(req.Name), secutils.SanitizeForLog(string(req.Type)))
2025-08-05 07:08:07 +00:00
// SSRF validation for model BaseURL
if req.Parameters.BaseURL != "" {
if err := secutils.ValidateURLForSSRF(req.Parameters.BaseURL); err != nil {
logger.Warnf(ctx, "SSRF validation failed for model BaseURL: %v", err)
fix(initialization): surface upstream and SSRF errors verbatim in test-connection responses Test-connection's "连接失败" was nearly useless for debugging because two layers conspired to swallow the actual upstream error: Backend swallow --------------- checkChatModelConnection and CheckASRModel pattern-matched on the upstream error string (401 / 404 / timeout / ...) and replaced it with a fixed Chinese hint. The original SDK message (request URL, response body, provider's error code description) was discarded. Refactor: extract classifyConnectionError(errMsg) — returns just the short Chinese hint. Callers combine "<hint>:<raw err>" so the hint indicates "where to start looking" and the raw error shows "what actually happened". The status-400 special-case (endpoint reachable, parameter mismatch only) still returns success. Frontend swallow ---------------- ModelEditorDialog.testConnection ignored result.message on success paths (fine), but also overwrote it with a generic i18n string on failure paths and 4xx rejections — even though axios's interceptor already lifts the backend's error.message to error.message. Failure branch now uses error?.message || result.message as the toast content; i18n fallback only when neither is present. SSRF allowlist hint ------------------- SSRF validation failures previously returned "Base URL 未通过安全校验: <reason>" with no mention of how to recover (the allowlist is server-side, not configurable in the UI). Operators hit it and were stuck. New utils.FormatSSRFError(label, rawURL, err) wraps the error with guidance to add the host to SSRF_WHITELIST_EXTRA (the operator-facing append-only escape hatch — SSRF_WHITELIST itself is the project's baseline). The hint echoes the failed host so it can be copy-pasted straight into the env var: Base URL 未通过安全校验:DNS resolution failed for hostname test.local.com ... 如该地址确实可信,请联系运维在服务端环境变量 SSRF_WHITELIST_EXTRA 中加入该主机(支持精确域名 / *.example.com 通配 / IP / CIDR),示例:SSRF_WHITELIST_EXTRA=test.local.com, *.example.com,10.0.0.0/8 All SSRF call sites switch to the helper: initialization (CheckRemoteModel / TestEmbedding / CheckRerank / CheckASR / TestMultimodal / batch validation), model CRUD, mcp_service CRUD, knowledge URL ingest, system ReconnectDocReader.
2026-05-17 07:10:42 +00:00
c.Error(errors.NewBadRequestError(secutils.FormatSSRFError("Base URL", req.Parameters.BaseURL, err)))
return
}
}
2025-08-05 07:08:07 +00:00
model := &types.Model{
TenantID: tenantID,
Name: secutils.SanitizeForLog(req.Name),
2026-05-28 10:36:38 +00:00
DisplayName: secutils.SanitizeForLog(req.DisplayName),
Type: types.ModelType(secutils.SanitizeForLog(string(req.Type))),
2025-08-05 07:08:07 +00:00
Source: req.Source,
Description: secutils.SanitizeForLog(req.Description),
2025-08-05 07:08:07 +00:00
Parameters: req.Parameters,
}
if err := h.service.CreateModel(ctx, model); err != nil {
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
logger.Infof(
ctx,
"Model created successfully, ID: %s, Name: %s",
secutils.SanitizeForLog(model.ID),
secutils.SanitizeForLog(model.Name),
)
2025-08-05 07:08:07 +00:00
c.JSON(http.StatusCreated, gin.H{
"success": true,
"data": dto.NewModelResponse(ctx, model),
2025-08-05 07:08:07 +00:00
})
}
// GetModel godoc
// @Summary 获取模型详情
// @Description 根据ID获取模型详情
// @Tags 模型管理
// @Accept json
// @Produce json
// @Param id path string true "模型ID"
// @Success 200 {object} map[string]interface{} "模型详情"
// @Failure 404 {object} errors.AppError "模型不存在"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models/{id} [get]
2025-08-05 07:08:07 +00:00
func (h *ModelHandler) GetModel(c *gin.Context) {
ctx := c.Request.Context()
logger.Info(ctx, "Start retrieving model")
id := secutils.SanitizeForLog(c.Param("id"))
2025-08-05 07:08:07 +00:00
if id == "" {
logger.Error(ctx, "Model ID is empty")
c.Error(errors.NewBadRequestError("Model ID cannot be empty"))
return
}
logger.Infof(ctx, "Retrieving model, ID: %s", id)
2025-08-05 07:08:07 +00:00
model, err := h.service.GetModelByID(ctx, id)
if err != nil {
if err == service.ErrModelNotFound {
logger.Warnf(ctx, "Model not found, ID: %s", id)
2025-08-05 07:08:07 +00:00
c.Error(errors.NewNotFoundError("Model not found"))
return
}
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
logger.Infof(ctx, "Retrieved model successfully, ID: %s, Name: %s", model.ID, model.Name)
2025-08-05 07:08:07 +00:00
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": dto.NewModelResponse(ctx, model),
2025-08-05 07:08:07 +00:00
})
}
// ListModels godoc
// @Summary 获取模型列表
// @Description 获取当前空间的所有模型
// @Tags 模型管理
// @Accept json
// @Produce json
// @Success 200 {object} map[string]interface{} "模型列表"
// @Failure 400 {object} errors.AppError "请求参数错误"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models [get]
2025-08-05 07:08:07 +00:00
func (h *ModelHandler) ListModels(c *gin.Context) {
ctx := c.Request.Context()
logger.Info(ctx, "Start retrieving model list")
tenantID := c.GetUint64(types.TenantIDContextKey.String())
2025-08-05 07:08:07 +00:00
if tenantID == 0 {
logger.Error(ctx, "Tenant ID is empty")
c.Error(errors.NewBadRequestError("Workspace ID cannot be empty"))
2025-08-05 07:08:07 +00:00
return
}
models, err := h.service.ListModels(ctx)
if err != nil {
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
logger.Infof(ctx, "Retrieved model list successfully, Tenant ID: %d, Total: %d models", tenantID, len(models))
2025-08-05 07:08:07 +00:00
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": dto.NewModelResponses(ctx, models),
2025-08-05 07:08:07 +00:00
})
}
const modelDebugMaxInputBytes = 64 * 1024
// ModelDebugOptions contains the cross-provider parameters exposed by the
// model debugger. Pointer fields preserve explicit zero/false values.
type ModelDebugOptions struct {
SystemPrompt string `json:"system_prompt,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
Thinking *bool `json:"thinking,omitempty"`
}
func parseModelDebugOptions(raw string) (ModelDebugOptions, error) {
var opts ModelDebugOptions
if strings.TrimSpace(raw) == "" {
return opts, nil
}
if err := json.Unmarshal([]byte(raw), &opts); err != nil {
return opts, fmt.Errorf("invalid options: %w", err)
}
if opts.MaxTokens != nil && (*opts.MaxTokens < 1 || *opts.MaxTokens > 8192) {
return opts, fmt.Errorf("max_tokens must be between 1 and 8192")
}
if opts.Temperature != nil && (*opts.Temperature < 0 || *opts.Temperature > 2) {
return opts, fmt.Errorf("temperature must be between 0 and 2")
}
if opts.TopP != nil && (*opts.TopP <= 0 || *opts.TopP > 1) {
return opts, fmt.Errorf("top_p must be greater than 0 and at most 1")
}
return opts, nil
}
func redactedDebugConfig(config map[string]string) map[string]string {
if len(config) == 0 {
return nil
}
out := make(map[string]string, len(config))
for key, value := range config {
lower := strings.ToLower(key)
if strings.Contains(lower, "secret") ||
strings.Contains(lower, "token") ||
strings.Contains(lower, "password") ||
strings.Contains(lower, "api_key") ||
strings.Contains(lower, "apikey") ||
strings.Contains(lower, "authorization") {
out[key] = "[REDACTED]"
continue
}
out[key] = value
}
return out
}
func modelDebugRequestPreview(model *types.Model, input string, documents []string, opts ModelDebugOptions, fileName string, fileSize int64) gin.H {
preview := gin.H{
"model_id": model.ID,
"model_name": model.Name,
"model_type": model.Type,
"source": model.Source,
"provider": model.Parameters.Provider,
"input": input,
"options": opts,
}
if len(documents) > 0 {
preview["documents"] = documents
}
if fileName != "" {
preview["file"] = gin.H{"name": fileName, "size": fileSize}
}
if model.Parameters.ExtraConfig != nil {
preview["model_extra_config"] = redactedDebugConfig(model.Parameters.ExtraConfig)
}
if len(model.Parameters.CustomHeaders) > 0 {
headerNames := make([]string, 0, len(model.Parameters.CustomHeaders))
for name := range model.Parameters.CustomHeaders {
headerNames = append(headerNames, name)
}
preview["custom_header_names"] = headerNames
}
return preview
}
func writeModelDebugResult(c *gin.Context, started time.Time, request gin.H, response any, callErr error, observations gin.H) {
data := gin.H{
"ok": callErr == nil,
"elapsed_ms": time.Since(started).Milliseconds(),
"request": request,
"raw_response": response,
"observations": observations,
}
if callErr != nil {
data["error"] = callErr.Error()
}
c.JSON(http.StatusOK, gin.H{"success": true, "data": data})
}
type modelDebugChatStreamResponse struct {
Content string `json:"content"`
ReasoningContent string `json:"reasoning_content,omitempty"`
ToolCalls []types.LLMToolCall `json:"tool_calls,omitempty"`
FinishReason string `json:"finish_reason,omitempty"`
Usage *types.TokenUsage `json:"usage,omitempty"`
StreamEvents []types.StreamResponse `json:"stream_events"`
}
func consumeModelDebugChatStream(stream <-chan types.StreamResponse) (*modelDebugChatStreamResponse, error) {
result := &modelDebugChatStreamResponse{
StreamEvents: make([]types.StreamResponse, 0),
}
for event := range stream {
result.StreamEvents = append(result.StreamEvents, event)
switch event.ResponseType {
case types.ResponseTypeThinking:
result.ReasoningContent += event.Content
case types.ResponseTypeAnswer:
result.Content += event.Content
if len(event.ToolCalls) > 0 {
result.ToolCalls = event.ToolCalls
}
if event.FinishReason != "" {
result.FinishReason = event.FinishReason
}
if event.Usage != nil {
result.Usage = event.Usage
}
case types.ResponseTypeToolCall:
if len(event.ToolCalls) > 0 {
result.ToolCalls = event.ToolCalls
}
case types.ResponseTypeError:
return result, fmt.Errorf("%s", event.Content)
}
}
return result, nil
}
// DebugModel executes a saved model through the same service constructors used
// by production calls and returns the complete normalized response. Credentials
// stay server-side; the request preview contains only non-secret fields.
func (h *ModelHandler) DebugModel(c *gin.Context) {
ctx := c.Request.Context()
started := time.Now()
id := secutils.SanitizeForLog(c.Param("id"))
if id == "" {
c.Error(errors.NewBadRequestError("Model ID cannot be empty"))
return
}
model, err := h.service.GetModelByID(ctx, id)
if err != nil {
if err == service.ErrModelNotFound {
c.Error(errors.NewNotFoundError("Model not found"))
return
}
c.Error(errors.NewInternalServerError(err.Error()))
return
}
input := c.PostForm("input")
if len(input) > modelDebugMaxInputBytes {
c.Error(errors.NewBadRequestError("input is too long"))
return
}
opts, err := parseModelDebugOptions(c.PostForm("options"))
if err != nil {
c.Error(errors.NewBadRequestError(err.Error()))
return
}
var documents []string
if rawDocuments := c.PostForm("documents"); strings.TrimSpace(rawDocuments) != "" {
if err := json.Unmarshal([]byte(rawDocuments), &documents); err != nil {
c.Error(errors.NewBadRequestError("documents must be a JSON string array"))
return
}
if len(documents) > 100 {
c.Error(errors.NewBadRequestError("documents cannot exceed 100 items"))
return
}
}
var (
fileBytes []byte
fileName string
fileSize int64
)
if file, header, fileErr := c.Request.FormFile("file"); fileErr == nil {
defer file.Close()
fileName = header.Filename
fileSize = header.Size
maxFileBytes := secutils.GetMaxFileSize()
maxFileSizeMB := secutils.GetMaxFileSizeMB()
if fileSize > maxFileBytes {
c.Error(errors.NewBadRequestError(fmt.Sprintf("file cannot exceed %d MB", maxFileSizeMB)))
return
}
fileBytes, err = io.ReadAll(io.LimitReader(file, maxFileBytes+1))
if err != nil {
c.Error(errors.NewBadRequestError("failed to read uploaded file"))
return
}
if int64(len(fileBytes)) > maxFileBytes {
c.Error(errors.NewBadRequestError(fmt.Sprintf("file cannot exceed %d MB", maxFileSizeMB)))
return
}
fileSize = int64(len(fileBytes))
}
requestPreview := modelDebugRequestPreview(model, input, documents, opts, fileName, fileSize)
observations := gin.H{}
switch model.Type {
case types.ModelTypeKnowledgeQA:
if strings.TrimSpace(input) == "" {
c.Error(errors.NewBadRequestError("query cannot be empty"))
return
}
instance, callErr := h.service.GetChatModel(ctx, id)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
messages := make([]chat.Message, 0, 2)
if strings.TrimSpace(opts.SystemPrompt) != "" {
messages = append(messages, chat.Message{Role: "system", Content: opts.SystemPrompt})
}
messages = append(messages, chat.Message{Role: "user", Content: input})
chatOpts := &chat.ChatOptions{}
if opts.Temperature != nil {
chatOpts.Temperature = *opts.Temperature
}
if opts.TopP != nil {
chatOpts.TopP = *opts.TopP
}
if opts.MaxTokens != nil {
chatOpts.MaxTokens = *opts.MaxTokens
}
chatOpts.Thinking = opts.Thinking
chatConfig := chat.ConfigFromModel(model, "", "")
thinkingControl := chat.EffectiveThinkingControl(chatConfig)
observations["stream"] = true
observations["requested_thinking"] = opts.Thinking != nil && *opts.Thinking
observations["thinking_control"] = thinkingControl
observations["thinking_parameter_sent"] = opts.Thinking != nil && thinkingControl != "none"
stream, callErr := instance.ChatStream(ctx, messages, chatOpts)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
resp, callErr := consumeModelDebugChatStream(stream)
if resp != nil {
observations["reasoning_returned"] = strings.TrimSpace(resp.ReasoningContent) != ""
observations["reasoning_characters"] = len([]rune(resp.ReasoningContent))
observations["answer_characters"] = len([]rune(resp.Content))
}
writeModelDebugResult(c, started, requestPreview, resp, callErr, observations)
case types.ModelTypeEmbedding:
if strings.TrimSpace(input) == "" {
c.Error(errors.NewBadRequestError("input cannot be empty"))
return
}
instance, callErr := h.service.GetEmbeddingModel(ctx, id)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
vector, callErr := instance.Embed(ctx, input)
observations["dimension"] = len(vector)
writeModelDebugResult(c, started, requestPreview, vector, callErr, observations)
case types.ModelTypeRerank:
if strings.TrimSpace(input) == "" || len(documents) == 0 {
c.Error(errors.NewBadRequestError("query and documents cannot be empty"))
return
}
instance, callErr := h.service.GetRerankModel(ctx, id)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
results, callErr := instance.Rerank(ctx, input, documents)
observations["result_count"] = len(results)
writeModelDebugResult(c, started, requestPreview, results, callErr, observations)
case types.ModelTypeVLLM:
if len(fileBytes) == 0 {
c.Error(errors.NewBadRequestError("image file is required"))
return
}
instance, callErr := h.service.GetVLMModel(ctx, id)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
result, callErr := instance.Predict(ctx, [][]byte{fileBytes}, input)
observations["answer_characters"] = len([]rune(result))
writeModelDebugResult(c, started, requestPreview, result, callErr, observations)
case types.ModelTypeASR:
if len(fileBytes) == 0 {
c.Error(errors.NewBadRequestError("audio file is required"))
return
}
instance, callErr := h.service.GetASRModel(ctx, id)
if callErr != nil {
writeModelDebugResult(c, started, requestPreview, nil, callErr, observations)
return
}
result, callErr := instance.Transcribe(ctx, fileBytes, fileName)
if result != nil {
observations["text_characters"] = len([]rune(result.Text))
observations["segment_count"] = len(result.Segments)
}
writeModelDebugResult(c, started, requestPreview, result, callErr, observations)
default:
c.Error(errors.NewBadRequestError("unsupported model type"))
}
}
2025-08-05 07:08:07 +00:00
// UpdateModelRequest defines the structure for model update requests
// Contains fields that can be updated for an existing model
type UpdateModelRequest struct {
Name string `json:"name"`
2026-05-28 10:36:38 +00:00
DisplayName *string `json:"display_name"`
2025-08-05 07:08:07 +00:00
Description string `json:"description"`
Parameters types.ModelParameters `json:"parameters"`
Source types.ModelSource `json:"source"`
Type types.ModelType `json:"type"`
2025-08-05 07:08:07 +00:00
}
// UpdateModel godoc
// @Summary 更新模型
// @Description 更新模型配置信息
// @Tags 模型管理
// @Accept json
// @Produce json
// @Param id path string true "模型ID"
// @Param request body UpdateModelRequest true "更新信息"
// @Success 200 {object} map[string]interface{} "更新后的模型"
// @Failure 404 {object} errors.AppError "模型不存在"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models/{id} [put]
2025-08-05 07:08:07 +00:00
func (h *ModelHandler) UpdateModel(c *gin.Context) {
ctx := c.Request.Context()
logger.Info(ctx, "Start updating model")
id := secutils.SanitizeForLog(c.Param("id"))
2025-08-05 07:08:07 +00:00
if id == "" {
logger.Error(ctx, "Model ID is empty")
c.Error(errors.NewBadRequestError("Model ID cannot be empty"))
return
}
var req UpdateModelRequest
if err := c.ShouldBindJSON(&req); err != nil {
logger.Error(ctx, "Failed to parse request parameters", err)
c.Error(errors.NewBadRequestError(err.Error()))
return
}
logger.Infof(ctx, "Retrieving model information, ID: %s", id)
2025-08-05 07:08:07 +00:00
model, err := h.service.GetModelByID(ctx, id)
if err != nil {
if err == service.ErrModelNotFound {
logger.Warnf(ctx, "Model not found, ID: %s", id)
2025-08-05 07:08:07 +00:00
c.Error(errors.NewNotFoundError("Model not found"))
return
}
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
// Update model fields if they are provided in the request
if req.Name != "" {
model.Name = req.Name
}
2026-05-28 10:36:38 +00:00
if req.DisplayName != nil {
model.DisplayName = secutils.SanitizeForLog(*req.DisplayName)
}
model.Description = req.Description
// SSRF validation for updated model BaseURL
if req.Parameters.BaseURL != "" {
if err := secutils.ValidateURLForSSRF(req.Parameters.BaseURL); err != nil {
logger.Warnf(ctx, "SSRF validation failed for model BaseURL: %v", err)
fix(initialization): surface upstream and SSRF errors verbatim in test-connection responses Test-connection's "连接失败" was nearly useless for debugging because two layers conspired to swallow the actual upstream error: Backend swallow --------------- checkChatModelConnection and CheckASRModel pattern-matched on the upstream error string (401 / 404 / timeout / ...) and replaced it with a fixed Chinese hint. The original SDK message (request URL, response body, provider's error code description) was discarded. Refactor: extract classifyConnectionError(errMsg) — returns just the short Chinese hint. Callers combine "<hint>:<raw err>" so the hint indicates "where to start looking" and the raw error shows "what actually happened". The status-400 special-case (endpoint reachable, parameter mismatch only) still returns success. Frontend swallow ---------------- ModelEditorDialog.testConnection ignored result.message on success paths (fine), but also overwrote it with a generic i18n string on failure paths and 4xx rejections — even though axios's interceptor already lifts the backend's error.message to error.message. Failure branch now uses error?.message || result.message as the toast content; i18n fallback only when neither is present. SSRF allowlist hint ------------------- SSRF validation failures previously returned "Base URL 未通过安全校验: <reason>" with no mention of how to recover (the allowlist is server-side, not configurable in the UI). Operators hit it and were stuck. New utils.FormatSSRFError(label, rawURL, err) wraps the error with guidance to add the host to SSRF_WHITELIST_EXTRA (the operator-facing append-only escape hatch — SSRF_WHITELIST itself is the project's baseline). The hint echoes the failed host so it can be copy-pasted straight into the env var: Base URL 未通过安全校验:DNS resolution failed for hostname test.local.com ... 如该地址确实可信,请联系运维在服务端环境变量 SSRF_WHITELIST_EXTRA 中加入该主机(支持精确域名 / *.example.com 通配 / IP / CIDR),示例:SSRF_WHITELIST_EXTRA=test.local.com, *.example.com,10.0.0.0/8 All SSRF call sites switch to the helper: initialization (CheckRemoteModel / TestEmbedding / CheckRerank / CheckASR / TestMultimodal / batch validation), model CRUD, mcp_service CRUD, knowledge URL ingest, system ReconnectDocReader.
2026-05-17 07:10:42 +00:00
c.Error(errors.NewBadRequestError(secutils.FormatSSRFError("Base URL", req.Parameters.BaseURL, err)))
return
}
2025-08-05 07:08:07 +00:00
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
// Credentials (api_key, app_secret) NEVER flow through this endpoint —
// they live behind the /credentials subresource. Force-preserve them by
// snapshotting the stored values before copying request fields in, so
// that even a misbehaving caller that puts api_key in the body cannot
// clobber a stored credential. Log a warning to spot stale callers.
storedAPIKey := model.Parameters.APIKey
storedAppSecret := model.Parameters.AppSecret
if req.Parameters.APIKey != "" && req.Parameters.APIKey != storedAPIKey {
logger.Warnf(ctx,
"deprecated: api_key in PUT /models/%s body is ignored; use PUT /credentials instead", id)
fix: redact sensitive fields and prevent credential loss on edit Closes #988 Unify the "write-only secrets" pattern across MCP services, Models, WebSearch providers, and DataSources. Fix two related defects: 1. GET responses leaked plaintext credentials — GetMCPServiceByID while List masked, all Model / WebSearchProvider / DataSource GET responses unconditionally. 2. When the frontend echoed a masked placeholder back on save, the update path overwrote the stored secret with the mask — silently wiping the real credential. MCP surfaced this most visibly: ListMCPServices masked Token to "first4****last4" and McpServiceDialog round-tripped that masked string back into the DB. - internal/types/secret.go introduces the shared sentinel RedactedSecretPlaceholder = "***" and two helpers: - IsRedactedOrEmpty(s): true for "" or the sentinel - PreserveIfRedacted(incoming, existing): merge predicate - Each resource adds a RedactSensitiveData (mutates-in-place) or Redacted (returns-a-copy) method and a MergeUpdate contract: empty or placeholder preserves, any other value replaces, and a write-only Clear* bool flag (gorm:"-", json omitempty) wipes with precedence over preserve / replace. Clear* never persists. - Every GET / List / Create / Update response path now redacts. VectorStore's pre-existing "***" literal is refactored onto the shared constant for consistency. All four dialogs — McpServiceDialog, ModelEditorDialog, WebSearchSettings, DataSourceEditorDialog — never pre-fill secret inputs. The stored-state signal is collapsed into the input placeholder: "•••••••• (Enter new value to replace)" when the server reports a stored value, falling back to the connector-specific placeholder otherwise. The input itself carries the "something is there" cue, matching how password dots render. An opt-in "Remove this credential" checkbox appears in edit mode when a value is stored; checking it disables the input and triggers a danger-themed confirmation dialog before submit. DataSource rolls the clear up to a connector-level "Remove all credentials" flag since its credentials are per-connector atomic bundles (e.g. Feishu app_id + app_secret). i18n updates across en-US, zh-CN, ko-KR, ru-RU: secret.storedPlaceholder / clearHint / confirmClearTitle / confirmClearBody. - MCP update handler's manual map-to-struct projection was silently dropping clear_api_key / clear_token / custom_headers on partial updates. Pull them off the map explicitly. - MCP configChanged detection now snapshots URL / StdioConfig / TransportType before the merge so connection recycling still triggers on real transport changes (not just whenever AuthConfig is present). - MCP CustomHeaders: nil preserves existing, non-nil replaces. Was always overwriting, silently wiping headers on any auth-bearing update. - GetMCPServiceByID now redacts (was plaintext while List masked). - WebSearchProvider update preserves Name / Description when empty instead of clobbering them. - WebSearchProvider TestProviderRaw rejects the "***" placeholder up front with an actionable message rather than forwarding it to the upstream provider API as a literal key. - Model builtin hideSensitiveInfo now preserves non-secret metadata (SupportsVision, Provider, ExtraConfig, InterfaceType, ParameterSize, EmbeddingParameters) that were previously zeroed out for builtin model cards. - Replace local equalStringSlices / equalStringMaps helpers with stdlib slices.Equal / maps.Equal (Go 1.21+). - internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go: table-driven coverage of merge, redact, clear, and write-only flag invariants for every resource. - internal/application/service/mcp_service_test.go: full service layer with a fake repo; covers ListMCPServices / GetMCPServiceByID redaction, UpdateMCPService partial-update honoring of Clear*, CustomHeaders preserve-on-nil / replace-on-non-nil, and clear-flag precedence over submitted values. Additive only. No changes to request or response JSON shape beyond the write-only Clear* flags (omitted by default via omitempty). No DB schema changes. No SDK struct field removals. Existing clients that don't know about the new flags continue to function unchanged. - DataSource.Config jsonb at-rest encryption. Pre-existing residual risk; tracked as a follow-up. - Renaming VectorStore.MaskSensitiveFields to match the new RedactSensitiveData / Redacted naming convention. Left alone to minimize diff; a follow-up rename PR can unify.
2026-04-18 12:48:50 +00:00
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
if req.Parameters.AppSecret != "" && req.Parameters.AppSecret != storedAppSecret {
logger.Warnf(ctx,
"deprecated: app_secret in PUT /models/%s body is ignored; use PUT /credentials instead", id)
fix: redact sensitive fields and prevent credential loss on edit Closes #988 Unify the "write-only secrets" pattern across MCP services, Models, WebSearch providers, and DataSources. Fix two related defects: 1. GET responses leaked plaintext credentials — GetMCPServiceByID while List masked, all Model / WebSearchProvider / DataSource GET responses unconditionally. 2. When the frontend echoed a masked placeholder back on save, the update path overwrote the stored secret with the mask — silently wiping the real credential. MCP surfaced this most visibly: ListMCPServices masked Token to "first4****last4" and McpServiceDialog round-tripped that masked string back into the DB. - internal/types/secret.go introduces the shared sentinel RedactedSecretPlaceholder = "***" and two helpers: - IsRedactedOrEmpty(s): true for "" or the sentinel - PreserveIfRedacted(incoming, existing): merge predicate - Each resource adds a RedactSensitiveData (mutates-in-place) or Redacted (returns-a-copy) method and a MergeUpdate contract: empty or placeholder preserves, any other value replaces, and a write-only Clear* bool flag (gorm:"-", json omitempty) wipes with precedence over preserve / replace. Clear* never persists. - Every GET / List / Create / Update response path now redacts. VectorStore's pre-existing "***" literal is refactored onto the shared constant for consistency. All four dialogs — McpServiceDialog, ModelEditorDialog, WebSearchSettings, DataSourceEditorDialog — never pre-fill secret inputs. The stored-state signal is collapsed into the input placeholder: "•••••••• (Enter new value to replace)" when the server reports a stored value, falling back to the connector-specific placeholder otherwise. The input itself carries the "something is there" cue, matching how password dots render. An opt-in "Remove this credential" checkbox appears in edit mode when a value is stored; checking it disables the input and triggers a danger-themed confirmation dialog before submit. DataSource rolls the clear up to a connector-level "Remove all credentials" flag since its credentials are per-connector atomic bundles (e.g. Feishu app_id + app_secret). i18n updates across en-US, zh-CN, ko-KR, ru-RU: secret.storedPlaceholder / clearHint / confirmClearTitle / confirmClearBody. - MCP update handler's manual map-to-struct projection was silently dropping clear_api_key / clear_token / custom_headers on partial updates. Pull them off the map explicitly. - MCP configChanged detection now snapshots URL / StdioConfig / TransportType before the merge so connection recycling still triggers on real transport changes (not just whenever AuthConfig is present). - MCP CustomHeaders: nil preserves existing, non-nil replaces. Was always overwriting, silently wiping headers on any auth-bearing update. - GetMCPServiceByID now redacts (was plaintext while List masked). - WebSearchProvider update preserves Name / Description when empty instead of clobbering them. - WebSearchProvider TestProviderRaw rejects the "***" placeholder up front with an actionable message rather than forwarding it to the upstream provider API as a literal key. - Model builtin hideSensitiveInfo now preserves non-secret metadata (SupportsVision, Provider, ExtraConfig, InterfaceType, ParameterSize, EmbeddingParameters) that were previously zeroed out for builtin model cards. - Replace local equalStringSlices / equalStringMaps helpers with stdlib slices.Equal / maps.Equal (Go 1.21+). - internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go: table-driven coverage of merge, redact, clear, and write-only flag invariants for every resource. - internal/application/service/mcp_service_test.go: full service layer with a fake repo; covers ListMCPServices / GetMCPServiceByID redaction, UpdateMCPService partial-update honoring of Clear*, CustomHeaders preserve-on-nil / replace-on-non-nil, and clear-flag precedence over submitted values. Additive only. No changes to request or response JSON shape beyond the write-only Clear* flags (omitted by default via omitempty). No DB schema changes. No SDK struct field removals. Existing clients that don't know about the new flags continue to function unchanged. - DataSource.Config jsonb at-rest encryption. Pre-existing residual risk; tracked as a follow-up. - Renaming VectorStore.MaskSensitiveFields to match the new RedactSensitiveData / Redacted naming convention. Left alone to minimize diff; a follow-up rename PR can unify.
2026-04-18 12:48:50 +00:00
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
newParams := req.Parameters
newParams.APIKey = storedAPIKey
newParams.AppSecret = storedAppSecret
// Preserve backend-managed fields not sent by the frontend either.
newParams.ParameterSize = model.Parameters.ParameterSize
if newParams.InterfaceType == "" {
newParams.InterfaceType = model.Parameters.InterfaceType
}
if newParams.AppID == "" {
newParams.AppID = model.Parameters.AppID
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
if newParams.ExtraConfig == nil {
newParams.ExtraConfig = model.Parameters.ExtraConfig
}
refactor(credentials): introduce /credentials subresource pattern Replace the legacy "redacted placeholder + Clear* boolean" pattern with dedicated per-resource credential subresources across MCP services, Models, Web search providers, and Data sources. Why --- The previous design had three problems: 1. Main PUT body carried secret fields. The frontend echoed back a redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty defense in the service layer tried to detect "user did not change this" vs "user wants to clear this". A regression in that defense (or a new frontend forgetting it) silently overwrites the stored secret with the placeholder. 2. The "remove this credential" UX was a red checkbox under a pre-filled password input. Three intents (preserve / replace / clear) collapsed onto one field, and credential changes were bundled with unrelated config edits in the same submit. Users wiped working keys by mistake. 3. Secret presence was inferred from "did the response come back with a '***' placeholder", which couples the contract to a magic string. Design ------ Each of the four resources now exposes: PUT /{resource}/{id}/credentials # write one or more fields DELETE /{resource}/{id}/credentials/{field} # clear a single field "Is this configured?" metadata travels on the main resource response as a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET endpoint. The frontend reads the configured boolean from the main GET and never sees secret values at all. Main PUT endpoints now ignore any secret fields in the request body and log a deprecation warning if they appear, so legacy clients fail loudly rather than overwriting silently. Frontend -------- - New reusable <CredentialResource> component renders a three-state card (unconfigured / configured / editing) and drives the dedicated endpoints. Used by MCP, Model, Web search; DataSource has a bespoke card with the same behaviour because its credentials are a single atomic per-connector map. - Cancel from edit mode now restores state synchronously from the meta prop. The previous async refresh() was a no-op while state was still 'editing', leaving the input frozen open. - Remove is single-click + toast. The danger-themed button is the deterrent; a modal confirm adds friction without adding safety (the plaintext is irrecoverable client-side either way — recovery means re-typing). DTOs (internal/handler/dto/) are deliberately separate from the GORM models so "no secret in response" is a compile-time invariant: a future contributor cannot leak a secret without explicitly adding the field to the DTO, which is review-able in a single diff. Storage is unchanged — credentials still live in the existing jsonb / parameters columns. No schema migration. Cleanup ------- - types.MCPService / types.Model / types.WebSearchProviderEntity / types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData(). - utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty. RedactedSecretPlaceholder constant is retained because VectorStore still uses the old pattern and is out of scope here. - Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 07:09:30 +00:00
model.Parameters = newParams
model.Source = req.Source
model.Type = req.Type
2025-08-05 07:08:07 +00:00
logger.Infof(ctx, "Updating model, ID: %s, Name: %s", id, model.Name)
2025-08-05 07:08:07 +00:00
if err := h.service.UpdateModel(ctx, model); err != nil {
if appErr, ok := errors.IsAppError(err); ok {
c.Error(appErr)
return
}
2025-08-05 07:08:07 +00:00
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
logger.Infof(ctx, "Model updated successfully, ID: %s", id)
2025-08-05 07:08:07 +00:00
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": dto.NewModelResponse(ctx, model),
2025-08-05 07:08:07 +00:00
})
}
// DeleteModel godoc
// @Summary 删除模型
// @Description 删除指定的模型
// @Tags 模型管理
// @Accept json
// @Produce json
// @Param id path string true "模型ID"
// @Success 200 {object} map[string]interface{} "删除成功"
// @Failure 404 {object} errors.AppError "模型不存在"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models/{id} [delete]
2025-08-05 07:08:07 +00:00
func (h *ModelHandler) DeleteModel(c *gin.Context) {
ctx := c.Request.Context()
logger.Info(ctx, "Start deleting model")
id := secutils.SanitizeForLog(c.Param("id"))
2025-08-05 07:08:07 +00:00
if id == "" {
logger.Error(ctx, "Model ID is empty")
c.Error(errors.NewBadRequestError("Model ID cannot be empty"))
return
}
logger.Infof(ctx, "Deleting model, ID: %s", id)
2025-08-05 07:08:07 +00:00
if err := h.service.DeleteModel(ctx, id); err != nil {
if err == service.ErrModelNotFound {
logger.Warnf(ctx, "Model not found, ID: %s", id)
2025-08-05 07:08:07 +00:00
c.Error(errors.NewNotFoundError("Model not found"))
return
}
if appErr, ok := errors.IsAppError(err); ok {
c.Error(appErr)
return
}
2025-08-05 07:08:07 +00:00
logger.ErrorWithFields(ctx, err, nil)
c.Error(errors.NewInternalServerError(err.Error()))
return
}
logger.Infof(ctx, "Model deleted successfully, ID: %s", id)
2025-08-05 07:08:07 +00:00
c.JSON(http.StatusOK, gin.H{
"success": true,
"message": "Model deleted",
})
}
// ModelProviderDTO 模型厂商信息 DTO
type ModelProviderDTO struct {
Value string `json:"value"` // provider 标识符
Label string `json:"label"` // 显示名称
Description string `json:"description"` // 描述
DefaultURLs map[string]string `json:"defaultUrls"` // 按模型类型区分的默认 URL
ModelTypes []string `json:"modelTypes"` // 支持的模型类型
}
// modelTypeToFrontend 将后端 ModelType 转换为前端兼容的字符串
// KnowledgeQA -> chat, Embedding -> embedding, Rerank -> rerank, VLLM -> vllm
func modelTypeToFrontend(mt types.ModelType) string {
switch mt {
case types.ModelTypeKnowledgeQA:
return "chat"
case types.ModelTypeEmbedding:
return "embedding"
case types.ModelTypeRerank:
return "rerank"
case types.ModelTypeVLLM:
return "vllm"
case types.ModelTypeASR:
return "asr"
default:
return string(mt)
}
}
// ListModelProviders godoc
// @Summary 获取模型厂商列表
// @Description 根据模型类型获取支持的厂商列表及配置信息
// @Tags 模型管理
// @Accept json
// @Produce json
// @Param model_type query string false "模型类型 (chat, embedding, rerank, vllm)"
// @Success 200 {object} map[string]interface{} "厂商列表"
// @Security Bearer
// @Security ApiKeyAuth
// @Router /models/providers [get]
func (h *ModelHandler) ListModelProviders(c *gin.Context) {
ctx := c.Request.Context()
modelType := c.Query("model_type")
logger.Infof(ctx, "Listing model providers for type: %s", secutils.SanitizeForLog(modelType))
// 将前端类型映射到后端类型
// 前端: chat, embedding, rerank, vllm
// 后端: KnowledgeQA, Embedding, Rerank, VLLM
var backendModelType types.ModelType
switch modelType {
case "chat":
backendModelType = types.ModelTypeKnowledgeQA
case "embedding":
backendModelType = types.ModelTypeEmbedding
case "rerank":
backendModelType = types.ModelTypeRerank
case "vllm":
backendModelType = types.ModelTypeVLLM
case "asr":
backendModelType = types.ModelTypeASR
default:
backendModelType = types.ModelType(modelType)
}
var providers []provider.ProviderInfo
if modelType != "" {
// 按模型类型过滤
providers = provider.ListByModelType(backendModelType)
} else {
// 返回所有 provider
providers = provider.List()
}
// 转换为 DTO
result := make([]ModelProviderDTO, 0, len(providers))
for _, p := range providers {
// 转换 DefaultURLs map[types.ModelType]string -> map[string]string
// 使用前端兼容的 key (chat 而不是 KnowledgeQA)
defaultURLs := make(map[string]string)
for mt, url := range p.DefaultURLs {
frontendType := modelTypeToFrontend(mt)
defaultURLs[frontendType] = url
}
// 转换 ModelTypes 为前端兼容格式
modelTypes := make([]string, 0, len(p.ModelTypes))
for _, mt := range p.ModelTypes {
modelTypes = append(modelTypes, modelTypeToFrontend(mt))
}
result = append(result, ModelProviderDTO{
Value: string(p.Name),
Label: p.DisplayName,
Description: p.Description,
DefaultURLs: defaultURLs,
ModelTypes: modelTypes,
})
}
logger.Infof(ctx, "Retrieved %d providers", len(result))
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": result,
})
}