Add web search capabilities

This commit is contained in:
HugeFrog24
2026-07-17 10:49:24 +02:00
parent 0543283b8a
commit 7fd7818142
6 changed files with 344 additions and 52 deletions
+29
View File
@@ -76,9 +76,38 @@ Each bot is one JSON file in `config/` (see `config/default.json` for the templa
| `max_tokens` | number | `1000` | Maximum output tokens per reply. **Thinking tokens count toward this limit** — raise it (e.g. `4000`+) whenever `thinking` is `"adaptive"`, or a turn can spend the whole budget on reasoning and produce no text. | | `max_tokens` | number | `1000` | Maximum output tokens per reply. **Thinking tokens count toward this limit** — raise it (e.g. `4000`+) whenever `thinking` is `"adaptive"`, or a turn can spend the whole budget on reasoning and produce no text. |
| `thinking` | string | *(omitted)* | Reasoning mode: `"adaptive"` (the model decides when and how much to think) or `"disabled"`. Omit the key entirely to use the model's own API default. If the configured model doesn't support the chosen mode, the API rejects the request with a 400 — owners/admins see the raw error, regular users get the generic fallback. Check [Anthropic's model docs](https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking) for per-model support. | | `thinking` | string | *(omitted)* | Reasoning mode: `"adaptive"` (the model decides when and how much to think) or `"disabled"`. Omit the key entirely to use the model's own API default. If the configured model doesn't support the chosen mode, the API rejects the request with a 400 — owners/admins see the raw error, regular users get the generic fallback. Check [Anthropic's model docs](https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking) for per-model support. |
| `thinking_display` | string | *(omitted)* | `"summarized"` or `"omitted"`. Only valid together with `"thinking": "adaptive"`. Controls whether the API returns a readable summary of the reasoning (logged, never sent to chat). Thinking is billed the same either way; when omitted, the API's per-model default applies. | | `thinking_display` | string | *(omitted)* | `"summarized"` or `"omitted"`. Only valid together with `"thinking": "adaptive"`. Controls whether the API returns a readable summary of the reasoning (logged, never sent to chat). Thinking is billed the same either way; when omitted, the API's per-model default applies. |
| `web_search` | object | *(omitted)* | Enables Anthropic's server-side web search (and, optionally, web fetch), sandboxed to a domain allowlist. Omit the key entirely to leave both tools off — an absent block sends byte-identical requests to before. See [Web search / fetch](#web-search--fetch) below. |
Every reply logs one accounting line — `[usage] model=... in=... out=... thinking=... stop=...` — so thinking spend (billed even when its text is omitted) stays visible in `journalctl`/`docker compose logs`. A `stop=max_tokens` line is accompanied by an error-level warning that the reply was truncated. Every reply logs one accounting line — `[usage] model=... in=... out=... thinking=... stop=...` — so thinking spend (billed even when its text is omitted) stays visible in `journalctl`/`docker compose logs`. A `stop=max_tokens` line is accompanied by an error-level warning that the reply was truncated.
### Web search / fetch
The `web_search` block wires in Anthropic's server-side `web_search` (and optionally `web_fetch`) tools, mirroring how `mcp_servers` wires in MCP toolsets: the engine provides the generic mechanism, and the per-bot config carries the policy. **No domains are hardcoded in the engine** — each profile declares its own allowlist. When the model decides a question needs current information, it runs a search server-side; results and any fetch targets are confined to the domains you list.
```json
"web_search": {
"allowed_domains": [
"example.com/help",
"docs.example.com"
],
"max_uses": 3,
"fetch": true,
"max_content_tokens": 50000
}
```
| Field | Type | Description |
| -------------------- | -------- | ----------- |
| `allowed_domains` | string[] | Search results and fetch targets are restricted to these. Each entry is a **domain with an optional path prefix and no scheme** — `example.com` or `example.com/help`. Mutually exclusive with `blocked_domains`. |
| `blocked_domains` | string[] | Excludes these domains instead of allowlisting. Mutually exclusive with `allowed_domains` (setting both fails validation at boot, since the API 400s). |
| `max_uses` | number | Caps how many searches the model may run per turn. Omit for no cap. |
| `fetch` | bool | Also enable `web_fetch`, which pulls full page content for URLs already surfaced by a search or pasted by the user (it cannot fetch model-invented URLs). Shares the same allow/block list. Citations are always on when fetch is enabled, so fetched passages are sourceable. |
| `max_content_tokens` | number | Caps the tokens a single `web_fetch` may pull into context. Only meaningful with `fetch: true`. |
The allowlist is a **hard, server-side sandbox**, not a prompt request — off-list results are dropped and an off-list fetch target returns `url_not_allowed`, which is the exfiltration mitigation Anthropic recommends for bots processing untrusted input. Keep the list tight. Note that `web_fetch` cannot read JavaScript-rendered pages, so for dynamic sites the search snippet (plus citations) is the reliable signal and a full fetch is a bonus, not a dependency.
Web search is **not free** — Anthropic bills per search (plus the tokens the results add) — so scope the allowlist and `max_uses` deliberately, and lean on a prompt that tells the bot to search only when a question actually turns on current or authoritative information. Search activity logs as `[web] ...` lines, and long multi-search turns are handled transparently (the engine follows Anthropic's `pause_turn` continuations up to a small cap, so a slow search doesn't truncate the reply).
> [!TIP] > [!TIP]
> For deep request/response debugging, the Anthropic Go SDK ships `option.WithDebugLog(...)` (dumps full HTTP bodies with auth headers redacted). It is not wired into the bot — dev-only, add it temporarily to the client constructor if you ever need wire-level traces. > For deep request/response debugging, the Anthropic Go SDK ships `option.WithDebugLog(...)` (dumps full HTTP bodies with auth headers redacted). It is not wired into the bot — dev-only, add it temporarily to the client constructor if you ever need wire-level traces.
+124 -52
View File
@@ -17,6 +17,8 @@ var ErrModelNotFound = errors.New("model not found or deprecated")
const maxFileNotFoundRetries = 3 const maxFileNotFoundRetries = 3
const maxPauseTurnContinuations = 5
const defaultMaxTokens = 1000 const defaultMaxTokens = 1000
const mcpUnsupportedSentinel = "format not currently supported by the Anthropic API" const mcpUnsupportedSentinel = "format not currently supported by the Anthropic API"
@@ -65,9 +67,10 @@ func (b *Bot) getAnthropicResponse(ctx context.Context, chatID int64, messages [
params.Thinking = thinking params.Thinking = thinking
} }
var tools []anthropic.BetaToolUnionParam
if len(b.config.MCPServers) > 0 { if len(b.config.MCPServers) > 0 {
mcpServers := make([]anthropic.BetaRequestMCPServerURLDefinitionParam, 0, len(b.config.MCPServers)) mcpServers := make([]anthropic.BetaRequestMCPServerURLDefinitionParam, 0, len(b.config.MCPServers))
tools := make([]anthropic.BetaToolUnionParam, 0, len(b.config.MCPServers))
for _, s := range b.config.MCPServers { for _, s := range b.config.MCPServers {
srv := anthropic.BetaRequestMCPServerURLDefinitionParam{ srv := anthropic.BetaRequestMCPServerURLDefinitionParam{
Name: s.Name, Name: s.Name,
@@ -95,32 +98,99 @@ func (b *Bot) getAnthropicResponse(ctx context.Context, chatID int64, messages [
tools = append(tools, anthropic.BetaToolUnionParam{OfMCPToolset: toolset}) tools = append(tools, anthropic.BetaToolUnionParam{OfMCPToolset: toolset})
} }
params.MCPServers = mcpServers params.MCPServers = mcpServers
params.Tools = tools
params.Betas = append(params.Betas, anthropic.AnthropicBetaMCPClient2025_11_20) params.Betas = append(params.Betas, anthropic.AnthropicBetaMCPClient2025_11_20)
} }
for attempt := 0; attempt < maxFileNotFoundRetries; attempt++ { tools = append(tools, webSearchTools(b.config.WebSearch)...)
joined, streamErr := b.streamMessages(ctx, params, onSegment)
if streamErr == nil { if len(tools) > 0 {
return joined, nil params.Tools = tools
}
var apiErr *anthropic.Error
if !errors.As(streamErr, &apiErr) || apiErr.StatusCode != http.StatusNotFound {
return "", fmt.Errorf("error creating Anthropic message: %w", streamErr)
}
missingFileID := extractMissingFileID(streamErr)
if missingFileID == "" {
return "", fmt.Errorf("%w: %s", ErrModelNotFound, b.config.Model)
}
ErrorLogger.Printf("[%s] self-heal: stripping dead file_id %s from chat %d (attempt %d/%d)",
b.config.ID, missingFileID, chatID, attempt+1, maxFileNotFoundRetries)
b.stripDeadFileIDFromMemory(chatID, missingFileID)
if _, cleanupErr := b.markFilesPendingCleanup(ctx, chatID, []string{missingFileID}); cleanupErr != nil {
ErrorLogger.Printf("[%s] mark files pending cleanup: %v", b.config.ID, cleanupErr)
}
params.Messages = b.prepareContextMessages(b.getOrCreateChatMemory(chatID))
} }
return "", fmt.Errorf("max self-heal retries (%d) exceeded: too many file_ids gone from anthropic", maxFileNotFoundRetries)
var fullText strings.Builder
var lastMsg anthropic.BetaMessage
fileRetries, pauseContinuations := 0, 0
for {
joined, msg, streamErr := b.streamMessages(ctx, params, onSegment)
if streamErr != nil {
var apiErr *anthropic.Error
if !errors.As(streamErr, &apiErr) || apiErr.StatusCode != http.StatusNotFound {
return "", fmt.Errorf("error creating Anthropic message: %w", streamErr)
}
missingFileID := extractMissingFileID(streamErr)
if missingFileID == "" {
return "", fmt.Errorf("%w: %s", ErrModelNotFound, b.config.Model)
}
fileRetries++
if fileRetries > maxFileNotFoundRetries {
return "", fmt.Errorf("max self-heal retries (%d) exceeded: too many file_ids gone from anthropic", maxFileNotFoundRetries)
}
ErrorLogger.Printf("[%s] self-heal: stripping dead file_id %s from chat %d (attempt %d/%d)",
b.config.ID, missingFileID, chatID, fileRetries, maxFileNotFoundRetries)
b.stripDeadFileIDFromMemory(chatID, missingFileID)
if _, cleanupErr := b.markFilesPendingCleanup(ctx, chatID, []string{missingFileID}); cleanupErr != nil {
ErrorLogger.Printf("[%s] mark files pending cleanup: %v", b.config.ID, cleanupErr)
}
params.Messages = b.prepareContextMessages(b.getOrCreateChatMemory(chatID))
continue
}
lastMsg = msg
if joined != "" {
if fullText.Len() > 0 {
fullText.WriteString("\n\n")
}
fullText.WriteString(joined)
}
if msg.StopReason == anthropic.BetaStopReasonPauseTurn {
pauseContinuations++
if pauseContinuations > maxPauseTurnContinuations {
ErrorLogger.Printf("[%s] pause_turn continuations exceeded (%d); returning partial answer",
b.config.ID, maxPauseTurnContinuations)
break
}
params.Messages = append(params.Messages, msg.ToParam())
continue
}
break
}
if fullText.Len() == 0 {
return "", emptyStreamError(string(lastMsg.StopReason),
lastMsg.Usage.OutputTokensDetails.ThinkingTokens, params.MaxTokens)
}
return fullText.String(), nil
}
func webSearchTools(cfg *WebSearchConfig) []anthropic.BetaToolUnionParam {
if cfg == nil {
return nil
}
search := &anthropic.BetaWebSearchTool20250305Param{
AllowedDomains: cfg.AllowedDomains,
BlockedDomains: cfg.BlockedDomains,
}
if cfg.MaxUses > 0 {
search.MaxUses = param.NewOpt(int64(cfg.MaxUses))
}
tools := []anthropic.BetaToolUnionParam{{OfWebSearchTool20250305: search}}
if cfg.Fetch {
fetch := &anthropic.BetaWebFetchTool20250910Param{
AllowedDomains: cfg.AllowedDomains,
BlockedDomains: cfg.BlockedDomains,
Citations: anthropic.BetaCitationsConfigParam{Enabled: param.NewOpt(true)},
}
if cfg.MaxUses > 0 {
fetch.MaxUses = param.NewOpt(int64(cfg.MaxUses))
}
if cfg.MaxContentTokens > 0 {
fetch.MaxContentTokens = param.NewOpt(int64(cfg.MaxContentTokens))
}
tools = append(tools, anthropic.BetaToolUnionParam{OfWebFetchTool20250910: fetch})
}
return tools
} }
func buildUserContext(username, firstName, lastName string, isPremium bool, languageCode string, messageTime int) string { func buildUserContext(username, firstName, lastName string, isPremium bool, languageCode string, messageTime int) string {
@@ -179,7 +249,7 @@ func thinkingParamFromConfig(mode, display string) (anthropic.BetaThinkingConfig
} }
} }
func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNewParams, onSegment func(string) error) (string, error) { func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNewParams, onSegment func(string) error) (string, anthropic.BetaMessage, error) {
stream := b.anthropicClient.Beta.Messages.NewStreaming(ctx, params) stream := b.anthropicClient.Beta.Messages.NewStreaming(ctx, params)
defer func() { defer func() {
if err := stream.Close(); err != nil { if err := stream.Close(); err != nil {
@@ -188,6 +258,7 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
}() }()
var ( var (
message anthropic.BetaMessage
allSegments []string allSegments []string
currentKind string currentKind string
currentText strings.Builder currentText strings.Builder
@@ -197,31 +268,24 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
currentTResultUseID, currentTResultServer string currentTResultUseID, currentTResultServer string
currentTResultIsError bool currentTResultIsError bool
currentTResultContent string currentTResultContent string
currentServerToolName, currentServerToolID string
currentServerResult string
mcpCalls = map[string]mcpCall{} mcpCalls = map[string]mcpCall{}
startInputTokens int64
finalUsage anthropic.BetaMessageDeltaUsage
stopReason string
) )
for stream.Next() { for stream.Next() {
e := stream.Current() e := stream.Current()
if accErr := message.Accumulate(e); accErr != nil {
ErrorLogger.Printf("[stream] accumulate failed: %v", accErr)
}
switch e.Type { switch e.Type {
case "message_start":
startInputTokens = e.AsMessageStart().Message.Usage.InputTokens
case "message_delta":
md := e.AsMessageDelta()
finalUsage = md.Usage
if md.Delta.StopReason != "" {
stopReason = string(md.Delta.StopReason)
}
case "content_block_start": case "content_block_start":
cbs := e.AsContentBlockStart() cbs := e.AsContentBlockStart()
currentKind = cbs.ContentBlock.Type currentKind = cbs.ContentBlock.Type
currentText.Reset() currentText.Reset()
currentThinking.Reset() currentThinking.Reset()
currentInputJSON.Reset() currentInputJSON.Reset()
currentServerResult = ""
switch currentKind { switch currentKind {
case "mcp_tool_use": case "mcp_tool_use":
currentTUseName = cbs.ContentBlock.Name currentTUseName = cbs.ContentBlock.Name
@@ -232,6 +296,11 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
currentTResultServer = cbs.ContentBlock.ServerName currentTResultServer = cbs.ContentBlock.ServerName
currentTResultIsError = cbs.ContentBlock.IsError currentTResultIsError = cbs.ContentBlock.IsError
currentTResultContent = cbs.ContentBlock.JSON.Content.Raw() currentTResultContent = cbs.ContentBlock.JSON.Content.Raw()
case "server_tool_use":
currentServerToolName = cbs.ContentBlock.Name
currentServerToolID = cbs.ContentBlock.ID
case "web_search_tool_result", "web_fetch_tool_result":
currentServerResult = cbs.ContentBlock.JSON.Content.Raw()
} }
case "content_block_delta": case "content_block_delta":
@@ -246,7 +315,7 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
currentThinking.WriteString(cbd.Delta.Thinking) currentThinking.WriteString(cbd.Delta.Thinking)
} }
case "input_json_delta": case "input_json_delta":
if currentKind == "mcp_tool_use" { if currentKind == "mcp_tool_use" || currentKind == "server_tool_use" {
currentInputJSON.WriteString(cbd.Delta.PartialJSON) currentInputJSON.WriteString(cbd.Delta.PartialJSON)
} }
} }
@@ -285,6 +354,15 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
"(total=%d): server=%q tool=%q input=%s tool_use_id=%q", "(total=%d): server=%q tool=%q input=%s tool_use_id=%q",
b.config.ID, total, call.server, call.name, call.input, currentTResultUseID) b.config.ID, total, call.server, call.name, call.input, currentTResultUseID)
} }
case "server_tool_use":
InfoLogger.Printf("[web] %s id=%q input=%s",
currentServerToolName, currentServerToolID, currentInputJSON.String())
case "web_search_tool_result", "web_fetch_tool_result":
preview := currentServerResult
if len(preview) > 500 {
preview = preview[:500] + "...(truncated)"
}
InfoLogger.Printf("[web] %s content=%s", currentKind, preview)
case "thinking", "redacted_thinking": case "thinking", "redacted_thinking":
if summary := strings.TrimSpace(currentThinking.String()); summary != "" { if summary := strings.TrimSpace(currentThinking.String()); summary != "" {
if len(summary) > 500 { if len(summary) > 500 {
@@ -296,7 +374,7 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
} }
default: default:
if currentKind != "" { if currentKind != "" {
InfoLogger.Printf("[mcp] block type=%q (unhandled)", currentKind) InfoLogger.Printf("[stream] block type=%q (unhandled)", currentKind)
} }
} }
currentKind = "" currentKind = ""
@@ -304,27 +382,21 @@ func (b *Bot) streamMessages(ctx context.Context, params anthropic.BetaMessageNe
} }
if err := stream.Err(); err != nil { if err := stream.Err(); err != nil {
return "", err return "", message, err
} }
if stopReason != "" || finalUsage.OutputTokens > 0 { stopReason := string(message.StopReason)
inputTokens := finalUsage.InputTokens if stopReason != "" || message.Usage.OutputTokens > 0 {
if inputTokens == 0 {
inputTokens = startInputTokens
}
InfoLogger.Printf("[usage] model=%s in=%d out=%d thinking=%d stop=%s", InfoLogger.Printf("[usage] model=%s in=%d out=%d thinking=%d stop=%s",
params.Model, inputTokens, finalUsage.OutputTokens, params.Model, message.Usage.InputTokens, message.Usage.OutputTokens,
finalUsage.OutputTokensDetails.ThinkingTokens, stopReason) message.Usage.OutputTokensDetails.ThinkingTokens, stopReason)
if stopReason == "max_tokens" { if message.StopReason == anthropic.BetaStopReasonMaxTokens {
ErrorLogger.Printf("[usage] response truncated at max_tokens=%d - raise max_tokens (thinking counts toward it)", ErrorLogger.Printf("[usage] response truncated at max_tokens=%d - raise max_tokens (thinking counts toward it)",
params.MaxTokens) params.MaxTokens)
} }
} }
if len(allSegments) == 0 { return strings.Join(allSegments, "\n\n"), message, nil
return "", emptyStreamError(stopReason, finalUsage.OutputTokensDetails.ThinkingTokens, params.MaxTokens)
}
return strings.Join(allSegments, "\n\n"), nil
} }
func emptyStreamError(stopReason string, thinkingTokens, maxTokens int64) error { func emptyStreamError(stopReason string, thinkingTokens, maxTokens int64) error {
+84
View File
@@ -127,6 +127,90 @@ func TestBackwardCompatibleParams(t *testing.T) {
} }
} }
func TestWebSearchTools(t *testing.T) {
t.Run("nil config yields no tools", func(t *testing.T) {
if tools := webSearchTools(nil); tools != nil {
t.Errorf("webSearchTools(nil) = %v, want nil", tools)
}
})
t.Run("search only when fetch off", func(t *testing.T) {
tools := webSearchTools(&WebSearchConfig{
AllowedDomains: []string{"example.com/hc"},
MaxUses: 3,
})
if len(tools) != 1 {
t.Fatalf("got %d tools, want 1 (search only)", len(tools))
}
if tools[0].OfWebSearchTool20250305 == nil {
t.Fatalf("tools[0] is not a web_search tool")
}
if tools[0].OfWebFetchTool20250910 != nil {
t.Error("web_fetch tool present but fetch is off")
}
})
t.Run("search + fetch with allowlist and citations", func(t *testing.T) {
tools := webSearchTools(&WebSearchConfig{
AllowedDomains: []string{"example.com/hc", "docs.example.com"},
MaxUses: 3,
Fetch: true,
MaxContentTokens: 50000,
})
if len(tools) != 2 {
t.Fatalf("got %d tools, want 2 (search + fetch)", len(tools))
}
search := tools[0].OfWebSearchTool20250305
if search == nil {
t.Fatalf("tools[0] is not a web_search tool")
}
if len(search.AllowedDomains) != 2 {
t.Errorf("search AllowedDomains = %v, want 2 entries", search.AllowedDomains)
}
if search.MaxUses.Value != 3 {
t.Errorf("search MaxUses = %d, want 3", search.MaxUses.Value)
}
fetch := tools[1].OfWebFetchTool20250910
if fetch == nil {
t.Fatalf("tools[1] is not a web_fetch tool")
}
if len(fetch.AllowedDomains) != 2 {
t.Errorf("fetch AllowedDomains = %v, want 2 entries", fetch.AllowedDomains)
}
if fetch.MaxContentTokens.Value != 50000 {
t.Errorf("fetch MaxContentTokens = %d, want 50000", fetch.MaxContentTokens.Value)
}
if fetch.Citations.Enabled.Value != true {
t.Error("fetch citations not enabled")
}
})
t.Run("wire shape carries allowed_domains", func(t *testing.T) {
tools := webSearchTools(&WebSearchConfig{
AllowedDomains: []string{"thatgamecompany.helpshift.com/hc"},
MaxUses: 2,
Fetch: true,
})
raw, err := json.Marshal(tools)
if err != nil {
t.Fatalf("marshal: %v", err)
}
body := string(raw)
for _, want := range []string{
"web_search_20250305",
"web_fetch_20250910",
"thatgamecompany.helpshift.com/hc",
"allowed_domains",
} {
if !strings.Contains(body, want) {
t.Errorf("wire body missing %q:\n%s", want, body)
}
}
})
}
func TestEmptyStreamError(t *testing.T) { func TestEmptyStreamError(t *testing.T) {
err := emptyStreamError("max_tokens", 3900, 4000) err := emptyStreamError("max_tokens", 3900, 4000)
for _, want := range []string{"output budget exhausted", "3900", "4000"} { for _, want := range []string{"output budget exhausted", "3900", "4000"} {
+26
View File
@@ -15,6 +15,14 @@ type MCPServer struct {
AllowedTools []string `json:"allowed_tools,omitempty"` AllowedTools []string `json:"allowed_tools,omitempty"`
} }
type WebSearchConfig struct {
AllowedDomains []string `json:"allowed_domains,omitempty"`
BlockedDomains []string `json:"blocked_domains,omitempty"`
MaxUses int `json:"max_uses,omitempty"`
Fetch bool `json:"fetch,omitempty"`
MaxContentTokens int `json:"max_content_tokens,omitempty"`
}
const ( const (
ThinkingModeAdaptive = "adaptive" ThinkingModeAdaptive = "adaptive"
ThinkingModeDisabled = "disabled" ThinkingModeDisabled = "disabled"
@@ -43,6 +51,7 @@ type BotConfig struct {
ElevenLabsModel string `json:"elevenlabs_model"` ElevenLabsModel string `json:"elevenlabs_model"`
DebugScreening bool `json:"debug_screening"` DebugScreening bool `json:"debug_screening"`
MCPServers []MCPServer `json:"mcp_servers,omitempty"` MCPServers []MCPServer `json:"mcp_servers,omitempty"`
WebSearch *WebSearchConfig `json:"web_search,omitempty"`
ConfigFilePath string `json:"-"` ConfigFilePath string `json:"-"`
} }
@@ -112,6 +121,11 @@ func loadAllConfigs(dir string) ([]BotConfig, error) {
config.ID, config.MaxTokens) config.ID, config.MaxTokens)
} }
if ws := config.WebSearch; ws != nil && len(ws.AllowedDomains) == 0 && len(ws.BlockedDomains) == 0 {
InfoLogger.Printf("[%s] web_search enabled with no allowed_domains/blocked_domains: the model may search the open web",
config.ID)
}
config.ConfigFilePath = validPath config.ConfigFilePath = validPath
configs = append(configs, config) configs = append(configs, config)
} }
@@ -168,6 +182,18 @@ func validateConfig(config *BotConfig, ids, tokens map[string]bool) error {
return fmt.Errorf("'max_tokens' must be greater than 0 when set") return fmt.Errorf("'max_tokens' must be greater than 0 when set")
} }
if ws := config.WebSearch; ws != nil {
if len(ws.AllowedDomains) > 0 && len(ws.BlockedDomains) > 0 {
return fmt.Errorf("'web_search' cannot set both allowed_domains and blocked_domains (the API rejects that)")
}
if ws.MaxUses < 0 {
return fmt.Errorf("'web_search.max_uses' must be greater than 0 when set")
}
if ws.MaxContentTokens < 0 {
return fmt.Errorf("'web_search.max_content_tokens' must be greater than 0 when set")
}
}
if config.MessagePerHour <= 0 { if config.MessagePerHour <= 0 {
return fmt.Errorf("'messages_per_hour' must be greater than 0") return fmt.Errorf("'messages_per_hour' must be greater than 0")
} }
+81
View File
@@ -875,3 +875,84 @@ func TestThinkingConfigLoad(t *testing.T) {
t.Errorf("MaxTokens = %d, want 4096", cfg.MaxTokens) t.Errorf("MaxTokens = %d, want 4096", cfg.MaxTokens)
} }
} }
func TestWebSearchConfig(t *testing.T) {
t.Run("absent leaves nil", func(t *testing.T) {
var cfg BotConfig
if err := json.Unmarshal([]byte(`{}`), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.WebSearch != nil {
t.Errorf("WebSearch = %+v, want nil when absent", cfg.WebSearch)
}
})
t.Run("loads allowlist", func(t *testing.T) {
jsonData := `{
"web_search": {
"allowed_domains": ["example.com/hc", "docs.example.com"],
"max_uses": 3,
"fetch": true,
"max_content_tokens": 50000
}
}`
var cfg BotConfig
if err := json.Unmarshal([]byte(jsonData), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.WebSearch == nil {
t.Fatalf("WebSearch = nil, want populated")
}
if len(cfg.WebSearch.AllowedDomains) != 2 {
t.Errorf("AllowedDomains = %v, want 2 entries", cfg.WebSearch.AllowedDomains)
}
if cfg.WebSearch.MaxUses != 3 {
t.Errorf("MaxUses = %d, want 3", cfg.WebSearch.MaxUses)
}
if !cfg.WebSearch.Fetch {
t.Error("Fetch = false, want true")
}
if cfg.WebSearch.MaxContentTokens != 50000 {
t.Errorf("MaxContentTokens = %d, want 50000", cfg.WebSearch.MaxContentTokens)
}
})
cases := []struct {
name string
ws *WebSearchConfig
wantErr string
}{
{"nil ok", nil, ""},
{"allowlist ok", &WebSearchConfig{AllowedDomains: []string{"example.com"}, MaxUses: 3}, ""},
{"blocklist ok", &WebSearchConfig{BlockedDomains: []string{"evil.com"}}, ""},
{"empty ok", &WebSearchConfig{}, ""},
{"both allow and block rejected",
&WebSearchConfig{AllowedDomains: []string{"a.com"}, BlockedDomains: []string{"b.com"}},
"cannot set both allowed_domains and blocked_domains"},
{"negative max_uses rejected",
&WebSearchConfig{AllowedDomains: []string{"a.com"}, MaxUses: -1},
"'web_search.max_uses' must be greater than 0"},
{"negative max_content_tokens rejected",
&WebSearchConfig{AllowedDomains: []string{"a.com"}, MaxContentTokens: -1},
"'web_search.max_content_tokens' must be greater than 0"},
}
for i, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cfg := thinkingTestConfig(fmt.Sprintf("bot-web-%d", i))
cfg.WebSearch = tc.ws
err := validateConfig(&cfg, map[string]bool{}, map[string]bool{})
if tc.wantErr == "" {
if err != nil {
t.Fatalf("validateConfig(web_search=%+v) = %v, want nil", tc.ws, err)
}
return
}
if err == nil {
t.Fatalf("validateConfig(web_search=%+v) = nil, want error containing %q", tc.ws, tc.wantErr)
}
if !contains(err.Error(), tc.wantErr) {
t.Errorf("error %q does not contain %q", err.Error(), tc.wantErr)
}
})
}
}
Binary file not shown.