Skip to content

feat: add Tavily as optional search backend alongside Bing scrape#67

Open
manisrinivasan2k1 wants to merge 1 commit into
QuintinShaw:mainfrom
Tavily-FDE:feat/tavily-migration/bing-scrape-web-search
Open

feat: add Tavily as optional search backend alongside Bing scrape#67
manisrinivasan2k1 wants to merge 1 commit into
QuintinShaw:mainfrom
Tavily-FDE:feat/tavily-migration/bing-scrape-web-search

Conversation

@manisrinivasan2k1

Copy link
Copy Markdown

Summary

Adds Tavily as a configurable, higher-quality search backend for the web_search tool. When TAVILY_API_KEY is present in the environment, searches are routed through the Tavily API; otherwise the existing Bing HTML scrape continues to work as before (zero-config fallback).

What changed

  • src/web-tools.ts: Added a tavilySearch() helper using dynamic import("@tavily/core"). Modified createWebSearchTool() to check process.env.TAVILY_API_KEY at call time and branch accordingly. Added a new createTavilySearchTool() factory for callers who want Tavily exclusively.
  • src/index.ts: Exported the new createTavilySearchTool factory.
  • package.json: Added @tavily/core as an optional dependency.

Files changed

  • src/web-tools.ts
  • src/index.ts
  • package.json

Dependency changes

  • Added @tavily/core (^0.6.0) as an optional dependency

Environment variable changes

  • TAVILY_API_KEY (optional) — when set, activates Tavily backend; when absent, Bing scrape is used as before

Notes for reviewers

  • The dynamic import() ensures @tavily/core is only loaded when actually needed, so environments without it installed won't break.
  • The tool interface ({url, title} result shape) is unchanged — downstream consumers like deep-research require no modifications.
  • All existing tests remain unaffected since the Bing scrape path is preserved as the default.

Automated Review

  • Passed after 1 attempt(s)
  • Final review: The migration is well-implemented and correct. It adds Tavily as an optional, additive search backend: createWebSearchTool() gains a runtime branch that uses Tavily when TAVILY_API_KEY is set (falling back to Bing scrape otherwise), and a new createTavilySearchTool() factory is added for callers who want Tavily exclusively. The optional dependency pattern (dynamic await import("@tavily/core") + optionalDependencies in package.json) is the right approach. All three reported files are modified correctly, and existing behavior is fully preserved when the env var is absent. Minor issues only: no guard or clear error when createTavilySearchTool is called without the API key set, and the file-level JSDoc comment was not updated to mention Tavily.

@QuintinShaw QuintinShaw left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @manisrinivasan2k1 — nice idea, and the zero-config Bing fallback plus the secrets handling (key only ever sent as a Bearer header, never logged or in a URL) are both done right. A couple of things before merge:

Blocking:

  1. npm test currently fails on biome (src/web-tools.ts): a noNonNullAssertion error + a format diff, so CI won't go green. The ! is actually unnecessary — @tavily/core's tavily() already falls back to process.env.TAVILY_API_KEY when you pass no apiKey, so plain tavily() drops the !; then npx biome check --write src/web-tools.ts for the format.
  2. It's added via optionalDependencies, but that doesn't make it opt-in — npm still installs it for everyone (I measured +27 packages / +25M in node_modules). Since we ship as a library, every consumer pays that even with no TAVILY_API_KEY. Could you drop the @tavily/core entry from package.json entirely and rely purely on the dynamic import("@tavily/core"), wrapped so a missing package gives a clear "run npm install @tavily/core" message? That's the right shape for a truly optional backend.

Non-blocking, would be great:

  • Tests for the TAVILY_API_KEY branch (success + error paths) — no coverage of the new logic yet.
  • On a Tavily error (bad key, rate limit) it hard-fails rather than falling back to Bing — either's fine, but worth a deliberate choice + a note.
  • The Tavily call inherits the SDK's 60s timeout while Bing uses 15s — worth passing { timeout: 15 }.
  • createTavilySearchTool is exported but never wired in — drop it or hook it up.

Happy to look again as soon as CI's green. Thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants