Activity

1. thesujai

   added the

   Feature

   issue type on Sep 23, 2026
2. thesujai

   added

   help wantedMaintainers will accept community code contributions for this issueMaintainers will accept community code contributions for this issue

   on Sep 23, 2026
3. thesujai commented on Sep 23, 2026

   

   thesujai

   on Sep 23, 2026

   ContributorAuthor

   More actions

   For the folks picking this up let us know the implementation plan before we can assign you and you can begin the work
4. Aman-goel-04 commented on Sep 23, 2026

   

   Aman-goel-04

   on Sep 23, 2026

   Contributor

   More actions

   Would like to pick this up! Letting you know about the implementation plan in a bit.
5. Aman-goel-04 commented on Sep 23, 2026

   

   Aman-goel-04

   on Sep 23, 2026

   Contributor

   More actions

   Here's the implementation plan, following the existing Parallel provider as the template (packages/trueforge-core/src/core/web-search/ParallelWebSearchProvider.ts, schemas/webSearchProvider.ts, catalog/web-search-catalog.yaml, websearch/providers.ts):

   1. Add WebSearchProviders.Exa and an ExaWebSearchProvider implements IWebSearchProvider in trueforge-core, calling Exa's search API with an injected ssrfFetch (matching Parallel's SSRF guard usage), mapped into the existing normalized WebSearchHits/WebFetchPages shapes.
   2. Extend WebSearchProviderManifestSchema's discriminated union with an exa variant (type: 'exa' plus auth.api_key), using the same redact-on-read and rotate-or-keep-on-PUT semantics as Parallel. No changes to the singleton settings model.
   3. Add exa to web-search-catalog.yaml so it's discoverable via the catalog endpoint.
   4. Wire resolveWebSearchProvider() to construct ExaWebSearchProvider when manifest.type === 'exa', so it's picked up in agent turns and gates capabilities the same way Parallel does today.
   5. Add provider-level tests (search/fetch mapping, missing/invalid-key failure) and route-level tests (PUT/GET redaction and rotation, catalog listing) mirroring the Parallel test coverage.
   6. Regenerate the OpenAPI-derived SDK types affected by the manifest union change.

   One open question I'll resolve during implementation: Exa's content-extraction endpoint doesn't map 1:1 to Parallel's extract API, so I'll either adapt it to the fetch() contract or return a clear per-URL "unsupported" error if it doesn't fit, following whatever's closest to existing provider-parity expectations. Not touching the TrueFoundry-managed env-var synthesis path (TrueFoundryWebSearchProviderStore) since that's enterprise-only and outside this issue's stated scope.

   Happy to adjust based on feedback.
6. Aman-goel-04 commented on Sep 23, 2026

   

   Aman-goel-04

   on Sep 23, 2026

   Contributor

   More actions

   @thesujai, Let me know if I can start working on it!
7. xinpingzh commented on Sep 23, 2026

   

   xinpingzh

   on Sep 23, 2026

   Contributor

   More actions

   Hi @thesujai, I'd like to work on this issue. Is it still available, and would you approve me taking it?

   I've reviewed the current Parallel integration and propose to:

   • add Exa to the catalog and settings manifest;
   • resolve the configured provider at runtime;
   • implement Exa search and, if expected by the current interface, Exa-backed web_fetch;
   • preserve secret redaction/rotation and require a fresh key when switching provider types;
   • add focused provider, settings, resolver, and catalog tests, with no UI changes.

   Before implementation, could you confirm three scope details?

   1. Should this issue include web_fetch through Exa Contents? IWebSearchProvider currently requires both search() and fetch(), and the capability exposes both tools.
   2. For multiple search_queries, is one Exa request per query followed by URL deduplication and a global result cap acceptable? Exa's additionalQueries is limited to deep-search variants.
   3. Should invalid credentials fail clearly on first use, or should settings PUT validate them remotely?

   I'll wait for approval and scope confirmation before starting implementation.
8. Aman-goel-04 commented on Sep 25, 2026

   

   Aman-goel-04

   on Sep 25, 2026

   Last edited by Aman-goel-04

   Contributor

   More actions

   @thesujai, Implementation is ready locally and tested. I'll open the PR if you're okay with the following implementation details:

   What's implemented

   • ExaWebSearchProvider calls Exa /search and /contents through ssrfFetch, with no new dependency.
   • exa is added to the manifest union and to web-search-catalog.yaml, and resolveWebSearchProvider() is wired for it.
   • A changeset is included. I did not touch the generated SDK, since CI regenerates it.
   • Bug fix beyond the original plan: a PUT that switched provider type with a redacted key reused the previous provider's stored key. It now returns 400 and requires a fresh key.

   Scope decisions

   1. fetch(): implemented via Exa /contents, because IWebSearchProvider requires both methods. Per-URL failures come from statuses, and successful pages are kept.
   2. Multiple search_queries: one /search request per query, results interleaved, duplicate URLs dropped, and a total cap of 5 hits with 500-character snippets.
   3. Invalid credentials: they fail clearly on first use (401 and 402 have dedicated messages). PUT does not validate remotely.

   Testing

   • Unit tests cover the provider, the settings API (upsert, redaction, rotation, type switch), the resolver, and the catalog.
   • Typecheck, lint, format, and the full core and server suites pass.
   • Live run against Exa with a real key: search, contents, a bad URL, and the invalid-key error all behaved as expected.
   • Not covered: an end-to-end agent turn using Exa.

   Notes

   • trueforge-ui's catalog adapter still only handles parallel. I left it alone as UI work is out of scope.
9. heerambavi1998

   assigned

   Aman-goel-04

   on Sep 26, 2026
10. heerambavi1998 commented on Sep 26, 2026

    

    heerambavi1998

    on Sep 26, 2026

    Last edited by heerambavi1998

    Contributor

    More actions

    Hi @Aman-goel-04, I have assigned the issue to you. Could you check the current web search prompt -
    Operators such as site:domain, filetype:pdf, intitle:word, -term, and "exact phrase" may work. And add details in the implementation, on which of the following are supported and hence tested for Exa?
    Also, for the contract differences, please note the exact differences, and the finalised approach before raising the PR. Thanks!
11. Aman-goel-04 commented on Sep 26, 2026

    

    Aman-goel-04

    on Sep 26, 2026

    Contributor

    More actions

    @heerambavi1998, Thanks for assigning it to me. I checked the search prompt operators against Exa and pinned down the contract differences.

    Operator support (tested live against Exa /search)

    Constraint-satisfying hits out of 10 to 30 per operator. "Operator" means the operator written in the query text.

    Operator	Without operator	With operator	Result
    site:domain	2/10	10/10	Supported
    -site:domain	6 hits from the excluded domain	0	Supported
    -term	2/10	10/10	Supported
    "exact phrase"	23/30	30/30	Supported
    filetype:pdf	13/30	16/30	Weak, not reliable
    intitle:word	19/30	18/30	Not supported

    Exa's docs describe no operator syntax, but it honors most of these from the query text. I also tried the native parameters. includeDomains and excludeDomains match the query-text results, while excludeText and includeText are deprecated and approximate (excludeText matched 7/10 versus 10/10 for -term in the query).

    Finalised approach

    • Queries are sent to Exa verbatim. The provider does not parse operators into API parameters, since the query text performs as well as or better than the native parameters.
    • The shared tool prompt is unchanged. It already says operators "may work when the backend supports them".
    • The supported and unsupported operators are documented on ExaWebSearchProvider, and a unit test asserts that operator syntax reaches Exa unmodified.

    Contract differences vs Parallel

    • Objective: Parallel has an objective field. Exa has none, so it is passed as the highlights query. This focuses snippets but does not affect ranking.
    • Multiple queries: Parallel accepts them in one call. Exa needs one request per query (up to 5), merged with interleaving, URL dedupe, and a 5-hit cap.
    • Snippets: Exa highlights, capped at 500 characters each to match Parallel.
    • published_at: often null on Exa, since publishedDate is missing for many pages.
    • fetch(): Exa /contents returns full text and ignores objective. Per-URL errors come from statuses[].error (tag plus HTTP code) instead of Parallel's errors[].
    • Auth: x-api-key header over REST, no SDK, still routed through ssrfFetch.
    • Credential failures: raised on first use with a clear message (401 and 402 have their own messages). PUT does not validate the key remotely.

    Happy to adjust anything before I raise the PR.
12. thesujai commented on Sep 28, 2026

    

    thesujai

    on Sep 28, 2026

    ContributorAuthor

    More actions

    @zhangxinping666 since this is already being worked on, you can look at #853 if you are interested. It is similar
13. thesujai commented on Sep 28, 2026

    

    thesujai

    on Sep 28, 2026

    ContributorAuthor

    More actions

    @Aman-goel-04 Please open a PR
14. Aman-goel-04 commented on Sep 28, 2026

    

    Aman-goel-04

    on Sep 28, 2026

    Contributor

    More actions

    @thesujai, done! Please review it when you get the time.
15. thesujai commented on Sep 29, 2026

    

    thesujai

    on Sep 29, 2026

    ContributorAuthor

    More actions

    We will review it next week
16. Aman-goel-04 commented on Oct 6, 2026

    

    Aman-goel-04

    on Oct 6, 2026

    Contributor

    More actions

    @thesujai @heerambavi1998, Could you please review it now?
17. Aman-goel-04 commented on Oct 21, 2026

    

    Aman-goel-04

    on Oct 21, 2026

    Contributor

    More actions

    @thesujai @heerambavi1998 hey?