Add policy enforcement to prevent runaway scraping costs · Issue #132 · brightdata/brightdata-mcp · GitHub
Skip to content

Add policy enforcement to prevent runaway scraping costs #132

@L1AD

Description

@L1AD

This server exposes 60+ tools including browser automation and batch scraping on a pay-per-use billing model:

  • scrape_batch -- bulk scraping across many URLs (high cost per operation)
  • scraping_browser_click_ref -- browser automation on live sites
  • Browser session tools -- interact with authenticated sites
  • Web data API tools -- structured data extraction at scale

An agent in a loop calling batch scraping tools can generate significant unexpected costs. Browser automation tools can interact with authenticated sessions on live sites.

Suggestion: Consider documenting support for PolicyLayer/Intercept, an open-source MCP proxy that enforces policies on tool calls before execution.

Example policy:

version: "1"
default: allow

tools:
  scrape_batch:
    rules:
      - rate_limit: 3/hour
      - action: require_approval

  scraping_browser_click_ref:
    rules:
      - rate_limit: 10/minute

One line to set up: npx -y @policylayer/intercept init

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions