Policy

Yosoi policy is the control plane for how a scrape, search, or crawl runs. It covers model selection, fetch behavior, web-search defaults, discovery, output, downloads, crawl limits, telemetry, and trust gates.

A policy is a frozen Pydantic object in Python and a JSON/YAML document on disk. The same shape works in both places.

atom_reads: false
trust_tier: strict
scrape:
  fetcher_type: auto
search:
  backend: google,bing,brave
  region: us-en
  safesearch: "moderate"
  max_results: 10
output:
  formats: [json]
  flat_files: false
page:
  timeout_seconds: 30

Why policy exists

Yosoi has many knobs, but they are not all contract fields. A contract defines what data means. Policy defines how Yosoi is allowed to get it.

Use policy for:

• model and provider settings
• cache and trust gates
• fetcher and page acquisition behavior
• web-search provider and regional defaults
• crawl limits and safety controls
• output shape and debug artifacts
• download permissions
• observability settings

Policy surfaces

Surface	Use it for
Python ys.Policy(...)	Library code, tests, per-call overrides.
Policy.from_env()	Legacy env compatibility and secrets.
JSON/YAML policy files	Global/project defaults and repeatable CLI runs.
CLI flags	Narrow call-site overrides.

Main sections

Section	Purpose
model	Provider, model name, temperature, token cap, credential reference.
scrape	Replay/discovery behavior for one scrape run.
search	Web-search provider, backend, region, safesearch, page, and result limit.
page	Generic page acquisition settings shared by scrape and crawl.
crawl	Crawl budget, scheduler, safety, escalation, and targets.
output	Terminal/file output, debug HTML, logs, flat files.
download	Default-deny file download side effects.
discovery	Selector discovery backend and concurrency.
telemetry	Langfuse references and host.
fingerprint	Off-path fingerprint signal lane.

Quick commands

uvx yosoi policy init --local
uvx yosoi policy validate .yosoi/policy.yaml
uvx yosoi policy inspect .yosoi/policy.yaml --format yaml
uvx yosoi policy effective --format yaml
uvx yosoi scrape https://example.com --policy .yosoi/policy/20-page.yaml
uvx yosoi search "cascading labs yosoi" --policy .yosoi/policy/search.yaml

Start with Policy Files and Layering & Precedence, then use the section reference pages for individual options.