---
name: web-scraping-patterns
slug: web-scraping-patterns
score: 49
rubric: 1.0
provenance: community
publisher: scraping-skills
license: MIT
capability: read-only
canonical: https://top-agent-skills.com/skill/web-scraping-patterns
---

# web-scraping-patterns

Ethical scraping at scale. robots.txt respect, rate-limiting, structured extraction.

## Install

### Claude Code

```bash
npx skills add scraping-skills/patterns
```

## Best for

Research workflows where you need to extract structured data without breaking sites.


## About

Ethical web scraping: robots.txt respect, rate-limiting, retry-with-backoff, structured extraction with schema.

## Compatibility

claude-code, cursor, codex

## Metadata

- Categories: browser-automation, data-analytics
- Tags: Scraping, Ethics, Schema
- Setup time: 2 min
- Complexity: Low
- Last update: 2026-04-19
- Verified: 2026-04-30



---

_Indexed by Top Agent Skills. Score breakdown: https://top-agent-skills.com/about/methodology_