unclecode/crawl4ai
A Python crawler built on Playwright. It opens pages, including JavaScript-heavy ones, and returns clean Markdown ready for an LLM. It can crawl deep with BFS or DFS, resume after a crash, and extract fields with a model.
Idea adaptedKey files read, not run
I learnt this repo's idea and adapted it into my own workspace setup.
How we know
We read the README and key files, not every source file, and did not run it, so the most we can say is Idea adapted or Risky.
Good for
Anyone feeding many web pages into RAG or an LLM who needs clean Markdown.
Not for
Teams who want a hosted crawling service and do not want to run their own browser.
Repo
- Stars
- 82,084
- License
- Apache-2.0Use it, change it, sell it, closed-source included. Keep the author's copyright notice.
- Maturity
- active
- Last push
- 2026-09-09
- Last checked
- 2026-11-14