Cockroach Crawler

About

Enable AI agents to access web content without providing them open network permissions. Cockroach Crawler is an open-source Node.js framework designed for static and dynamically rendered crawling. It features adaptive traversal, screenshot capture, PDF generation, structured data extraction, and reads from public GitHub and YouTube sources. The toolkit supports MCP, Docker, and Cloudflare Workers. It delivers results in Markdown and JSON formats alongside verifiable evidence records, all governed by clear origin attribution, usage budgets, and runtime controls.