A simple trap for web crawlers
-
Updated
Aug 2, 2023 - Rust
A simple trap for web crawlers
A containerized AI Scraping Defense Stack using Nginx+Lua for edge filtering and Python backend services (FastAPI/Flask) to detect, tarpit, and block malicious bots. Features Redis blocklisting, heuristic/ML analysis, PostgreSQL tarpit data, and alerting. Deployable via Docker/Kubernetes.
An IIS-native implementation of the AI Scraping Defense Stack, utilizing ASP.NET Core middleware and Python backend services (FastAPI/Flask) to detect, tarpit, and block malicious bots and AI scrapers in a Windows environment. Features include Redis-based blocklisting, heuristic analysis, ML model integration, and configurable alerting.
JavaScript Email Protection (JEP) is a simple and lightweight way to keep your email safe from spam bots. Instead of leaving your email exposed in plain HTML (where bots can easily grab it), this script dynamically inserts it using JavaScript—making it way harder for scrapers to detect.
Digital media protection and web scraping mitigation for creators at Patreon (or similar services)
Add a description, image, and links to the anti-scraping topic page so that developers can more easily learn about it.
To associate your repository with the anti-scraping topic, visit your repo's landing page and select "manage topics."