eafbc1ac52
The crawler subdirectory was the only active project. Moving it to the repo root simplifies paths and removes the unnecessary nesting. The vqa/ and immoweb/ directories were legacy/unused and have been removed. Updated .drone.yml, .gitignore, .claude/ docs, and skills to reflect the new flat structure.
4 lines
59 B
Text
4 lines
59 B
Text
# Block ALL bots from ALL pages
|
|
User-agent: *
|
|
Disallow: /
|
|
|