Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

That's the site owner demanding payment via cloudflare, not cloudflare unilaterally deciding to charge money (as far as I can tell at least).

https://blog.cloudflare.com/introducing-pay-per-crawl/

Looks like The Verge either set up an excessively tight pay-per-crawl policy or doesn't want IA scraping their stuff.





Cloudflare enabled blocking by default. People were on X complaining about it.

Blocking access to everyone or to scraping crawlers?

They have detailed stats about the behavior of all visitors, including how bot-like they are and how likely they are to scrape your (their users’) content.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: