Hacker News new | past | comments | ask | show | jobs | submit login

A couple months ago, reddit started refusing requests from my web scraper. Figured out they started checking the user agent and refusing connections that didn't look like they came from a user's browser. Unless I missed an announcement somewhere, it doesn't seem like they're overly friendly about allowing web scrapers.



They just don't want to be abused.

"We're happy to have API clients, crawlers, scrapers, and Greasemonkey scripts, but they have to obey some rules:"

https://github.com/reddit/reddit/wiki/API


They are fine with it as long as you abide to their terms, they have a subreddit dedicated to reddit development and the reddit api which has discussion of scraping: http://www.reddit.com/r/redditdev




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: