rudyharrelson

joined 2 years ago
[–] rudyharrelson@lemmy.radio 2 points 4 hours ago (1 children)

The "scraping" part becomes unethical when the scraping is so aggressive that it takes down the website (or severely impacts its ability to serve actual clients).

Archive.org scrapes the web all the time, but it doesn't do it so aggressively that it becomes an issue for the websites they're scraping. The same cannot be said for AI scrapers.

[–] rudyharrelson@lemmy.radio 1 points 4 hours ago

The scraping wouldn’t be a problem if Reddit simply provided an RSS feed or other data-efficient API

Reddit does provide RSS feeds, e.g.: https://www.reddit.com/r/SonicTheHedgehog/.rss

Frankly, I'm surprised they still offer RSS feeds. They've been slowly but surely killing off all ways of accessing their content for years. One day they'll disable them, but for now they still work.