Arc Forumnew | comments | leaders | submitlogin
2 points by shader 1628 days ago | link | parent

Ah. About the same as me then.

Maybe we should make an auto scraper/notifier? Though iirc, the arc server doesn't really appreciate frequent queries. On closer look, it should be fine; the default throttle window should be 10s. It could query once a minute, and update an rss feed for arc forum posts and comments. Or provide webhook + email support.

1 point by akkartik 1628 days ago | link

I actually built one back in the day: Trouble was, it kept getting banned/blocked even when I scaled back to crawl HN every five minutes. Eventually I was crawling so infrequently that I was missing stories that dropped off /newest in the meantime, so I stopped crawling.

I actually had an arc version for a brief period[1], but it didn't seem like there was as much need for it here. But perhaps the email notifications would be useful. I'll see if I can bring it back.

[1] I was certain I'd showed it here, but no I only showed it over email to a couple of folks. I really should bring it back just so y'all can look at it. Or at least locate the (rails) sources.