Benjamin Golub’s blog

Compare the current top story at RSSmeme and ReadBurner: J-Walking with Reader.  At the moment RSSmeme has 20 shares while ReadBurner has only 12.  I can’t say how fast ReadBurner is but I can go through 50 feeds per minute with my current script.  The feed with the oldest update time as of writing this post at 6:48pm is 5:46pm; so it currently takes me an hour to go through all 600 feeds.

When I first launched RSSmeme I went twice as fast; parsing 100 feeds every 5 minutes; but now I’ve toned it down to 50 feeds every 5 minutes to keep the load down on the server.  So if it takes me 1 minute to parse 50 feeds and I only parse every 5 minutes then that means 20% of the time my server is grabbing new data; I think that’s a happy medium.

Pages (without GET parameters like the search or pagination) are cached for 5 minutes.  If you want fresher data I could decrease the cache and parse more feeds at a time; just let me know!