Wednesday, May 12, 2004

RSS and scaling

A Wired article expresses concern that RSS polling will crush sites from heavily traffic, but scaling RSS is really pretty straightforward using caching. Web-based RSS readers like My Yahoo!'s RSS Beta cache RSS feeds to share among several readers, reducing load on the server. Because immediate updates aren't important when delivering RSS content, it's also straightforward to set up intermediatary caching servers, just like many do for static web content.

Unfortunately, personalized RSS feeds can't easily be cached and shared. Each RSS feed contains different content; there's effectively millions of different RSS feeds. This is a more serious scaling challenge. Findory News has heavily optimized its personalized RSS feeds because of this issue.

No comments: