Friday, 19 May 2006
Discovering Feed Agents 
« Web 2.0 Traffic Breakdown: Machines vs. Humans | Main | Announcement: Simpy Firefox Extension »
If you read my previous post about Men vs. Machines, and are interested in that kind of stuff, try this with your web server logs:
grep 'FOO' /log/apache/access_log | cut -d' ' -f12 | sort | less
Of course, "FOO" is just a placeholder. Use the portion of the URL that marks your feed (e.g. on Simpy it would be "/rss", because all feed URLs start with "/rss").
The above will give you a full list of feed agents. If you want to see a list of unique agent names, use this:
grep 'FOO' /log/apache/access_log | cut -d' ' -f12 | sort | uniq | less
What is this list good for? It's good for figuring out feed agent names, so you can redirect their request to different log file, for instance. It's also good for discovering new feed-eating services and software out there. Simpy currently distinguishes 30+ different feed readers.
Posted by at 12:37 PM in Tips & Tricks
