Update: I took the time to look more closely and it looks like Netvibes is using client side Javascript to fetch feeds. I guess that’s no different than if the user had a dedicated news client, but it sure would be nice if (a) Netvibes cached my feed locally and had all its users pull it from the cache, or (b) set the Referrer header to “-”.
I took a look at my Web server stats for the first time in a while and saw this.
Now, I know that Netvibes is getting some traction out there, but they can’t be this popular. Heck, I’m not this popular. Looking at the logs directly, it seems that Netvibes is setting the Referrer header when they’re just getting my feed. Not only that, the User-Agent varies from request to request (various versions of Firefox on Windows, OS X, and Ubuntu). But wait there’s more, Netvibes also does a normal feed fetch (empty Referrer, User-Agent set to “Netvibes”) every 45 minutes or so.
So here’s my theory. From the looks of it I have 21 subscribers via Netvibes. I suspect that Netvibes is not only fetching my feed as you would expect it to, but it is also fetching it 21 additional times every X amount of time. And for these fetches it’s setting the Referrer to netvibes.com and using the actual browser and OS combination of the subscriber in the UA header. At first I thought that Netvibes might be serving up some local JavaScript to do the fetch, but the IP addresses aren’t random enough. Anyone have any other thoughts on this?
I included a few extra lines in the stats picture so you can see that last entry. Apparently some robot that’s trying to crack phpBB (which I don’t use) sets its referrer field to “forum.{your_domain}” when it connects to your site. That’s annoying.
Post a Comment