Comments on: Googling for graphs http://bit-player.org/2007/googling-for-graphs An amateur's outlook on computation and mathematics. Tue, 07 Feb 2012 08:14:54 +0000 http://wordpress.org/?v=2.6.3 By: New Year’s To-Do List « Learning Computation http://bit-player.org/2007/googling-for-graphs#comment-2836 New Year’s To-Do List « Learning Computation Fri, 30 Apr 2010 23:47:34 +0000 http://bit-player.org/?p=125#comment-2836 [...] every month or so, so that you can shame me into staying on target. (Hat tip to Brian Hayes at bit-player for [...] [...] every month or so, so that you can shame me into staying on target. (Hat tip to Brian Hayes at bit-player for [...]

]]> By: mersin web tasarım hosting http://bit-player.org/2007/googling-for-graphs#comment-1562 mersin web tasarım hosting Wed, 19 Dec 2007 00:47:35 +0000 http://bit-player.org/?p=125#comment-1562 thanks thanks

]]> By: brian http://bit-player.org/2007/googling-for-graphs#comment-1560 brian Fri, 14 Dec 2007 04:26:54 +0000 http://bit-player.org/?p=125#comment-1560 Yes, if you load the page 50,000 times, I get away clean but Google puts you on the no-fly list. Yes, if you load the page 50,000 times, I get away clean but Google puts you on the no-fly list.

]]> By: Kurt http://bit-player.org/2007/googling-for-graphs#comment-1559 Kurt Fri, 14 Dec 2007 02:31:25 +0000 http://bit-player.org/?p=125#comment-1559 Well, this raises more questions for me. The IP number from which the chart request comes from belongs to the person browsing, not the host site. Of course, the web browser is supposed to pass along the referring URL along with other info about the requester, so Google can still keep track of which host sites are generating a lot of requests. And I suppose that any site which generates 50,000 hits per day can afford to generate their own graphs in-house. Well, this raises more questions for me. The IP number from which the chart request comes from belongs to the person browsing, not the host site. Of course, the web browser is supposed to pass along the referring URL along with other info about the requester, so Google can still keep track of which host sites are generating a lot of requests. And I suppose that any site which generates 50,000 hits per day can afford to generate their own graphs in-house.

]]> By: brian http://bit-player.org/2007/googling-for-graphs#comment-1558 brian Thu, 13 Dec 2007 15:02:50 +0000 http://bit-player.org/?p=125#comment-1558 Yes, they could surely cache the most popular graphs. The question I'm curious about is whether it's worthwhile to do so. Obviously there's a storage cost for keeping things, but more important there's a computational cost. You've got to examine every URL that comes in to see if it matches one of the saved graphs. The routine might go like this: 1. Receive a URL. 2. Apply a hash function. 3. Look up the hash in a table.       3a. If you've never seen this hash before, set a counter to 1.       3b. If you've seen the hash before, increment its counter. 4. Compare the count with the popularity threshold:       (count < threshold) --> generate a fresh graph       (count = threshold) --> generate a fresh graph and cache it       (count > threshold) --> retrieve the cached copy (I'm ignoring the possibility of hash collisions, which could make matters worse.) Note that the overhead of hashing and table lookup and counting applies to <em>every</em> URL received, not just the popular ones. Is it worth the bother? I don't know. Perhaps we could find out by experiment: Try requesting the same graph repeatedly, and see if at some point the response time drops significantly. But note the following "usage policy": <blockquote>Use of the Google Chart API is subject to a query limit of 50,000 queries per user per day. If you go over this 24-hour limit, the Chart API may stop working for you temporarily. If you continue to exceed this limit, your access to the Chart API may be blocked.</blockquote> This suggests that they are at least bothering to keep track of the IP number that requests come from. And of course many of us suspect that Google knows all and keeps <em>everything</em>. Yes, they could surely cache the most popular graphs. The question I’m curious about is whether it’s worthwhile to do so. Obviously there’s a storage cost for keeping things, but more important there’s a computational cost. You’ve got to examine every URL that comes in to see if it matches one of the saved graphs. The routine might go like this:

1. Receive a URL.
2. Apply a hash function.
3. Look up the hash in a table.
      3a. If you’ve never seen this hash before, set a counter to 1.
      3b. If you’ve seen the hash before, increment its counter.
4. Compare the count with the popularity threshold:
      (count < threshold) –> generate a fresh graph
      (count = threshold) –> generate a fresh graph and cache it
      (count > threshold) –> retrieve the cached copy

(I’m ignoring the possibility of hash collisions, which could make matters worse.)

Note that the overhead of hashing and table lookup and counting applies to every URL received, not just the popular ones. Is it worth the bother? I don’t know. Perhaps we could find out by experiment: Try requesting the same graph repeatedly, and see if at some point the response time drops significantly. But note the following “usage policy”:

Use of the Google Chart API is subject to a query limit of 50,000 queries per user per day. If you go over this 24-hour limit, the Chart API may stop working for you temporarily. If you continue to exceed this limit, your access to the Chart API may be blocked.

This suggests that they are at least bothering to keep track of the IP number that requests come from. And of course many of us suspect that Google knows all and keeps everything.

]]> By: Kurt http://bit-player.org/2007/googling-for-graphs#comment-1557 Kurt Thu, 13 Dec 2007 01:26:14 +0000 http://bit-player.org/?p=125#comment-1557 <blockquote>If a thousand people read the page, the graph will be recreated a thousand times.</blockquote>I'm willing to bet that Google has some clever caching strategies so that high-volume web pages get their graphs stored so they don't have to be regenerated for each hit.

If a thousand people read the page, the graph will be recreated a thousand times.

I’m willing to bet that Google has some clever caching strategies so that high-volume web pages get their graphs stored so they don’t have to be regenerated for each hit.

]]>