Comments on: Smart Polling http://925html.com/code/smart-polling/ For those of us who work with the web daily. Sun, 16 Jan 2011 12:41:12 +0000 hourly 1 http://wordpress.org/?v=3.0.4 By: Eric Ferraiuolo http://925html.com/code/smart-polling/comment-page-1/#comment-624 Eric Ferraiuolo Sat, 31 Oct 2009 07:28:55 +0000 http://925html.com/?p=288#comment-624 @EJ I have since over the months reduced the features of this utility, and the newest code is up on GitHub: http://github.com/ericf/yui3-gallery/tree/master/build/gallery-io-poller/ <a href="#comment-91" rel="nofollow">My comment above</a> explains the reason for using the Last-Modified date and Etag headers and making Conditional GET requests. There's a fundamental difference in your approach using Max-Age, and my approach using Conditional GET requests. I <em>want</em> to check if there's a change on the server and know when that change happens ASAP. I have no way to know when the change is going to happen on the server, therefore have no way to accurately set the Max-Age header. I see what you're saying about the modern browsers respecting Max-Age and evicting their cached XHR-requested resource representations; this is the exact problem I had, browsers are good at caching the responses of XHR requests. My question for you is: How do you know what to set the Max-Age response header to? How can you predict when something like a sports score will change? I want to <em>be in control</em> of how the browsers are caching all resource representations on my applications (static UI files, XHR-requested resources, etc.); this was the motivating factor behind sending over Conditional GET requests on each XHR-poll of a resource. The server should be quick at determining if what the client has cached is stale or not by comparing an Etag; saving the hit of generating the entity body and sending the payload over the wire by simply responding with a 304 Not Modified response. @EJ I have since over the months reduced the features of this utility, and the newest code is up on GitHub: http://github.com/ericf/yui3-gallery/tree/master/build/gallery-io-poller/

My comment above explains the reason for using the Last-Modified date and Etag headers and making Conditional GET requests.

There’s a fundamental difference in your approach using Max-Age, and my approach using Conditional GET requests. I want to check if there’s a change on the server and know when that change happens ASAP. I have no way to know when the change is going to happen on the server, therefore have no way to accurately set the Max-Age header.

I see what you’re saying about the modern browsers respecting Max-Age and evicting their cached XHR-requested resource representations; this is the exact problem I had, browsers are good at caching the responses of XHR requests.

My question for you is: How do you know what to set the Max-Age response header to? How can you predict when something like a sports score will change?

I want to be in control of how the browsers are caching all resource representations on my applications (static UI files, XHR-requested resources, etc.); this was the motivating factor behind sending over Conditional GET requests on each XHR-poll of a resource. The server should be quick at determining if what the client has cached is stale or not by comparing an Etag; saving the hit of generating the entity body and sending the payload over the wire by simply responding with a 304 Not Modified response.

]]>
By: EJ http://925html.com/code/smart-polling/comment-page-1/#comment-623 EJ Sat, 31 Oct 2009 07:11:12 +0000 http://925html.com/?p=288#comment-623 And I also agree that it's great to promote this standards based way of refreshing content on the page. E-tags and conditional gets are quite powerful and can solve many use cases with off the shelf software. And I also agree that it’s great to promote this standards based way of refreshing content on the page. E-tags and conditional gets are quite powerful and can solve many use cases with off the shelf software.

]]>
By: EJ http://925html.com/code/smart-polling/comment-page-1/#comment-622 EJ Sat, 31 Oct 2009 06:33:52 +0000 http://925html.com/?p=288#comment-622 At sports.yahoo.com, we let the browser do conditional gets and with a proper Max-Age header emitted by the server, all modern browsers properly make the request at the frequency we specify. IE 6 has some gotcha's so we're planning on using a random variable to bust through caches on that browser, but everywhere else, just relying on the browser to do the right thing has worked well. See implementation here (when games are going on): http://rivals.yahoo.com/ncaa/football/scoreboard So I question the need for this library. JS here: http://l.yimg.com/j/static/versioned_asset/v7/js/editorial/js/scorestream.js At sports.yahoo.com, we let the browser do conditional gets and with a proper Max-Age header emitted by the server, all modern browsers properly make the request at the frequency we specify.

IE 6 has some gotcha’s so we’re planning on using a random variable to bust through caches on that browser, but everywhere else, just relying on the browser to do the right thing has worked well.

See implementation here (when games are going on):
http://rivals.yahoo.com/ncaa/football/scoreboard

So I question the need for this library.

JS here: http://l.yimg.com/j/static/versioned_asset/v7/js/editorial/js/scorestream.js

]]>
By: orip http://925html.com/code/smart-polling/comment-page-1/#comment-603 orip Mon, 24 Aug 2009 15:49:38 +0000 http://925html.com/?p=288#comment-603 Your concept is awesome, thanks! It's given me inspiration for a polling JS component I'm making. Your concept is awesome, thanks! It’s given me inspiration for a polling JS component I’m making.

]]>
By: Eric Ferraiuolo http://925html.com/code/smart-polling/comment-page-1/#comment-577 Eric Ferraiuolo Mon, 29 Jun 2009 19:24:28 +0000 http://925html.com/?p=288#comment-577 @Andy, I'm not a PHP guy; so I'm unsure about your code you posted. May I ask what your goals are here; high-level what are you hoping to accomplish? @Andy, I’m not a PHP guy; so I’m unsure about your code you posted. May I ask what your goals are here; high-level what are you hoping to accomplish?

]]>
By: Andy D http://925html.com/code/smart-polling/comment-page-1/#comment-576 Andy D Mon, 29 Jun 2009 19:17:18 +0000 http://925html.com/?p=288#comment-576 Hmm, so I haven't used Etag values before, but I tried to set it with the PHP header method: <pre><code class="php">include("include/database.php"); $q = "Select * From qryLatestPost"; $result = $database->query($q); $num_rows = mssql_num_rows($result); if($num_rows > 0){ $DatePosted = mssql_result($result,0,"DatePosted"); $DisplayName = mssql_result($result,0,"DisplayName"); $PageText = mssql_result($result,0,"PageText"); $PageText = addslashes($PageText); // create ETAG (hash of feed) // (HTTP_IF_NONE_MATCH has quotes around it) $eTag = md5($DisplayName.$DatePosted); header('Etag: '.$eTag); $text = "{ \"label\":\"$DisplayName\", \"time\":\"$DatePosted\" }"; // compare Etag to what we got if ($eTag == $_SESSION["HTTP_IF_NONE_MATCH"]) { header("HTTP/1.0 304 Not Modified"); header('Content-Length: 0'); } else { // dump feed echo $text; } $_SESSION["HTTP_IF_NONE_MATCH"] = $eTag; }</code></pre> It is technically a dynamic source, but if there hasn't been a new post, it doesn't need to update every 5 seconds. If there is a better way to do this, I'd love to hear about it, otherwise I'll just keep researching Etags. Hmm, so I haven’t used Etag values before, but I tried to set it with the PHP header method:

include("include/database.php");

$q = "Select * From qryLatestPost";
$result = $database->query($q);
$num_rows = mssql_num_rows($result);

if($num_rows > 0){
	$DatePosted = mssql_result($result,0,"DatePosted");
	$DisplayName = mssql_result($result,0,"DisplayName");
	$PageText = mssql_result($result,0,"PageText");
	$PageText = addslashes($PageText);

	// create ETAG (hash of feed)
	// (HTTP_IF_NONE_MATCH has quotes around it)
	$eTag = md5($DisplayName.$DatePosted);
	header('Etag: '.$eTag);

	$text = "{ \"label\":\"$DisplayName\", \"time\":\"$DatePosted\" }";

	// compare Etag to what we got
	if ($eTag == $_SESSION["HTTP_IF_NONE_MATCH"]) {
		header("HTTP/1.0 304 Not Modified");
		header('Content-Length: 0');
	} else {
	// dump feed
		echo $text;
	}
	$_SESSION["HTTP_IF_NONE_MATCH"] = $eTag;
}

It is technically a dynamic source, but if there hasn’t been a new post, it doesn’t need to update every 5 seconds.

If there is a better way to do this, I’d love to hear about it, otherwise I’ll just keep researching Etags.

]]>
By: Eric Ferraiuolo http://925html.com/code/smart-polling/comment-page-1/#comment-574 Eric Ferraiuolo Fri, 26 Jun 2009 21:20:43 +0000 http://925html.com/?p=288#comment-574 @Andy The code will preform conditional GET requests to the resource via: If-None-Match (if it has a cached Etag vale) or If-Modified-Since (if it has a cached LastModified date), in that order; Etag takes priority over LastModified. In the case where responses from the resource do not provide either a LastModified or Etag, the code will assume it's changed, firing the Modified event (useful for polling dynamic resources). @Andy The code will preform conditional GET requests to the resource via: If-None-Match (if it has a cached Etag vale) or If-Modified-Since (if it has a cached LastModified date), in that order; Etag takes priority over LastModified. In the case where responses from the resource do not provide either a LastModified or Etag, the code will assume it’s changed, firing the Modified event (useful for polling dynamic resources).

]]>
By: Andy D http://925html.com/code/smart-polling/comment-page-1/#comment-573 Andy D Fri, 26 Jun 2009 21:08:42 +0000 http://925html.com/?p=288#comment-573 well, never mind, for some reason the data.json file wasn't uploading using dreamweaver. I logged into the server and changed it with notepad manually and it did update. Now I just have to make it read something besides timestamp. Thanks for the cool API well, never mind, for some reason the data.json file wasn’t uploading using dreamweaver.

I logged into the server and changed it with notepad manually and it did update.

Now I just have to make it read something besides timestamp.

Thanks for the cool API

]]>
By: Andy D http://925html.com/code/smart-polling/comment-page-1/#comment-572 Andy D Fri, 26 Jun 2009 20:52:25 +0000 http://925html.com/?p=288#comment-572 I'm trying to get this working, and the polling seems to be doing what it's supposed to, but I haven't been able to make it detect that a change has been made. The data.php file isn't downloadable (comes down as a blank page), so I've been simply changing the timestamp in the data.json file by hand. however, it doesn't seem to be noticing that the file has been updated. So either it checks something other than the timestamp, or I'm completely missing something important by not seeing the php file. Would you mind posting the contents for ajax newbies like me? I’m trying to get this working, and the polling seems to be doing what it’s supposed to, but I haven’t been able to make it detect that a change has been made.

The data.php file isn’t downloadable (comes down as a blank page), so I’ve been simply changing the timestamp in the data.json file by hand. however, it doesn’t seem to be noticing that the file has been updated.

So either it checks something other than the timestamp, or I’m completely missing something important by not seeing the php file.

Would you mind posting the contents for ajax newbies like me?

]]>
By: Eric Ferraiuolo http://925html.com/code/smart-polling/comment-page-1/#comment-97 Eric Ferraiuolo Mon, 06 Apr 2009 18:23:03 +0000 http://925html.com/?p=288#comment-97 <strong>@Dan</strong> I like your thinking here. This would effectively allow the server to tell the client when to ping back to see if there has been an update to the resource. I'm unsure how common this type of behavior would be used by either the client developer to let the sever tell what to do, or by a server developer to have an accurate prediction of when <em>dynamic</em> data would change. It seems, in practice, that max-age Cache-Control values are on the magnitude of hours or greater. That being said, I feel your suggestion falls into the <em>smart</em> category here; effectively making the component know how to listen to the server in yet another way. I've started looking into how to implement this functionality and will try to include it in the next rendition of this component. My caution is to make sure it's clear to the client API that checking of the max-age Cache-Control value might be going on in the background if no <code>interval</code> configuration value is set. Currently it is clear to the client that if they don't set an <code>interval</code> value that 10,000ms (10 seconds) will be used. <strong>@Chris</strong> thanks for the heads-up, fixed the typo. @Dan I like your thinking here. This would effectively allow the server to tell the client when to ping back to see if there has been an update to the resource. I’m unsure how common this type of behavior would be used by either the client developer to let the sever tell what to do, or by a server developer to have an accurate prediction of when dynamic data would change. It seems, in practice, that max-age Cache-Control values are on the magnitude of hours or greater. That being said, I feel your suggestion falls into the smart category here; effectively making the component know how to listen to the server in yet another way. I’ve started looking into how to implement this functionality and will try to include it in the next rendition of this component. My caution is to make sure it’s clear to the client API that checking of the max-age Cache-Control value might be going on in the background if no interval configuration value is set. Currently it is clear to the client that if they don’t set an interval value that 10,000ms (10 seconds) will be used.

@Chris thanks for the heads-up, fixed the typo.

]]>