Transport Options in Javascript

by Ben McMahan - January 24, 2011

Recently I had to play around with lots of different methods of communication between the browser and a server. In this document I wanted to put down some of that hard earned information. More specifically I'm interested in times when you want to do this progressively. Doing so is tricky because of various browser constraints, but hopefully this can help you navigate the waters.

XHR

Bread and butter of transport mechanisms. This is standard across all browsers but pushing it to its limit exposes some browsers are better suited to modern use than others. See http://www.w3.org/TR/XMLHttpRequest/ for more information.

var request = new XMLHttpRequest();
request.onreadystatechange = function() {
  // Typical variables to be concerned about are request.readyState, request.status, and request.responseText (or responseXML)
  if (request.readyState == 4) {
    // Handle the response.
    // NOTE: you should never just dump the response into the DOM without
    // validating it first.
    document.getElementById('fillin').innerHTML = request.responseText;
  }
};
request.open('GET', '/request/url?data=1');
request.send();

Progressive XHR

Sometimes you would like to receive multiple bits of data from a single request but you don't want to have to wait for the whole request to use the data. A common example is the xhr is pulling in a new page that we will display for the user. We could just wait for the whole thing then show the response, but it would be better, especially for our modem toting friends, to show the page as it comes in just like the browser does for a page request.

To use progressive XHRs you simply look at the response while in readyState 3. The onreadystatechange callback will be called with each new chunk of data and the responseText will contain all the data you've seen so far. If you need nicely seperated data, a delimiter can be used to quickly detect the data boundaries. See Kyle's post for more information.

var index = 0;
var delimiter = '/*delimiter*/;

request.onreadystatechange = function() {
  if (request.readyState == 3) {
    // Data download is in progress.  Look for a delimiter and process all the information up to that point.
    var temp = request.responseText.indexOf(delimiter, index);
    while (temp > index) {  // Loop in case more than one chunk came in.
      var data = request.responseText.substring(index, temp);
      index = temp + delimiter.length;
      handleChunk(data);
      temp = request.responseText.indexOf(delimiter, index);
    }
  } else if (request.readyState == 4) {
    // There shouldn't be anything left for us to do since the last chunk would have also have had one last readyState == 3 pass.
  }
};

Caveats (client side): This just works out of the box in Firefox. For Chrome the response needs to not have a text/html response type. Typical types to use are application/x-javascript or text/plain. Also note that this DOES NOT work in IE no matter how hard you try. In IE7 or lower, the callback will crash with an error when accessing the responseText in readyState 3. In IE8+ it will not crash but it won't have any text until the entire response has been returned. According to this post this seems like it might be a hard limitation of IE. So they introduced a Cross Domain Request (XDR) but this has plenty of faults as well (it's cross domain and doesn't allow cookies to be sent).

Caveats (server side): The server needs to implement server side buffer flushing to send the data back in chunks. This requires HTTP1.1, but note that proxies can (and do) downgrade requests so even if you are doing everything perfectly, some people may not be getting the experience you expect.

Cross Domain requests using XHR

XDR

This is IE's response to progressive XHRs, but it's also their response to needing a cross domain request, so you can't have one without the other. If your responses don't rely on cookies and you have your security issues solved, this is actually quite easy to adapt to once you're already using XHRs. See more details here

var xdr = new XDomainRequest();
if (xdr) {
  xdr.onprogress = function() {
    // Handle the data progressively by accessing xdr.responseText.
  }
  xdr.onload = function() {
    // See the final data in xdr.responseText.
  }
  xdr.onerror = function() {
    // Handle any errors.
  }
  xdr.open("GET", url);
  xdr.send();
}

Note: your server will have to allow these requests by returning a "Access-Control-Allow-Origin" header. See http://www.w3.org/TR/cors/ for more details.

CORs

Cross-Origin Request specification

To implement this, you use an xhr and you just need the server to return a "Access-Control-Allow-Origin" header. The Cross-Origin Request specification has more information. Added bonus, if you want to send cookies, just set xhr.withCredentials to true before sending. In fact, checking if this property exists is a good way to determine if the browser can handle CORs.

IFrames

This is the easiest way to get progressive communication from the server to the client in all browsers, but it does come with a host of little issues. The main idea is to add an iframe with the src set to the request you want. The response comes down wrapped with some script that gives the payload to the main document. To get more progressiveness, have it call the function with chunks of the data.

Main Document:
<html>
<script>
function requestData(url) {
// Just change the src of the iframe to what you want.
window.frames['transport'].src = url;
}
function responseCallback(data) {
  // handle data here.
}
</script>
<body>
<iframe name=transport style="display:none;" src="about:blank"></iframe>
</body>
</html>
Response:
<script>
parent.responseCallback('<data chunk1 - use quotes if a string, if json data just pass the object.>');
parent.responseCallback('<data chunk2 - use quotes if a string, if json data just pass the object.>')
</script>

Note: IFrames have a lot of little quirks. If requesting multiple urls, you'll need a new iframe for each parallel request you want. Once the data comes, you won't have strong indication what url it is from unless you have that send up with the callback (or name the callback appropriately). Further, in IE, whenever an iframe loads data it creates an opportunity for a character to be eaten from an input box, so this also makes this inappropriate to use if you load data on every keystroke.

JSONP

This is the easiest way to do cross domain requests, but note that JSONP is never progressive (and can act slowly in Firefox if you use multiple requests at the same time). The idea is to add a new <script> tag with the src set to your request. The response, like with IFrames, comes down wrapped in a function call that can handle the response payload. One major difference is that even if you break it up as chunks (multiple function calls) it all gets processed at once when the data is fully loaded so you lose progressiveness.

Main Document:
<html>
<script>
function requestData(url) {
// Create a new script element with src pointing the way.
var script = document.createElement('script');
script.src = url;
document.body.appendChild(script);
// Note you may want to give the element an id so you can clean up later.
}
function responseCallback(data) {
// handle data here.
}
</script>
<body>
</body>
</html>
Response:
<script>responseCallback('<data - use quotes if a string, if json data just pass the object.>')</script>

Note: This has similar issues that the IFrames have. You'll need multiple elements for parallel requests and you might need to echo the url up to the responseCallback to know what request it corresponds to. Also Firefox will have issues once you try to do parallel requests.

Flash

Alternatively, you can create a flash object that will do the transport for you. It will do progressive chunks, though it seems to need a pretty large chunk in IE before it splits it so padding it to fill up to 2k might be needed. This will work similar to IFrames and JSONP, the flash object will call a javascript callback with the data as it processes it. One last caveat is Flash seems to always unescape the data when you retrieve it so you may also have to double escape your output.

Websockets

Websockets are being added as part of the HTML5 spec. It allows you to open one channel to the server, then have either the server or client send data to each other. Progressiveness can be achieved by simply sending the chunks individually from the server. It looks like all the browsers are implementing it except IE (which will probably add it once there is enough demand). See the w3c spec for more details.

Conclusion

If you want to examine your responses as they come in, it's not as easy as it should be (I'm looking at you IE). Without progressive XHRs, you have to get creative but you should still be able to achieve your goals.

Name Progressive Cross Domain IE Firefox Webkit (Chrome/Safari) Opera
Base XHR Yes if not IE No Yes Yes Yes Yes
CORS XHR Yes Yes No Yes Yes Yes
XDR Yes Yes Yes No No No
IFrames Yes No Yes Yes Yes Yes
JSONP No Yes Yes Yes Yes Yes
Flash Yes Yes Yes Yes Yes Yes
Websockets No but doesn't need to be ? No Yes Yes Yes