matrix.js introduction

Essentially, matrix.js is like other script loader, but it does other things such extensible resource type, dependencies registration and reload, resource locations, resource release and others. In the following, I will explain what the library can do. All these example presented here can be found at GitHub

The most used method in matrix is the "matrix" method. Here is the definition.

//returns a promise
//resourceKeys is a string or an array, e.g "a.css, b.css" 
//or ["a.css", "b.css"] or ["c.js", "a.css, b.css"]
matrix(resourceKeys[, loadByOrder])

If you have no configuration, the resource key is is treated as url of the resource. Resource key follow the format of [resourceName].[resourceType]. There are three build-in resource handler in matrix, "module", "js", and "css", resourceKeys can be a string or array. Most of the time, you should use string format resourceKeys. If it is a string, this means resource are fetched in parallel, if it is an array, this means resources are fetched in serials, most of time, we use resourceKeys as string. I will discuss more about it later. Just like any other script loader, you can load javascript and css like the following.

var promise = 
matrix("/css/reset.css, /css/base.css, /js/utilities.js, /js/app.js", 
).done(fn);

Behind the scene, the loader will fetch two css and two javascript files using ajax call in parallel, and evaluate each source file into the page when it is ready at client side. and run fn1, fn2, fn3 after all resources are loaded. It is important to know that, the order of making ajax call is not necessary the order when ajax call is finished, so the order of evaluation is not guaranteed. If the order of evaluation is important, you can use the following code.

var promise = 
matrix("/css/reset.css, /css/base.css, /js/utilities.js, /js/app.js", 
true).done(fn);

Behind scene, the load method actually convert it into the following call.

//reset depends nothing
matrix.depend("/css/reset.css", null); 

//base.css depends on reset.css
matrix.depend("/css/base.css", "/css/reset.css"); 

//utitlies.css depends on base.css
matrix.depend("/js/utilities.css", "/css/base.css");  

//app.css depends on utilities.css
matrix.depend("js/app.css", "/js/utilities.css");  
var promise = matrix("/js/app.js", true).done(fn);

Dependencies between the module is important in resource loading, sometimes is more complicated than serialized dependencies like above case. When loading resource, matrix follows a couple steps. Firstly, matrix determines the type of resource and the correct handler for processing the resource request. After that, matrix use the handler to execute three tasks(fetch, parse and evaluation). Fetch is to get the content ready at the client side. Parsing is an optional step, matrix currently extract dependencies from the content in this step, and this is necessary only when dependencies information is not pre-configured, and parsing method is implemented by the handler. Evaluation is to integrate the resource into the page. These three tasks are implemented by a resource handler.

Let's try a more complicated case. jQuery UI library offer lots of widgets. It is a highly modularized library, some components(like ui.mouse.js) are reused over the library like. If you only want to use a certain widget,e.g dialog, you can go to their home page, deselect all, and just select dialog, and it create a a package that include just the css, js file that dialog widget use. This is a great feature. But there is still a problem. If I have two page, page one need a,b,c widget, and page two need b,c,d widget, so I need to create two js package. To save the trouble, I might better off just create a package file that include all the modules. However this might load too many modules that I don't need. We can use matrix to load the minimum set of resource to be able to use a widget. First I will show you the hard way, later I will show you easy way. But regardless of which, we need to solve two problem, resource dependencies and and resoure locations.

To understand dependencies in jQuery UI, you need to download all the source of jQury UI. Open the source file in jQuery.ui.dialog.min.js, the header says

/*
 * jQuery UI Dialog 1.8.10
 *
 * Copyright 2011, AUTHORS.txt (http://jqueryui.com/about)
 * Dual licensed under the MIT or GPL Version 2 licenses.
 * http://jquery.org/license
 *
 * http://docs.jquery.com/UI/Dialog
 *
 * Depends:
 * jquery.ui.core.js
 * jquery.ui.widget.js
 *  jquery.ui.button.js
 * jquery.ui.draggable.js
 * jquery.ui.mouse.js
 * jquery.ui.position.js
 * jquery.ui.resizable.js
 */

Now we can see what dialog depends, dialog also depends jquery.ui.dialog.css. Let's change it for now and repeat this for other module accordion, autocomplete, button, datepicker, dialog, progressbar, resizable, selectable, slider, tabs, for example, add jquery.ui.accordion.css in jQuery.ui.accordion.min.js header. We will need it later in using next method.

/*
..
* Depends:
 * jquery.ui.dialog.css
 * jquery.ui.core.js
 * jquery.ui.widget.js
 *  jquery.ui.button.js
 * jquery.ui.draggable.js
 * jquery.ui.mouse.js
 * jquery.ui.position.js
 * jquery.ui.resizable.js

*/

To convert this into code, we can write the following. Please note that I am not using the url of resource as the resource key, later I will map the reource key to url.

matrix.depend( "ui.dialog.js": "ui.dialog.css, ui.core.js, ui.widget.js, ui.button.js, ui.draggable.js, 
ui.mouse.js,ui.position.js, ui.resizable.js");

But this is not enough, because inner resources have their own dependencies as well. So we need to open all direct and indirect referenced js files and find out their dependencies. If there a file has no dependencies, we still need to use a null value. For example, ui.core.js does not have any dependencies, so its dependencies is null. Finally, we can write something like the following.

matrix.depend( {
 "ui.dialog.js": "ui.dialog.css, ui.core.js, ui.widget.js, ui.button.js, ui.draggable.js, ui.mouse.js, 
ui.position.js, ui.resizable.js",
 "ui.core.js":null,
 "ui.widget.js":null,
 "ui.button.js": "ui.button.css, ui.core.js, ui.widget.js",
 "ui.draggable.js": "ui.core.js, ui.mouse.js, ui.widget.js",
 "ui.mouse.js": "ui.widget.js",
 "ui.position.js":null,
 "ui.resizable.js":"ui.resizable.css, ui.core.js, ui.mouse.js, ui.widget.js",
 //ui.redmond.css is my favorite skin for jQuery ui, :)
 "dialog.module" : "ui.core.css, ui.redmond.css, ui.dialog.js"
} );

Now we need to solve the second problem, resolving the url of resource, and we can use the following

matrix.url( {
 "ui.core.css": "js/jquery.ui/css/base/jquery.ui.core.css",
 "ui.dialog.css": "js/jquery.ui/css/base/jquery.ui.dialog.css",
 "ui.resizable.css": "js/jquery.ui/css/base/jquery.ui.resizable.css",
 "ui.button.css": "js/jquery.ui/css/base/jquery.ui.button.css",
 "ui.smoothness.css": "js/jquery.ui/css/smoothness/jquery.ui.theme.css",
 "ui.redmond.css": "js/jquery.ui/css/redmond/jquery.ui.theme.css",
 "ui.dialog.js":"js/jquery.ui/jquery.ui.dialog.min.js",
 "ui.core.js":"js/jquery.ui/jquery.ui.core.min.js",
 "ui.widget.js":"js/jquery.ui/jquery.ui.widget.min.js",
 "ui.button.js":"js/jquery.ui/jquery.ui.button.min.js",
 "ui.draggable.js":"js/jquery.ui/jquery.ui.draggable.min.js",
 "ui.mouse.js":"js/jquery.ui/jquery.ui.mouse.min.js",
 "ui.position.js":"js/jquery.ui/jquery.ui.position.min.js",
 "ui.resizable.js":"js/jquery.ui/jquery.ui.resizable.min.js"
} );

matrix( "dialog.module", function () {
 $( "#linkDialog" ).click( function() {
  $( "<h1>hello</h1>" ).dialog();
  return false;
 } );
} );

Let's how the resources are actually process.

So far so good, the fetching is parallel, and the evaluation is serial and in the correct order of dependencies. But the above configuration is too tedious. But if we want to use accordion widget, we need to repeat this process, obviously it violate the principle "Don't Repeat Yourself". Let's do "Convention over configuration". As I mentioned before, resource is extensible, you can extend handling of resource by adding a new resource type, which map to a new handler. For example, and x.mytype is mapped to a handler matrix.handlers.mytype. Here is methods that a handler can implement.

1. load(resourceKey) [must implement]

   This method is to fetch and evaluate the resource and its dependencies into memory.

2. release(resourceKey) [optional]
   

   This method is to release single resource (not its dependencies)

3. url(resourceKey) [optional]

   This method is to build a url using a naming convention

4. parseDepends(sourceCode) [optional]

   how to parse source code to build dependencies

Only the load method need to be implemented, other methods are optional. In the case here however, we want to reuse the default load method of javascript handler and css handler, all we need to do use extend url, and parseDepends. The following shows how to use the addHandler method

// it return back the new handler
//newHandlerName is also the name of new resource type
//matrix.addHandler(newHandlerName, baseHandlerName [,newBehavior])

matrix.addHandler( "uicss", "css", {
 url: function ( resourceKey ) {
  return matrix.fullUrl( matrix.baseUrl + "jquery.ui/css/base/jquery.ui." + 
matrix.resourceName( resourceKey ) + ".css" );
 }
} );

matrix.addHandler( "uitheme", "css", {
 url: function ( resourceKey ) {
  return matrix.fullUrl( matrix.baseUrl +  "jquery.ui/css/" + 
matrix.resourceName( resourceKey ) + "/jquery.ui.theme.css" );
 }
} );

var rdependencies = /^\s*\/\*[\w\W]*Depends:([\w\W]+)\*\//,
 ruiModule = /jquery\.ui\.(\w+?)\.(js|css)/gi;

//extend js handler
matrix.addHandler( "uijs", "js", {

 url: function ( resourceKey ) {
  return matrix.fullUrl( matrix.baseUrl +  "jquery.ui/jquery.ui." + 
matrix.resourceName( resourceKey ) + ".min.js" );
 },

 parseDepends: function ( sourceCode ) {
  var dependencies = [],
   module,
   dependText = rdependencies.exec( sourceCode );


  if ( dependText = dependText && dependText[1] ) {
   //dependText is something like
/*
   * jquery.ui.tabs.css
   * jquery.ui.core.js
   * jquery.ui.widget.js
*/
   while ( module = ruiModule.exec( dependText ) ) {
    //find jquery.ui.xxx.css and convert it to xxx.uicss
    //find jquery.ui.xxx.js and convert it to xxx.uijs
    
    // "xx" + ".ui" + "js" == "core.uijs"
    // "xx" + ".ui" + "css" == "core.uicss"
    dependencies.push( module[1] + ".ui" + module[2] );
   }
   return dependencies.length ? dependencies.toString() : null;
  }

  return null;
 }
} );

In the above code, the url method is straight forward, it is used to map a resource key to a url, since jQuery UI team follow a naming convention of their components, we can use this, and use a function to calculate the url based on resource key. What is interesting this the parseDepends method. The idea is that if we can parse it by human eye, we can parse it by code. The method basically extract the headers we modified before, and convert them into resource keys, and then build dependencies. With all this code in place, we can throw away the previous dependencies and url configuration. Let's write a bit of code, to register the dependencies and the locations of all jQuery ui files.

matrix.baseUrl = "js/";
matrix.debug();

matrix.depend( {
 "uicss.module": "core.uicss, redmond.uitheme"
} );

$.each( 
("accordion,autocomplete,button,datepicker,dialog," + 
"progressbar,resizable,selectable,slider,tabs," + 
"draggable,droppable,mouse,position,sortable").split( "," ),
 function (index, value) {
  var resourceKey = value + ".module";
  var depends = value + ".uijs, uicss.module";
  matrix.depend(resourceKey, depends);
 } );

I agree that I need to spend more time to do this kind of convention over configuration. But I think it is worthy, because I can dynamically load the the resources of just of my widget, any widget in jQuery UI, not just dialog. In the next post, I will explain dependencies refresh, release release, performance and other advanced topic in using matrix.

tweaking HTTP header to imporve performance

Lately, I am doing some client side optimization. One of the techniques is cache. When browser firstly request a resource and get the response, the cache subsystem of the browser need to determine whether to cache resource. It is based on the response header Cache-Control, and Expires. When neither of them exists in the response header, browsers use their own strategy to cache, these strategies vary from browser to browser, so they are not predictable, we should explicitly set these headers. To explicitly disable cache in browser, we can use the following

Pragma: no-cache 
or
Cache-Control: no-cache 
or
Cache-Control: no-store

The Pragma: no-cache header is included in HTTP 1.1 for backward compatibility with HTTP 1.0+. It is technically valid and defined only for HTTP requests; however, it is widely used as an extension header for both HTTP 1.0 and 1.1 requests and responses. HTTP 1.1 applications should use Cache-Control: no-cache, except when dealing with HTTP 1.0 applications, which understand only Pragma: no-cache. According RFC 2616, Cache-Control: no-cahce is semantically different from Cache-Control: no-store. non-cache still allows browser cache a response, and the cache needs to re-validate the response with the origin server before serving it. no-store request browser not to cache response at all. However most browser treat "no-cache" as "no-store".
To implement the semantics of "no-cache", we should use Cache-Control: max-age=0 or Cache-Control: s-maxage=0 or Cache-Control: must-revalidate.

To explicitly to cache resource, we should use the following.

Cache-Control: max-age=3600
Cache-Control: s-maxage=3600
//or
Expires: Fri, 05 Jul 2002, 05:00:00 GMT

The Cache-Control: max-age header indicates the number of seconds since it came from the server for which a document can be considered fresh. There is also an s-maxage header (note the absence of a hyphen in "maxage") that acts like max-age but applies only to shared (public) caches. The deprecated Expires header specifies an actual expiration date instead of a time in seconds. The HTTP designers later decided that, because many servers have unsynchronized or incorrect clocks, it would be better to represent expiration in elapsed seconds, rather than absolute time. An analogous freshness lifetime can be calculated by computing the number of seconds difference between the expires value and the date value.

If browser hit the same url again, browser will firstly ask the cache sub-system to get a cache copy. If the cache is still fresh based on "Cache-Control" or "Expires" header, the cache is returned, and no request is sent to server. If the cache expired, determined whether a validator was send from the previous response, different request will be sent. If a validator of previous response is "Last-Modified", the following will apply.

//validator of previous response
Last-Modified: Sun, 03 Apr 2011 14:34:43 GMT

//validator of next request
If-Modified-Since: Sun, 03 Apr 2011 14:34:43 GMT

If a validator of previous response is "Etag", the following will apply

//validator of previous response
ETag: 100

//validator of next request
If-None-Match: 100

Normally we should use either "Last-Modified" or "ETag" in response from server. If neither of them is available in the previous response, then a new request will make. You should note that validator is independent with "Cache-Control" and "Expires" header, which means it can be used without these two header.

In asp.net, if you don't do anything, the following default header will be generated.

HTTP/1.1 200 OK
Server: ASP.NET Development Server/10.0.0.0
Date: Sun, 03 Apr 2011 14:47:30 GMT
X-AspNet-Version: 4.0.30319
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 509
Connection: Close

Because the Cache-Control header does not specify a max-age, by default is zero, and because there is no validator available, is semantically "Cache-Control: no-store". So we don't need to do anything if we want the client not to cache the response, because it is by default. The following is the default header of next response.

GET http://localhost:1491/WebSite1/Cache.aspx HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: localhost:1491
Cookie: ASP.NET_SessionId=imdo5jrzxkpw2ohy24guaw0b

The following is a typical header to enable client side cache.

Cache-Control: max-age=3600
Last-Modified: Sun, 03 Apr 2011 14:34:43 GMT

or 
Cache-Control: max-age=3600
ETag: 100

We should prefer "Cache-Control" over "Expires" because it is smarter. When the resource is static, we should config web server to add a Last-Modified header but no using ETag, because it simply and easier. And ETag is problematic in IIS web farm. When the resource is dynamic, we should use dynamically generate an ETag as hash to compare change. Here is some server side code.

private void AddCacheControlHeader(int seconds)
    {
        Response.AddHeader("Cache-Control", string.Format("max-age={0}", seconds));
    }

    private void AddETagValidator()
    {
        string hash = "xyz"; //calcuate a hash match the request
        Response.AddHeader("ETag", hash);
    }

    private void ProgessETagValidator()
    {
        string hash = "xyz"; //calcuate a hash match the request
        if (Request.Headers["If-None-Match"] == hash)
        {
            Response.StatusCode = 304;
            Response.End();
        }
    }

A enhanced curry method

JavaScript is a functional language, it supports function composition, we can do curry with JavaScript. Douglas Crockford has defined "curry" method in his book JavaScript: The Good Parts to facilitate the steps to define curried function.

Function.prototype.method = function ( name, func ) {
 if ( !this.prototype[name] ) {
  this.prototype[name] = func;
 }
};

Function.method( 'curry', function () {
 var slice = Array.prototype.slice,
   args = slice.apply( arguments ),
   that = this;
 return function () {
  return that.apply( null, args.concat( slice.apply( arguments ) ) );
 };
} );

//so I can write the following code

test( "test curry", function () {
 function cancat_abc( a, b, c ) {
  return a + b + c ;
 }

 var cancatBc = cancatAbc.curry( "a" );
 var result = cancatBc( "b", "c" );
 equal( result, "abc", "the first parameter is preloaded" );

} );

Nice. But there is problem here, the preloaded arguments have to be the first consecutive arguments. So if I have want to preload the second and third parameter. The curry method does not support that.

Of course, I can do it manually instead of using the curry method, but this is not quite DRY (Don't Repeat Yourself). So I modify the curry method as follow to do the work.

Function._ = {};

Function.method( 'curry', function () {
 var slice = Array.prototype.slice,
   preloads = slice.apply( arguments ),
   _ = Function._,
   fn = this;
 return function () {
  var args = slice.apply( arguments );
  for ( var i = 0; i < preloads.length; i++ ) {
   if ( preloads[i] == _ ) {
    preloads[i] = args.shift();
   }
  }

  return fn.apply( null, preloads.concat( args ) );
 };
} );

var _ = Function._;

test( "test curry", function () {
 ok( _, "an dummy object has been defined" );
 function cancat_abcd( a, b, c, d ) {
  return a + b + c + d;
 }

 var cancatBd = cancatAbc.curry( "a", _, "c", _ );
 var result = cancatBd( "b", "d" );
 equal( result, "abcd", "curry should work in order" );

} );

the when method

In jQuery 1.5, there is a "when" method.

function when(objects) {}

//we can call with like
jQuery.when(object1, object2, object3, ..).then(done, fail);

The semantics of the function is when ALL the objects'(object1, object2, ..) value is realized, then call the done function with all the realized value, when one of them fail, call the fail function. These objects can be a "realized" value or a promise (promising value). If all the value are already "realized" initially, the done function will be called synchronously.

the promise object

The jQuery.Deferred object has a function .promise(), it returns a promise object. Some other people call it future. In the async framework of .net, it is called Task<T> . It an object that promise to deliver a value in the future. So if believe that promise, you can express your believing like this

promise.done(function () {
  var valuesOfFuture = [].slice.call(arguments, 0);
   alert();
});

The promise object does not has method of "resolve" and "reject". Because these method belong to server side. This pattern is very popular, but jQuery stream line this pattern. It is very useful pattern.

jQuery.Deferred Object

Internally, the Deferred object is implemented with two _deferred object. The Deferred object offer two set of functions, done/resolveWtih/resolve/isResolved and fail/rejectWith/reject/isRejected. The are basically the functions of two internal _deferred object. The failed set function is the done/resolveWith/resoved/isREsolved function of fail object. But we don't have cancel function, because it is unnecessary, it is replaced with reject function. The done function will empty the failed list, and the reject function will empty the done list.

It also introduced two new functions, then and promise. The then function is short cut to push a done function and failed function at one shot. We care about the success and compensation at the same time, we should consistently use then function instead of interweaving done or fail method. But if we don't care about that, we can interweave done and fail method. However, I think the then method should always be used, as it adds very little cost.

Another new method is promise method. It basically return a new object or decorate an existing object with "promise prospect" or promise methods then, done, fail, isResolved, isRejected, promise. The promise object does the same thing as Deferred object, except it can not resolve, reject, which should be used at the server side. When we first call the d.promise(obj) with a real obj, then the obj is augmented with the promise methods and cached internally, and this pattern should be used at the first time. The subsequent call d.promise() will return the cached object.

The constructor Deferred(initialize) can be passed with an initialize function. Your function can be defined as

function (deferred) {
  //this == deferred
   this.then(x, y);
   this.m1 = "m1";
}

jQuery._Deferred Object

The jQuery._Deferred is an object maintaining a task list. To understanding it, help you to understand how the jQuery.Deferred and jQuery.when works. The _Deferred object has the following use case.

• Push your task into the list to be run(resolved) in the future. It has been resolved, call the resolved method again.
  
• 
  Resolve(run) the tasks in the list, the task will be run in the order you push it, [].shift method is used to dequeue from the list. If everything runs fine, the task will be empty.
  
• 
  Push the task into task list during the task is run. But the new task will the last one to run.
  
• 
  Exception handling inside of your task. If the current task is run synchronously, you can optionally use exception handling, an exception is not caught, the remaining tasks will be canceled.
  
• 
  You have an option to eagerly cancel the remaining task, in the current running task. This is an advanced task.
  
• 
  Once you resolve the task list one a set of values, then the resolve context, and values are cached. If you resolve it again with new values, the new task still be run, but run in the old context, and values.