geansai gorm

Archive for the 'javascript' Category

I’m currently working on removing MooTools from KFM. While doing that, I’m also trying to speed up KFM as much as possible. The reason I’m removing MooTools is that I feel that it is inefficient in some things, and I don’t like how it tries to attach itself to everything in the document - if you retrieve an element with ID ‘test1′ using $('test1'), then inspect the element, you’ll find that MooTools has added itself to the element. This happens a lot. They call it a feature.

I’m replacing it with jQuery. jQuery has a strong community and a database of plugins. I like it. While it is still possible to write inefficient jQuery code, I feel it is a bit more difficult than in MooTools.

change your events code

John Resig’s tutorial recommends adding events to objects using something like this:

$j('#the_element').click(eventToHandleClicks);

While that is very neat, and perfectly fine in most cases, it’s not very efficient when you’re adding it to hundreds of elements on a page when the elements are built up using functions as happens in KFM. (ie; the elements are added ad-hoc, and the events are added at the same time as the element creation)

The method I would recommend is this:

$j.event.add(document.getElementById('the_element'),'click',eventToHandleClicks);

In this case, we avoid the internal stuff that happens in $j() by replacing it with the fast browser function document.getElementById(), and we also jump straight to the addition of the event with the static $j.event.add() instead of $j().click() which is meant to work on an arbitrary number of elements and is therefore slightly slower.

The same applies to generic addEvent() code:

$j('#the_element').addEvent('mouseover',mouseOverEvent);

Again, replace that with:

$j.event.add(document.getElementById('the_element'),'mouseover',mouseOverEvent);

defer creation of events

Let’s say you have a hundred elements to create. Each element has a mouseover, click and mouseout event to be added. That’s 300 operations.

function inLink(){
  window.status='in link';
}
function outLink(){
  window.status='out of link';
}
function clickLink(){
  window.status='link clicked';
}
var i,el;
for(i=0; i<100; ++i){
  el=document.getElementById('link'+i);
  $j.event.add(el,'mouseover',inLink);
  $j.event.add(el,'mouseout',outLink);
  $j.event.add(el,'click',clickLink);
}

This is a very common piece of code. However, it is inefficient, both speed- and memory-wise. If you have 100 links, and are likely to only click one, then why bother adding the mouseout and click events to the rest?

The answer is to add those events upon mouseover and make sure they’re only added once.

function inLink(){
  window.status='in link';
  if(this.eventsAdded)return;
  $j.event.add(this,'mouseout',outLink);
  $j.event.add(this,'click',clickLink);
  this.eventsAdded=true;
}
function outLink(){
  window.status='out of link';
}
function clickLink(){
  window.status='link clicked';
}
var i,el;
for(i=0; i<100; ++i){
  el=document.getElementById('link'+i);
  $j.event.add(el,'mouseover',inLink);
}

Here’s a method to create minified JS files on-the-fly, without incurring the time cost of the minification process.

To explain, minification is the act of removing all whitespace from JavaScript so the file can be downloaded and parsed as quickly as possible.

A useful part of minification is that during the act of compiling your minified source, you can also pull in other JavaScript files and compiled them all into one single source. This has a major advantage that there is only one file to download. This is very important because even if those small files were small, the lag involved in sending multiple HTTP requests can sometimes be worse than if a single file which was twice the size of all the small files in total was downloaded. (which is why good web devs use Sprites when possible)

Right. That’s the explanation of /why/ to do it. Now how?

First off, here’s a very simple piece of source to show it (in a file called “all.php”).

  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  echo $js;

You can get jsmin-php from here. the other files are JQuery files but only used for illustrative purposes.

Anyone spot the problem? It does the job, yes, but the minify() script takes a few seconds to run, ruining the advantage it created with the minification.

The next step towards perfection is to cache the file. The obvious solution to that is to save the file in a location. If the file exists when the files are requested next, then serve the cached version instead of going through the minification process.

That’s not good enough, though. What if a file in the list was changed? You’d never know because the cached file will always be served.

The solution is to save the cached file in a file named using an MD5 of the last modified datetimes (genius idea - I’d love to shake the author’s hand). That way the cache will always be correct, and will automatically update itself when a file is changed.

$writabledir='/path/to/caches/';

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  file_put_contents($writabledir.$name,$js);
  echo $js;
}

Better. This time, the delay is only on the first load. All subsequent loads will have an instant download of the cached file.

However, if you’re a developer, then almost every reload will involve the minification process - you’d never get your work done.

A solution is to check to see if the current MD5 cache exists, and if it doesn’t, then download the files as a bundle without minifying them. Instead, before sending the files, you tag on a little bit of javascript which will do the minification in the background after the script has been sent to the client.

Here is a complete solution, along with a little cleanup which removes caches older than an hour.

$writabledir='/path/to/caches/';

function delete_old_md5s($folder) {
  $olddate=time()-3600;
  $dircontent = scandir($folder);
  foreach($dircontent as $filename) {
    if (strlen($filename)==32 && filemtime($folder.$filename) && filemtime($folder.$filename)<$olddate) unlink($folder.$filename);
  }
}

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

header('Content-type: text/javascript');
header('Expires: '.gmdate("D, d M Y H:i:s", time() + 3600*24*365).' GMT');

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  if(isset($_REQUEST['minify'])){
    require 'jsmin-1.1.1.php';
    $js=JSMin::minify($js);
    file_put_contents($writabledir.$name,$js);
    delete_old_md5s($writabledir);
    exit;
  }
  else{
    $js.="setTimeout(function(){var a=document.createElement('img');a.src='all.php?minify=1';a.style.display='none';document.body.appendChild(a);},5000);";
  }
  echo $js;

A pseudo-code version of the above would be:

if file named after md5 of scripts directory exists, echo it and exit.
else{
  concatenate requested scripts into one large string.
  if request URI does not have "minify" as a parameter{
    add a javascript instruction to the string to load this script again with the minify parameter in 5 seconds
    print the string and exit.
  }
  else{
    minify the string.
    save the string to a file named after the MD5
    exit without printing anything (no reason to send this to the client)
  }
}

Almost every time someone says to me that something I was working on is broken, the answer is “clear your cache”, and that automagically fixes everything. However, that’s not an ideal solution - ideally, the problem would never happen in the first place.

So, why does the problem happen?

Let’s say that there is a HTML file which calls a JS function like this: showImage(); - the JS function is included from the external file /j/images.js.

Browsers are usually set to cache .js files, and that’s the correct behaviour for the most part. Unfortunately, when a file needs to be fixed, it can cause problems.

For example, let’s say that I’ve corrected the function name to match my usual naming scheme - images_show();. I change the reference in both places. The browser reads the new HTML file from the net, but loads the JavaScript from the cache - suddenly there’s a mismatch which causes a problem.

So, how to get around this?

The solution I’m using at the moment involves a little bit of mod_rewrite and PHP.

Sticking with the contrived example, let’s rewrite /j/images.js so it is accessible from /j/images (using /.htaccess):

ExpiresActive On
ExpiresDefault A259200
RewriteEngine on
RewriteRule ^j/images$ /j/images.js [L]

Now, we add a little magic. We want to change the URL if the file has changed. The only way to know this is to look at the modified date of the file.

In your PHP, you could do it like this:

<script type="text/javascript" src="/j/images/<php? echo md5(`ls -l j/images.js`); ?>"></script>

and then change the .htaccess file to allow that:

ExpiresActive On
ExpiresDefault A259200
RewriteEngine on
RewriteRule ^j/images/(.*)$ /j/images.js [L]

Now, if no file changes happen, then the MD5 hash (and therefore the URL) will be cacheable, and if the file changes, then the URL will automatically change as well.

…and that’s not all!

I like to aggregate my JavaScript files to reduce the network pain felt by the browser. In my CMS, it’s done with a /j/js.php file. Here’s a short excerpt:

<?php
$js=file_get_contents('jquery-1.2.3.min.js');
$js.=file_get_contents('js.js');
$js.=file_get_contents('tabs.js');
$js.=file_get_contents('addrow.js');
$js.=file_get_contents('formhide.js');
/* more files */

header('Cache-Control: max-age=2592000');
header('Expires-Active: On');
header('Expires: Fri, 1 Jan 2500 01:01:01 GMT');
header('Pragma:');
header('Content-type: text/javascript; charset=utf-8');

echo $js;

That’s then pointed to with this line in my .htaccess:

RewriteRule ^js/(.*)$ /j/js.php [L]

And it’s referenced in the browser like this:

echo '<script type="text/javascript" src="/js/'.md5(`ls -l j`).'"></script>';

Simple, innit! That simple trick now keeps track of a number of files, and the browser knows immediately if there are any changes.

BTW: The same trick can be used with images, css, and any number of other “static” objects.

I don't have a geansai gorm, but if I did, I might sometimes wear it.