klog

Here’s a method to create minified JS files on-the-fly, without incurring the time cost of the minification process.

To explain, minification is the act of removing all whitespace from JavaScript so the file can be downloaded and parsed as quickly as possible.

A useful part of minification is that during the act of compiling your minified source, you can also pull in other JavaScript files and compiled them all into one single source. This has a major advantage that there is only one file to download. This is very important because even if those small files were small, the lag involved in sending multiple HTTP requests can sometimes be worse than if a single file which was twice the size of all the small files in total was downloaded. (which is why good web devs use Sprites when possible)

Right. That’s the explanation of /why/ to do it. Now how?

First off, here’s a very simple piece of source to show it (in a file called “all.php”).

  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  echo $js;

You can get jsmin-php from here. the other files are JQuery files but only used for illustrative purposes.

Anyone spot the problem? It does the job, yes, but the minify() script takes a few seconds to run, ruining the advantage it created with the minification.

The next step towards perfection is to cache the file. The obvious solution to that is to save the file in a location. If the file exists when the files are requested next, then serve the cached version instead of going through the minification process.

That’s not good enough, though. What if a file in the list was changed? You’d never know because the cached file will always be served.

The solution is to save the cached file in a file named using an MD5 of the last modified datetimes (genius idea - I’d love to shake the author’s hand). That way the cache will always be correct, and will automatically update itself when a file is changed.

$writabledir='/path/to/caches/';

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  file_put_contents($writabledir.$name,$js);
  echo $js;
}

Better. This time, the delay is only on the first load. All subsequent loads will have an instant download of the cached file.

However, if you’re a developer, then almost every reload will involve the minification process - you’d never get your work done.

A solution is to check to see if the current MD5 cache exists, and if it doesn’t, then download the files as a bundle without minifying them. Instead, before sending the files, you tag on a little bit of javascript which will do the minification in the background after the script has been sent to the client.

Here is a complete solution, along with a little cleanup which removes caches older than an hour.

$writabledir='/path/to/caches/';

function delete_old_md5s($folder) {
  $olddate=time()-3600;
  $dircontent = scandir($folder);
  foreach($dircontent as $filename) {
    if (strlen($filename)==32 && filemtime($folder.$filename) && filemtime($folder.$filename)<$olddate) unlink($folder.$filename);
  }
}

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

header('Content-type: text/javascript');
header('Expires: '.gmdate("D, d M Y H:i:s", time() + 3600*24*365).' GMT');

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  if(isset($_REQUEST['minify'])){
    require 'jsmin-1.1.1.php';
    $js=JSMin::minify($js);
    file_put_contents($writabledir.$name,$js);
    delete_old_md5s($writabledir);
    exit;
  }
  else{
    $js.="setTimeout(function(){var a=document.createElement('img');a.src='all.php?minify=1';a.style.display='none';document.body.appendChild(a);},5000);";
  }
  echo $js;

A pseudo-code version of the above would be:

if file named after md5 of scripts directory exists, echo it and exit.
else{
  concatenate requested scripts into one large string.
  if request URI does not have "minify" as a parameter{
    add a javascript instruction to the string to load this script again with the minify parameter in 5 seconds
    print the string and exit.
  }
  else{
    minify the string.
    save the string to a file named after the MD5
    exit without printing anything (no reason to send this to the client)
  }
}

Fresh from the success of their recent release, Ghosts, which Bronwyn has been playing non-stop since our CD arrived (and for weeks before using the MP3s downloaded from the Interweb), Nine Inch Nails has released a new album and gone even further.

Yes, this album is totally free for you to download. All they ask is that you sign up for it via their site. However, as the album is licensed via Creative Commons Non-Commercial, it is also legal for you to download via bittorrent if you prefer.

The album is available in MP3, M4A, FLAC and even 24/96 WAV (for freaks that want a 1.2GB album taking up space).

NIN are recommending that people take the album and produce remixes from it. (That site doesn’t work well in Firefox in Linux btw - fix it, please - remember I bought a ticket for your concert in the RDS way back in ‘96? Go on Trent, be a pal!)

As for the sound, I think it’s more accessible than Year Zero (although I think over time, I will prefer Year Zero) on a first listen. There is the usual layered approach to the music, where textured “landscapes” are drawn up. Interestingly, in some tracks, the layers appear to be deliberately off-time from each other (Echoplex, for example).

…doesn’t work properly (at least on my laptop, using a GeForce 7000M). the reason for this is that Fedora 9 uses the latest version of xorg and NVidia’s driver does not yet work with it (3D is not fully working).

To get stuff at least minimally working, you need to downgrade your xorg RPMs to the Fedora 8 versions.

Unfortunately, KDE4 appears to require fully working GLX in order to display the kicker (or whatever the hell they’re calling it now), so my KDE is slightly crippled at the moment. A pity, really, as KDE4 is beautiful and almost as usable as 3.5 was.