klog

Here’s a method to create minified JS files on-the-fly, without incurring the time cost of the minification process.

To explain, minification is the act of removing all whitespace from JavaScript so the file can be downloaded and parsed as quickly as possible.

A useful part of minification is that during the act of compiling your minified source, you can also pull in other JavaScript files and compiled them all into one single source. This has a major advantage that there is only one file to download. This is very important because even if those small files were small, the lag involved in sending multiple HTTP requests can sometimes be worse than if a single file which was twice the size of all the small files in total was downloaded. (which is why good web devs use Sprites when possible)

Right. That’s the explanation of /why/ to do it. Now how?

First off, here’s a very simple piece of source to show it (in a file called “all.php”).

  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  echo $js;

You can get jsmin-php from here. the other files are JQuery files but only used for illustrative purposes.

Anyone spot the problem? It does the job, yes, but the minify() script takes a few seconds to run, ruining the advantage it created with the minification.

The next step towards perfection is to cache the file. The obvious solution to that is to save the file in a location. If the file exists when the files are requested next, then serve the cached version instead of going through the minification process.

That’s not good enough, though. What if a file in the list was changed? You’d never know because the cached file will always be served.

The solution is to save the cached file in a file named using an MD5 of the last modified datetimes (genius idea - I’d love to shake the author’s hand). That way the cache will always be correct, and will automatically update itself when a file is changed.

$writabledir='/path/to/caches/';

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  require 'jsmin-1.1.1.php';
  $js=JSMin::minify($js);
  file_put_contents($writabledir.$name,$js);
  echo $js;
}

Better. This time, the delay is only on the first load. All subsequent loads will have an instant download of the cached file.

However, if you’re a developer, then almost every reload will involve the minification process - you’d never get your work done.

A solution is to check to see if the current MD5 cache exists, and if it doesn’t, then download the files as a bundle without minifying them. Instead, before sending the files, you tag on a little bit of javascript which will do the minification in the background after the script has been sent to the client.

Here is a complete solution, along with a little cleanup which removes caches older than an hour.

$writabledir='/path/to/caches/';

function delete_old_md5s($folder) {
  $olddate=time()-3600;
  $dircontent = scandir($folder);
  foreach($dircontent as $filename) {
    if (strlen($filename)==32 && filemtime($folder.$filename) && filemtime($folder.$filename)<$olddate) unlink($folder.$filename);
  }
}

function md5_of_dir($folder) {
  $dircontent = scandir($folder);
  $ret='';
  foreach($dircontent as $filename) {
    if ($filename != '.' && $filename != '..') {
      if (filemtime($folder.$filename) === false) return false;
      $ret.=date("YmdHis", filemtime($folder.$filename)).$filename;
    }
  }
  return md5($ret);
}

header('Content-type: text/javascript');
header('Expires: '.gmdate("D, d M Y H:i:s", time() + 3600*24*365).' GMT');

$name=md5_of_dir('./');
if(file_exists($writabledir.$name))readfile($writabledir.$name);
else{
  $js=file_get_contents('jquery-1.2.3.min.js');
  $js.=file_get_contents('jquery.dimensions.pack.js');
  $js.=file_get_contents('jquery.impromptu.js');
  $js.=file_get_contents('jquery.iutil.pack.js');
  $js.=file_get_contents('jquery.idrag.js');
  $js.=file_get_contents('jquery.grid.columnSizing.js');
  $js.=file_get_contents('jquery.tablesorter.js');
  if(isset($_REQUEST['minify'])){
    require 'jsmin-1.1.1.php';
    $js=JSMin::minify($js);
    file_put_contents($writabledir.$name,$js);
    delete_old_md5s($writabledir);
    exit;
  }
  else{
    $js.="setTimeout(function(){var a=document.createElement('img');a.src='all.php?minify=1';a.style.display='none';document.body.appendChild(a);},5000);";
  }
  echo $js;

A pseudo-code version of the above would be:

if file named after md5 of scripts directory exists, echo it and exit.
else{
  concatenate requested scripts into one large string.
  if request URI does not have "minify" as a parameter{
    add a javascript instruction to the string to load this script again with the minify parameter in 5 seconds
    print the string and exit.
  }
  else{
    minify the string.
    save the string to a file named after the MD5
    exit without printing anything (no reason to send this to the client)
  }
}

For those that don’t know, KFM (Kae’s File Manager) is an application which allows a user to edit/move/delete files online. With some plugins, you can even view videos, listen to your music files, etc.

There are a number of interesting improvements in 1.3.

The configuration methods have been improved. To set up your new KFM, simply copy configuration.dist.php to configuration.php, remove the bits you won’t be changing, and edit what’s left. As an example, here is my entire configuration.php:

<?php
$kfm_db_type = 'sqlitepdo';
$kfm_userfiles_address = '/home/kae/websites/kfm/1.3/files';
$kfm_userfiles_output = '/get.php';

Plugins!

The point of plugins is to enable actions for various file types. We have plugins for the default actions such as image resize, editing text with codepress, etc. You can download new plugins from here to supplement the default ones (I recommend jwplayer and malsup_media_player) - just unzip them in your /plugins/ directory and refresh your KFM.

The most important part about plugins is that they are easy to write. You don’t need to understand all of KFM to write your own plugin. Just copy an existing plugin that’s similar to what you want to do, and then edit it until it’s like what you wanted.

Another important part is that it helps to keep the size of KFM’s core down. We are aware that KFM is “bloating”, and are trying to keep an eye on that while still increasing its capabilities.

The list-view has been improved, along with a vast number of other things.

We have more new languages - Swedish (thanks to Aram Mäkivierikko), Persian (thanks to Ghassem Tofighi) and Hungarian (thanks to Ujj-Mészáros István).

There is a new strings translator on the way in 1.4, along with a new method of submitting translations. I’ve learned a lot in the last few months and feel I can do this better so I’m rewriting the translation methods. I’ll announce more as it becomes more complete.

A lot of work went into allowing KFM to be templated. This is not quite mature yet, so I won’t say too much about it, but it is now possible to design KFM the way you want it. If you have a design you’d like KFM to fit into, please contact me so I can help out. Documentation will come soon.

To download KFM 1.3, get the tar bzip (840k) or the zip (1M).

Please report any bugs to the usual place.

Packt publishing emailed me a week ago and asked me if I was interested in writing a review of a book on phpMyAdmin.

In short, my review is this: overall, the book detailed everything I can think of (and more) about phpMyAdmin - I learned about some features in phpMyAdmin that I was not already aware of. I would ask for less screenshots, more how-tos and less “we/our” speech for the next revision, but if there are any questions I have about phpMyAdmin, the answers are in this book.

I use phpMyAdmin only occasionally, when I can’t get direct access to the database in question (I’m a console jockey - there’s nothing like the thrill of tappity-tapping away, knowing that you’re doing Important Stuff and …well, yeah - geek).

I’ve used phpMyAdmin since forever, so was familiar with its common uses - importing/exporting databases, running simple queries, etc. I was interested to see if there was anything that I might have missed. And who better to show me than one of the developers, Marc Delisle.

The table of contents was large, spread over 12 pages. I feel it would have been easier to read if there was one page detailing just h1 listings, followed by 2-3 pages of h2 listings. The h3 and h4 listings could have been shown at the beginning of the concerned chapters.

Disregarding that, scanning the contents showed some features that I was not aware of. Did you know that phpMyAdmin has a built-in relations manager? Even if the engine used is MyISAM, which does not manage referential integrity, phpMyAdmin will protect you and manage it for you if queries are run through it.

Screenshots are provided for everything. This makes some things much easier to understand. Sometimes it made interesting features stand out even more. Did you know that phpMyAdmin can generate PDFs of the relational schema of a database? Some screenshots in chapter 15 show examples of this feature.

Marc obviously knows his stuff. As a major contributor to phpMyAdmin since the early days, he is an ideal person to talk us through configuration details and the various tools that phpMyAdmin offers.

There are, however, a number of things which detract from the readability of the book.

This is the first edition for this title, so the following are probably “teething” problems for the author. However, I think they should be taken care of for the second edition, to make the book easier to read. Although, the second edition would most likely have a different title (why is the version number in the title?)

The book is written from a 1st-person-plural point of view - now we do this, now we do that, our table, our server. This distracted and annoyed me while reading the book.

It is too detailed in places, at the expense of readability. For example, chapter one goes through a blow-by-blow account of how phpMyAdmin came to be, with version numbers and even dates! I feel this should have been left to an appendix. The first chapter of any book should introduce a reader gently to the subject; not pummel the reader with historical facts.

The author sometimes forgets that the reader does not know the subject and its community’s habits. As an example, on page 28 the configuration value “PmaAbsoluteUri” is described, but afterwards, I was not sure a) what “Pma” is, or b) what the variable was supposed to do (the book described how it should be configured, but not what it was supposed to do). It isn’t until page 33 that the book pointed out that “pma” is an acronym for “phpMyAdmin” used by the community. I don’t feel stupid for not getting this immediately, as there is an awkward mixture of letter-cases going on there. By rights, “Pma” should stand for “Phpmyadmin”, not “phpMyAdmin”.

The sheer number of screenshots was a bit confusing. I feel that screenshots should only be used where the subject is not clear from the text. Two images on every page (almost) has the effect of breaking your concentration when you’re trying to read.

The book exclusively deals with how to use phpMyAdmin, but I think there should have been more about how to deal with MySQL itself, using phpMyAdmin. For example, chapter 17 details phpMyAdmin’s coverage of character sets and collations. As a person who knows the pain of upgrading a database from MySQL 4 to MySQL 5 while switching character sets, I was looking forward to seeing how this case is handled (which I am sure is pretty common in one form or another). However, the chapter describes the problems with character sets and does not offer any solutions to those problems.

Overall, the book was detailed, and I learned a lot of new stuff from it. I’d recommend it for administrators that are not comfortable with CLI MySQL and want a reference for the best web-based MySQL administration tool out there.