What I’ve been up to

Being self-employed is hard work!

Naive person that I am, I thought it would be easy enough – I get to work on my own ideas all day long, occasionally selling something to customers who are happy every time.

Of course, that’s not true.

I spend a lot of my time working on jobs for my previous employer, and some other clients that have known me for years. The large pool of new clients that I expected would magically appear, never magically appeared.

Of course, it’s not all doom and gloom.

Being out on my own has taught me quite a lot, and in a very short time.

  • Cash is king – there’s no point having a huge job which may potentially pay you thousands of euro in two months, when you need the money right now to go buy food for the house.
  • Advertising is not easy! I hate to pester, so advertising is not a natural thing for me, but it’s tricky to get new people to come in and buy from me if they don’t know where I am. Especially if they don’t know whether to trust my work! Conveying trust in ads is not a very easy thing to do.
  • Word-of-mouth is a brilliant thing. It is much easier to sell to someone if they’ve already been half-sold by their friends.
  • Low prices don’t always pull people in. Even though I charge about a fifth of other local web designers, I think I must also be getting only a fifth of the customers ;-)
  • Salary is a wonderful thing, when it’s someone else’s job to make sure it gets paid.

Of course, there are also the upsides.

I get to work on my own projects without worrying that they my conflict with the company’s plans. Basically, I am the company, so anything I want to work on is the company plan.

To that end, I’ve created 20eurowebsites.com, a site-creator which is built on my kvWebME CMS (open source, PHP – download it yourself if you’re techie!). €20 to buy a website (including the domain name), and €10 per month hosting after that.

I have other plans in the works as well, such as a local odd-jobs finder which has a pretty good twist, and a free face-book-like chat application which people will be able to add to their own websites with just a few keystrokes.

multi-tenanted CMS architecture

Last week, I did a talk at the Dublin Google buildings titled “Multi-tenanted CMS Architecture using PHP”.

Here are the slides that I used:

While talking with Google’s Brian Brazil, he explained that it is actually more efficient to use one database and many separate tables, than to separate each installation into a separate database, so one point I made (that KV-WebME uses separate databases per site) will change in the future.

I think the talk went down well, by the number of questions afterwards.

Last year, I gave a similar talk, and made the mistake of including way too much PHP in it – I had assumed that the audience would be composed of PHP developers. This year, there is just one slide of PHP, and that’s just to illustrate one possible way to build a proxy config.

Lesson’s learned for this time:

  • Talk slower. When I’m explaining something, I tend to try to get as much in as possible, so speak very fast. This makes it hard to hear what I’m saying.
  • More pictures, less words!
  • Stats. Some of the questions were around how efficient certain parts of the method were – particularly on the overhead of piping a file through a script as opposed to simply delivering it via Apache. I need to come up with numbers for that.

Overall, I was happy with this presentation.

script for shrinking videos

For this tip, you’ll need mencoder, ffmpeg and php installed. Only tested on Linux.

I’ve a load of videos, because I don’t like deleting anything in case I want to watch it again later (hoarding!).

I’ve three 1TB drives stuffed full of what I’ve collected over the last 20 years, but still I keep running out of room.

So, choice – buy more drives, or shrink what I have? I chose to shrink what I have, because I’m totally happy with VHS-quality video in stereo – I don’t need to watch fan-subbed Bleach in HD in 5.1 surround sound!

So, I wrote a script which will search the current directory (and its sub-directories) for all videos, will figure out their bit-rates, and will shrink them if they’re too large.

The script also reduces the videos so they match a maximum width and height.

It’s written in such a way that it will start with the most wasteful files (those that are taking up the most room), and work from that to the least wasteful.

It’s also written to only do 100 videos in a run. There’s no point shrinking absolutely every video you have, as most will probably save only a few kilobytes. The script does 100 of the most wasteful ones.

Of course, you could always run it again and again afterwards, and it should choose different videos each time to shrink.

How to use: copy the file into the root directory of your video repository. Go to that directory. Type “php shrink.php” into the console. Now go do something for a few days. Simple as that!

<?php

$maxwidth=624;
$maxheight=352;

function list_files($dir, $list) {
  $files=new DirectoryIterator($dir);
  foreach ($files as $f) {
    if ($f->isDot()) {
      continue;
    }
    $fname=$f->getPathname();
    if (is_dir($fname)) {
      $list=list_files($fname, $list);
      continue;
    }
    if (in_array(
      strtolower(preg_replace('/.*\./', '', $fname)),
      array('avi', 'mpg', 'mp4')
    )) {
      if (preg_match('/-shrink.avi$/', $fname)) {
        unlink($fname);
        continue;
      }
      $arr2=array();
      // { get duration of video
        exec('ffmpeg -i "'.$fname.'" 2>&1 | grep Duration', $arr2);
        $bits=explode(':', preg_replace('/.*Duration: ([^,]*),.*/', '\1', $arr2[0]));
        if (count($bits)!=3) {
          echo 'failed to get duration of "'.$fname.'"'."\n";
        }
        $duration=(int)preg_replace('/^0/', '', $bits[2])
          +((int)preg_replace('/^0/', '', $bits[1]))*60
          +((int)preg_replace('/^0/', '', $bits[0]))*3600;
      // }
      // { get width/height of video
        exec('ffmpeg -i "'.$fname.'" 2>&1 | grep "Video:"', $arr2);
        $bits=explode('x', preg_replace('/.*\s([0-9]+x[0-9]+)[\s,].*/', '\1', $arr2[1]));
        if (count($bits)!=2) {
          echo 'failed to get dimensions of "'.$fname.'"'."\n";
          continue;
        }
        $width=$bits[0];
        $height=$bits[1];
      // }
      $fsize=filesize($fname);
      $bps=$fsize/$duration;
      echo '.';
      $list[]=array($bps, $fsize, $width, $height, $duration, $fname);
    }
  }
  return $list;
}

echo "\ngetting file details\n";
$list=list_files('.', array());

echo "\nsorting by bits per second\n";
for ($i=0; $i<count($list)-1; ++$i) {
  echo '.';
  for ($j=$i+1; $j<count($list); ++$j) {
    if ($list[$j][0]>$list[$i][0]) {
      $tmp=$list[$j];
      $list[$j]=$list[$i];
      $list[$i]=$tmp;
    }
  }
}

echo "\nshrinking\n";
for ($i=0; $i<100&&$i<count($list); ++$i) {
  $fname=$list[$i][5];
  echo $fname."\nold size: ".$list[$i][1]."\n";
  @unlink('divx2pass.log');
  $resize='';
  if ($list[$i][2]>$maxwidth || $list[$i][3]>$maxheight) {
    $rx=$maxwidth/$list[$i][2];
    $ry=$maxheight/$list[$i][3];
    $ratio=$rx<$ry?$rx:$ry;
    $width=(int)($list[$i][2]*$ratio);
    $height=(int)($list[$i][3]*$ratio);
    $resize='-vf scale='.$width.':'.$height;
  }
  exec('mencoder "'.$fname.'" -ovc xvid -xvidencopts bvhq=1:chroma_opt:quan'
    .'t_type=mpeg:bitrate=400:pass=1 '.$resize.' -alang en -oac copy -o "'
    .$fname.'-shrink.avi" 2>&1');
  exec('mencoder "'.$fname.'" -ovc xvid -xvidencopts bvhq=1:chroma_opt:quan'
    .'t_type=mpeg:bitrate=400:pass=2 '.$resize.' -alang en -oac mp3lame '
    .' -o "'.$fname.'-shrink.avi" 2>&1');
  echo 'new size: '.filesize($fname.'-shrink.avi')."\n\n";
  if (filesize($fname.'-shrink.avi')<$list[$i][1]) {
    unlink($fname);
    rename($fname.'-shrink.avi', preg_replace('/\.[^\.]*$/', '.avi', $fname));
  }
  else {
    unlink($fname.'-shrink.avi');
  }
}

2010

I’ve the most awful memory.

While trying to remember what the hell I’d done in the last year, I came up with nothing.

Luckily, I have a spare brain in the form of my facebook friends, who came up with this list for me:

  • I started a new company, KV Sites, which will be up and running properly within a month or so, and will be selling affordable CMS websites and programming.
  • I got grade 2 in piano. I’m still waiting for an examiner for grade 3 (which I wanted to do in September). I’ll be doing grade 4 in March.
  • I got my first grading in Genbukan Ninjutsu.
  • I finished another book, CMS Design using PHP and jQuery. I hope it is as well-received as the previous book, jQuery 1.3 with PHP. btw, Packt would like me to remind people that the book “Mastering phpMyAdmin [...] for effective mysql management” (reviewed here and here) has been updated to version 3.3.x.
  • I am building up to a new release of my CMS, WebME, which, despite the last downloadable version being from early 2009, has actually been very actively updated. It should be ready for release tomorrow, right on time for 2011!
  • I wrote and released two jQuery plugins: k3dCarousel, and SaorFM (which I hope to vastly improve in 2011).
  • I also built a first attempt at a clavichord made from plywood. I’ve got some new tuning pegs and redesigned the keyboard, so will hopefully be able to record on it soon.

I’m hoping 2011 turns out to be awesomer and that my head will be able to remember it all!

jQuery stars plugin

I was asked to replicate a “star” effect, where stars appear in various areas around a page and then disappear after a while. I won’t bother linking to the original site as it will be gone shortly, but this is what I came up with:

demo

To use this on your own site, simply download the script, link to it in your page, then add this piece of JavaScript.

$('body').stars();

If you want to use the star image I created, download it to the same directory and tell the plugin where it is:

$('body').stars({
  "i":"stars.png"
});

clavichord fretting

My clavichord project stalled when I realised it was just not going to work the way I’d done it.

This is partly because I’d naively gone for the full un-fretted design in the beginning, then later realised this would put too much pressure on the cheap bodywork and cause it to implode.

Changing the design afterwards to a fretted design wasn’t going to work either, because of how the keys were laid out.

So, the new plan is to rebuild the keyboard on the current clavichord, and hopefully get the thing finished as a triple-fretted, single-strung design.

Now, to explain…

If you have one string per note, this is called “unfretted”. In this design, every string is only ever hit by only one key.

The usual way to design a clavichord is “double-strung”, this means that every note actually has two strings for it. This makes double-strung clavichords louder than single-strung clavichords, because the combination of the two strings’ waves tends to alternately strengthen and dampen what’s happening at the soundboard.

Back to fretting – consider a guitar. Despite only having six strings, a guitar can play many more than only six notes. This is accomplished by “fretting” the strings. When you play a “G” on an “E” string, what happens is that you are shortening the wavelength of the overall string (with the fret and your finger), causing it to play a different note (G) than it would play if it was unfretted (E).

With a clavichord, the very act of striking the string with the key (or “tangent”, as the striking edge is called) causes fretting. The strings are damped at the ends with cloth or felt so when the key is not touching the string, the string doesn’t vibrate.

When you design your keyboard to multi-fret the strings, you need to do some calculation – let’s say you have a note, C, which is struck on the strings 100cm (let’s say) from the bridge. If your fretting involves the C# hitting the same string, then that key’s tangent must hit the string at about 94.4cm.

This is quite a small distance between the two tangents (5.6 mm), meaning that if you decide to triple-fret all your notes, then the keys for the high notes will be very close together, and the lower notes will be further apart (lower notes have larger wavelengths, so the distance between semi-tone frets increases as well as you get lower).

That explains the following image (a double-strung, triple-fretted clavichord – click for a larger image):

Note that the keys are all squashed together on the right side where the high notes are, and the spaces gradually increase as you move further left.

Notice as well that at the extreme left, the increase in spacing stops and all the keys are together again.

The reason for this is that when the notes get too low, there’s simply no more room for multi-fretting, so instead, the lower notes are all one per string.

There’s one more point to make about the keys.

Let’s say you create a key, which has its tangent 25cm from the fulcrum (a clavichord key is a lever). When the key is pressed, the tangent arcs up and strikes the string. It is still 25cm from the fulcrum in a 3D sense, but when measuring x/y from a top-down view of the clavichord, if the string is 4cm above the tangent (with key at rest), then the tangent strikes the string about 22.5cm from the fulcrum.

This must be taken into account when you design where the strings will contact the bridge and the hitchpins, as getting this wrong will cause the tangents to miss. Yes, you could just place the tangents after doing the strings, but my goal here is to be as perfect as possible. (there’s also the added problem that the tangent’s top is a certain height (3cm, say) above the level of the fulcrum, but you get the picture)

I’ve explained some of the problems to do with designing a fretted keyboard and string layout. Now, I’m off to write a program to design one automatically!

3d carousel bug-fix and slight improvement

plugins.jquery.com page.

I’ve improved the plugin slightly. It now pauses when you mouse-over it, I’ve improved the left/right arrows, and I’ve fixed a bug to do with image aspect ratios.

I’ve yet to upgrade the demo. See here for the original description of the plugin

jQuery k3dCarousel plugin

I needed a “3D” carousel to rotate images. The currently available jQuery plugins for this purpose were either too large, too restricted (showing only 4 at a time is not good enough), or too old (not working in jQuery 1.4+).

So I wrote my own.


demo

In our CMS, WebME, this is available in the Image-Transition plugin. (If you’re interested in using WebME, you can either download it through SVN, or talk to us about our reseller deals.)

How the thing works, is that you add a list of images to an element like this:

<div id="k3dCarousel">
	<img src="images/concrawood.jpg" />
	<img src="images/fernheasty.jpg" />
	<img src="images/mcnallys.jpg" />
	<img src="images/northernstandard.jpg" />
	<img src="images/soccertours.jpg" />
	<img src="images/soundinsulation.jpg" />
	<img src="images/textaroom.jpg" />
	<img src="images/windhoist.jpg" />
</div>

Then link to the jQuery library and the jquery.k3dCarousel plugin:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script src="jquery.k3dCarousel.min.js"></script>

And then tell jQuery to run the plugin after all the images have loaded:

<script>
	$(window).load(function(){
		$('#k3dCarousel').k3dCarousel();
	});
</script>

If you want, you can vary the speed of the cycle by changing the sT (spin-time) variable, or the wT (wait time), which govern how long it takes for the images to move from point to point, and how long they pause once reaching there:

<script>
        $(window).load(function(){
                $('#k3dCarousel').k3dCarousel({
                        wT: 500,
                        sT: 100
                });
        });
</script>

Interested yet? Download it (tar.bz2, zip)

SaorFM progress – the Hidden Files plugin

In the last post, I mentioned that a jQuery plugin had been created.

At the time, it could only be used as a file or directory selector.

It can now also be used to upload, download, and delete files.

I’ve also enhanced the core engine so that it now has plugin support.

The first plugin is a Hidden Files plugin, which lets you hide files or directories depending on their names. The default pattern is to hide nodes with names that begin with a ‘.’ – this is the default for most systems out there.

I’ll explain how plugins work, in the hope that someone out there may be interested in creating one ;-)

For this example, I’ll use the Hidden Files plugin, as it’s already created, and uses everything in the engine so far.

First off, you create a directory in the /plugins/ directory of the plugin. In this case, I created the directory /plugins/hidden-files/.

In that, the only essential file is config.php, which holds a configuration for the plugin:

<?php
$config=array(
  'triggers'=>array(
    'checkfilename'=>'HiddenFiles_checkFilename'
  )
);

When you tell SaorFM to install the plugin (using $SaorFM->installPlugin('hidden-files');), it will read through that array and add the details to its own configuration file, so it doesn’t have to open up multiple files every time it boots up.

As an example, here’s my test config before the plugin is installed:

<?php
$SaorFM_config='{"user_files_directory":"\/home\/kae\/websites\/saorfm\/trunk\/tests\/\/userfiles"}';

And here’s how it appears after the installation of the plugin:

<?php
$SaorFM_config='{"user_files_directory":"\/home\/kae\/websites\/saorfm\/trunk\/tests\/\/userfiles","plugins":["hidden-files"],"hiddenfiles":"#^\/?\\.#","triggers":{"checkfilename":["HiddenFiles_checkFilename"]}}';

There are three additions:

  • "plugins":["hidden-files"] – this is an array of installed plugins.
  • "triggers":{"checkfilename":["HiddenFiles_checkFilename"]} – the $config array above had a trigger mentioned – that’s converted into an object and added to the SaorFM config.
  • "hiddenfiles":"#^\/?\\.#" – this is to tell the hidden files plugin what pattern to apply to file names to decide whether they should be hidden or not.

The first two are obvious – the first is the name of the plugin, and the second is the name of a trigger named in the config array.

For the third, we need to run an installation script when Hidden Files is installed, so SaorFM checks the /plugins/hidden-files/ directory to see if a file exists named install.php:

<?php
if (!isset($this->_config->hiddenfiles)) {
  // the default pattern hides files that begin with .
  $this->_config->hiddenfiles='#^/?\.#';
}

That’s run by SaorFM when the plugin is installed. It adds a default hiddenfiles value to the config. This can be changed at any time afterwards.

Ok – so we have the config installed – now how does it work?

Let’s say we ask SaorFM to list the files in ‘/’: $SaorFM->listFiles('/');

Inside the listFiles method, there is a loop which adds filenames to an array. For each one of the filenames, we check it before adding to the array:

    foreach ($files as $f) {
      if ($f->isDot()) {
        continue;
      }
      if ($this->trigger('checkfilename', $f->getFilename())) {
        continue;
      }
      $arr[$f->getFilename()]=array(
        $f->getFilename(),
        $f->getSize(),
        $f->isDir()?1:0 // record as 1 or 0 to save http traffic
      );
    }

Notice the $this->trigger('checkfilename', $f->getFilename()) – if that returns anything other than false, then the file is not added to the list.

The trigger method cycles through any functions which have been named for that trigger, and calls each one in turn. If any return a result other than false, the method returns that result immediately:

  public function trigger($trigger, $vals) {
    if (!isset($this->_config->triggers->{$trigger})) {
      return false;
    }
    foreach ($this->_config->triggers->{$trigger} as $function) {
      $ret=$function($this, $vals);
      // if the trigger gives any response other than false, then return it
      if ($ret) {
        return $ret;
      }
    }
    return false;
  }

Note that we haven’t yet defined the function HiddenFiles_checkFilename.

Upon installation, we also load up the file /plugins/hidden-files/functions.php if it exists, and this file is loaded every time that SaorFM is instantiated after installation as well:

function HiddenFiles_checkFilename(&$saorfm, &$filename) {
  return preg_match($saorfm->get('hiddenfiles'), $filename);
}

This is a very very simple function – it could get a lot more complex in other plugins. For example, an Authentication plugin would be much more verbose.

I hope this is all very clear. It’s very very simple to add arbitrary triggers to SaorFM, so if you want to write a plugin and the trigger doesn’t exist, just tell me and I’ll put it in place.

After all this, let’s say you want to uninstall the plugin.

Within SaorFM, you just call $SaorFM->uninstallPlugin('hidden-files');.

This automatically removes the ‘hidden-files’ value from the plugins array (and removes that array if it’s empty), then removes any added filters as well.

Finally, it also checks to see if /plugins/hidden-files/uninstall.php exists and runs that if so:

<?php
unset($this->_config->hiddenfiles);

After this, the SaorFM config file is back to the exact state it was in before we installed the plugin. Clean.

If you’re interested in hacking on SaorFM, please download a copy from SVN here.

There are a number of things we need to create before it’s usable by the general public. An installer, for example, and some widgets for popular RTEs like CKeditor or TinyMCE. If you feel you’d like to create those, please contact me.

saorfm: first visible results

The first visible project using SaorFM is a jQuery file-selection widget:


demo

Here is the source of that page:

<html>
	<head>
	</head>
	<body>
		<p>Select a file</p>
		<input id="file-select" />

		<link rel="stylesheet" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/base/jquery-ui.css" type="text/css" />
		<link rel="stylesheet" href="/saorfm/jquery-widgets/jquery.saorfm/jquery.saorfm.css" type="text/css" />

		<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js"></script>
		<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/jquery-ui.js"></script>
		<script src="/saorfm/jquery-widgets/jquery.saorfm/jquery.saorfm.js"></script>
		<script>
			$(function(){
				$('#file-select').saorfm({
					"rpc":"/saorfm/core/rpc.php"
				});
			});
		</script>
	</body>
</html>

In the above demo, we link in the jQuery and jQuery-UI scripts, and load up the widget’s script.

The only content elements are a <p> telling the user to choose a file, and a normal text <input> box which the user can use as a combobox.

The magic happens with this little bit of JavaScript:

$(function(){
	$('#file-select').saorfm({
		"rpc":"/saorfm/core/rpc.php"
	});
});

That tells jQuery to convert the #file-select input box into a saorfm widget. The only parameter in this demo is rpc, which tells the widget where the SaorFM server is to be found online.

To install this on your own server, all you need to do is download the SaorFM engine from Google Code, and add a config.php file to the core directory. Here’s mine:

<?php
$SaorFM_config='{"user_files_directory":"\/home\/verens\/domains\/demo.verens.com\/saorfm","language":"en","json_errors":false}';

In that, a JSON-encoded hash object contains the configuration. Change the user_files_directory parameter to your own files directory, and you are done with the configuration!

I’ll be adding this to my WebME project next week, after I’ve finished writing the tests (Selenium and PHPUnit).