27 Oct

what am I up to?

My wife is in hospital with an absess on her eye, so I’m stuck at home minding Jareth, my 3yo son.

I don’t have time to work on KFM at the moment, but work is progressing anyway – Benjamin Ter Kuile has been hacking away at it feverishly – he’s currently working on the Image code. We plan to have all thumbnails contained within the sqlite database for version 0.7. I received an Italian language file from Stefano Luchetta, which, together with Hannu Nemi’s Finnish translation, makes a total of ten languages!

I’m busy with a rather large project which I can’t reveal details about, but can say that it involves Perl and a few different DB versions. Perl is a bit of a bugger, after you’ve been used to PHP, and PostgreSQL is just a pain in the behind…

I translated my Kaejax framework from PHP into Perl last night, after much hardship, and am currently strugging with JavaScript – a previous developer on this project made a decision to use the Prototype JS library. That’s not a bad decision, but it is interfering with my own scripts such that some simple things are not working as expected.

All in all, I’m incredibly busy. Thank $DEITY monday is a bank holiday, as I think I’ll need it, in order to catch up with everything!

19 Oct

pdo::sqlite gotcha

This one caught me a few days ago, but I didn’t have to time to concentrate on it (I was being harassed by screaming kids at the time). I came across the same problem today, and managed to figure it out with the help of some useful prompts from the ILUGgers.

The problem manifests as a query failing to run for no apparent reason.

An example of some code that exhibits the bug (taken from the KFM project:

function _moveDirectory($from,$to){
	global $db;
	$q=$db->query('select * from directories where id="'.$from.'"');
	$from=$q->fetch();
	$q=$db->query('select * from directories where id="'.$to.'"');
	$to=$q->fetch();
	if(strpos($to['physical_address'],$from['physical_address'])===0)return 'error: cannot move a directory into its own sub-directory'; # TODO: new string
	if(file_exists($to['physical_address'].'/'.$from['name']))return 'error: "'.$to['physical_address'].'/'.$from['name'].'" already exists'; # TODO: new string
	rename($from['physical_address'],$to['physical_address'].'/'.$from['name']);
	if(!file_exists($to['physical_address'].'/'.$from['name']))return 'error: could not move directory'; # TODO: new string
	$len=strlen(preg_replace('#/[^/]*$#','',$from['physical_address']));
	$fugly='update directories set physical_address=("'.addslashes($to['physical_address']).'"||substr(physical_address,'.($len+1).',length(physical_address)-'.($len).')) where physical_address like "'.addslashes($from['physical_address']).'/%" or id="'.$from['id'].'"';
	$db->exec($fugly) or die('error: '.print_r($db->errorInfo(),true));
	$db->exec('update directories set parent="'.$to['id'].'" where id="'.$from['id'].'"') or die('error: '.print_r($db->errorInfo(),true));
	return _loadDirectories(1);
}

The problem is that the SQL statement $fugly (so named because it’s fucking ugly) and the following one do not run, and return an error saying the database is locked. This is despite the fact that earlier in the same function, we’ve read from the database with no problems.

After much banging of heads, I found the problem – the variable $q holds a lock on the database, but it’s a read lock. In order to change to ‘write’ mode, you need to free up that variable before you attempt the write.

function _moveDirectory($from,$to){
	global $db;
	$q=$db->query('select * from directories where id="'.$from.'"');
	$from=$q->fetch();
	$q=$db->query('select * from directories where id="'.$to.'"');
	$to=$q->fetch();
	$q=null;
	if(strpos($to['physical_address'],$from['physical_address'])===0)return 'error: cannot move a directory into its own sub-directory'; # TODO: new string
	if(file_exists($to['physical_address'].'/'.$from['name']))return 'error: "'.$to['physical_address'].'/'.$from['name'].'" already exists'; # TODO: new string
	rename($from['physical_address'],$to['physical_address'].'/'.$from['name']);
	if(!file_exists($to['physical_address'].'/'.$from['name']))return 'error: could not move directory'; # TODO: new string
	$len=strlen(preg_replace('#/[^/]*$#','',$from['physical_address']));
	$fugly='update directories set physical_address=("'.addslashes($to['physical_address']).'"||substr(physical_address,'.($len+1).',length(physical_address)-'.($len).')) where physical_address like "'.addslashes($from['physical_address']).'/%" or id="'.$from['id'].'"';
	$db->exec($fugly) or die('error: '.print_r($db->errorInfo(),true));
	$db->exec('update directories set parent="'.$to['id'].'" where id="'.$from['id'].'"') or die('error: '.print_r($db->errorInfo(),true));
	return _loadDirectories(1);
}

The conclusion is that you can make a load of read requests in a row, or a load of write requests in a row, but if you are mixing the query types, you need to free the result each time.

08 Oct

kfm acquires a search engine

Over the last few weeks, I’ve gradually been shifting the KFM project over to using sqlite for its file meta-data organisation, a database engine which keeps everything in one file and doesn’t require any pesky authentication.

The logical next step was a search engine, so I wrote that today – and wrote some workarounds for SQLite – my home version is 3.3, and work server runs 3.1.3. 3.1.3 doesn’t support auto_increment, which bloody annoyed me.

demo
kfm screenshot

At the bottom left, open up Search, and type “jpg”. No need to press “enter” – it’s a live search – it will ask for the results 500ms after your last keypress. You’ll get a result set of all files with “jpg” in the name.

That’s one of the major new things for KFM 0.6, which I hope to push out the door by next weekend.

Thanks to Benjamin ter Kuile for his help in getting the system to work on servers that don’t have PDO-SQLite built in – he supplied a different flat-file database. I haven’t seen it in action, but it works on his server (which is running MacOSX!)

Also, thanks to Vse Do Feni, who supplied a Russian translation! It looks strange to see KFM with Cyrrilic characters.

The next few weeks should be interesting. I’ll be implementing tagging very soon. KFM will be the most advanced ajax filemanager in existance!

30 Sep

materials arrived for my college course

Yesterday, a package arrived for me from Open University – a load of handbooks covering diverse database subjects, and a cd with copy of IBM’s DB2 on it.

Unfortunately for me, they seem to have neglected the Linux installation of DB2. It’s mentioned in the readme.txt file, but the installation files are simply not there. I feel a complaint coming on.

So, yeah… me doing college!? My highest qualification so far was my Junior Certificate. I didn’t bother with the Leaving Cert. – got bored, then got a job. That was a bad decision on my part, but hopefully it’s not too late to rectify that.

M877 is “Advanced Database Technology”, a post-graduate course, which I hope to use as a stepping stone to an MSc or even MBA.

The tutor contacted me to ask if I was actually qualified to do this course. I gave him my job history, and he was satisfied – I’ve been working with databases professionally for at least 7 years, and if I don’t know something straight-out, I’ve at least used it or read about it it.

So yesterday (Jareth’s 3rd birthday by the way), I had a quick glance through the handbooks and was relieved to find that I understood what I read, and was able to answer some of the sample exam questions correctly.

I’m looking forward to this -and eventually being called “Dr Kae” 🙂