07 Jun

book review: Learning PHP Data Objects

Overview: Learning PHP Data Objects, by Dennis Popel, is an introduction to PDO, which walks through the building of a believable test example – a library manager for your home library. Each chapter introduces a new facet of PDO and shows how to rewrite the appropriate parts of the application to slot the new ideas in. Very clear and easy to read. Non-PDO subjects are appropriately kept to the appendices.

I really couldn’t find very much about this book that I didn’t like. Ignoring the appendices, the book is 154 pages purely devoted to teaching PDO through examples, including error handling, working with BLOBs, even the creation of the M in MVC (Models).

I mentioned MVC there. One of my gripes with most tutorials of MVC is that they introduce the concept simply, then provide pages and pages of code with the end product which is “hello world”. Why I should go to all that trouble instead of simply writing <php echo 'hello world'; ?> to the screen usually escapes me. Dennis, however, concentrates solely on the Model and shows exactly why it’s a great idea. I think some more separation of concerns would have been better (don’t mix Author and Book SQL in the same object, for example), but the ideas were all good.

I think that if Dennis was going to show how the Model works, he should also have gone a little further and showed an example of an Active Record pattern as well. But I guess the point of showing MVC was more to show /an/ example of abstraction of the DB code, and that was sufficient.

The book covers a Library manager application all the way through from conception to implementation, demonstrating at all points that the code works with SQLite and MySQL (and by implication, all other DBMS’s) with a change of only the connection string.

Possible problems are explained clearly and solutions are provided. For example, Dennis explains why, after you compile the query select * from books, PDO (and indeed the database itself) does not know how many rows it will return. A solution, in the form of a very smart getRowCount() function shows a query-agnostic method for counting results of an arbitrary line of SQL.

Other areas that are covered in the book include error-handling, prepared statements and transaction-handling.

PDO can handle Prepared Statements even if the underlying DBMS cannot handle it, so it is possible to write your code in a cross-platform way. Examples of why you should use this are provided. One of the examples shows an efficient way to handle insertion or updating of a table using the same parameters for both cases, with the row-handling function deciding whether to use update or insert based on whether an ID was provided.

I feel the Transactions section could have been expanded a bit further. It is not explained how PDO handles this for DBMS’s that don’t internally support transactions, and I wouldn’t like to assume that they work all the time, only to find after deleting critical data that it’s not supported.

Overall, I enjoyed reading this book. Dennis is a good writer and I think he explained his thoughts very clearly.

On an aside, my four-year-old son Jareth loves Packt Publishing‘s books. Sometimes when I go to read another chapter, I need to covertly steal the book I’m reading back from him. For a while, he made it a bed-time ritual to grab all the Packt books he could find around and bring them up with him to read in bed. I think he loved the screen-shots and the frequent code samples. He’s high-functioning autistic and likes literary constructs, and programming books are perfect for him in that regard. Thanks Packt, you’ve made my son (and therefore me) happy.

03 Nov

today's ANN goals

I’ve been doing well over the last two weeks – I started with an ANN which can balance a pole, and upped that by then creating a net which could recognise letters.

The plan for today is a bit more ambitious. I’m writing it down here in case it takes longer than a day to write it. In general, I want to write a PHP application which will allow you to upload images, which the ANN will then try to recognise. If it gets it right, all well and good. If it gets it wrong, you can correct it.

Some milestones for the project:

  • readers can upload images and have them tested and/or added to the training sequence
  • extraction of image pixels using <canvas>
  • automatic creation of new neurons as they are required
  • best net is stored on server, so new readers always start with a working net

I think this may grow into a damned cool thing – I’m already thinking of other cool features like distributed nets, or background nets which can be placed in other pages of a site so the thing can continue training even though the user is not actively viewing the thing (that might be a bit cheeky though).

Anyway – now that I’ve written what I intend to do, I suppose I’d better actually do it.

29 Aug

KFM 1.1 released

It’s not even been a month yet, but I think 1.1 is ready. 1.0 was the real test, and a load of bug fixes went into it on an almost daily rate until a week ago, when no more bugs were reported. I managed to squeeze in a number of the requested improvements, and KFM 1.1 is now ready for release.

I think the most important part of this current release is that SQLite PDO is now supported. This means that a person who does not have a MySQL or PostGres database as part of their hosting package might still be able to use KFM. This involved building a database abstraction layer abstraction layer (not a typo), so MDB2 can be used for the “normal” databases.

It was fun to solve the IE7 problem where normal JavaScript dialogs are considered to be “popups”. The way I solved it was to create a modal window which passes its value forward to a referred function. Those of you that are good JavaScript writers might be interested to read the source and see my solution – I’m damned proud of it!

There’s a lot of work to do to prepare KFM for the next big thing (changeable view types – icon view, details view, etc), and I’m “rearing at the bit” to get started on that!

Anyway, please download and try it!, and if you feel very happy that I’ve improved your customers’ online experiences, there is a handy PayPal button just below the big green Download button.

05 Aug

KFM 1.0 released

Today, I released KFM 1.0, a web-based file manager powered by php/ajax with mysql/postgresql/sqlite.

This is an important release for me, as it marks the completion of the original feature-set that I had in mind when I first started the project – match and exceed FCKeditor‘s default file manager, and provide extras such as multiple file upload and tags.

I was considering giving this a rest for a while before starting on version 2, which will have a lot more “desktop” tricks in it – different view modes, live streaming of videos, vector graphics, plugins – but I feel energised right now, so may just get to work on it.

ohloh considers this project to be worth about $889,277, and you get it for free, so please download it, try it, and comment about it.

09 Feb

KFM 0.8

function getKfmFile(version){
window.SetUrl=function(value){
value=value.replace(/[a-z]*:\/\/[^\/]*/,”);
document.getElementById(‘filename’).value=value;
}
var lang=document.getElementById(‘lang’).value
window.open(‘http://kfm.verens.com/demo/0.8/?lang=’+lang,’kfm’,’modal,width=700,height=500′);
}

Demo English
Bulgarian
Danish
German

Spanish
Finnish
French
Irish
Italian
Dutch
Romanian
Russian

Browse…

download it at the KFM website

New Features

  • New language: Romanian (thanks to Andrei Suscov)
  • Return thumbnails to the opener (35, 142)
  • Tag engine (12, 13, 143), allowing files to be categorised, and searched by category.
  • Client-side syntax highlighted text editing (125, 126) using CodePress.
  • Rename multiple files (103). This allows you to select a group of files, and rename them all to a common format.

Improvements

  • Multi-line captions (89)
  • ImageMagick used where possible (152, 111), allowing huge images to be edited without PHP running out of memory.
  • KFM will return either a normal URL (155), or a URL which allows the get.php script to be used to retrieve the file. The advantage to using get.php is that in the future, we will be able to apply authentication, so only authorised users can read a file. This also allows thumbnails to be built on-the-fly.
  • The database tables can use a specified prefix now (129).
  • Double-click can be configured to either send the file back to the opener (FCKeditor, your CMS, etc), or to open the file directly (120).
  • config.php split into two files, allowing easier upgrades (146).
  • when moving through files with the arrow keys, the selected file will always be visible (110).
  • KFM can now be used to manage multiple sites through one instance (154).
  • A metrics logger has been added (134). This will allow us to know what versions of KFM are in use, and whether it is safe to deprecate certain features if need be.
  • Comments can now be extracted from .gif files to be used as captions (88).
  • Documentation written on how to transfer a KFM archive from one machine to another (161).
  • Many bug fixes (159, 97, 156, 149, 68, 157, 147, 148, 144, 102, 145, 118, 140, 141, 98, 139, 135, 150, 151, 165, 167, 164, 162, 160, 113, 163).
  • Some ideas were rejected (112, 21, 131, 19, 18).

As usual, this release has been helped along by the many testers in the forum, testers who have contacted me by email, and all of the translators.

Very large thanks to Benjamin Ter Kuile, who has been very actively hacking away at the KFM code and produced such gems as the CodePress integration.

22 Dec

KFM 0.7

demo, download (828k .tbz2, 1.1M .zip)

New Features

  • New Languages
    • Bulgarian, thanks to Tondy (tondy.com)
  • Unzip zipped files (84). This allows users to zip up multiple files offline, upload them as one file, and unzip once they are uploaded.
  • Multiple Databases (127, 122). We now support PostGreSQL, MySQL and SQLite.

Improvements

  • Files may be located anywhere on the system at all. They do not need to be within a web-readable area (33)
  • bugfixes (117, 100)
  • Long directory names are now truncated, using the same method as long filenames (80)
  • Directories with many files are now displayed quicker (106)
  • Download From Url has been combined with File Upload (108)
  • KFM has been tested and is known to work on PHP4.3+ and PHP5.1+

As usual, this release has been helped along by the many testers in the forum, testers who have contacted me by email, and all of the translators.

Development for version 0.7 was sponsored by the infinitely glorious web development company, Webworks.ie. We’re really quite good.

14 Dec

kfm 0.7 in beta

No versioned release zip yet, but I just finished the last of the features scheduled for this release. You can download via SVN using the details mentioned on the KFM site.

I’ll be announcing the string-freeze to the translators later today. We have one more language this time, Bulgarian. The official release will be in one week’s time. I need to give the translators time to do their work, and also, will spend that time looking through the code for bits that I can make more efficient.

New features for 0.7:

  • you can now upload a zipped archive of a few files, and extract the archive. this allows you to upload a load of files at the same time.
  • there were a lot of problems with SQLite in version 0.6. to help alleviate this problem, KFM now supports MySQL, Postgres and SQLite, using the MDB2 Pear library.
  • instead of returning links which point directly to the requested images/files, we now return a link which retrieves the requested file via KFM. this allows your file repository to be held outside the web root, and will allow file authentication and other tricks (logging, uri-based thumbs) in the future.
  • long directory names are now truncated similar to long file names.
  • “file upload”, and “copy from Internet” now use the same form.
  • lots of speed issues have been fixed.

enjoy. The main release will be next Tuesday. I’ll write up a quick article then detailing what features I think will be in 0.8, 0.9, and on up to 1.0.

If there are any problems using this beta, please mention them using the KFM forum.

In related news, Webworks, my great and glorious company will be using KFM in a very large project next year, which will mean a lot of work will be put into it. I am still committed to providing the improvements to the great unwashed, so you’ll all benefit from our hard work.

19 Oct

pdo::sqlite gotcha

This one caught me a few days ago, but I didn’t have to time to concentrate on it (I was being harassed by screaming kids at the time). I came across the same problem today, and managed to figure it out with the help of some useful prompts from the ILUGgers.

The problem manifests as a query failing to run for no apparent reason.

An example of some code that exhibits the bug (taken from the KFM project:

function _moveDirectory($from,$to){
	global $db;
	$q=$db->query('select * from directories where id="'.$from.'"');
	$from=$q->fetch();
	$q=$db->query('select * from directories where id="'.$to.'"');
	$to=$q->fetch();
	if(strpos($to['physical_address'],$from['physical_address'])===0)return 'error: cannot move a directory into its own sub-directory'; # TODO: new string
	if(file_exists($to['physical_address'].'/'.$from['name']))return 'error: "'.$to['physical_address'].'/'.$from['name'].'" already exists'; # TODO: new string
	rename($from['physical_address'],$to['physical_address'].'/'.$from['name']);
	if(!file_exists($to['physical_address'].'/'.$from['name']))return 'error: could not move directory'; # TODO: new string
	$len=strlen(preg_replace('#/[^/]*$#','',$from['physical_address']));
	$fugly='update directories set physical_address=("'.addslashes($to['physical_address']).'"||substr(physical_address,'.($len+1).',length(physical_address)-'.($len).')) where physical_address like "'.addslashes($from['physical_address']).'/%" or id="'.$from['id'].'"';
	$db->exec($fugly) or die('error: '.print_r($db->errorInfo(),true));
	$db->exec('update directories set parent="'.$to['id'].'" where id="'.$from['id'].'"') or die('error: '.print_r($db->errorInfo(),true));
	return _loadDirectories(1);
}

The problem is that the SQL statement $fugly (so named because it’s fucking ugly) and the following one do not run, and return an error saying the database is locked. This is despite the fact that earlier in the same function, we’ve read from the database with no problems.

After much banging of heads, I found the problem – the variable $q holds a lock on the database, but it’s a read lock. In order to change to ‘write’ mode, you need to free up that variable before you attempt the write.

function _moveDirectory($from,$to){
	global $db;
	$q=$db->query('select * from directories where id="'.$from.'"');
	$from=$q->fetch();
	$q=$db->query('select * from directories where id="'.$to.'"');
	$to=$q->fetch();
	$q=null;
	if(strpos($to['physical_address'],$from['physical_address'])===0)return 'error: cannot move a directory into its own sub-directory'; # TODO: new string
	if(file_exists($to['physical_address'].'/'.$from['name']))return 'error: "'.$to['physical_address'].'/'.$from['name'].'" already exists'; # TODO: new string
	rename($from['physical_address'],$to['physical_address'].'/'.$from['name']);
	if(!file_exists($to['physical_address'].'/'.$from['name']))return 'error: could not move directory'; # TODO: new string
	$len=strlen(preg_replace('#/[^/]*$#','',$from['physical_address']));
	$fugly='update directories set physical_address=("'.addslashes($to['physical_address']).'"||substr(physical_address,'.($len+1).',length(physical_address)-'.($len).')) where physical_address like "'.addslashes($from['physical_address']).'/%" or id="'.$from['id'].'"';
	$db->exec($fugly) or die('error: '.print_r($db->errorInfo(),true));
	$db->exec('update directories set parent="'.$to['id'].'" where id="'.$from['id'].'"') or die('error: '.print_r($db->errorInfo(),true));
	return _loadDirectories(1);
}

The conclusion is that you can make a load of read requests in a row, or a load of write requests in a row, but if you are mixing the query types, you need to free the result each time.

08 Oct

kfm acquires a search engine

Over the last few weeks, I’ve gradually been shifting the KFM project over to using sqlite for its file meta-data organisation, a database engine which keeps everything in one file and doesn’t require any pesky authentication.

The logical next step was a search engine, so I wrote that today – and wrote some workarounds for SQLite – my home version is 3.3, and work server runs 3.1.3. 3.1.3 doesn’t support auto_increment, which bloody annoyed me.

demo
kfm screenshot

At the bottom left, open up Search, and type “jpg”. No need to press “enter” – it’s a live search – it will ask for the results 500ms after your last keypress. You’ll get a result set of all files with “jpg” in the name.

That’s one of the major new things for KFM 0.6, which I hope to push out the door by next weekend.

Thanks to Benjamin ter Kuile for his help in getting the system to work on servers that don’t have PDO-SQLite built in – he supplied a different flat-file database. I haven’t seen it in action, but it works on his server (which is running MacOSX!)

Also, thanks to Vse Do Feni, who supplied a Russian translation! It looks strange to see KFM with Cyrrilic characters.

The next few weeks should be interesting. I’ll be implementing tagging very soon. KFM will be the most advanced ajax filemanager in existance!