Category Archives: ajax

04 Oct

KFM 1.4.2 released. now with CKeditor support

kfm

KFM 1.4.2 has been released. It’s a minor maintenance release, but it adds support for CKeditor. TinyMCE and FCKeditor have been checked as well.

demo using CKeditor

This release also adds a configurable restriction, where images larger than a certain resolution will simply not be allowed in the repository.

Very large images can cause servers to crash when the system tries to generate thumbnails, because of the large amount of RAM needed. For example, a 3000×2000 image will use up at least 24MB just to hold the thing in memory, and much more to handle the manipulation. 24MB is a large amount of RAM to a web app.

There are also some code additions which are not used by the KFM GUI, but can be used by tools that hook into the KFM engine. These allow uploaded files to be placed in directories specified at the time of upload, or to replace existing files, or even to take an uploaded image and apply HSL transformations to it (when ImageMagick is installed). These are not documented, but anyone that needs this functionality will be a good enough programmer to read the source and find it (or email me 😉 ).

Download KFM 1.4.2 here

30 Sep

KFM2: the beginning

KFM 1.x has reached the end of its development. It has fulfilled its original purpose; to improve on the default file-manager for FCKeditor, and well-exceeded it.

From my records, I can see that KFM has been installed on over 9000 separate domains. That is quite a lot of sites, and says quite loudly how important it is that KFM2 is at least as good as KFM1 (we all know about the KDE4.0 fiasco…).

KFM2 will blow KFM1 well out of the water. It will do everything that KFM1 does, but much quicker, and with a much less monolithic architecture, allowing it to be hugely flexible. I am planning on creating a download section similar to jQuery UI’s, where you can choose the components that you want in the system, and it will be built up and configured for you, ready for download.

See, KFM 2 will be modular. If you just want a simple upload/download facility, then you will only need the core, and plugins to handle uploads and downloads. If you want the whole shebang, you will find plugins for various ways of selecting files and directories, full-on graphical UIs such as the existing KFM1 forces on the user, plugins for hooking into external file systems using FTP or whatever other method people can imagine, plugins for multi-media, file editing, databases, user authentication, search, RSS, and so on.

Basically everything that KFM1 already does, but with the option to easily remove/add bits that you want. Make it as fast or as “bling” as you want.

KFM2 will have a number of new things which are not available in any other online file-systems, that I know of.

new file-system things

The file system will be modular. You will be able to attach symbolic links to any part of the file system, linking to other external systems.

As an example, let’s look at a URL: http://example.com/kfm/get/images/site1/logo.jpg.

One of the new ideas is that this will not necessarily be located on the server named example.com.

If you are running a very busy website, you might prefer to delegate image management to a different machine deeper in your network, so you add a symbolic link /images/ linking to the image-management machine. This causes the server to issue a redirect to the browser, redirecting it to http://images.example.com/kfm/get/site1/logo.jpg.

Now, imagine that the image management server doesn’t host the logo.jpg file on itself. Maybe it’s one of a number of images.example.com servers in a load-balancing cluster, and the file is actually located on a file-server accessible only by FTP. So, we add a symbolic link in the server’s KFM, telling it how to connect to the FTP server. The image server does this, gets the required file, and sends it to the browser.

This sounds a bit complex, but at least it’s possible with the new system. The old system simply would not be able to handle that at all.

API

At the moment, if you want to select a file or directory, you fire up KFM1, wait 15 seconds for it to finish booting itself up, and select the file or directory. That’s way too long. Really, you should only need to load up a tiny widget that’s designed specifically for picking a file.

The new system will have a simple core with an API that can be connected to by little JavaScript widgets. As an example, let’s say you just want to get the file-listing for the foo directory.

At the moment, you need to go through the whole business of booting up KFM’s GUI, navigating to the right directory, then waiting for the server to connect to the database, generate thumbnails and other unnecessary stuff.

You should just have to do something simple like load up http://example.com/kfm/rpc.php?a=get_file_listing&v=/foo through jQuery’s $.get() function. The RPC script does exactly what was asked, and nothing more.

This should allow us to write some incredibly fast and tiny scripts for all the file manipulation things you could possibly want.

Think of it as the “gnu” way of doing KFM – many small tools, each of which is designed to do one task well.

Of course, you will still be able to use the original GUI with this, after it’s been re-engineered to hook into the new RPC system.

Or, you can write your own – the RPC’s API will be well-documented, and will use simple HTTP parameters, so you can write your widgets in whatever language you want – JavaScript is what I will be writing in, but there’s no reason why you’re confined to that – write a GUI in Python, C, Flash – whatever you’re comfortable with.

plugins

I think the new plugin architecture will make this profoundly extensible. I’m writing the core to be as flexible and stable as possible. You will be able to write a plugin to just about anything:

  • authenticate a user before serving a file.
  • change a user’s root directory based on the server-name or the user’s authentication.
  • the entire system is held on a different machine accessible only by FTP? fine – let’s change how files are accessed.
  • record all files and directories in a database, allowing searches to be done.
  • manipulate files online – edit text files, rotate or crop or resize images, move or rename or delete, etc. standard KFM1 stuff, these.
  • create a log plugin, so all access is recorded.
  • create a quota plugin, for denying uploads or downloads when the quota is exceeded.

It was possible for us (Benjamin and myself) to write all of these into KFM1, but as we are only two people, and very busy people at that, it’s never been the absolute top priority to get all these things done.

But, when KFM2 is released, you won’t have to wait for us anymore. You will be able to write plugins, or download plugins created by other people, which do all of the above and I’m sure there will be plugins that I simply have not imagined yet, that inventive people will come up with (Conor, I’m sure you’ll be one of those!)

languages

Languages have always been one of the cool things about KFM. Version 1.3 was released in about 13 languages in total. Unfortunately, the code got too complex in 1.4 to easily add more, but KFM2 is starting from scratch with the learning-curve of KFM1 well behind me. I’ve already made most of the big mistakes I’m likely to make, so from now on, things should be much easier.

How languages will work is that a website will be created which allows people to help out open-source projects, not just KFM by providing translations of words and sentences.

The programmer will create a screen-shot showing the text in use, and will provide the untranslated text in a format such as .po. Translators will translate the text on-line, and the translation will be available to download by the programmer as soon as it’s done.

To ensure good translation, each line will be presented to a number of different translators. The translations which agree the most with each other, will be considered the “right” one.

Using this, it will be possible for KFM to organically add languages with no interaction from myself or any other programmers. When first loaded, a plugin would try to translate messages into the user’s language. Finding that it doesn’t have the language in its database, it will try downloading it from the language translation website. If it’s not available, a request will be added that it be created. When translators that speak that language come to the website, they will be automatically given the KFM text to translate to that language. The next time the plugin checks, the translation might be done, and it will be added to the KFM instance’s local language database.

Of course, people don’t just do this stuff for the hell of it. It’s more fun if there’s a competition or reward. It would be nice if I could get a few paying customers to add their own translation requests to the website – that way I could offer a reward to the best translators (those that do the most work, or are the most consistent).

The competition aspect comes in, where you give people points for every translation they do that is matched by someone else’s translation of the same text.

Google does a similar thing, for example, where it shows images to people and asks them to come up with keywords describing the image – the commonest keywords are considered to be correct. Same trick, different target.

Anyway! Without further ado, I have a core to design, and tests to create. I was going to talk about testing with PHPUnit, automatic upgrades, and the like, but to hell with ye all – I’ve work to do 😉

27 Sep

KFM 1.4 released

It’s been almost a full year since the last release, 1.3.1, and I haven’t done much on KFM in that time.

Benjamin ter Kuile, though, has! Today, we release KFM 1.4

In the time that I’ve been busy with other stuff, and procrastinating on KFM, Benjamin has been working diligently, adding features and making the core files more stable.

Here is a short list of improvements, gleaned from the SVN logs over the last year. I’m sure I’m missing a lot.

  • jQuery upgraded to 1.3.2 (faster, better, stronger)
  • you can edit multiple text files simultaneously
  • context menu code improved to help it work in Chrome, Opera and Safari
  • file icons are now created as a montage, improving display speed
  • a lot of functions converted to “lazyload” – improves startup speed
  • images are also lazyloaded – if an icon is not visible on the screen, its thumbnail is not loaded
  • lots of bug-fixes

Not many new features, as KFM was basically complete.

1.4 is the last 1.x version which will be released. We’re done with the code-base, and ready to start on a much more ambitious project.

We’re going to rebuilt the entire system from scratch in a modular way, adding functionality that would be very difficult to shoehorn into the present system. The new version will be called KFM 2, and I’m really excited about it. I will be starting the coding for it very soon, as soon as I’ve finished with the release of my book, jQuery 1.3 with PHP.

For those of you that are already using 1.3; 1.4 is /faster/, but 1.3 already completes the initial spec, so only upgrade if you find that 1.3 is slow. New users should install 1.4.

02 Aug

jquery remoteselectoptions plugin

Based on some work I did for chapter 3 of jQuery 1.3 with PHP, I’ve created a plugin to encapsulate the remote selectbox trick.

The idea of this is that in a lot of cases, there may be huge select boxes in your forms, but the selected value might not ever need to change. For example, if you have a country list, and you’re pretty sure that the user is in Ireland, then it might be pointless to have a full list of countries in there if it’s unlikely to be changed.

demo
country-select

Notice that when you click the select box, it’s populated with countries. However, the source of the page does not include those countries in the HTML. So how does it do it?

How it does it is to add a focus event to the selectbox, which populates it only when it’s actually about to be changed. In this case, the missing options are in a file named countries.html which is 9k in size. So, by using this trick, we’ve saved 9k in bandwidth. Multiply that by the amount of large selectboxes in your forms, and it could be considerable.

Look at the source of that page:

<html>
	<head>
		<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
		<script src="jquery.remoteselectoptions.min.js"></script>
		<script>
			$(document).ready(function(){
				$('select[name=countries]').remoteselectoptions({
					url:'countries.html'
				});
			});
		</script>
	</head>
	<body>
		<em>select a country</em>

		<select name="countries"><option value="IRL">Ireland</option></select>
	</body>
</html>

All that’s needed is to include a single option in there to indicate the default, then run .remoteselectoptions() on the selectbox, telling it what url it should grab the options from.

In case you need to do some checking on the server-side based on the current selected item, the script calls the option source with a selected GET parameter set to the current selected option’s value. You might set the url to a PHP script for example, which would build up the option list based on that selected item.

Download the source here (4.5k)

Rate it here.

22 Jul

jQuery 1.3 With PHP: chp8, drag/drop

For chapter 8 of jQuery 1.3 with PHP, I wrote about drag and drop. jQuery has a few different ways to do drag and drop. The most fundamental way is to use the Draggable and Droppable interfaces, but as it happens, most of the things you might want to do with a CMS can be done just with the Sortable interface.

The examples I wrote for this chapter are:

  • Sorting a list and saving the result. demo, download
  • Sorting a tree-based list (navigation menu, for example) and saving the result. demo, download
  • Dragging items from a list of contacts into a list of people to send emails to. demo, download

The second one worried me for a while, as it kept behaving erratically on me – I would sometimes drag an item and it would simply vanish. It turns out that it’s a bug in jQuery UI’s Sortable that should be fixed by the time the book is out, so it will work better then. I’m not worried about its behaviour right now as I’m certain this will be fixed.

I only used Sortable in this chapter, as I was trying to think of a case where Sortable couldn’t be used, and the only cases I thought of were a bit too advanced for the book – the book’s supposed to be for PHP developers that are not absolutely rock solid in JavaScript and those examples might be a bit daunting. Maybe if there’s a second edition I might expand on it with more advanced examples.

An advanced example might including dragging a shopping item into a basket. You could not use Sortable with that because usually, what you are dragging is a full description of something, and when you drop it into the basket, you want it to change to a compact description. Coding is involved there, and that’s when it gets bogged down. Writing about that sort of complex case would mean diving into JavaScript and away from the core subject of the book, which is the interaction of PHP and jQuery.

update: I fixed Sortable. Here’s an example of nested sortables using the corrected code: demo.

18 Jul

jQuery 1.3 With PHP: chp7, image manipulation

I’ve submitted chapter 7 of jQuery 1.3 with PHP to Packt, which involves image manipulation. The editor I build in that chapter allows you to non-destructively manipulate an image in the most commonly needed ways – resize, rotate and crop.

The idea is that when you upload an image, it’s usually not yet right to include in a website. People tend to upload massive photos (3000×2000 or so) and resize it down using the <img> attributes, which is the wrong way to do it (see here for a solution to that particular problem).

However, once you’ve resized an image down, you can’t change it back.

The solution is to make copies of the image with your manipulations applied to them. That lets you make multiple versions of the same image.

But again, there’s the problem that if you change your mind about the original image, you can replace that, but the others all need to be redone.

Yet another problem is that unless you’ve kept records, there’s no link from the new images back to the original, so if you need to make space on the server by removing files that might not be in use anymore, you can’t be certain what thumbnails are still relevant and what are not.

The solution is to not actually create copies, but to record the changes applied to the original image in the URL of your manipulated image. When the server is asked for those URLs, it runs the manipulations on the original image and gives you back the new image. Of course, we will cache the new manipulations so this doesn’t need to be done every time, but this now allows you to replace the original image, then clear the cache, forcing all the variants of it to refresh, without you needing to do it again.

What this means, is that when the manipulation is recorded in the URL, no new copy is actually created until some browser actually requests it. This means that you can periodically clear your old thumbnails, safe in the knowledge that if they were still relevant, they will be recreated from the archived originals automatically the next time they’re needed.

Enough talk.

demo

This is, as usual, just a demo. It’s not designed to be impressive, but just to show how to do manipulations.

A permanent URL is provided, in the same manner as google maps – it updates itself as you manipulate it. To see the actual cropped, resized and rotated image, try opening that permanent URL in a different tab.

The GUI here is a means to an end (the permanent URL). In your own CMS, you might force a certain width/height for uploaded images automatically using the URL for example.

The example image I manipulated in the chapter is the IMG_0134.JPG in chp7. Try rotating, resizing, and cropping (drag your mouse on the image).

example:

Little thing to note about that URL – there’s no query indicator. Standards-compliant browsers treat the ‘?’ mark in a URL as an indication that the result should never be cached, but that would be pretty expensive for the server, so we remove that. The get.php file rebuilds the query from the $_SERVER['REQUEST_URI'] and sends out the manipulated image with caching headers.

The graphics manipulation was done using Imagick, which is a PECL extension that allows PHP to run ImageMagick functions internally.

In Fedora, you can install Imagick with yum install php-pecl-imagick. In CentOS, follow these instructions.

CentOS users might have a problem with the latest Imagick (2.2.2), so change the pecl install imagick line to pecl install imagick-2.2.1.

download the demo

Make sure to edit the images_libs.php file to point to your own image repositories. The images don’t need to be in a web-accessible directory.

I’ve a bit of work to do now, and then I’ll be applying the tricks in this chapter to our own CMS, WebME.

29 Jun

jQuery 1.3 With PHP: Calendars

I was supposed to write about Datatables for chapter 6 of jQuery 1.3 with PHP, but the website of the plugin I was going to use (http://www.datatables.net/) was down for about a week, and so I wrote about Calendars instead.

I used Red3‘s jquery-week-calendar plugin for a recent project in work, and was so impressed I really had to write about it.

In chapter 6 of the book, I’ll walk through how to build a simple calendar, including creation and editing of events, and including once-off and recurring events.

demo

The demo is a session-based calendar, which records only for the duration of your browser session. It’s for demo purposes only, obviously. If you want to use it in a larger project, you would need to adapt the PHP so that it records to a database or files or something.

Download

Here’s an image of it in use:

fig_6_0

Only four or so chapters left and then I’m done with the hard part. After that, is rewrites, then you can all throw your money at me.

28 Mar

jQuery 1.3 With PHP: quick tricks

As mentioned before, I’m writing a book for Packt Publishing called jQuery 1.3 with PHP.

I think I’m just about done with the first chapter I’ll be submitting, tentatively entitled Quick Tricks.

The book will include full explanations of how they work, but here are the examples.

In the last two examples, the demo is designed to show what happens in a failed save or delete as well as in a successful one.

Source is, of course, available.

03 Dec

prompt

I wrote some JavaScript today which required using a prompt() to request a number. The idea was that you have a table full of numbers, and want to change one. So, you click it, and a prompt() asks for the replacement number.

In Firefox, the prompt is a small modal window which has an ugly title saying something like “The page at http://localhost/…”.

So, I wanted to know is there a way to avoid it. I was hoping that prompt(question,value) had a secret third parameter that set the title.

Unfortunately not, but what I did find was even more annoying. the first hit for javascript prompt is a “tutorial” which is so derisive of the prompt() that I have no idea why the person bothered writing anything at all.

The JavaScript prompt is a relic from the 1990’s

The JavaScript prompt is not very useful and many find it slightly annoying, but hey, this tutorial is here to educate you, so let’s learn how to make that prompt!

You can use a prompt for a wide variety of useless tasks, but below we use it for an exceptionally silly task.

Good job, Tizag – I wonder if all your students (if any can stand the method of teaching) learn to treat JavaScript with the derision that you do. I bet you go to bed chortling at how clever your insults are and how the people that use JavaScript day to day are all idiots.

By the way, you’ve made a few incorrect statements.

The JavaScript prompt is a relic from the 1990’s that you seldom see being used in modern day websites.

Ever heard of Ajax? It involves a lot of user interaction. That sometimes involves requesting a value from the user. The way you do that is prompt(). Now, you /could/ write your own custom version of prompt(), but when it comes down to it, that’s just a prettier version of something that you claim is a relic.

The JavaScript prompt is not very useful

Except for those very common cases where you might require input of a single value.

many find it slightly annoying

[citation needed]

but hey, this tutorial is here to educate you

If education means teaching to sneer, belittle, and misunderstand, then you are correct.

The point of the JavaScript prompt is to gather information from the user so that the information can be used throughout the site to give the visitor a personalized feel.

No! The “point” of prompt() is in its name – it prompts the user for a value. What you do with the value afterwards is of no bearing at all to the function itself.

By the way, your “tutorial” doesn’t even say what the two parameters of prompt() do.

Fail.

14 Nov

webworks to opensource its tools

I had a nice chat with John earlier. We‘re considering releasing our tools under an open source license.

We’re doing this for a number of reasons. I think that chief among them is the hope that with many developers (or at least, with more than just myself), we should be able to keep on top of all the modern net tricks that keep appearing – RSS, Ajax, etc. So far, I’ve been doing that myself, but as our CMS, WebME (Web Management Engine) has grown, the effort to keep it uptodate has become trickier.

So, we are planning on releasing the code as open-source while I am still on top of it, so I can help other developers join in on the coding while still working away on it myself.

I’m very excited about this.

A few years ago, we tried this very thing, open-sourcing WebME, but no-one took us up on the offer, and we brought it back in-house.

However; since then, I released KFM, which has developed into a pretty popular open-source project, and I feel that releasing WebME again will work based on whatever OS credit I might have earned from that project.

We are hoping to release our newsletter-management service as well, depending on how WebME does.

I’m very interested to see what other people think of this. I know most of you will not have seen WebME’s admin area, but I think it’s one of the most user-friendly ones out there (even if I say so myself), and will be a boon for developers who want a simple CMS for simple sites – page-based sites with galleries, simple online stores, etc.