12 Aug

AJAX in IE without ActiveX

What a day! Recently, one of our customers was saying that his site wasn’t working whenever he viewed it in his local net-cafe. This was incredible, to me, as every computer I’ve checked it on has displayed fine.

I finally had an inkling of an idea why this was happening, when I asked him to “click help->about in the browser”, to find out what exact version this was happening in. He said that the browser wasn’t allowing him to access that information.

I understood immediately that it was a security thing. Net-cafes would naturally hate to be infected with the myriad viruses out there, so they tend to belt-up their browsers tighter than the average user.

So, I tried to replicate what might be going on – I set my own Internet Explorer to stun – High security setting. When I tried the site, I was not surprised to find that it was not working. The fact that it relies on JavaScript made that a certainty anyway.

To make it work, I needed to turn on “Active Scripting” (what IE calls ECMAScript), and let it run safe ActiveX modules.

Naturally, I’m slightly ashamed that the site doesn’t work properly in a non-javascript environment, but the site is aimed at the overwhelmingly vast majority of people who are normal. The ActiveX thing, though, was a problem.

I began searching through the source of every AJAX library I could think of – Sarissa, JSPan, Sajax, ie7-xml-extras – every single one of them relied on ActiveX.

Surely there must be some XMLHttpRequest library out there that doesn’t rely on ActiveX?

I sat down, and started work on one. It’s Friday, and I’m exhausted after a very long day’s research, so I won’t go too in-depth into it…

Here’s the file

There are some problems with the above, which I’ll detail.

First off, the file is a drop-in emulation of the XMLHttpRequest object. I wanted to use the exact same API as every other library on the planet is using.

The short explanation of how it works: the library creates an <iframe>, populates it with the requested data, then returns the source of that object.

That’s not quite as simple as it sounds. I got 90% of the code working in 10% of my working time today, and spent the next 90% of the day trying, and failing, to solve the other 10%.

As it is, the file works – it should be bloody simple to add it to any library out there. The major problem, though, is white-space.

The innerHTML returned by IE is not exactly what it was given, but there is no other way of obtaining it.

Trust me – I spent hours crawling through the MSDN library today (in Firefox, because I had IE set on stun, so the navigation (which is ActiveX-based) wouldn’t work) – there is no other way.

I even tried dynamically creating a text/plain <object> to grab the data from it, but got alerts complaining that this was an ActiveX object!

If anyone out there in blogland has an inkling of an idea how to go forward from here, please do pipe up.

12 Aug

ajax-powered gallery with no server-side programming

This is just a demo, so excuse the rawness of it.

A client-side generated gallery. Probably does not work in IE, as IE does not have the XMLHTTPRequest object built into its JavaScript by default. I could adapt the code to make it work in IE, but I won’t.

Beware – bloody large! For broadband users only.

So – what’s special about it? Check the source.

This piece of code does not have any information about the images at all in the source, except for the location of them – i/. The script reads the directory of images, grabs the images, and displays thumbnails of them.

There are two innovative parts here:

  • The gallery grabs the list of images using AJAX. No server-side processing is involved – just plain old Apache directory indexing.
  • The gallery figures out the dimensions og the images, then displays thumbnails of them.

I won’t bother explaining how the AJAX works. If you read my blog at all, then you’re probably familiar with it.

The thumbnail generation, though. I haven’t seen it done before, so this may be the first time it has been done.

To display the thumbnail, the script must first grab the original image, then read the image information, then resize it.

Of course, you cannot expect the images to load immediately, even on a broadband connection, so the script checks the status of the images every second or so, to see how the download is getting on.

Each image is attached to the <body> with visibility:hidden and position:absolute;left:-2000px;top:-2000px to keep it out of the way. That way the server actually loads the image, but does not display it. As it’s attached, the image id is added to an array of images that are not finished yet.

When the images are all attached, a setTimeout is run so that every second or so, a function checks the status of the images, and builds the thumbnails, removing finished images from the incomplete array. If any images are left in the array, the function is scheduled again.

To build a thumbnail, the script grabs the offsetHeight and offsetWidth of the image, divides them by each other to get an aspect ratio, rescales the image to fit into a 100×100 block, and attaches it visibly to the document.

To use this with your own images, simply create an i/ directory, and place these files in the directory above it, then load up the gallery.html.

10 Aug

qmail refusing to create virtual domains

Had a weird problem here. I’m kinda new to qmail, so was working at it through webmin. I created a virtual server for myself in order to move my home server to an online server (see my last post for reasons), but webmin complained that I could not assign a password to my user, as the domain could not be created (!?).

This was weird – I wasn’t doing anything strange… so I tried it from the console, logged in through SSH:

[root@webworks domains]# ../bin/vadddomain verens.com
Please enter password for postmaster:
enter password again:
Failure while attempting to remove_line() the locals file
Error. Failed to add domain to assign file
Error: Could not update file

The solution is vaguely indicated there. All you need to do is this:

[root@webworks domains]# touch /var/qmail/control/locals

Then, remove the failed domain, ignoring the errors:

[root@webworks domains]# ../bin/vdeldomain verens.com
Warning: Could not access (/home/vpopmail/domains/verens.com)
Warning: Failed while attempting to delete domain from auth backend
Warning: Failed to delete directory tree: verens.com

And recreate it:

[root@webworks domains]# ../bin/vadddomain verens.com
Please enter password for postmaster:
enter password again:

All is as it should be.

Oh – also, I came across an errata in the o’reilly book on qmail. When I tried to submit it, though – I got an SQL error (tsk tsk). The error returned was this:

Unable to execute INSERT INTO submission VALUES(0, ‘qmail’, ‘128’, ‘Top information box’, ‘reader question or request for clarification’, ‘The error indicated in the information box – that of a local user being able to send to an address, but an external user not being able to, can also be caused by an MX record pointing to a domain name which is a CNAME of another host. This is pointed out here: http://cr.yp.to/im/cname.html’, ‘Kae Verens’, ‘kae@verens.com’)

The error looks like it was caused by someone changing the database table, but not updating all pages that access that. This could easily have been avoided by changing the syntax used to enter the data to something like this:

insert into submission set
  pagepart='Top information box',
  errortype='reader question or request for clarification',
  errordescription='The error indicated in the information box - that of a local user being able to send to an address, but an external user not being able to, can also be caused by an MX record pointing to a domain name which is a CNAME of another host. This is pointed out here: http://cr.yp.to/im/cname.html',
  name='Kae Verens',

Here’s the thing I tried to enter – page 128 of the oreilly qmail book:

The error indicated in the information box – that of a local user being able to send to an address, but an external user not being able to, can also be caused by an MX record pointing to a domain name which is a CNAME of another host. This is pointed out here: http://cr.yp.to/im/cname.html

10 Aug

the eircom blues

For the last few days, I’ve been grabbing a single webcam frame every minute from the AC/DC Big Ball convention, in order to create a short video detailing the entire convention, by stitching the .jpgs together with mencoder.

The last image was grabbed yesterday, at 13:47 – not because the convention had finished – but because that’s the time when Eircom went balls-up for the second time in two weeks.

It took them almost 9 hours to get Monaghan (the entire town – not just a house or two) back online. That is incompetence.

So, I’m moving all my servers off my home DSL link, to a more stable location. And, as soon as it’s feasible, I’m moving away from Eircom.

I know that in their opinions it’s probably something like “oops – there it goes again”, but the Internet is my livelihood – it’s what I work at – I need a reliable connection, and that is not Eircom.

08 Aug

Magic Menus

Another piece of unobtrusive javascript. I’ve been promising Ken, the webmaster of linux.ie a good look at this for a while.

Here is the demo. Works in (just testing it now…) IE7, Firefox, and Konqueror (although I see a tiny bug in the latter which I’ll have fixed before long). Its degradable, so should be fine as well in Links/Lynx, etc. I’m heading home now, so will work on it at home for Konqueror and Opera (haven’t tested that).

Basically, you just link to the script in your <head>, and it converts any <ul> trees that it finds into hierarchical menus – but only if the parent <ul> has a class name of either “magicmenuleft” or “magicmenutop”.

The code was originally based on work done by Mike Hall, but it’s been so worked over that I feel it’s safe to call this a different kettle of fish.

As usual, feel free to take this and work with it.

06 Aug

an ajax shopping cart

I had a simple-enough prototype which we (webworks.ie) released for a few clients a while back (Kipa Windows, Cavan Crystal, The Strats, CBO for example).

The main points of the existing prototype were:

  • AJAX used as much as possible, to reduce page refresh times.
  • Instant-update shoppingcarts – changing the number of items doesn’t involve a refresh of the page – the basket is updated instantly.
  • The cart is dynamic, in that there is nothing client-specific hard-coded into its source.

We got a fascinating job recently – inkjet.ie. The client had a few existing sites, but he wanted a new one which was more tailored to his needs. I took this as a chance to push my (well – Webworks’ 😉) shopping cart a lot further towards maturity.

Some of the points that needed to be addressed:

  • The design of the cart must become much-more flexible. Note that the examples I quoted above all have similar designs. The client in this case had some very specific knowledge of what worked, design-wise, and what doesn’t, when selling consumables.
  • In previous carts, items were located solidly in one parent category. The new site had consumables that may correctly be located in many parents. The code was updated, then, to allow items to be created anywhere, then “multiply-located” somewhere else. Because the items were in fact all the same instances, and not just copies, it was then possible to make mass-changes instantly (updating prices, for instance).
  • A “quick select” item was created, so that by choosing a parent category, you would get a list of all the items located in that category and its sub-categories. The client chose to have just the top-level categories shown, and then a recursive list of items. However, I have coded it in such a way that it’s a matter of moments to change it to show progressively deeper categories, instead of an instant jump to the items. The idea was directly modeled on this.
  • Another interesting development was that the code for this was so large that I just couldn’t release it like that! The JavaScript caused a very noticeable delay in first-time visiting browsers. This is not-so-bad with broadband, but I had the occasion to witness it on dialup, and the delay was horrendous. I managed to split the code up into many smaller ones, each of which loaded only when needed (I’ll write something up on this at some point, I swear!).
  • This cart was the first one I’d coded which had a direct API connection to a credit-card clearing server. Up until this point, the other versions of the cart led off variously to PayPal, WorldPay, or a simple email interface (again, dynamic, and instantly switchable).
  • I won’t bother going in-depth into the admin side, as it’s not really observable by the average user. But, again, some pretty funky AJAX was used (uploading of images, for example, through AJAX).
  • Turns out I had to do a bit of work on the AJAX API Sajax in order to get my work done. Sajax was not able to handle multiple AJAX-enabled scripts. It can now 🙂
  • Shipping prices were implemented. There was some coding to do in this, but it was all pretty simple. Note that the shipping price changes instantly if you change your order in certain ways (country of delivery, for instance, or ordering above a certain amount).

Anyway… there are a few items in the above that I’ll describe in detail at some point, as the methods are probably worth sharing and improving.

There is still some work to do on the site (doesn’t work properly yet in Opera or Konqueror yet, for instance, and most definitely does not work in Links), but that’ll all be fixed soon enough.

We (webworks.ie) will not be releasing this under GPL yet, although the previous version, as demonstrated in the example sites above, is freely available through the webme project. We will possibly release it at some point, though.

02 Aug

dynamically loading external JavaScripts with AJAX

I have a lot of JavaScript in my sites. In order to allow the user to view the site without having to download hundreds of K of scripts, though, I need to split the scripts up into many different files.

The files load upon certain conditions. For example, my multiselect code is only invoked when the document contains a <select> box, and my shopping cart code is only invoked when there is an element in the page with the class shoppingcart (btw: that’s a funky shopping cart, I think you’ll agree 😉 ).

So – I needed a function that could dynamically load the scripts that I need.

Up until today, I did this by adding a script element to the head of the document, but I found that this method crapped itself when I tried it in Konqueror.

I hit on the idea of loading up the code with AJAX, then using eval() to activate the code.

Here is the function I had:

var loadedScripts=[];
function loadExternalScript(url){
  var r=new XMLHttpRequest();
      var script=r.responseText;

I tried that out, with some alert()s placed in the external scripts to prove the files were being loaded, and that was fine.

Then I tried to run functions from those scripts… I got errors that they did not exist!

It turned out that the eval() from that function was running the code from the scripts with a scope local to that function – so when the context left the function, the external scripts were scrubbed from memory.

So – a quick think later, and I came up with this amendment:

function loadExternalScript(url){
  var r=new XMLHttpRequest();
      var script=r.responseText.replace(/function ([^\( ]*)/g,'window.$1 = function');

Note that this will affect all functions in the included script, including those which were intended to be used as class methods (I don’t do classes). Note also, that global variables will need to be addressed – but you get the idea.