Monthly Archives: May 2006 - Page 2

eurovision gets rocky

What a surprise! I was only half listening to the Eurovision contest yesterday. I loved the Lithuanian band (“We Are The Winners), and expected them to win through sheer cheekiness. I went out of the room to chat with the blokes at contactjuggling.org, and heard a rock band come on. They sounded kinda GWAR-ish. I took a look, and they looked GWAR-ish as well! A Finnish band called Lordi.

Finland has some damn cool bands, including Nightwish (I love their song Sleeping Sun (some samples from them) – but putting a creature-costumed band into the Eurovision? Crazy!

I thought – personally, I’d vote the Lithuanians first, and the Finnish second.

Stunningly, the Finnish won!

json notation in function calls

I’ve been using JSON to help move objects, arrays, etc. between the client and server.

Basically, JSON is a serialisation method, which allows an object to be compressed to a string description, sent, and decompressed on the far side.

For example, the following is a “Kae” object:

{
  name:          "Kae",
  surname:      "Verens",
  dependants:  [ "Bronwyn Verens", "Jareth Verens" ],
  age:            29
}

The above is easy to understand, and can be compressed by stripping the spaces:

{name:"Kae",surname:"Verens",dependants:["Bronwyn Verens","Jareth Verens"],age:29}

There are four types of variable in the above; numeric, string, array, and object.

When you call a function in JavaScript with the above, after receiving it from the server, you must first “eval” it into a usable state.

Assuming the received serialised string is called “ret”, the following should be all that’s needed, assuming the values are going to the function “call_me()”.

call_me(eval(ret));

The above will work for each of the following strings:

ret="name=\"Kae\"";
ret="[\"Bronwyn Verens\",\"Jareth Verens\"]";
ret="age=29";

But, this will not:

ret="{name:\"Kae\",surname:\"Verens\",dependants:[\"Bronwyn Verens\",\"Jareth Verens\"],age:29}";

The problem is that object notation ({ ... : ... }) fails. It took me a while to figure out why.

This is valid code:

var kae={name:"Kae",surname:"Verens",dependants:["Bronwyn Verens","Jareth Verens"],age:29};
call_me(kae);

But, the following is not:

call_me({name:"Kae",surname:"Verens",dependants:["Bronwyn Verens","Jareth Verens"],age:29});

Why? Who knows… Anyway – this works:

call_me(({name:"Kae",surname:"Verens",dependants:["Bronwyn Verens","Jareth Verens"],age:29}));

Remember, the original premise was that the JSON was to be eval’ed while the function call was being created, so this ends up as:

call_me(eval(ret.charAt(0)=='{'?'('+ret+')':ret));

Ugly, but it works.

oy! Buy my rock!

A few weeks ago, I bought a tumbled stone from my local weird and wonderful shop (East Of Eire). This stone is a form of metamorphic rock called “Picasso Jasper” (or Picasso Stone).

I’ll admit it’s no jesus in toast, but as it’s a rock, I don’t think it will eventually simply rot away.

Want it? Buy it!.

imagepng problem and solution

Just came across a problem where I was saving a PNG and the PHP engine silently exited. A message was logged in the httpd error_log, though:

[Wed May 10 11:45:41 2006] [error] [client 192.168.1.109] PHP Fatal error:  imagepng()[function.imagepng]: gd-png:  fatal libpng error: zlib error in /share/www/test/j/fckeditor/editor/filemanager/browser/default/connectors/php/commands.php on line 74

It turns out that the problem is that PHP 5.1 introduces a third parameter to the imagepng() function, for controlling the level of quality. This value is an integer between 0 and 9 inclusive.

The error I was experiencing was caused by me using 100 as a default third parameter, as imagejpeg() uses this to control quality output (as a percentage), and I was trying to be efficient with my code by calling the image creation function using a variable instead of a hard-coded call (allowing me to save to an arbitrary format, instead of a set format).

So – if you use a quality parameter in your PNG creation code blocks, remember that it is not similar to the JPEG creation function (annoying as that is). For example, the line I just corrected in my CMS now reads:

$save($im,$newname,($type=='png')?9:100);

very modest bounty

In an effort to get some work done on my robot, I’d like to offer a small bounty for help with some parts. I’m going to start off very small, but hopefully as time advances (and my finances improve), I’ll be able to offer larger sums.

The first bounty will be a very modest €50 for a C console application which can do the following:

  • Accept two filenames from the command line as input, which correspond to two photographs of one scene, taken parallel to each other, separated by about 6 inches. The cameras may not be exactly parallel, and my be slightly twisted or unfocused, relative to each other, so the application should be relatively intelligent about that.
  • From those files, extrapolate 3D shapes and textures, and echo them to STDOUT. The format of the output should be well known. I offer VRML and X3D as ideas, but if there is another more-apt language, then that will be acceptable.
  • Any textures needed should be output as images to the current directory as 1.png, 2.png, etc.
  • The application should run in Linux, and must not use up a huge amount of RAM.
  • The application may use other libraries to achieve the goal (in fact, I’d recommend this, to make maintenance easier), but those libraries must be open source.

The closing date for this is July 1. All solutions offered will be compared for efficiency and accuracy, against a few different models created in POV-Ray, and also against real-life photographs.

If no solutions are submitted, I will announce the project again, with a higher bounty (probably increasing by another 50, or 100).

All applications submitted should have the source-code readily available.

Once an application is accepted, it will immediately be released as open source. You will be asserted as the author of the code, but the ownership of the code will be public.

I know the bounty is kind of low, but it’s really just to drum up interest, and I’m not all that wealthy 😉

This project is really an experiment. If it is successful, I have many other projects that I want done, which I will also announce bounties for.

some utf8 problems and solutions

This week, I was tackling some character set issues in our CMS.

The biggest problem I had was that files uploaded and imported into MySQL were losing important characters through translation. The files included characters such as €, á, Á, etc.

The problem seemed simple at first – up until very recently, most servers were set up by default to use the Latin1 character set (ISO-8859-1). This character set is ideal for American use, and closely matches their ASCII character set.

ISO-8859-1 is a superset of ASCII, with western European characters added. This includes the accented characters such as á, é, í, etc, which are used in the Irish language.

Trouble starts, though, when you start considering more recent characters such as the Euro symbol, €.

The Euro, and other characters, wer added to the Latin character set in ISO-8859-15. This involved removing some other characters, though.

The ISO-8859 characters sets are all 8-bit sets, which means that each one can hold only 256 characters at the most. This might be okay when you only consider one language, but when you go beyond one language, you need to either start using other character sets, or consider using UTF8 or UTF16.

Using many different character sets would be a nightmare undertaking, so it is easier to simply use UTF8 for everything.

This sounds simple, but there is unfortunately a lot of work needed in quite a few areas before UTF8 can be easily used for all purposes.

For example, I mentioned the trouble I had importing files into MySQL. This was happening because the default character set used by PHP’s MySQL binding is Latin1. You can see this by echoing the output of mysql_client_encoding() on any RPM-installed PHP.

This encoding is just one point in a chain of string handlers, though. A string must pass from PHP itself into the MySQL binding, through it to the MySQL server. Each one of these steps can have any different default character set. You can see how many different settings are involved by opening your console, entering a MySQL client, and typing show variables like "c%";. The “character” and “collation” values are all different character sets that are currently active.

The manual page for this function hints that it is possible to change the connection character set to UTF8, but in practice, it is not simple.

A lot of further work revealed that, in fact, the default mysql_* functions simply cannot handle character sets very well. Instead, you must use the mysqli_* functions. There is simply no simple way around this!

Luckily for me, this was pretty straightforward for my own CMS. I had begone work a long time ago on abstracting the database calls away, so it was a simple matter to change the abstraction (I use Pear::DB) to use ‘mysqli’ instead of ‘mysql’.

To get the ‘mysqli’ bindings to speak in only UTF8, I had to hack the Pear::DB library slightly. Within the connect() function, immediately after the ini_set('track_errors', $ini); line, I added these lines:

@mysqli_query('set names "utf-8"');
@mysqli_query('set character set "utf8"');
@mysqli_query('set character_set_server="utf8"');
@mysqli_query('set collation_connection="utf8_general_ci"');

That was enough to solve the file import problems. Suddenly, everything worked! Please note the difference between “utf-8” and “utf8” in the above – I’m too tired to go digging deeper into finding a single standard way – the above works, and I’m happy enough with that.

A knowledgeable person might notice that the function calls there are not strictly correct. Ideally, they should be similar to @mysqli_query($this->connection,'set character set "utf8"');, but that was causing errors which I haven’t discovered the cause of yet.

Since I was working away on moving everything to UTF8, I decided to try sending all JavaScript as UTF8 as well.

To send all .js files as UTF8, simply add this line to your root .htaccess file:

AddCharset utf-8 .js

Hehe – funnily, to get a copy of that line (my memory isn’t perfect 😉 ), I ran a search in Google which returned one of my own pages as the top result.

This is fine for strictly JavaScript files. However, there is a very big gotcha when it comes to Ajax and UTF8… if you try send results back to an IE6 or IE7 (but apparently not IE5) XMLHTTPRequest object with UTF8 encoding (for example, using PHP’s header('Content-type: text/plain; Charset=utf-8');), IE6 will complain with a very confusing warning – “Error: System error: -1072896658” in IE6, or “msxml3.dll error ‘c00ce56e'” in IE7.

Long story short? IE still sucks.

Tired now. Think I’ll bring the kid to bed, then crack open a beer. I hope these notes are of use to someone!