11 May

quick method to clone a MySQL database

let’s say you have a MySQL database on db1.db and you want to clone it to db2.db

the “official” way to do this is to run a “mysqldump” on db1.db and then import the resulting .sql file into the db2.db server.

There are problems with this approach:

  • mysqldump locks the source database, making it inaccessible while the dump is happening.
  • mysqldump creates files which may be many times the size of the source database’s binary files, potentially exhausting the space on your source server before it’s even done.
  • the resulting file then needs to be imported into the target server, which could take hours depending on the size.

I needed to clone some databases in a hurry that are about 20G in size. The method I used ended up taking less than half an hour to complete, and the source database (db1.db) only had to be down for less than a minute, instead of the potential /hours/ in the mysqldump method.

  1. use rsync on db2.db to copy the data directories from db1.db to db2.db:
    cd /var/lib/ && rsync root@db1.db:/var/lib/mysql ./ -rva –progress –delete
  2. use rsync on db2.db to copy binary logs from db1.db to db2.db:
    cd /var/log/ && rsync root@db1.db:/var/log/mysql ./ -rva –progress –delete
  3. repeat 1&2 (the first time around would take some time. the second time around will be quick)
  4. on db1.db, stop the database
    service mysqld stop
  5. on db2.db, repeat 1&2 one last time
  6. on db1.db, start the database again, and start the slave service if you need to
    service mysqld start
  7. on db2.db, remove auto.cnf and any innodb log files
    cd /var/lib/mysql/ && rm -f auto.cnf ib_logfile*
  8. start the database, and start the slave if needed
    service mysqld start

With the above method, your source database will be down for only a minute or so (steps 4-6).

The reason that 1&2 are repeated 3 times:

  1. clone the db1.db database from scratch. this will take a while
  2. because it took so long to run #1, there are probably a lot of changes. repeat to get those changes
  3. when you stop db1.db, some files will get final changes as they are changed. grab those after db1.db has been stopped

You need to delete any existing innodb logs (step 7) which might cause the system to attempt to “fix” some tables it might think are broken. but, because we did a clean shutdown in step 4, this is not necessary. so delete the log files (they will be recreated automatically).

If you are doing the clone because you want to create a new slave database, then the database needs a new internal ID that it will send to the master. By deleting auto.cnf, you force the MySQL server to create a new unique ID.

25 Mar

front right top corner

I’ve done the back corners of the printer. Now, I can tackle the front.

The front corners are where I will put the motors that control the X/Y coordinates of the hot-end.

So far, everything I’ve printed is symmetrical, but the two belts are at different heights, so in this case, one motor will be higher than the other.

I’ve decided that the right motor (right when facing out from the printer. left when facing the printer) will be the top motor.

I’ve designed the model for this so that it can wrap over the end of the case edge (and you can screw into it) and bolt the motor into the model.

front right top corner, with model Nema-17 motor in place

front right top corner, with model Nema-17 motor in place

front right top corner model

front right top corner model

It’s best to print this one on its side, so there is no support needed, and less cleanup in that space between the walls. After printing this out for the first time, I found that the wall space in my print was too tight, so I adjusted the STL file to add 1mm more space. this should not matter much.

motor and printed model added to box

motor and printed model added to box

24 Mar

back top corners

CoreXY printers have two timing belts overlaid on each other around the box. To allow the belts to move, bearings are placed in various corners. Today, I’ll tackle the back top corners of the printer box.

In the image below (taken from a scene of this video), you can see how it’s handled usually:

image showing back corners of CoreXY belt system

image showing back corners of CoreXY belt system

Because I’m trying to avoid using any rods are other forms of complex structure, I decided to come up with a printed solution that I could attach to the wooden corners of the box.

The design with bearings and a washer in place will look like this:

back top corners of print, with two bearings and a washer in place

back top corners of print, with two bearings and a washer in place

This slots neatly over the wood at the back top corners of the box.

The design is not yet perfect. I anticipate there will be pressure towards the center of the box on the bottom bearing, so I should have screw holes at the bottom of those walls as well. But, I think this will do for the “bootstrap” printer.

An improvement I will be making as soon as the prototype is complete, is to replace the metal bearings with 3d-printed bearings, like in this video. That will get me closer to having a purely 3d-printed 3d printer. Also, 3d-printed bearings will be cheaper than metal bearings, reducing the cost for future printers.

So to create the corners, we will need to print out two each of the outer back top corners, and the inner back top corners. Don’t slot them together until you have your bearings. Otherwise you will find it difficult (or impossible) to separate them without breaking them.

inner back top corner. bearings and washer go on the pole

inner back top corner. bearings and washer go on the pole

Screenshot from 2016-03-23 20-13-32

outer back top corner. the hole on the top slots onto the inner corner’s pole to keep it still

Once your pieces are printed, place an LM8UU bearing on each pole, then a washer, and then another LM8UU bearing. Slot the bottom piece with the pole into the top piece so that the pole goes into its corresponding circular hole in the top piece. You might need to shave the top of the pole slightly to make this fit. Don’t shave too much.

Finally, place the corner pieces over the back top corners of the box and bolt them in place. For the other edge and corner pieces so far, you could use screws, but this one will need bolts because there will be inward pulling force on the pieces from the belts going through them.

where to put the inner back top corner pieces

where to put the inner back top corner pieces

I don’t yet have the bearings for the corners, so the photo below is of installation on one side without the bearings. When the bearings arrive, I’ll update this post.

back top corner. the belts loop around the pole on this (after bearings are added)

back top corner. the belts loop around the pole on this (after bearings are added)

23 Mar

putting the box together

KV Printer 1 will be basically a 50cm^3 cube, giving quite a large printable area.

Obtain a 5mm plywood sheet and cut 4 50cm^2 squares in it. These form the base and walls.

Next, we need to stick this together at the corners. To do that, print out 2 corner pieces and 6 edge pieces. Using these as templates, drill 2mm diameter holes in all corners of the wooden squares (they’ll be 20.5mm in from X and Y), then screw the squares together like in the third image below.

outer corner for 3d printer

outer corner for 3d printer

outer edge piece for 3d printer

outer edge piece for 3d printer

placement of outer corner and edge pieces

placement of outer corner and edge pieces

Notice that we have not yet fastened the back top edges together. That will be done in the next post.

The finished product at this stage looks like this:

printer box after installation of back bottom corner and side edge pieces

printer box after installation of back bottom corner and side edge pieces

21 Mar

building a new 3D printer

after working with the MakiBox 3D printer for 8 months, I think I’ve learned enough about its failings to start building my own.

I’ve started building a 3D printer of my own, based on the SmartCore idea, but with enough changes that this will be my own design.

Makibox (on the right) printing out pieces for the new KVPrinter version 1. The wood on the left is for the walls and base of KV Printer 1

Makibox (on the right) printing out pieces for the new KVPrinter version 1. The wood on the left is for the walls and base of KV Printer 1

The MakiBox printer’s major failing (as far as I’m concerned) is in how it controls the X/Y position of the hot-end.

To do this, it has two long horizontal threaded rods, against the back wall and the left wall. These rods have long arms positioned on the threads, extending out above the print bed. Where the arms cross each other, the hot-end hangs down. Thus, the position of the hot-end can be adjusted by turning the rods.

The problem with this method is easy to see when you consider an analogy. Hold a pencil normally, and draw a 1mm line. Now, hold the pencil by the eraser end and try draw a 1mm line. The precision is just not there. The further away from the fingers the pencil lead gets, the harder it is to control it precisely.

One solution to this which I thought of, is to use a Bowden cable (bicycle brake cables, for example) to fix the position of the arms at the screw side to the position of the arms at their opposite sides. This would work, and would increase the precision of prints drastically, but it’s a lot of work and would look ugly.

After seeing the SmartCore printer, I decided that instead of fixing what I have, I would use what I have to make a new printer. In a way, I am printing a new printer. At least, parts of one.

The SmartCore printer is based on the CoreXY positioning technology, which is similar to the Bowden solution I came up with. Here is a video showing CoreXY in motion

In CoreXY, the hot-end (or drawing thing in the video) is positioned on a moving platform. It can move in X along the platform, and the platform itself moves in Y along rods in the sides of the frame.

To reduce cost in my own printer, I will replace the Y and Z rods with ledges that the platform will slide along.

My calculations suggest that the material cost of my printer will end up being below €150. If this ends up being correct, and the printer is as good as I hope it to be, then I will sell kit packages of the printer for €200.

Bill of Materials:

itemamtcost per piecetotal
nema 17 motors2€18.61€37.22
nema 17 motors2€12.675€25.35
rods, 8x500mm2€4.58€9.16
lm8uu bearings12€0.5075€6.09
608 bearings10€0.237€2.37
timing belts (meters)5€1.004€5.02
controller board1€25.71€25.71
pfte bowden tube1€7.977.97
hot end1€8.53€8.53
psu, 12v 20a1€21.29€21.29
wood
Total€148.71

I’m working on construction at the moment. I’ll write more articles as I go.

16 Nov

jQuery stars plugin

I was asked to replicate a “star” effect, where stars appear in various areas around a page and then disappear after a while. I won’t bother linking to the original site as it will be gone shortly, but this is what I came up with:

demo

To use this on your own site, simply download the script, link to it in your page, then add this piece of JavaScript.

$('body').stars();

If you want to use the star image I created, download it to the same directory and tell the plugin where it is:

$('body').stars({
  "i":"stars.png"
});
20 Oct

clavichord fretting

My clavichord project stalled when I realised it was just not going to work the way I’d done it.

This is partly because I’d naively gone for the full un-fretted design in the beginning, then later realised this would put too much pressure on the cheap bodywork and cause it to implode.

Changing the design afterwards to a fretted design wasn’t going to work either, because of how the keys were laid out.

So, the new plan is to rebuild the keyboard on the current clavichord, and hopefully get the thing finished as a triple-fretted, single-strung design.

Now, to explain…

If you have one string per note, this is called “unfretted”. In this design, every string is only ever hit by only one key.

The usual way to design a clavichord is “double-strung”, this means that every note actually has two strings for it. This makes double-strung clavichords louder than single-strung clavichords, because the combination of the two strings’ waves tends to alternately strengthen and dampen what’s happening at the soundboard.

Back to fretting – consider a guitar. Despite only having six strings, a guitar can play many more than only six notes. This is accomplished by “fretting” the strings. When you play a “G” on an “E” string, what happens is that you are shortening the wavelength of the overall string (with the fret and your finger), causing it to play a different note (G) than it would play if it was unfretted (E).

With a clavichord, the very act of striking the string with the key (or “tangent”, as the striking edge is called) causes fretting. The strings are damped at the ends with cloth or felt so when the key is not touching the string, the string doesn’t vibrate.

When you design your keyboard to multi-fret the strings, you need to do some calculation – let’s say you have a note, C, which is struck on the strings 100cm (let’s say) from the bridge. If your fretting involves the C# hitting the same string, then that key’s tangent must hit the string at about 94.4cm.

This is quite a small distance between the two tangents (5.6 mm), meaning that if you decide to triple-fret all your notes, then the keys for the high notes will be very close together, and the lower notes will be further apart (lower notes have larger wavelengths, so the distance between semi-tone frets increases as well as you get lower).

That explains the following image (a double-strung, triple-fretted clavichord – click for a larger image):

Note that the keys are all squashed together on the right side where the high notes are, and the spaces gradually increase as you move further left.

Notice as well that at the extreme left, the increase in spacing stops and all the keys are together again.

The reason for this is that when the notes get too low, there’s simply no more room for multi-fretting, so instead, the lower notes are all one per string.

There’s one more point to make about the keys.

Let’s say you create a key, which has its tangent 25cm from the fulcrum (a clavichord key is a lever). When the key is pressed, the tangent arcs up and strikes the string. It is still 25cm from the fulcrum in a 3D sense, but when measuring x/y from a top-down view of the clavichord, if the string is 4cm above the tangent (with key at rest), then the tangent strikes the string about 22.5cm from the fulcrum.

This must be taken into account when you design where the strings will contact the bridge and the hitchpins, as getting this wrong will cause the tangents to miss. Yes, you could just place the tangents after doing the strings, but my goal here is to be as perfect as possible. (there’s also the added problem that the tangent’s top is a certain height (3cm, say) above the level of the fulcrum, but you get the picture)

I’ve explained some of the problems to do with designing a fretted keyboard and string layout. Now, I’m off to write a program to design one automatically!

07 Oct

jQuery k3dCarousel plugin

I needed a “3D” carousel to rotate images. The currently available jQuery plugins for this purpose were either too large, too restricted (showing only 4 at a time is not good enough), or too old (not working in jQuery 1.4+).

So I wrote my own.


demo

In our CMS, WebME, this is available in the Image-Transition plugin. (If you’re interested in using WebME, you can either download it through SVN, or talk to us about our reseller deals.)

How the thing works, is that you add a list of images to an element like this:

<div id="k3dCarousel">
	<img src="images/concrawood.jpg" />
	<img src="images/fernheasty.jpg" />
	<img src="images/mcnallys.jpg" />
	<img src="images/northernstandard.jpg" />
	<img src="images/soccertours.jpg" />
	<img src="images/soundinsulation.jpg" />
	<img src="images/textaroom.jpg" />
	<img src="images/windhoist.jpg" />
</div>

Then link to the jQuery library and the jquery.k3dCarousel plugin:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script src="jquery.k3dCarousel.min.js"></script>

And then tell jQuery to run the plugin after all the images have loaded:

<script>
	$(window).load(function(){
		$('#k3dCarousel').k3dCarousel();
	});
</script>

If you want, you can vary the speed of the cycle by changing the sT (spin-time) variable, or the wT (wait time), which govern how long it takes for the images to move from point to point, and how long they pause once reaching there:

<script>
        $(window).load(function(){
                $('#k3dCarousel').k3dCarousel({
                        wT: 500,
                        sT: 100
                });
        });
</script>

Interested yet? Download it (tar.bz2, zip)

26 Sep

SaorFM progress – the Hidden Files plugin

In the last post, I mentioned that a jQuery plugin had been created.

At the time, it could only be used as a file or directory selector.

It can now also be used to upload, download, and delete files.

I’ve also enhanced the core engine so that it now has plugin support.

The first plugin is a Hidden Files plugin, which lets you hide files or directories depending on their names. The default pattern is to hide nodes with names that begin with a ‘.’ – this is the default for most systems out there.

I’ll explain how plugins work, in the hope that someone out there may be interested in creating one 😉

For this example, I’ll use the Hidden Files plugin, as it’s already created, and uses everything in the engine so far.

First off, you create a directory in the /plugins/ directory of the plugin. In this case, I created the directory /plugins/hidden-files/.

In that, the only essential file is config.php, which holds a configuration for the plugin:

<?php
$config=array(
  'triggers'=>array(
    'checkfilename'=>'HiddenFiles_checkFilename'
  )
);

When you tell SaorFM to install the plugin (using $SaorFM->installPlugin('hidden-files');), it will read through that array and add the details to its own configuration file, so it doesn’t have to open up multiple files every time it boots up.

As an example, here’s my test config before the plugin is installed:

<?php
$SaorFM_config='{"user_files_directory":"\/home\/kae\/websites\/saorfm\/trunk\/tests\/\/userfiles"}';

And here’s how it appears after the installation of the plugin:

<?php
$SaorFM_config='{"user_files_directory":"\/home\/kae\/websites\/saorfm\/trunk\/tests\/\/userfiles","plugins":["hidden-files"],"hiddenfiles":"#^\/?\\.#","triggers":{"checkfilename":["HiddenFiles_checkFilename"]}}';

There are three additions:

  • "plugins":["hidden-files"] – this is an array of installed plugins.
  • "triggers":{"checkfilename":["HiddenFiles_checkFilename"]} – the $config array above had a trigger mentioned – that’s converted into an object and added to the SaorFM config.
  • "hiddenfiles":"#^\/?\\.#" – this is to tell the hidden files plugin what pattern to apply to file names to decide whether they should be hidden or not.

The first two are obvious – the first is the name of the plugin, and the second is the name of a trigger named in the config array.

For the third, we need to run an installation script when Hidden Files is installed, so SaorFM checks the /plugins/hidden-files/ directory to see if a file exists named install.php:

<?php
if (!isset($this->_config->hiddenfiles)) {
  // the default pattern hides files that begin with .
  $this->_config->hiddenfiles='#^/?\.#';
}

That’s run by SaorFM when the plugin is installed. It adds a default hiddenfiles value to the config. This can be changed at any time afterwards.

Ok – so we have the config installed – now how does it work?

Let’s say we ask SaorFM to list the files in ‘/’: $SaorFM->listFiles('/');

Inside the listFiles method, there is a loop which adds filenames to an array. For each one of the filenames, we check it before adding to the array:

    foreach ($files as $f) {
      if ($f->isDot()) {
        continue;
      }
      if ($this->trigger('checkfilename', $f->getFilename())) {
        continue;
      }
      $arr[$f->getFilename()]=array(
        $f->getFilename(),
        $f->getSize(),
        $f->isDir()?1:0 // record as 1 or 0 to save http traffic
      );
    }

Notice the $this->trigger('checkfilename', $f->getFilename()) – if that returns anything other than false, then the file is not added to the list.

The trigger method cycles through any functions which have been named for that trigger, and calls each one in turn. If any return a result other than false, the method returns that result immediately:

  public function trigger($trigger, $vals) {
    if (!isset($this->_config->triggers->{$trigger})) {
      return false;
    }
    foreach ($this->_config->triggers->{$trigger} as $function) {
      $ret=$function($this, $vals);
      // if the trigger gives any response other than false, then return it
      if ($ret) {
        return $ret;
      }
    }
    return false;
  }

Note that we haven’t yet defined the function HiddenFiles_checkFilename.

Upon installation, we also load up the file /plugins/hidden-files/functions.php if it exists, and this file is loaded every time that SaorFM is instantiated after installation as well:

function HiddenFiles_checkFilename(&$saorfm, &$filename) {
  return preg_match($saorfm->get('hiddenfiles'), $filename);
}

This is a very very simple function – it could get a lot more complex in other plugins. For example, an Authentication plugin would be much more verbose.

I hope this is all very clear. It’s very very simple to add arbitrary triggers to SaorFM, so if you want to write a plugin and the trigger doesn’t exist, just tell me and I’ll put it in place.

After all this, let’s say you want to uninstall the plugin.

Within SaorFM, you just call $SaorFM->uninstallPlugin('hidden-files');.

This automatically removes the ‘hidden-files’ value from the plugins array (and removes that array if it’s empty), then removes any added filters as well.

Finally, it also checks to see if /plugins/hidden-files/uninstall.php exists and runs that if so:

<?php
unset($this->_config->hiddenfiles);

After this, the SaorFM config file is back to the exact state it was in before we installed the plugin. Clean.

If you’re interested in hacking on SaorFM, please download a copy from SVN here.

There are a number of things we need to create before it’s usable by the general public. An installer, for example, and some widgets for popular RTEs like CKeditor or TinyMCE. If you feel you’d like to create those, please contact me.