08 May

Scaling FieldMotion

I wrote a quick overview of how I approach the scaling-up of FieldMotion, over at the FieldMotion blog.

Basically, we started off with a huge monolithic block of code and data, then looked carefully at it to see how we could tear it apart into logical chunks.

The simplest to start with was data, so we separated out the database into a MySQL master/slave cluster, and a MongoDB replicated shard cluster.

Afterwards, we looked at the actual services that were performed by the server and started separating those out so they were completely independent from the main block.

At the moment, FieldMotion runs across more than 60 servers, and can tolerate sudden catastrophes on pretty much any part of it. We actually had an incident recently where an entire datacentre went offline for about 8 hours, but no-one noticed because we make sure each of our replicas is in a different datacentre.

05 May

yet another update

Time flies. I keep on planning to do things, and then failing to do them because there isn’t enough time, in between working 12 hours a day and trying not to fall asleep as soon as I get home.

I finished the basics of my next book, Live Forever, which I put up in website form so I can figure out through statistics which pages (a lot of them!) need work. Tonight, I’m working on the Cancer chapter so haven’t put that in there yet.

Over the weekend, I hope to get a start on a new project, which will help to design 100% nutrition diets based on common supermarket produce. There are known recommended daily allowances (RDAs) for all nutrients, but when you make your dinner, you don’t calculate an optimal meal because it’s just not practical or easy. The new project is designed to get around that by offering meal plans that are affordable and personalisable (you will be able to put your preferences into it). We’ll see if that gets off the ground!

In CoderDojo, some of my students (I really mentor them, more than teach, but what do you call someone you mentor? Mentoree?) are working on some interesting projects for this year’s Coder Dojo conference and next year’s Young Scientist. Two examples: programmable magnetic levitation, and a laser harp.

In work, we’ve moved beyond the frantic development stage that all companies go through, and are now in stabilisation mode, making sure the system is bulletproof and can scale well beyond current needs. I still find it interesting, even though the work I’m doing at the moment is not flashy and user-visible. Today, for example, I was writing a logging system to make sure that even though users access our mobile servers in a “round robin” method at the moment and the logs of their visits are therefore scattered among the servers, I can still aggregate them on the other end into something that can be searched easily. Not flashy, but quietly satisfying.