Check this out.  Get to 30.  Your day is now shot :)

(I just finished.)

Happy fourth!

I just wanted to post a note bragging about my sister, Jill.  The Chicago Tribune just ran an article about her and her daughter Ellie.  Ellie has what they are calling "Sensory Integration Disorder."  She is basically really hyper and needs to jump, spin, rock, swing, etc.  I think I probably had a touch of something similar when I was small.  One of the things that helps Ellie has been a weighted vest.  Jill couldn't find any she and Ellie liked and so made one herself.  It turned out so well, she decided to start making them for others. 

I've helped her get a web site up and running to show off what she has done.  Check it out at www.customweightedvests.com. You can also check out some other custom sewing she does at www.stitchessosweet.com. 

Scott Ludwig and I have released a new version of his python based backup script, Link-Backup.  This new version ignores broken symlinks and has an option to ensure that only one backup at a time is going on.

Also with this release is a cgi script (viewlb.cgi) for exploring the backups.  This makes it easy to keep tabs on what is going on.  Scott wrote this script a while ago and I updated and improved it.

Details here.

I haven't been posting anything lately because I've been so busy with work and Annie.  It is funny how life gets in the way.

In any case, she really loves flowers.  We try and go on a walk in the Washington Park Arboretum every day.  She loves to hold and pick little flowers.  Here is a quick photo I snapped of her today:

Speaking of the Arboretum, almost every time I go there, I see the official bird of Seattle, the Great Blue Heron.  Here is a snap of one of those guys from a couple of weeks ago.  He has just caught himself some dinner.

Between photos of Annie, other photography and music, I have quite a bit of data.  Probably over 300GB now and growing fast.

Backing up this much data is a challenge.  I've set some requirements of for what I needed in a backup system:

• Automatic.  A backup system is useless if you don't actually use it.  I know that if I'm required to do anything manually it probably won't get done.
• Off site.  There are too many things that can happen if you have an on site backup.  Your house can burn down, you can get robbed, etc.
• Virus resistant. If somehow a virus were to try and wipe everything out, I'd like my data to survive.
• Versioned.  In case I delete something and don't realize right away, I'd like to keep versions of my data.
• Big. I have lots of data and I don't imagine that I'm going to stop accumulating it.  I'd like to plan for 800GB-1TB.
• On line.  I want to be able to get to my backup without mounting/moving anything.

This is a pretty tall order.  Here are some things that I considered but decided not to do:

• RAID. Mirroring is pretty secure in the face of hardware failure.  RAID 5 sounds great but stories I hear is that if one drive goes there is a serious chance another will go during the rebuilding.  Simply doing RAID also fails to satisfy many of the requirements above.
• DVD.  I started backing some stuff up on DVDs.  I got read errors a month later.  Too manual also.
• External Harddrives. I thought about having a couple of really big external drives and rotating them.  This solves a lot of the problems above but it is pretty manual.  A friend of mine is going to do this and keep a copy at the office.  He said he planned to backup/move every couple of months.  That is just too long for me.
• NT Volume Shadow Copy. This is a cool technology that can keep snapshots through time on a drive.  It looks like the Linux Volume Manager (LVM) can do some similar things.  This isn't really a backup solution as much as a versioning solution.  This plus RAID 0 is probably pretty good except it isn't off site.

I ended up going with a more brute force solution.  I bought and set up two Linux servers.  The first is a home server.  The second is in a datacenter in downtown Seattle.

The home Linux server is a dual core Pentium running Ubuntu 5.10 server.  I have a HW/SW raid card driving 5 250GB drives in RAID 5.  It servers a data share over Samba.  I backup that share to another directory every night.  That archive directory is shared out via Samba also, but as read-only.  This makes the situation fairly virus proof.  Since I have two copies of the data (one that is r/w and the other that is a backup) I might have to add more drives in the future.  I got a big honking case so I'll have room.  I'm running slimserver on this amoung other things.  I also use it to do long running batch enblend stitching jobs for panoramas.

The data center machine is a 1U Tyan server with 4x320GB drives in Linux SW RAID 5.  It is also running Ubuntu 5.10 server.  (Getting Ubuntu installed on a RAID5 drive array was a challenge.  I don't remember the exact steps I took or I would document them here.)  The cost of doing this can be high unless you have a friend that can get you hooked up.  Even if you can't, it might be worthwhile.  I'm backing up to this every night over my cable modem.  The cable modem upload speed is okay since I upgraded to Comcast's higher level of service.  I now have ~768kbps up.

There are a couple of choices for software for doing the backups.  The key is that it has to be bandwidth smart (only update diffs), handle versioning gracefully and be able to do partial copies based on a timer.  The last requirement is so that if I have a ton of data to upload it can go over a couple of nights.  The most obvious candidate for this is rsync.  rsync is an amazing tool for these types of things.  My MS friends think that robocopy is cool -- it can't hold a candle to rsync.  It also generally operates over ssh so it is also secure. It can also build version trees where unchanged files are hardlinked to previous versions.  The only thing it can't do is handle files that have moved but haven't changed, and stop itself after a certain number of minutes.

To solve this problem, Scott Ludwig (a coworker of mine who has a similar setup) developed a python script that does much of what rsync does but solves these two last problems.  It is called "link backup" and you can get it here.  Every night starting at 11:30pm, I run this backup script to backup my working directory to the backup directory on my home server.  This is usually pretty quick.  I then backup the latest snapshot of that backup set to the server in the datacenter.  This can take a little longer but at least I don't have to think about it.

Since I implemented this, Amazon's S3 has come on the scene.  While it might be interesting to back up using S3, I'm not sure how the economics work out.  At my current level (300GB) and Amazon's pricing model ($0.15 per GB per month) I would be paying $45 per month.  If I grow to 600GB, I'm up to $90 per month.  Bandwidth is extra, but I don't use much of that.  It should be easy to find colocation hosting for that amount of coin.  There is the one time cost of the hardware and the work of keeping it up to date also (I think my server came in around $1600 but I probably overbought).  The advantage is that you can run and do other things with your own server in a datacenter.

[April 16, 2006: Edited to fix Ubuntu version number.]

If you haven't heard yet, we shipped a new version of the Google Talk client to a small set of testers yesterday.  You can get a review over at customizetalk.com.  Hopefully we'll fix any outstanding issues and get this version out to everyone soon.

I wasn't super involved in this version but did manage to sneak a couple of features in.  The first was Friend Online notifications (defaults to off to avoid annoying people). I also did the new hover cards for your Friends.  I wanted people to be able to see all of the cool pictures in true 96x96 pixel alpha blended glory.

We hope you like it!  Let us know what features are important to you by filling out our suggestion form.

I was going to stay away from all of the April 1st shenanigans, but couldn't pass up linking to the new Jingle transport spec that Peter Saint-Andre put together:

JEP-0183: Jingle Telepathy Transport

Nice one!

I must be getting old because every week I look forward to listening to The Swing Years and Beyond on KUOW, our local NPR station.  It runs from 7-12 every Saturday and is always fun to listen to.  I'm not an expert in the era or anything but Amanda Wilde (and Cynthia Doyon before her) always seem to hit the right note.  When I got my Honda Element it came with a free trial for XM radio.  They have a 1930s big band/swing station but it just isn't the same.

If you don't live in Seattle you can listen to their streaming feed.  No podcast/mp3 available though.

My grandmother died recently and listening to this music always reminds me of her.  One of the songs they played a couple of weeks ago was "Chattanooga Choo Choo."  I can't hear it without thinking of her favorite joke (copied from here):

Roy Rogers had just purchased a new set of boots. Going out one day to check on Trigger, Roy stepped in a pile of ... well suffice it to say that Dale told him to leave his boots outside on the porch until he could clean them up. Roy did so and went inside for dinner. In the meantime a cougar wandering around on the property spied the boots, grabbed them up in his mouth and headed for the wilderness. Roy, seeing his new boots heading for the hills, pulled on a pair of old boots, ran out and saddled up Trigger and headed for the hills to cut off the escape path of the cougar. Several hours later Roy returned, new boots in hand and the cougar stretched over the back of Trigger. Dale looking at the scene in front of her said "Pardon me Roy is that the cat that chewed your new shoes?"

There are just so many ways my daughter won't get this joke.

I'm sorry that I missed Peter Saint-Andre's presentation at Etel this last week.  I wanted to be down there but it is hard to get to conferences with the baby and all.  You can check out his slides off of that link.

Peter used a presentation style that I really like a lot.  I'd love to see how he pulled it off.  Perhaps some video will surface?  If you want to see videos of talks done in a similar style check out Larry Lessig's talk on Google Print or Dick Hardt's talk on Identity 2.0.  Very cool.  I'm not sure I could pull it off though :)

Speaking of conferences, I am planning on being around for a day or two of Spring VON.

Last night was the first time Annie slept through the night.  We put her to bed around 8pm and it is now 6:20am and she is still asleep. We've been working over the past couple of nights to eliminate her late night feeding and help her to get back to sleep on her own.

I mentioned to my wife this morning I feel like we discovered the lost city of gold.  Even if it doesn't go like this every night it is good to know that it can be done.

Here is a recent picture of our little babe: