wow... this looks pretty cool. many of the things we scream about at work when using C or C++ seem to be some of the same things that this language is trying to fix. I especially like the concept of the gofmt command!

http://golang.org/

Finally managed to get the netbook onto my rsnapshot backup scheme. As I talked about in this post, there are issues trying to backup machines with rsnapshot that use WOL or are wireless netbooks that are sometimes on and sometimes not.

So, I wrote a script that constantly polls for a machine and backs it up if it is one the network (detailed in the linked post above).

But now... a new challenge. This netbook runs windows... and it is wireless so WOL is not really an option... AND.. it is XP home edition, so the samba sharing is not really what I would like. This means I need a rsync server on the windows netbook.

Downloaded and installed cwrsync server. Read through the documentation and this by itself just looks like a gigantic security hole, especially for a travelling netbook. So, I also installed copssh, an ssh server for windows, and then did the following:

Step 1: Install copssh and cwrsync server
Step 2: Configure cwrsync to only allow hosts "127.0.0.1" which is localhost
Step 3: Configure ssh keys for key based login
Step 4: Change copssh to run on non-standard ssh port
Step 5: Configure .ssh/config to specify the username and port to use when connecting
Step 6: Modify the rsnapshot script I presented in the linked post to open up an ssh tunnel before running rsnapshot, and then to kill the tunnel afterwards
Step 7: Create an rsnapshot configuration file but specify the host as "localhost" since the tunnel will provide the mechanism to the netbook

Ultimately, this seems to work pretty nice. When I catch the netbook powered on and on the network, and it hasn't been backed up recently, I back it up through an encrypted ssh tunnel. The rsync is protected by the windows firewall and the host blocking line. The ssh server is protected by running on a non-standard port, by having the nonstandard port firewall entry in Windows restricted to the two hosts I might ssh from, and using key based login. Lastly, all the transmitted info is encrypted.

For those interested, here is the mods to the previous script I presented:

rsnapshot_wrapper_some_machine.csh:

... previous script stuff ...

    ping -qnc 1 some_machine > /dev/null
    if ($?) then
        # If it has been more than a couple of days, alert the admin
        if ( $time_since_last_success > $warn_secs ) then
            echo "some_machine failed. Consider manual run"
            echo "some_machine failed. Consider manual run" >>/var/log/rsnapshot
        else
            echo "some_machine not awake.  Exiting..." >> /var/log/rsnapshot
        endif
    else
        echo "Found some_machine... Attempting backup..." >> /var/log/rsnapshot
        cd -

        # Kill any old zombie tunnels
        ps ax |grep "ssh" |grep "somehost |awk '{print $1}' |xargs -i kill {}

        # Establish the tunnel
        ssh -f -N -L 873:localhost:873 user@some_machine

        # Run the rsnapshot job
        rsnapshot -c /etc/rsnapshot.conf.some_machine $1

        # Kill the tunnel
        ps ax |grep "ssh" |grep "somehost" |awk '{print $1}' |xargs -i kill {}

... rest of script ...

Seems to be working!

very cool... might use this when discussing code with peers:

Codepad

September 30th! If this protocol catches on... being open like email is... it could revolutionize the web.



I've been watching this for some time, so it is exciting to see a date assigned.

If you have no idea what Google Wave and the Wave protocol is, watch the above video. Incredible stuff.

My favorite parts: Rosy the robot... the sample integration with a bug tracker and a blog... and the Initech wave client in an ansi terminal!

why oh why do people put up with this.

Got my updated firmware for my T-Mobile G1 phone yesterday. This is the infamous "Cupcake" version of the software.

I'm really happy about most of the fixed. Many of the new features are awesome, and many of the annoyances have been fixed. I like the way they fixed the issue of keeping the dialpad open during phone calls where you need to hit buttons to get through menus.

Phone orientation detection, virtual keyboard, updates to the browser, gmail, calendar, etc... they're all awesome! Although, I still can't believe that all of this work went into such an awesome phone and such awesome software, and there is still no way to alert someone that they missed a call. How this phone has went through multiple revisions of software without the capability to beep after a call was missed is beyond me.

But that really is the only main thing that annoys me. Everything else is great. I'm so glad I have a G1 and not an iphone. I'm really tired of the great development that Apple does overshadowing all the evil crap they get away with. Android... open, free, and awesome.

On the computing front, got around to trying a couple of things.

A while back, I set up some simple software to use talk to X10 plugs from the computer so that I could schedule things (such as the Christmas tree turning on and off). I detailed it on this project page.

But now that my windows machines on all hibernate and use power saving features, the only machine that is on all the time is my linux server. I needed to move over the scheduling to the linux machine with some simple cron jobs.

And with this small software called BottleRocket, this was super easy. Now, turning on the Christmas tree (device a1) is as simple as:

> /usr/local/bin/br a1 on

Awesome.

Next up is to fix another issue caused by my windows machines constantly hibernating. Backups.

My current backup solution uses a really sweet piece of software called rsnapshot which uses a backup concept of rsync combined with hard links for super fast and super efficient backups. The only problem is, if the machines aren't on, then the backups fail. I have some of them using Wake-On-Lan (WOL) so that I can wake them up before I back them up, but if one machines takes a long time to backup, then the others could go back to sleep. Also, I have one machine which does not support WOL.

So, I started looking into BackupPC. This is a very highly rated backup system that is open source and very useful in enterprise environments. It supports the same concept as rsnapshot (rsync and hard links to de-duplicate data), but it also is set up to find machines when they are available on the network and back them up, which is what I need.

Long story longer, I fought with this ALL weekend. I actually got everything working with BackupPC. It does what I needed. But, the last part of my backup solution that I need is access to the snapshots so that I can duplicate it to an external hard drive when we evacuate for a hurricane, or just for archival.

This is where the BackupPC finally convinced me to go back to rsnapshot. BackupPC mangles the filenames in the backups. They have a reason for it: it is so they can differentiate between backed up files and their own attrib files. Plus, since their access to the data is supposed to be through the website, this should be a non-issue.

But for me, I don't even need the website. I'm the only one at home accessing backups. Also, in a lab environment, it is nice to let people mount the backup volume as read only and have access to it. With the file name mangling that BackupPC does, this does not work.

So, after an entire weekend of configuring, messing, trying, and testing, I decided to go back to rsnapshot, which of course left me with my original problem. So, I had to add some wrappers around rsnapshot to make it smart.

For the machine that doesn't support WOL, I needed to do something exactly like BackupPC. That is, I need to run a job often and check and see if the machine is available on the network. If it is, and it has been a while since it was backed up, then it needs to get backed up. If it isn't online, or if it was backed up recently, then it needs to get skipped.

In case anyone is interested, here is the csh wrapper script I am using in a hourly cron job to accomplish this. Since it is being run from cron, anything printed to stdout will get mailed to the system administrator email. So, if I detect that it hasn't been backed up in quite a while, I alert the administrator. (NOTE: The script needs a little cleaning... a few of the items in here should be in configurable variables, such as the log file to append to and the machine name. Apologies.)

rsnapshot_wrapper_some_machine.csh:

#!/bin/csh -f

if $1 == "" then
    echo "usage: $0 "
    exit
endif

##########################################
#                                        #
#    some_machine                        #
#                                        #
##########################################

set current_time_string = `date`
echo " " >> /var/log/rsnapshot
echo "  some_machine $1 attempt at $current_time_string" >> /var/log/rsnapshot

if (-f /var/log/rsnapshot_some_machine_last) then
    set last_success=`cat /var/log/rsnapshot_some_machine_last`
else
    set last_success=0
endif

set current_time=`date +%s`
@ threshhold_secs = 60 * 60 * 23
@ warn_secs = 60 * 60 * 72
@ time_since_last_success = $current_time - $last_success

if ( $time_since_last_success > $threshhold_secs ) then
    # Wake up machine
    echo "Waking up some_machine..." >> /var/log/rsnapshot
    wakeonlan 00:00:00:00:00:00 >> /dev/null
    echo "Sleeping ..." >> /var/log/rsnapshot
    sleep 35
    echo "Done sleeping.  Attemping to find via ping..." >> /var/log/rsnapshot
    ping -qnc 1 some_machine > /dev/null
    if ($?) then
        # If it has been more than a couple of days, alert the admin
        if ( $time_since_last_success > $warn_secs ) then
            echo "some_machine failed. Consider manual run"
            echo "some_machine failed. Consider manual run" >>/var/log/rsnapshot
        else
            echo "some_machine not awake.  Exiting..." >> /var/log/rsnapshot
        endif
    else
        echo "Found ping.  Attemping to automount..." >> /var/log/rsnapshot
        cd /misc/some_machine/
        touch foo
        echo "Found automount... Attempting backup..." >> /var/log/rsnapshot
        cd -

        # Run the rsnapshot job
        rsnapshot -c /etc/rsnapshot.conf.some_machine $1

        # Save the successful backup time
        date +%s > /var/log/rsnapshot_some_machine_last

        set current_time_string = `date`
        echo "some_machine $1 done - $current_time_string" >>/var/log/rsnapshot
    endif
else
    echo "Not enough time has passed for some_machine." >> /var/log/rsnapshot
endif

Seems to be working. I'm a little less stressed now.