Read more →

I’m slowly migrating a bunch of awesome things from a really old server, it’s still running Ubuntu 10.04.. to a really nice and shiny one. Which has 2 new 3TB HDDs in RAID 1, which are syncing..

<br /> cat /proc/mdstat<br /> md3 : active raid1 sda4[0] sdb4[1]<br /> 1847478528 blocks super 1.2 [2/2] [UU]<br /> [>....................] resync = 0.1% (2061952/1847478528) finish=28857.9min speed=1065K/sec<br />

Read more →

Earlier today I was setting up a brand new server for a migration and just as I was typing scp .ssh/authorized_keys2 my brain went and asked a question..

What is the difference between authorized_keys and authorized_keys2?

I’ve been working with Linux for well over a decade and some of my practices stem from things I learned in the ’90s that still work, putting all my public keys in ~/.ssh/authorized_keys2 is one of those things.

Read more →

My HTPC is almost appliance like, in the way I never upgrade it, i.e. this morning it was still running Ubuntu 11.04 Natty Narwhal… It’s also not very appliance like in that it’s also full of random development stuff that probably shouldn’t be on it as well as several different types of databases, my backup infrastructure and tons of other things you really don’t care for.

But not today. Today I was bored and decided the best use of my time was to replace MythTV with XBMC. Not only that, I figured I should probably upgrade from

Read more →

I gave up drinking 3 months ago, but still have a fridge full of beer which I didn’t want to pour down the sink, so I decided to make something with it. I found My Fridge Food, which is an awesome little site that will take the contents of your fridge and cupboards and suggest recipes! A lot of the recipes were rubbish, poached eggs,

Read more →

Back in September Google released it’s new Hummingbird algorithm, apparently affecting 90% of search queries, as it strives for greater accuracy and more relevant results. Aimed at ‘conversational searches’, like what is the best cake? rather than simple terms such as best cake. (when is the last time you made a search request like that?)

Then on the 4th October they released Penguin 2.1, aimed at reducing the quality of spammy backlinks, I assume the theory is that better sites have better backlinks, (as they have been around longer?) and newer sites, (trying to use spammy backlinks to gain traction?) can’t be as good or as relevant. Fairly naive thinking, though I guess it might help in ‘cleaning up the web’?

Read more →

There are several things to think about when taking control of another computer, the Operating System running on it, the speed of your network connection and the tools you have at your disposal.

How To Take Control Of Another Computer

Operating System

There are 3 main choices of Operating system that the computer you want to take control of might be running, Windows, Mac OS X and Linux. Fortunately Mac OS X is based on BSD so the tools you would use to take control of it are the same as you would for Linux, simplifying things some what!

Read more →

We’re using an old version of Upstart, on Centos, to manage stopping and starting our Node.js daemons, and one of the things the script does, like any good deamon, is change the user of the deamon process from root to something more applicable, security and all that 😉

The scripts look a little like this

!upstart
description "Amazing Node.js Daemon"
author "idimmu"
start on runlevel [2345]
stop on shutdown
env PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
env NAME="amazing-daemon"
script
export HOME="/root"
cd /opt/idimmu/$NAME
echo $$ > /var/run/$NAME.pid
exec sudo -u idimmu /usr/bin/node /opt/idimmu/$NAME/server.js >> /var/log/$NAME/stdout.log 2>&1
end script
pre-start script
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (upstart) Starting $NAME" >> /var/log/$NAME/stdout.log
end script
pre-stop script
rm /var/run/$NAME.pid
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (upstart) Stopping $NAME" >> /var/log/$NAME/stdout.log
end script

Which is nice, as it means we can use Upstart to stop/start/status deamons really nicely. The equivalent init.d script looked really horrible.

Read more →

I’ve got a rather large dataset that I need to do a lot of processing on, over several iterations, it’s a 20gb zip file, flat text, and I’m impatient and don’t like not knowing things!

My new favourite Linux command line tool, pv (pipe viewer) is totally awesome. Check this out:

<br /> pv -cN source < urls.gz | zcat | pv -cN zcat | perl -lne '($a,$b,$c,$d) = split /\||\t/; print $b unless $b =~ /ac\.uk/; print $c unless $c =~ /ac\.uk/' | pv -cN perl | gzip | pv -cN gzip > hosts.gz<br /> zcat: 93.4GiB 1:33:18 [26.6MiB/s] [ <=> ]<br /> perl: 85.7GiB 1:33:18 [25.3MiB/s] [ <=> ]<br /> source: 13.2GiB 1:33:17 [3.57MiB/s] [===============================================> ] 67% ETA 0:44:41<br /> gzip: 12.7GiB 1:33:18 [3.51MiB/s] [ <=> ]<br />

Read more →

One of our applications (Freeswitch) just randomly crashed for no apparent reason and didn’t write anything to it’s log files. The service we’re trialling is currently in Beta so there’s room to muck about and do some diagnostics. I want to make the kernel dump a core file whenever Freeswitch dies, in case it happens again, so that we have some stuff to work with after the fact. It’ll also shut up my QA manager.

Read more →