I came across an article today that lifted my spirits:

100% Verifiable Bug-Free Code Is Possible (Dobbs Code)

The executive summary: ok-labs.com and nicta.com.au have developed a microkernel that is provably correct, in defiance of the conventional wisdom that formal methods are only practical for the simplest of projects. While I take my hat off to the developers for their heroic efforts, I firmly believe this is only the first step in a process that will revolutionise the industry. The ultimate goal should be a complete, provably correct computing platform. This will not come about quickly or easily, but there are practical things we can do in the meantime.

Recently, the SANS Institute published a list of the 25 most dangerous programming errors. Most of these, e.g. buffer overflows, are rookie mistakes that every apprentice programmer is warned about, yet even with better training and increased collective experience, the population of errors in the wild seems to increase daily. The reason is simple: as the number of programmers grows, and their productivity increases, the number of opportunities for errors to creep in grows faster than the ability of the human mind to be vigilant. Or to paraphrase Murphy’s Law: if a programmer can make mistakes, he will make mistakes.

The conventional wisdom is to accept that such errors will be made, and be vigilant in testing. But no amount of post-facto investigation can give us 100% confidence that a catastrophic error doesn’t lurk somewhere in the code. With software increasingly responsible for people’s lives (fly-by-wire aircraft, nuclear power), the potential consequences of code errors will become unmanageable. Let me demonstrate.

The standard equation of risk analysis is

risk = probability × severity

What this means in practice is that in order to maintain risk below a given maximum, any event that has N times more severe consequences (e.g. a plane crash that kills 10 times more people than a car crash) should happen N times less often. But going from 99.999% (five nines) certainty to 99.9999% (six nines) is a lot harder than going from 99% to 99.9%, due to diminishing returns. Throwing man-hours of testing at such a problem does not scale.

Formal methods give us a way around this impasse. But we do not even need the expense of formal testing to eliminate the most common errors – design choices in our programming languages can do a lot of the work for little cost. Consider the state of the art before the introduction of structured programming in Algol-60. Without nested loops or multi-line conditions, the GOTO statement was king. GOTO-based programming languages are Turing-complete, in that they can perform any conceivable algorithm, but they are too flexible. It is quite possible (and easy) to write nonsense algorithms where loops would start or end inside the ELSE block of an IF statement, for example*. No modern programmer would contemplate such absurdities – there was a very good reason why GOTO statements were banished from polite society. And yet programming languages without GOTO statements are still Turing-complete.

What the above history lesson shows us is that, given sufficiently intelligent design paradigms, we can remove once-thought-necessary features from our lexicon without sacrificing Turing completeness, and thereby eliminate entire classes of potential programming errors.**

Now consider buffer overflow – this can be generalised to all kinds of array bounds violation; a buffer being merely a 1D array. The array index is constrained between two values, typically 0 and some integer N. If the algorithm attempts to access the array with an index outside this range, we have an array bounds violation. If this is not tested for explicitly by the algorithm (easy for a stressed programmer to forget) or implicitly in the execution environment (inefficient and not always enabled) undesirable behaviour can easily result. We could laboriously check every subscription operation in our algorithm to ensure that sane values were being passed at all times; or we could change our programming paradigm so that it is impossible to write an invalid subscription operation.

This is not as outlandish as it sounds – we already have a commonly-used language feature that does most of the work for us: FOREACH. A FOREACH loop iterates over the elements of an array, and automatically stops when there are no more elements. We cannot pass an invalid value into the array subscription operation, because there is no such operation; subscription is handled implicitly by the FOREACH. If all buffer operations used FOREACH statements, there would never be an overflow.

Now say that at some point in the near future we find similar, implicit, methods for array subscription that cover all sensible use cases (in the same way we found alternatives for all sensible use cases of GOTO). We could then eliminate the explicit array subscription operation from our lexicon, and buffer overflows would be a historical curiosity.

I think such a development is within reach. Like many programming geeks I have been working, on and off, for some time on a pet language project which, amongst other things, features strict limitations on the array subscription operator, as well as a plethora of implicit alternatives to handle common use cases. I haven’t yet been able to eliminate subscription completely, but all ideas would be gratefully received.

* This is a real example from personal experience, which shockingly did work but was (unsuprisingly) impossible to understand.
** Of course, if the language is still Turing-complete, it will therefore still be possible to make the same error; but you will have to go out of your way to deliberately do so.

Despite having played with Democracy Miro, I’m still rather attached to Azureus Vuze for one or two of its useful features*.

Being an addict of US scifi TV, I was excited to discover feedmytorrents.com, which provides vodcast RSS feeds for most current popular US shows, such as my current favourites BSG and Lost. The killer advantage over other RSS sources (such as the excellent EZTV) is that one can subscribe to a particular feed in Miro or Vuze and get precisely one copy of every episode of that show. No more remembering to search mininova to see if the new episode was up yet! This was exactly what I had been waiting for since discovering Democracy Miro’s channels system.**

Of course, it didn’t take long for the system to break. Vuze’s RSS FeedScanner plugin would display a new episode in the list but not start downloading, instead giving a constantly shifting error message: “Init” > “Scanning 1/64…” >>> “Scanning 39/64…” > “Init” (it never seemed to get past 39). This would eventually time out saying “No data”. This affliction would only affect those particular episodes that feedmytorrents had sourced from mininova, but not those from the pirate bay (easily distinguished by the torrent URL). There was nothing wrong with the torrent file itself, as I could right-click > copy link URL and paste it into firefox, which would happily download the torrent and open it in Vuze, the default application. But of course this completely defeated the seamless experience.

I searched on the Vuze forums and found several similar problems reported but few solutions. One suggestion was that the setting “Use URL as referer” might need to be turned off. Sure enough, this worked. For whatever reason, it seems that mininova doesn’t like the HTTP referer field to be set to the “URL”, whichever URL that may be (itself?). Turning this option off (thus sending no referer) didn’t seem to break piratebay torrents (and why should it: this is the behaviour when one types in a URL by hand), so I disabled it on all my feeds. This makes me ask two questions. 1: why isn’t this option off by default and 2: what the hell does the HTTP referer field have to do with it anyway?

* coughcoughtorcoughcough! Interestingly, the latest version (4.1) of Vuze has reskinned itself to look almost exactly like Miro…
** sorry guys, I don’t watch Rocketboom…

Spreading the meme…

serenity:~ andrewg$ history | awk ‘{a[$2]++} END {for(i in a)print a[i] ” ” i}’ | sort -rn | head -10
108 telnet
63 rscreen
43 ping
24 sudo
24 ssh
20 host
16 scp
16 more
14 ls
13 xdvi

Hm. I seem to be using the command line mainly as a gateway into remote systems – which reflects my average working day. The stray ‘xdvi’ is due to my recent heavy use of TextMate  to write a paper in LaTeX. Not sure why I’ve been using sudo so much on my Mac though.

Similarly, on my work Linux laptop:

andgal@nbgal185:~$ history | awk ‘{a[$2]++} END {for(i in a)print a[i] ” ” i}’ | sort -rn | head -10
68 rscreen
52 ping
45 host
36 sudo
35 rdesktop
32 xrandr
21 startmenu
20 ssh
18 ifconfig
15 telnet

rscreen is merely a wrapper for ssh:

function rscreen() { /usr/bin/ssh -t $1 ’screen -dr || /usr/bin/screen || /bin/bash’; }

and startmenu is a cool but dodgy hack to get into my windows virtual machine:

alias startmenu=’nohup rdesktop -A -s “c:\program files\seamlessrdp\seamlessrdpshell.exe explorer.exe” 192.168.185.128 -u andgal -p xxxxxxxx&

xrandr reflects the fact that I have to configure my dual-screen setup by hand after each boot under Ubuntu 7.10, as the GUI configurator just Doesn’t Work. I had to test this many many times. Apparently the latest Ubuntu beta fixes most of these problems.

I’m a Firefox user, but I used to like Opera back in the days before Firefox was available. I changed over once I discovered TabMixPlus and All-In-One Gestures, which collectively replicated most of Opera’s cool features – all except side-docked tabs, which doesn’t seem to be available in Opera any more either.

There are far too many things which like to dock with the top or bottom of my screen. Given that I use a Mac, I already have an extra menu bar to start with anyway. Computer screens are increasingly in widescreen aspect ratio, but documents read best in portrait mode. This is why most blog software (including this one) puts links and metadata at the side of the page rather than at the top, but there are still webapps out there (Google Maps being one of the worst offenders) that have a huge, wasteful banner across the top of the page.

So imagine my joy today when I found this Firefox chrome CSS hack:

Johnathon Weare – Widefox

The dropdown menu placement doesn’t match the tab bar location, but that’s a minor problem. I can now see Google Maps in a slightly less letterboxy format. Woohoo.

Error changing computer name to “NBGAL159″: the domain was joined under the old computer name “NBGAL159″. Account already exists.

<rabc>Wha’?</rabc>

I finally got spam filtering working again on my personal mail server. This was after I’d (re)discovered that I didn’t have a backup of my per-user spam filtering control program (see this post). Luckily I still have a working account on the machine that I’d initially developed the system for.

It’s a little utility called automail, which I wrote so that users who didn’t have shell access on a Linux mail server could still make basic changes to their procmail configuration. You can find information (and downloads) for it here.

I wrote a post bitching about gld, and then my wordpress login timed out and I lost it. I then wrote a post bitching about wordpress, but wordpress then said did I really want to edit the post. Thinking that I might have a chance to salvage the original post, I said no.

Now I’ve lost the text of both posts. (*censored*)

I first got into Skype properly when working at Banjax, where the CEO was a big fan and encouraged its use. If anything, my current office is even more dependent on the little program from Estonia. So I have been impatiently waiting for the release of Skype 2.5 for Mac, in order to get some of the features currently only available in the Windows client.

Unfortunately, it crashed on me several times in succession on trying to start. I was just about to resort to downgrading when I tried killing the chat window as soon as it appeared, which seemed to do the trick. Not sure why restoring a chat window caused it to barf, but it seems reasonably stable now.

But annoyingly, the feature that I was really looking forward to – contact groups – is still missing. Instead I am treated to a work colleague’s birthday reminder. Come on guys, how hard can it be?

Of course, I should be thankful. Linux users are still stuck on version 1.3.