andgal@nbgal185:~$ host -t any peracton.com
peracton.com has SOA record ns1.novara.ie. hostmaster.host.ie. 2011080416 10800 3600 604800 14400
peracton.com name server ns1.novara.ie.
peracton.com name server ns2.novara.ie.
peracton.com name server ns3.nameserver.ie.
peracton.com mail is handled by 10 smail3.host.ie.
peracton.com has address 80.93.17.28

andgal@nbgal185:~$ host peracton.com
peracton.com has address 78.137.166.130
peracton.com mail is handled by 10 mail2.hosting.digiweb.ie.

The first set of results is the “correct” one, so why is host (and nslookup, and dig, and firefox…) still going to the old address by default? I suspect it is something to do with cache locking on our Server 2008 DNS forwarder. It seems that even after I have forced a fresh request by using “-t any”, the stale cached A record is being returned for normal searches. This is apparently a security measure to protect against cache poisoning. It would appear that the TTL on the old A record was unusually long, which means that I had to flush the cache on the primary DNS forwarder (the backup DNS forwarder is fine, presumably because the old record was never in its cache).

Sure enough, running “dnscmd /clearcache” on the offending server fixed the problem.

I started using the desktop package manager – it had been prompting me with “a new version of Ubuntu is available” for quite some time, so I pushed the button and let it do its thing. After an initial false start (out of disk space) it downloaded the upgrades and went to work. About half an hour in, it decided to restart kdm, which is where the fun started.

Now I’m at a command login and the upgrade is in a bad state. A few apt-get -f installs later I find that there’s a file missing in the latex hyphenation config and one of the postinst scripts keeps failing. The postrm script fails too, so I can’t even remove the offending package. After sleeping on the problem, grep -r comes to my rescue and I find the reference to the missing file. Commenting that out just bumped me to the next missing file, so I rmed the lot. I know I now have a badly broken latex system but dammit, I just want this upgrade to finish.

A few more apt-get -f installs and apt-get dist-upgrades later, and the system is ready to reboot. But grub can’t find the root partition, and it drops me to an initramfs which can’t see any hardware. So I had to fire up a 10.04 install cd that I fortunately had to hand, and use it to chroot into the root partition and rebuild the initrd.

Finally, it boots. But I can’t login graphically because gnome-session can’t be found. Back into the command line and apt-get install ubuntu-desktop, which takes another half an hour because it’s all missing, the lot of it. At this point, I notice something odd – I can use my thinkpad fingerprint reader to log in on the command line, but not graphically – scanning the finger when in X gives nothing, not even an error.

Anyway, my xorg.conf file is apparently no longer valid, as it doesn’t recognise the dual screen setup. I rename it and let it run on auto-detect and the screen comes back, but the EmulateWheel on my trackball now doesn’t work. So I run X :2 -configure to get a skeleton xorg.conf file, save that in /etc/X11/xorg.conf and cut&paste my old mouse settings into it. This doesn’t work.

At this point, having used sudo several times, I accidentally discover how to make the fingerprint sensor work while inside X. When prompted, scan your finger. Wait two seconds, hit Ctrl-C and then enter. Don’t ask me why.

It turns out that in recent versions of xorg, you need to set the option “AllowEmptyInput off” in the ServerFlags section or else it ignores any mouse or keyboard configuration sections. Sure enough, this allows EmulateWheel to work again, but the mouse pointer also moves while the emulated scroll wheel is turning. I’m now running very late and this sorry saga will have to continue in the new year.

Merry Christmas, software developers.

DNAME records are supported in Server 2008′s DNS service, but you cannot add them (or edit them) in the graphical tool. You need to use the command line. Right-click “Command Prompt” and run as administrator. Then you can type in the following:

DNSCMD /RECORDADD myzone.example.com mynewrecord DNAME myoldrecord.example.com

If you now refresh the graphical DNS tool you will see a new record with a blank type and contents “Unknown – view properties for more info”. If you do this, you will see the raw hex data for the DNAME RR (type 0×27). The only thing you can do with it in the graphical tool is delete it.

The advantage of using JIRA issues to store account codes is twofold: firstly I can define whatever custom fields I require to hold structured data (e.g. budget holder userids, timestamps) in a format that JIRA can easily understand; and secondly the financial administrators and I can manage this data through a familiar interface. The big disadvantage is that issue linking in JIRA is done through a separate interface from issue creation/editing, meaning a two-stage process for my users. The ideal solution would be a drop-down menu in the issue creation/editing screen, but this means using a custom field – and one I would probably have to write myself.

Being a typical sysadmin, I decided to hack something together with sticky tape and string. I found a likely-looking custom field in the form of the JIRA Database Values Plugin. Using this to tap directly into JIRA’s back-end database I can populate a drop-down menu using some horrendous left joins:

sql.query=select i.id, cv1.stringvalue, cv2.stringvalue, cv3.stringvalue from (select id from jiraissue where project=10200 and issuetype=13) as i left join (select issue, stringvalue from customfieldvalue where customfield=10140) as cv1 on i.id=cv1.issue left join (select issue, stringvalue from customfieldvalue where customfield=10141) as cv2 on i.id=cv2.issue left join (select issue, stringvalue from customfieldvalue where customfield=10142) as cv3 on i.id=cv3.issue order by cv2.stringvalue, cv3.stringvalue;

primarykey.column.number=0
rendering.viewpattern={1,(none)} {2} {3}

The joins are necessary because JIRA uses a relational back end which assumes a fixed number of columns in each table. It handles issues with custom-defined fields by storing the custom field values in a separate table with back-references to the appropriate row in the main table.

The JDVP plugin provides a pretty front end to a custom field that stores the primary key of a database row, which in this case is jiraissue.id. I can then recall the account code issue in a JSS script by passing the primary key to getIssueObject() – the example below is a post-function that I use in an approval workflow to ensure that nobody can approve his own request:

from com.atlassian.jira import ComponentManager

# Only perform functions on Travel Request issues
if(issue.getIssueTypeObject().getId()=="8") :
  cm = ComponentManager.getInstance()
  cfm = cm.getCustomFieldManager()
  im = cm.getIssueManager()

  accountCodeID = issue.getCustomFieldValue(cfm.getCustomFieldObject('customfield_10134'))
  acIssue = im.getIssueObject(int(accountCodeID))
  acBudgetHolder = acIssue.getCustomFieldValue(cfm.getCustomFieldObject('customfield_10070'))
  acEmergencyBudgetHolder = acIssue.getCustomFieldValue(cfm.getCustomFieldObject('customfield_10151'))

  # Set the budget holder to the account code's budget holder, unless that is the reporter.
  # The Java API for custom fields is crap. I'm using a trick here.
  # http://confluence.atlassian.com/pages/viewpage.action?pageId=160835&focusedCommentId=224398519&#comment-224398519
  # DO NOT USE issue.setCustomFieldValue() - it does NOT persist. This is a feature (!)

  cf=cfm.getCustomFieldObject('customfield_10070')
  if issue.getReporter() == acBudgetHolder :
    cf.getCustomFieldType().updateValue(cf,issue,acEmergencyBudgetHolder)
  else :
    cf.getCustomFieldType().updateValue(cf,issue,acBudgetHolder)

I have two things to say about the above. Firstly, scripts in a language where leading whitespace is significant are very difficult to post to blogs. Secondly, I meant everything I said about the custom fields API. In my previous post, I complained about the Java standard library requiring double references – well here’s another example. Read the last ten lines and try to tell me that the developer in question was sober when he designed the interface.

This does exactly what I want (like all good sticky-tape solutions should). The only thing the JDVP did that annoyed me was to always render “Unknown” (or the localisation equivalent) in the case that one of my database fields contained NULL (some of our accounting codes have optional subcodes, for example). After discussions with the developer, I submitted a patch which made the NULL behaviour configurable, and he has included it in the latest release. Thanks, Wim.

You can use validators on the Create Issue transition (if you can find it -  go to the initial step of your workflow and look for “incoming transitions”), but the technique I was using doesn’t work, as the Created field isn’t updated until after the validator is processed and I was thus comparing with Null. I have Misc Workflow Extensions, Jira Suite Utilities and Jira Toolkit installed, but none provided a validator that would compare a date field with anything other than another date field. Luckily I also have the Jira Scripting Suite installed, and I thought this would be a trivial use case. Grab the custom field data using a jython one-liner and compare with the system date.

It wasn’t so simple. Jira returns date custom field values as java.sql.Timestamp objects, which are nontrivial to compare with jython date objects. I then tried using java.util.Calendar instead of the native jython datetime library, thinking that a java.util.Date would be more easily compared with a java.sql.Timestamp because they implement the same interface. This is technically true, but I didn’t want to compare instantaneous times, only dates, and there is no native method for extracting a date-only component (as a Julian date, say) from a java.util.Date object. One might expect to be able to obtain the raw timestamp data and discard the fractional day components (say, by performing an integer division), but this falls foul of time zones.

Jira treats dates as timestamps with zeroed time-of-day components, but the database backend stores this as milliseconds since the Epoch, and this is what is returned in the Timestamp object. A bare date is thus stored as the timestamp of the immediately previous local midnight. In the case of Ireland in summertime, this will cause an off-by-one error in the algorithm if one naïvely performs an integer division by the number of milliseconds in a day. To find the correct number of local days since the Epoch, one must add the current timezone offset to both timestamps before discarding fractional days. The resulting algorithm is thus far from the desired one-liner:

from com.atlassian.jira import ComponentManager
from java.util import Calendar

cfm = ComponentManager.getInstance().getCustomFieldManager()
leavingDate = issue.getCustomFieldValue(cfm.getCustomFieldObject('customfield_10072'))
today = Calendar.getInstance()
tzoffset = today.getTimeZone().getOffset(today.getTimeInMillis())
leavingDateEpoch = (leavingDate.getTime()+tzoffset)//86400000
todayEpoch = (today.getTimeInMillis()+tzoffset)//86400000

if leavingDateEpoch < todayEpoch :
 result = False
 description = "Leaving date cannot be in the past"
 invalid_fields['customfield_10072'] = u"Leaving date cannot be in the past"

It works, but it’s damn ugly. I particularly love the double reference to the today object in line 7. How the Java standard library manages to be so verbose and still so lacking in basic functionality is beyond me. They actually seem to be going in the wrong direction – I found this wonderful piece of documentation on my travels:

int     Date.getTimezoneOffset()
Deprecated. As of JDK version 1.1, replaced by
-(Calendar.get(Calendar.ZONE_OFFSET) + Calendar.get(Calendar.DST_OFFSET)) / (60 * 1000)

Says it all, really.

But not impossible! You just can’t do it from within a running system. Instructions follow:

1. add the lenny (or later) repositories to /etc/apt/sources.list and run `apt-get update`
2. reboot into a lenny (or later) install CD
3. choose advanced -> rescue mode
4. answer the usual install/config questions – don’t worry about networking for now
5. execute a shell in the target environment *
6. `ifup eth0` (or whatever you need to get networking running)
7. `export TERM=vt100` (because bterm is badly broken)
8. `apt-get install linux-image-2.6.26-2-686` (or whatever kernel is appropriate)
9. reboot and do `apt-get dist-upgrade`

(*) The installer may not automatically mount your root partition – if so then you won’t be able to execute a shell in the target environment. In that case:

1. start a shell in the installer environment
2. mount your root partition somewhere by hand (this may be nontrivial if you’re using LVM!)
3. cd into it
4. `mount -t proc proc proc`
5. `chroot .`

You now have a shell in the target environment.

By contrast, “equal temperament” systems define the frequency ratio between any two adjacent notes to be a constant, but this constant cannot be a rational number and note frequencies which were once simple ratios of each other are now very slightly detuned. Analogue instruments have a natural variability in their tuning, so this slight “error” usually goes unnoticed.

The division of the octave into twelve equal intervals is a natural progression of the 12-interval Pythagorean system, but is not the only one. The “ideal” tuning system is one where all intervals are based on simple frequency ratios, but this is impossible to realise in practise without either providing an infinite number of notes or severely limiting the musical range of one’s compositions. Many of these ideal intervals are very badly approximated by 12-TET, notably the minor third (the interval between C and E♭). 31-TET is a better match overall, however it more than doubles the number of notes required on an instrument, and many of these will not be used in the vast majority of compositions because they do not approximate a useful interval.

Compromise tunings which do not require so many redundant notes include the alien-sounding 22-TET and the more mellow 19-TET. The latter gives improved major (C-E) and minor (C-E♭) thirds compared to 12-TET, at the expense of a slightly less accurate fifth (C-G). It also has few enough notes that 19-TET instruments are quite practical to construct and play. I had to try.

19TET refretted Encore strat copy

I refretted this guitar two years ago and forgot to blog about it at the time. I was fully convinced that I had, up until the point when I tried to find the appropriate link to help spice up my entry into this competition. I’m addressing my earlier failing now.

I have three guitars, though technically one of them belongs to my brother. The cheapest and crappest of my two was duly sacrificed in the name of science. I bought a fretsaw and obtained a length of fretwire from a local instrument repair shop. The existing frets on the guitar were easy enough to prise out of their grooves, and new grooves in the appropriate places* were easy to cut. To cut down my workload, I only refretted one octave worth of the neck – this isn’t as big a sacrifice as it sounds, as the 19-TET fret spacing close to the body would be far too narrow to play comfortably. The biggest difficulty came in cutting and shaping the new frets – I ended up having to file off a few offending corners and in doing so scratched the edges of the fretboard. Overall I’m pleased with the result though – it plays easily and looks just like a “real” guitar until you get too close.

How do you adjust your playing style to a completely different fretboard?  The pearl inlays from its 12-TET days are still present, and provide a useful visual clue for where the fingers should go. If that doesn’t work, I usually just fiddle with the chords until it sounds right. As a rule of thumb, anything requiring a one-fret interval should now use two, and a two-fret interval becomes three.

One advantage that 19-TET has over (say) 22-TET is that standard music terminology can be (ab)used. The white-note names are unchanged, black flats and sharps are now distinct notes (C♯ is not D♭), and B♯=C♭ and E♯=F♭ are now notes in their own right:

C C♯ D♭ D D♯ E♭ E (E♯=F♭) F F♯
  G♭ G G♯ A♭ A A♯ B♭ B (B♯=C♭) C

If you are familiar with the piano, imagine that each black note has been replaced by two black notes, and the gaps between B-C and E-F are now filled by one black note each.

How does it sound, I hear you ask? Unfortunately I can’t make you a recording at the moment, as the bridge recently came apart (someone remind me to go to the repair shop this weekend please?). However, some fantastic examples of 19-TET music can be found on Jeff Harrington‘s site – my favourite is this one. He’s a much better musician than me anyway.

* D(n) = D(0)/2^(n/19) where the distances D are measured from the bridge and D(0) is the distance to the nut.

- As you are aware of the activities of terrorist attacked at our airport,during our daily routine on 100% inspection of incoming and our going flights, we arrested a man with a consignment that claiming to be yourpartner, after we scan thetrunk box, we discovered the trunk box contain liquid cash in [US] Note and other documents attached to the consignmentdid not carry the suspect name, but carry your name and address as the receivers of the trunk box and this is very SUSPICIOUS.Get back to me immediately to explain your intention; before I proceed to contact my superior and your embassyfor more investigation, hope you arenot sponsoring terrorism with this fund? be very sincere to me please.Yours in service,a Office of Thomas joe,Metropolitan police chief.Heathrow Airport UK.EMAIL mrallen92@hotmail.com

100% Verifiable Bug-Free Code Is Possible (Dobbs Code)

The executive summary: ok-labs.com and nicta.com.au have developed a microkernel that is provably correct, in defiance of the conventional wisdom that formal methods are only practical for the simplest of projects. While I take my hat off to the developers for their heroic efforts, I firmly believe this is only the first step in a process that will revolutionise the industry. The ultimate goal should be a complete, provably correct computing platform. This will not come about quickly or easily, but there are practical things we can do in the meantime.

Recently, the SANS Institute published a list of the 25 most dangerous programming errors. Most of these, e.g. buffer overflows, are rookie mistakes that every apprentice programmer is warned about, yet even with better training and increased collective experience, the population of errors in the wild seems to increase daily. The reason is simple: as the number of programmers grows, and their productivity increases, the number of opportunities for errors to creep in grows faster than the ability of the human mind to be vigilant. Or to paraphrase Murphy’s Law: if a programmer can make mistakes, he will make mistakes.

The conventional wisdom is to accept that such errors will be made, and be vigilant in testing. But no amount of post-facto investigation can give us 100% confidence that a catastrophic error doesn’t lurk somewhere in the code. With software increasingly responsible for people’s lives (fly-by-wire aircraft, nuclear power), the potential consequences of code errors will become unmanageable. Let me demonstrate.

The standard equation of risk analysis is

risk = probability × severity

What this means in practice is that in order to maintain risk below a given maximum, any event that has N times more severe consequences (e.g. a plane crash that kills 10 times more people than a car crash) should happen N times less often. But going from 99.999% (five nines) certainty to 99.9999% (six nines) is a lot harder than going from 99% to 99.9%, due to diminishing returns. Throwing man-hours of testing at such a problem does not scale.

Formal methods give us a way around this impasse. But we do not even need the expense of formal testing to eliminate the most common errors – design choices in our programming languages can do a lot of the work for little cost. Consider the state of the art before the introduction of structured programming in Algol-60. Without nested loops or multi-line conditions, the GOTO statement was king. GOTO-based programming languages are Turing-complete, in that they can perform any conceivable algorithm, but they are too flexible. It is quite possible (and easy) to write nonsense algorithms where loops would start or end inside the ELSE block of an IF statement, for example*. No modern programmer would contemplate such absurdities – there was a very good reason why GOTO statements were banished from polite society. And yet programming languages without GOTO statements are still Turing-complete.

What the above history lesson shows us is that, given sufficiently intelligent design paradigms, we can remove once-thought-necessary features from our lexicon without sacrificing Turing completeness, and thereby eliminate entire classes of potential programming errors.**

Now consider buffer overflow – this can be generalised to all kinds of array bounds violation; a buffer being merely a 1D array. The array index is constrained between two values, typically 0 and some integer N. If the algorithm attempts to access the array with an index outside this range, we have an array bounds violation. If this is not tested for explicitly by the algorithm (easy for a stressed programmer to forget) or implicitly in the execution environment (inefficient and not always enabled) undesirable behaviour can easily result. We could laboriously check every subscription operation in our algorithm to ensure that sane values were being passed at all times; or we could change our programming paradigm so that it is impossible to write an invalid subscription operation.

This is not as outlandish as it sounds – we already have a commonly-used language feature that does most of the work for us: FOREACH. A FOREACH loop iterates over the elements of an array, and automatically stops when there are no more elements. We cannot pass an invalid value into the array subscription operation, because there is no such operation; subscription is handled implicitly by the FOREACH. If all buffer operations used FOREACH statements, there would never be an overflow.

Now say that at some point in the near future we find similar, implicit, methods for array subscription that cover all sensible use cases (in the same way we found alternatives for all sensible use cases of GOTO). We could then eliminate the explicit array subscription operation from our lexicon, and buffer overflows would be a historical curiosity.

I think such a development is within reach. Like many programming geeks I have been working, on and off, for some time on a pet language project which, amongst other things, features strict limitations on the array subscription operator, as well as a plethora of implicit alternatives to handle common use cases. I haven’t yet been able to eliminate subscription completely, but all ideas would be gratefully received.

* This is a real example from personal experience, which shockingly did work but was (unsuprisingly) impossible to understand.
** Of course, if the language is still Turing-complete, it will therefore still be possible to make the same error; but you will have to go out of your way to deliberately do so.

Error: System error: -2146697191
Code: 800C0019
Source: msxml3.dll

Turns out that a custom VBScript was sending a form to one of our secure web servers and the certificate chain couldn’t be verified up to the root (even though we bought it from a reputable source). Adding the offending root certificate to the trusted root certificate authorities database fixed the problem.

This does not seem to be documented anywhere on the internet yet, so here it is.