The Overwhelming Loss of Self

I suppose it had never occurred to me before. I mean, people died when I was younger but I guess I never really put two and two together. Maybe nobody does. I suppose it actually had much more to do with the type of relationship I had with those who died. I'm not talking about the quality of the relationship; no, this has more to do with socio-geography than it does with long walks on the beach. The first people that I remember dying were not people that I spent a lot of time with, either physically or in my head. In fact, I probably would have learned this earlier had I spent more time with some or all of them. I heard something on the radio today that I liked quite a bit: when you lose a parent, you lose the past; when you lose a child, you lose the future; when you lose a spouse, you lose the present. Losing the past is something terrible, but as children or young adults, we are pretty used to losing the past; we welcome it in most cases. Even so, we still lose that part of ourselves that existed only in the intersection of our lives with the other person's. Losing a child or a spouse (or sibling or friend) is even more terrible in the sense that you lose not only the part of you that existed in conjunction with that person, but also all the moments that were to come. Horrifically, it is the death of potential. Even though your potential may be realised eventually, the dance with that person is over. Insofar as that part of you is defined by its relation to the dance, you die too. To all those who have died but are still alive, I'm sorry for not understanding sooner.

How to add trusted keys to apt

The aptitude manual has a little section on how add trusted keys to apt.

The list of keys that apt will trust is stored in the keyring file /etc/apt/trusted.gpg. Once you have the GPG key, you can add it to this file by executing the command gpg --no-default-keyring --keyring /etc/apt/trusted.gpg --import newkey.asc . aptitude will then trust any archive that is signed with the key contained in newkey.asc.

Qemu networking goodness

It took forever but I finally got all of my Qemu networking working. My setup uses VDE, a TUN/TAP device to connect to my LAN, dnsmasq to give my QEMU hosts IP addresses and handle DNS requests, and IP Masquerading because apparently my wifi card can't spoof MAC addresses.

There follows a (probably incomplete) description of the setup

Setting up the TAP device

First thing to do is create the TAP device we will use to connect the VDE network to the LAN.

sudo modprobe tun
sudo chmod 666 /dev/net/tun     # I'm all alone on this box so...
sudo tunctl                     # This should create a device called tap0
sudo ifconfig tap0 10.0.0.1 up  # This is the IP for the VDE network

We're all done with TAP stuff.

Set up IP Masqerading through the TAP device

Now we need to make sure that traffic coming through the TAP device gets sent out over the LAN.

sudo su -c "echo 1 > /proc/sys/net/ipv4/ip_forward"       # Enable IP forwarding
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE -v          # wlan0 is my wifi card

OK, that's it for IP Masquerading.

dnsmasq for DNS requests and DHCP

This one was easy; just install the package and modify the conf where it says #interface= to say interface=tap0 (without the comment mark and substituting whatever you got back from tunctl above.

VDE setup

First we'll create a virtual switch

sudo vde -s /tmp/switch1

Then give everybody access to the VDE

sudo chmod -R a+rwx /tmp/vde.ctl

OK, that's it for VDE.

Qemu hosts

This was a trickier bit. The stumbling block for me was that if you specify a MAC address for the host (I'm using Debian Etch as the guest OS), a new eth device is created. So make sure you specify the right MAC address from the start. If you screw up, you can always edit /etc/udev/rules.d/z25_persistent-net.rules and remove the extra eth devices. The reason this happens is that udev figures out that you added a new card (because of the new MAC) and so it configures another device. There's probably a more elegant way around this, I just haven't figured it out yet.

Boot your Qemu host

After all of the above, booting should go smoothly; remember to specify -net nic,macaddr=XX:XX:XX:XX:XX:XX -net vde. If you don't get an IP automatically, just run dhclient ethX on the guest and you should be set.

Frustrated by UML

I recently have been experimenting with User Mode Linux. My goal is to be able to run some of my tests on my local machine without needing to setup a whole EC2 instance.

At the same time I am evaluating the possibility of using UML on my instance to separate out the various services.

The main problem I am having at the moment is with the networking. Although I am able to connect to the host machine, I seem to be unable to create a route to the rest of the network.

They were once so shiny...

I was just looking at this picture of my pipes. I had forgotten how shiny they were when I first got them. Now the brass has started to develop a patina so it is quite a bit duller and a little splotchy in places.

The patina is actually desirable (for a lot of pipers) as it makes the pipes look antique. We would probably all play 100-year-old sets if we could.

The wood (which is cocobolo - a type of rosewood) has also darkened considerably since I got the set. The maple in the mainstock hasn't changed much though, nor have the mounts. There is a slight discoloration of the chanter mounts since I keep it in a red leather case I made that has tinted the wood somewhat.

The three things that make us great: culture, culture and culture

My last post was in response to George Dinwiddie's "What would you like your software developers to learn" question. Here is a follow-up exchange that I wanted to share.

George wrote back to me asking:

Are your developers learning more how to take a long-term view? What steps are you currently taking to help them learn? What steps do you think you should take, but aren't yet, for some reason?

The rest of this post is my answer.

I think that one of the key roles that manager/architect types need to play is that of strategist. As one of my senior developers put it recently: "I know about tactics, what I need is strategy." An ability to predict the outcome of various actions is, in my experience, an indispensable quality in those who are responsible for guiding the overall direction of a development effort.

Some developers care about learning how to do this and some don't. Not everybody on the team needs to be a superstar, and there is usually no shortage of trash to be taken out by those who want to do that (or who lack the chops to do much else). On the other hand, those developers that do want to grow should be encouraged and given the appropriate tools.

On my team, I have a pretty heterogenous group where about 30% of them are interested in learning this stuff. They come from a variety of backgrounds and so some are starting from further back than others. I find that a good strategy for winnowing the grain from the chaff is to offer some of the basic tenets to everybody and see who picks up on it.

We have a 1.5 hour workshop once a week where we go over various parts of our codebase, learn about new techniques, have different developers do presentations on code they have written or had to maintain, etc. Generally I follow up on those workshops with additional attention to those whose eyes do not glaze over when we start talking about certain things.

We also have a very strong team culture that encourages developers to watch each others' backs and make sure that the "social contract" we have all agreed to is upheld. Most of the questions that I listed in my initial answer are things that my guys and gals check each other on all the time. It's a bit of a game: who can find the mistake in the other person's work. This, of course, is all done in a spirit of cooperation and mutual acceptance of constructive criticism (critical components in any team).

I am fortunate to have the leeway to do pretty much whatever I want to improve the current performance of my team. In that respect, I rarely find that there are things I want to do but cannot. This is not the same in every environment, of course, but I select the companies I work for specifically keeping in mind the fact that the most successful companies are the ones that pick the right people, put them in the right place, and get out of the way while they do their job.

At the end of the day I would say that the single most important factor in turning competent developers into excellent ones is culture. I try to lead by example and I expect my seniors to do the same. Every senior on my team had the role and responsibilities long before they had the title. Every junior on my team is capable of training up new juniors (and even some more intermediate-level developers) to be effective within two iterations (four weeks). The culture on our team is one of constant improvement and the juniors frequently teach the seniors lessons (which the seniors take in stride). With the right culture, everything else just seems logical. Without it, you are constantly swimming against the current.

From competence to excellence: what developers should know

One of the things that most developers have a lot of trouble with is a long-term view. I think it is critical that there be a very good understanding of the impact of current decisions and design choices on the future of the project and the company. A lot of developers (and in my experience this is only worse with contractors and consultants) have the impression that once the code is written, the job is done. They frequently omit details such as sustainability of the overall system and how easy it will be to make future changes.

One good indicator of a developer's current level of understanding of this concept is to look at the following:

Coding Practices

Does the developer write idiomatic code that would be easy for any other developer to understand given a basic comprehension of the development environment? Are the interfaces to their objects clean, minimal and easily understood? Are the techniques used to modularize their code sparingly and appropriately applied? Do they consistently strive for high coherence and low coupling in the codebase? Do they understand the value of a properly written test suite?

Source Management

Does the developer understand more than the basics of their source control tool? Do they know how to do branching and merging properly? Do they leave clear, concise comments for each commit? Common errors when using SCM tools include incorrect spelling for changeset names (which makes searching for a particular changeset difficult), bundling of unrelated changes into a single changeset (which makes reverting only one of the changes difficult/impossible), linking changesets to the incorrect development task (which makes it difficult to assert that a particular task has been completed successfully) and fumbling with branch and merge (which causes problems when a developer needs to work on some separate task - how do I fork the codebase? how do I re-integrate my changes? etc.)

Development Environment

Is the development environment properly documented and easy to re-build? Can each developer work on any other developer's machine or do they need their "own setup"? Does the development style employed by the developer depend on a specific tool that may not be supported in the future (i.e. web developers who do everything in DreamWeaver, Java developers who are lost without Eclipse's auto-complete and refactoring tools, etc.)? Is the project neatly organized such that it's physical layout (on the filesystem) is easy to understand/explain?

Developers who have a good grasp of the above concepts also usually have a more complete understanding of the lifecycle of a software product (as a product and not as a project). By looking ahead, and by applying certain principles now, we can often have a huge impact on the future maintainability of a system. Maybe I'm just lucky to work with guys and gals that have the basics down pat but I would say that the above is what separates the "software developers" from the "coders". Obviously, there are a number of more basic items that come first but a long-term view of the evolution of a codebase is something that lifts developers from competence to excellence.

Asterisk moved into EC2 cloud

I moved the Asterisk server into the EC2 cloud today. It was pretty quick.

Running it off my home box was no longer an option since we have people testing the service these days and I want to make sure there are no outages. I still need to setup a test server so we can test out new configurations before putting them live. I had previously hacked up a little configuration script that shuffles stuff around and performs a couple of emerges. I used the "eminent" AMI that is based on a simple Gentoo install. I plan on putting together my own clean base image at some point in the future but right now this one is good enough. I also plan on building a nice overlay that I can use with portage to update my boxen quickly and efficiently.

FWIW, I figured out how to get Asterisk working from behind a NAT. It was as simple as setting the externip variable in the sip.conf file.

Using encrypted SSL keys and certs in twill

Recent versions of mechanize support SSL connections and so, by extension, does twill. Recently I have been trying to open an HTTPS connection to a site that requires both a certificate and a client key. To try this at home, you will need a version of openssl.

The key and cert came to me packaged in a PKCS#12 format file. The first thing I needed to do was unpack the key and the cert like so:

openssl pkcs12 -clcerts -nokeys -in cert.p12 -out cert.pem
openssl pkcs12 -nocerts -in cert.p12 -out key.pem

After doing this, I plugged in the filenames of the cert and key in my twill code. Note that you need to access the underlying mechanize.Browser object when setting the client certificate:

import twill
from twill.commands import *

host = 'mysecurehost.com:443'
b = twill.get_browser()
b._browser.add_client_certificate(host, 'key.pem', 'cert.pem')

go('https://mysecurehost.com/secured_url.html')
show()

The problem you run into when doing this is that the SSL libraries prompt the user to enter the password for encrypted keys at runtime. This makes automating the interaction tricky at best. I tried fumbling with the PyOpenSSL library but it seems that setting a callback for the passphrase retrieval does not actually work. The set method call returns but the callback is never called during key decryption.

My (hackish) solution was to remove the encryption on the key before using it to connect. You can do this by re-exporting the key from the PKCS#12 file:

openssl pkcs12 -nodes -nocerts -in cert.p12 -out unencrypted_key.pem

Now if you use unencrypted_key.pem in the twill code above, you will be able to connect without providing a password for the key.

Centralized version control blues

I am in the process of migrating a SVN repository to Darcs. A couple of the changesets in the SVN repo make baby Jesus cry. They create conflicts such that checking them out in order fails spectacularly and you need to go hands with the working directory to clean everything up before moving on. All of this highlights a rather interesting (though totally predictable) issue that plagues centralized version control.

In Darcs, if a changeset is no good, you can just refuse to pull it. Doing so excludes that patch from your repo and no harm is done (unless, of course, a subsequent patch depends on it). In SVN, given that everybody checks into the same repo, by the time you realise that a patch is going to trip you up, it's already in the bloody database. Now, SVN provides no tools for deleting a specific revision (although I hacked together something that I'll try to toss out there someday) so you can't exclude the changesets that put the DB in an inconsistent state. All of this means that I have been wrestling for the past week and a half trying to get this stupid repository to ignore the "corrupted" changesets. Arrrrgh!

I just called...

Two weeks ago I purchased my first wake-up call. It was late at night and I had a particularly important meeting the next morning that I really couldn't afford to miss. So I slapped "wake-up call" into Google and hit the first site that popped up. If I'm not mistaken, it cost me $1.20 USD for that call.

The second time I wanted a call I did a little more research and turned up a site that offered calls at $0.75 USD with a "snooze package" of three snoozes for an additional $1.20 USD or something. I used this site several times trying to work out how the different options work.

Then (as usual) I got frustrated by the fact that I was essentially paying to use somebody else's script when I could write one myself. I spent and evening getting cozy with the Skype API and finally (after quite a bit of hair-pulling) figured out how to get SkypeOut to call my home phone and play a sound file (in WAV format) to me.

This weekend I put the finishing touches on my new wake-up call script. I can now set a date/time and have my computer call me to either remind me of something or to wake me up. I could also do other, wacked-out things like have my computer monitor an inbox for an important email while I'm at the country and then have it call me and read (via Text To Speech) the email's contents to me over the phone. This morning I used the scheduling to wake up and it worked like a charm!

The only problem now is that Skype charges me $0.024 CDN per minute for each call as well as a $0.05 CDN connection fee for SkypeOut. Still, it's much better than the $0.75 USD which was the cheapest I had found before (that is, without a subscription).

Performance Tuning - applying a function to a list

>>> import timeit
>>> timeit.Timer('map(f, range(10))', 'f=lambda x: str(6+x)').timeit()
15.467374484745967
>>> timeit.Timer('[f(x) for x in range(10)]', 'f=lambda x: str(6+x)').timeit()
16.062227741235269
>>> timeit.Timer('for x in range(10): f(x)', 'f=lambda x: str(6+x)').timeit()
14.686095299821623

And so, we can see that map is still faster than list comprehensions and the for loop beats them both. If you don't need the return value of that function, don't create the list: friends don't let friends create unnecessary objects. On the other hand, if performance is critical and you need the return values, you should prefer map over a list comprehension.

Picking teeny-tiny cherries

I would love to use Bazaar or Mercurial as a DVCS; just the fact that they are written in Python (and therefore eminently hackable as far as I'm concerned) is worth making the move. I just can't get past the fact that Darcs (my current favourite) provides support for hunk cherry-picking. This is a killer feature. It's great to be able to cherry-pick files into patches, but Darcs actually allows me to package up my hunks that relate to different changes into different patches. This gives me much more control over the way I decide to describe the modifications I am making to my project. When I can only operate at a higher (coarser) level of granularity (like Mercurial or, God forbid, SVN) I get stuck fiddling around removing hunks from files before committing the patches.

Another project to take up my time

I've just tossed out the first version of a small web-stack for Python called Khepri - a clever bit of word-play on the name "Apophis". There are a million others out there but I'm very picky. This one is mine and I like the way it is so far. Obviously, it is far from being really usable but if you are a developer, you can judge for yourself. Check out the darcs repo.

The cat came back the very next day

Well, not quite, but he is back. Tygger finally came home on Friday night. I came back from an evening of recording stuff at Justin's and called out to him out of habit. I heard somebody yelling at me from the neighbour's porch and got angry for a moment at Jack for looking so much like Tygger. I headed towards him to pat him anyway and he fled. It was then that I noticed that he wasn't wearing his bell. I followed him for a bit then gave up when he continued to elude me. As I headed upstairs, I thought once again "wouldn't it be stupid..." So I picked up the flashlight and went back outside. I crooned and cajoled for about ten minutes before he finally bolted past me and into the house. He was super-thin and even once he was inside I had to open his mouth to check for the broken tooth that confirmed his identity. He still smells a bit off but that's nothing a couple of baths won't cure. I guess that for now everything is right in my little world!

Tygger still missing

Our little guy is still missing and although everybody seems to have a story about how so-and-so's cat took 18 days to come home and their cousin's dog once crossed all the continents to get back to his owners I feel less and less confident every day that he will return. Marquis is all depressed and doesn't play; he just wanders around looking for Tygger and trying to get attention from us since he has nobody to play with. I certainly hope that he is huddled under a porch not twenty yards from our front door but I fear he is wounded somewhere or that he has been "taken in" by some well-meaning people who have discovered what a wonderful cat he is.

Linking into the weeb

So Justin spent the weekend setting up a bunch of accounts for himself at blogger and del.icio.us. I figured he needs a bit of link love since he can't manage to get himself onto the first page of Google's results. Now, I'm not sure that Google still indexes blog posts the way they used to but you can check out Justin's blog or go directly to his website.

Missing

Our cat, Tygger, got out the window Saturday night while I was sleeping on the couch in front of the TV. We spent the day yesterday looking for him and calling his name but to no avail. Marquis is starting to feel a bit lonesome. We put up posters this morning around the neighbourhood but I'm not really all that optimistic. We worry that he may have tried to go back to our place on Stratford. The tricky bit is that the wife of the guy who is renting from us isn't supposed to know that we had a cat living there before they moved in. Anyway, we'll go and put up more posters this evening. The apartment seems very empty without our little guy snoozing away on his blanket on the couch.

2nd April Menu

April is cool with frequent showers, hail and just a little wet snow. But spring is just around the corner. You can smell earth in the humid, expectant air. Lightness to match the season coming is in order, but these past weeks have been quite chilly, so a little warmth would not go amiss. Thus I propose to you the 2nd April menu with a hint of the exuberance of youth and just enough heat to drive winter's final tendrils back into the darkness. All dishes listed are low calorie unless indicated (with a *).



Entrées & Sidedishes



Cheesy Broccoli Bake

Oriental Broccoli Salad (soy, honey and sesame)

Broccoli with Herbed Breadcrumbs



Roasted Zucchini with Fresh Thyme



Asparagus Tips with Roasted Red Peppers

Orange Ginger Asparagus



Spicy Nutmeg Carrots

Ruby Carrots (cranberry)

Carrot and Coriander Soup



Cheesy Chicken Chowder



Main Courses



Zesty Chicken Sauté

Apricot Glazed Chicken Breasts

Sesame Chicken Salad



Skillet Shepherd's Pie



Bell Pepper Stir-Fry (on white rice)

Risotto Napoletana (sun-dried tomatoes, salami and parmesan)



Broiled Sesame Salmon

Oven-Baked Tandoori Salmon



Desserts



Raspberry Sorbet

Fruit Crisp*

Last know good configuration

Darcs has a feature (that I have never really tried) that allows you to locate the last working version of your source. I am currently shlepping through changesets in SVN applying them one by one in order to figure out which one breaks a series of tests that depend on our mocking library. It seems like it would be so simple to just repeatedly run the tests, stepping back through revisions until the tests pass.