Performance Tuning - applying a function to a list

>>> import timeit
>>> timeit.Timer('map(f, range(10))', 'f=lambda x: str(6+x)').timeit()
15.467374484745967
>>> timeit.Timer('[f(x) for x in range(10)]', 'f=lambda x: str(6+x)').timeit()
16.062227741235269
>>> timeit.Timer('for x in range(10): f(x)', 'f=lambda x: str(6+x)').timeit()
14.686095299821623

And so, we can see that map is still faster than list comprehensions and the for loop beats them both. If you don't need the return value of that function, don't create the list: friends don't let friends create unnecessary objects. On the other hand, if performance is critical and you need the return values, you should prefer map over a list comprehension.

Picking teeny-tiny cherries

I would love to use Bazaar or Mercurial as a DVCS; just the fact that they are written in Python (and therefore eminently hackable as far as I'm concerned) is worth making the move. I just can't get past the fact that Darcs (my current favourite) provides support for hunk cherry-picking. This is a killer feature. It's great to be able to cherry-pick files into patches, but Darcs actually allows me to package up my hunks that relate to different changes into different patches. This gives me much more control over the way I decide to describe the modifications I am making to my project. When I can only operate at a higher (coarser) level of granularity (like Mercurial or, God forbid, SVN) I get stuck fiddling around removing hunks from files before committing the patches.

Another project to take up my time

I've just tossed out the first version of a small web-stack for Python called Khepri - a clever bit of word-play on the name "Apophis". There are a million others out there but I'm very picky. This one is mine and I like the way it is so far. Obviously, it is far from being really usable but if you are a developer, you can judge for yourself. Check out the darcs repo.

The cat came back the very next day

Well, not quite, but he is back. Tygger finally came home on Friday night. I came back from an evening of recording stuff at Justin's and called out to him out of habit. I heard somebody yelling at me from the neighbour's porch and got angry for a moment at Jack for looking so much like Tygger. I headed towards him to pat him anyway and he fled. It was then that I noticed that he wasn't wearing his bell. I followed him for a bit then gave up when he continued to elude me. As I headed upstairs, I thought once again "wouldn't it be stupid..." So I picked up the flashlight and went back outside. I crooned and cajoled for about ten minutes before he finally bolted past me and into the house. He was super-thin and even once he was inside I had to open his mouth to check for the broken tooth that confirmed his identity. He still smells a bit off but that's nothing a couple of baths won't cure. I guess that for now everything is right in my little world!

Tygger still missing

Our little guy is still missing and although everybody seems to have a story about how so-and-so's cat took 18 days to come home and their cousin's dog once crossed all the continents to get back to his owners I feel less and less confident every day that he will return. Marquis is all depressed and doesn't play; he just wanders around looking for Tygger and trying to get attention from us since he has nobody to play with. I certainly hope that he is huddled under a porch not twenty yards from our front door but I fear he is wounded somewhere or that he has been "taken in" by some well-meaning people who have discovered what a wonderful cat he is.

Linking into the weeb

So Justin spent the weekend setting up a bunch of accounts for himself at blogger and del.icio.us. I figured he needs a bit of link love since he can't manage to get himself onto the first page of Google's results. Now, I'm not sure that Google still indexes blog posts the way they used to but you can check out Justin's blog or go directly to his website.

Missing

Our cat, Tygger, got out the window Saturday night while I was sleeping on the couch in front of the TV. We spent the day yesterday looking for him and calling his name but to no avail. Marquis is starting to feel a bit lonesome. We put up posters this morning around the neighbourhood but I'm not really all that optimistic. We worry that he may have tried to go back to our place on Stratford. The tricky bit is that the wife of the guy who is renting from us isn't supposed to know that we had a cat living there before they moved in. Anyway, we'll go and put up more posters this evening. The apartment seems very empty without our little guy snoozing away on his blanket on the couch.

2nd April Menu

April is cool with frequent showers, hail and just a little wet snow. But spring is just around the corner. You can smell earth in the humid, expectant air. Lightness to match the season coming is in order, but these past weeks have been quite chilly, so a little warmth would not go amiss. Thus I propose to you the 2nd April menu with a hint of the exuberance of youth and just enough heat to drive winter's final tendrils back into the darkness. All dishes listed are low calorie unless indicated (with a *).



Entrées & Sidedishes



Cheesy Broccoli Bake

Oriental Broccoli Salad (soy, honey and sesame)

Broccoli with Herbed Breadcrumbs



Roasted Zucchini with Fresh Thyme



Asparagus Tips with Roasted Red Peppers

Orange Ginger Asparagus



Spicy Nutmeg Carrots

Ruby Carrots (cranberry)

Carrot and Coriander Soup



Cheesy Chicken Chowder



Main Courses



Zesty Chicken Sauté

Apricot Glazed Chicken Breasts

Sesame Chicken Salad



Skillet Shepherd's Pie



Bell Pepper Stir-Fry (on white rice)

Risotto Napoletana (sun-dried tomatoes, salami and parmesan)



Broiled Sesame Salmon

Oven-Baked Tandoori Salmon



Desserts



Raspberry Sorbet

Fruit Crisp*

Last know good configuration

Darcs has a feature (that I have never really tried) that allows you to locate the last working version of your source. I am currently shlepping through changesets in SVN applying them one by one in order to figure out which one breaks a series of tests that depend on our mocking library. It seems like it would be so simple to just repeatedly run the tests, stepping back through revisions until the tests pass.

Twittering tools

Last night I installed Twitterbar, an extension for Firefox that adds the ability to post to Twitter straight from the address bar. In conjunction with the Jabber extension, JabBar, this provides a really nice little interface for using Twitter. I have a small, single line rolling feed from Jabber and since Twitter updates me via IM I have a mix of lines from any conversations I'm currently having with my contacts and any updates from people I'm following on Twitter. Now, I also have a way of easily posting twitters without changing the current window I'm using. Previously, I would have to open up the conversation panel with the Twitter IM bot, now I just fire and forget.

A reality check for old party lines

Last night Quebeckers overwhelmingly told both the PLQ and the PQ that they no longer want to focus discussions along sovereignty/federalism lines. The ADQ was boosted into a position as the official opposition and I think that, this morning, people are looking for meaning in the moves from yesterday.



It is important to remember that a fairly low proportion of the population should be voting "strategically". The reason for this is simple: unless the results in your riding are really a forgone conclusion (and last night should serve as a reminder that nothing is as certain as we think), you may end up splitting the vote against your party. To have the foresight required to strategically align one's vote so that an overall message is communicated is equally problematic: in trying to send a message to my party by issuing a protest vote I may actually participate in their defeat (given the extremely thin margins between candidates in some ridings...cough...Sherbrooke...cough).



A common error is to ascribe some sort of intelligence or design to the results of an election. Phrases like "the voters have spoken" or "Quebeckers are sending a strong message" are not false, but they only hint at the underlying truth: the message delivered is a composite one. To boil it down to any one factor or issue would be to imply some sort of external collusion on behalf of the voting population. To look only at the votes that counted towards a win is equally treacherous, especially when one notes that most ridings had at least two if not three candidates neck and neck all evening long. Indeed, one of the more cogent reflections of the evening remarked that in addition to winning 41 seats, the ADQ managed to come in second in over 40 ridings. Wow. Think of what could happen if they managed to attract a couple of extra votes! Dumont was absolutely correct in saying "on connait la prochaine etape; je vous y donne rendez-vous."



All in all, I am pleased with the results of this election; a lot more pleased than I have been in recent years. I was disappointed to not see any Green Party members garner seats but noted with satisfaction that QS did not do much better. An unfocused and inexperienced campaign may have hurt these two parties - certainly QS did not exactly shine in a post-election speech last night where all they could muster was some vague ranting about proportional representation (a point that should be conceded but that hardly belongs exclusively to small struggling political parties).



I am happy that Quebeckers have decided to keep a leader that has experience. I am happy that they have placed a young, dynamic and energetic "watch dog" opposite Mr. Charest at the National Assembly. I am happy that the rise of the ADQ shows that a party can amount to something even if it is not one of the province's "natural" parties. I am happy that the election did not get polarized along "nationalist/federalist" lines. I am happy with our decision to have a minority government: in the future I think we'll see a lot more of these as we move from strongly polarised discussions to shades of grey. The only time we should have a majority government is when everybody agrees and the talent and experience exist to support that arrangement. Mr. Dumont is not ready for power but the last 12 years have certainly shaped his character; I look forward to seeing him push for a government of his own 2 years from now.



Mostly, I am happy with our commitment to the future of our province: by providing for an experienced guiding hand and a powerful young movement, we have the best of both worlds. By smacking down the PQ we have said, in no uncertain terms, "your platform is not for us this year, you will play king-maker and nothing else; when we are ready to discuss sovereignty again we have no doubt you will be waiting."

Term "visible minority" considered harmful

A UN committee released a report today on Canada's efforts to combat racism. The report praises Canada on some aspects of it's anti-racism programs but says that the term "visible minorities" may not be "in accordance with the aims and objectives of the [International Convention on the Elimination of All Forms of Racial Discrimination]". The convention says that "distinction based on race, colour, descent, or national or ethnic origin is discriminatory".

Great. Now we can no longer tell the difference between a black man and a Ukrainian because to make the distinction between them (that is, to employ another definition of the word, to discriminate) based on the colour of one's skin and the other's nationality is "discriminatory". Um yeah. Aren't we pushing this a little too far? I mean, how far am I from being an "individual with low amounts of skin pigmentation from a city near the coast with parents and in posession of a penis"? Come on! Sometimes we need to differentiate between two people or two groups.

Now, I understand that penalizing one group or another based on these distinctions is what is really at issue here. That doesn't stop me from being a bit frustrated that there are actually people somewhere in the world paid to come up with this shit, write it down and hand it out to other people who are paid to read it. We could be solving "real" problems but instead we are haggling over what to call somebody who is black. How to talk about a deaf person without mentioning the fact that they can't hear.

Talking about people's differences or referring to them in some way is not discriminatory. There is only a problem when those differences cause you to judge based on those differences instead of based on personal merit. Now go home, make friends with a black deaf retarded gay woman and call it a day.

Version control through the ages

It just occurred to me that over the past six or seven years I have gone from using CVS to track all my code and projects to using Subversion and from there to using Darcs. In general I find that each one has really been an improvement over the previous tool. I miss certain Subversion features in Darcs (specifically svn:externals) and get a bit confused sometimes when dealing with multiple repositories but the extra functionality is well worth the effort to learn.

Quebec: a question of definition

Prime Minister Stephen Harper has said he will introduce a motion to recognize Quebec as a "nation within a united Canada". This was immediately pounced upon by Gilles Duceppe who demanded that the motion be reformulated in order to recognize a "nation that is currently within Canada".

The unfortunate thing is that we are spending our time debating the wrong thing. That a certain socio-ethnic group of people exists in Canada is not a question. That the people in that group have inherited a certain culture and certain beliefs is not at issue. The real problem is not that nobody "recognizes" Quebec. The real problem is the divide between those who believe that Quebec should have a certain level of sovereignty and those who disagree.

It all comes down to control; and a lot of separatists/sovereignists feel that Quebec should have full and complete control over everything that goes on inside its borders; including immigration/emigration and any other kind of import/export. This is not a question of culture, nor is it a question of inclusion/exclusion: this is a question of power. We can see it when people suggest sending delegates from Quebec to international assemblies; as if we really wanted to debate the north american missile shield with the "ambassador" of the state of New Mexico. Countries exist for a reason. They are a grouping of geographically and socio-economically like-minded people who get together for the same reasons as any other community forms. They provide a tool for dealing with other groups of people and, as such, are very useful.

In the case of Canada, the country also provides a way to re-distribute wealth to attenuate the issues encountered by one or another constituent population going through hard times. When the farmers of western Canada need help, the fishers of eastern Canada provide it (albeit indirectly). With regards to this particular issue, Quebec has historically felt hard done-by because of its own richness and other provinces relative poverty.

Meta-communities also provide the opportunity to set certain guidelines and enforce them at a higher level. We can see some of the effects of this by looking at organizations like the U.N. where a number of participants agree to live by the rules established by all. Difficulties arise however when participants in such a scheme attempt to assert their independence or sovereignty. A perfect example is when the U.S. decided unilaterally to invade Iraq against the recommendations of the U.N. security council. Unfortunately, the sanctions available to the U.N. is very limited. It comes down to "might makes right" and in Quebec's case we can expect more of the same: because of its socio-economic place in the country, Quebec stands to gain a significant amount of "bullying power" by declaring its independence (or at least having it recognized). The simple fact that a political party exists at the federal level to represent and defend the concerns of a single province is proof enough of the fact that Quebeckers are prepared to do what it takes in order to guarantee that their concerns become matters of national importance.

Do I believe that Quebec is distinct? Absolutely! I was born here and will always be a Quebecker in my heart. Do I think that this "distinctness" (as opposed to the distinctness of Ontarians or Nova Scotians) merits the declaration of the independence of the province from the "rule" of the rest of the dominion? No. Absolutely not. So let's stop debating our uniqueness and start debating what's really at stake here: whether or not Quebeckers (and to a certain extent the rest of Canada) are entitled to a greater amount of "self-determination" than other provinces; I mean, at some level, each city could declare itself sovereign and we could start all over again building counties from cities, provinces from counties and (gasp) countries from provinces. If the only goal here is to get more power in parliament I say "let it be." You already have a whole national-level political party to yourself. Isn't that enough bullying for one nation? If not, then let's get the Mohawks and the Haida and the Inuit in there too so that they can represent their people.

pkg_resources.VersionConflict

This exception is raised when you update the version of a setuptools package without deleting the egg-info folder. The test command ends up expecting that the version is whatever is listed from the previous build whereas the value has changed in the meantime. Just run setup.py clean --all from your "root" directory to get rid of the error.

Some long-awaited updates

I took some time this weekend to revamp a lot of the infrastructure around this site. I really would like to start documenting my progress in pipe-making as well as some of the other things I'm currently juggling.

I have set up my own little proprietary engine to pipe the contents of different RSS feeds of mine onto the front page of my site. This should allow me to collect the various things I'm up to into a single spot.

Packaging Python

Ian Bicking writes about packaging Python libraries. He says simply that his advice is: "you shouldn't".

I really can't say that I agree on this one. While I do see the benefits of packaging libraries with the applications they go with, I have to say that after years of working in Java, where JARs are distributed and need to be included in an application-specific classpath, I really find the Python site-packages to be a breath of fresh air.

Users don't care about you

Jeff Atwood has posted an entry titled "Users don't care about you". I will use this soapbox to pick at a semantic nit and explain why not instead of trying to explain how to get them to care.

The reason is pretty simple (as all good reasons are in hindsight): they are users. There are no "users" reading my blog. There never will be. And this is really the crux of the whole revival-of-the-cult-of-personality-but-on-the-web thing. Unless your audience feels they have a personal relationship with you, they won't read your blog. I read all of my friends' blogs (if you are my friend and I don't read your blog, please send me a link to it). Why? Because even if the crap they write is boring, pompous, puerile, silly, humiliating, myopic and self-centered it's familiar crap. There's something about the odour that reminds me of that person. It's probably why you're reading this right now.

The majority of people writing blogs are doing it for their personal acquaintances. The "long tail" of blogdom is the really real world. You all know where I work, what I do, what my hobbies are and so on. Because you care about me. If you didn't, you wouldn't be reading this. What causes so many broken hearts (and wasted electrons) is that a huge number of bloggers sort of expect that they will clickety-click over to blogspot.com, tappity-tap to fill in the fields in the registration form, select a template, dash off an article or two on what they had for dinner and skyrocket to Number One On Google. Everybody wants to be the next Scoble (or whatever). Don't worry, the podcasters all want to be Adam Curry (much to Dave Winer's dismay); the real-estate agents all want to be Donald Trump; the investors all want to be Warren Buffet; the programmers all want to be Bill Gates or Linus Torvalds (depending on whether they want to be rich and famous or just plain famous).

Of course everybody wants to be just like those heroes and anti-heroes that epitomize and characterize their environments. That's normal. It's OK to wanna be cool. And part of being cool is being you. I know, I know, it's trite (oops, I missed that adjective up there in the rant about crap) but it's so true: who is Bill Gates like? Who did Michael Jordan copy? Do they have heroes? Sure! But they didn't rush out to try to be carbon copies of whatever they thought was the best thing since sliced bread.

I just watched a video of Seth Godin presenting at Google. In it he talks about making a remarkable product as being the first step in a marketing campaign. Of course, we're talking about blogs (or are we?) but the same principle applies: if you want people to make remarks and tell their friends to go and read your blog, you need to be remarkable. It's sort of like getting a girl to go with you to the dance in high school. You need to be remarkable if you want people to notice you. Truism after truism.

I would like to segue briefly here into a comment about attention. It's not enough to get attention, you need to get the kind of attention you want. I was talking to a friend recently and he mentioned that (verbally at least) I am able to hold people's attention for a lot longer than most. I credit this to the incredibly competitive environment at the dinner table in the house where I grew up. I am the eldest of three sons and if you are the eldest of your siblings, you know what it is to need to fight for that spotlight. I employ a range of techniques to keep people's attention and none of them involve brutally assaulting my listeners verbally or physically. I do things like only pausing for breath at a place in my sentence where I know that everybody wants to hear the end; raising my voice just enough to be heard and then lowering it again as attention focuses on me, rewarding my listeners by not continuing to bludgeon them with a loud, crass delivery of my idea(s); shifting my physical position to be in more people's line of sight or so that there are less obstructions between me and my listeners - every person in the vicinity not listening to me is not only not a part of my audience but may even begin to sabotage me by luring other listeners into separate conversations. What I'm getting at here is that throwing big temper tantrums and gesticulating wildly will make people sidle quietly away to talk to me until you are left with only the most spineless of people standing around you desperately trying to finish their drinks so they can escape to the punch table before you start another long-winded story. Or maybe that's me...

OK, so what's the bottom line here? Well, if you really want people to read your crap (I know I said I wouldn't explain how just why but this is my blog so...) they need to really know you. I remember when I first got into podcasting and I listened a lot to this one guy who was rude, crude and whose podcast was really just him blabbing around a couple of pieces of "underground" hip-hop and techno music. I fast-forwarded through the music. What I really liked about this guy was:

1. How raw he was: there was nothing polished about his performance (lots of "ums" and "ahs" and plenty of swearing), his podcasts (plenty of snap, whistle and pop) or his distribution (a simple RSS feed without much around it)
2. He came from Montreal (yay locals!)
3. He talked about the shit going on in his life and made me feel his pain

I identified, ultimately, with how crappy this guy was - there was something so human about it. Adam Curry, in contrast, with his silky voice (run through 18 compressors) and his trailers (recorded by adoring fans and mixed on donated software from companies hoping for a plug), seemed more of a confection than anything else.

So, at the end of the day I just want to read a blog, written by you, to me, for us and our friends. If you can slip in some stuff that only we know (it's like a club) or that I don't know yet ('cause I still like to learn - even when I'm killing time reading my friends' blogs) then that's a real bonus. But really, please, just be you and I'll be me, and hopefully, because we're friends, we'll actually have something to say to each other.

Update: After posting this I realized how huge that first item in the list is. This is both because I am being clever (see Ma, no hands) and because it is really the most important. I like it raw.

Pivot points

I just completed a webservice that calculates pivot points for major FX pairs. Try it out with USDJPY. The data backing it is downloaded and cached from Forexite Ltd. You can swap the symbol in the URL for different sets of pivots. You can also append .txt or .html at the end of the URL to specify the output format - the default is HTML.

Foreign Exchange

Over the Christmas holidays I discovered forex trading. This is the practice of buying and selling large amouints of foreign and domestic currencies and speculating on changes in the price. I have spent much of the last several weeks reading books on the subject of technical analysis and accumulating a small hoard of resources to help me understand how to trade. I have opened a demo account with Alpari, a russian forex site (don't worry Mom, they don't have my money yet) and am attempting to learn how to play with the big boys.

I have started a blog to track my trading progress. You can watch my progress as I battle my way through a $1.5 trillion market.