Pivot points

I just completed a webservice that calculates pivot points for major FX pairs. Try it out with USDJPY. The data backing it is downloaded and cached from Forexite Ltd. You can swap the symbol in the URL for different sets of pivots. You can also append .txt or .html at the end of the URL to specify the output format - the default is HTML.

Foreign Exchange

Over the Christmas holidays I discovered forex trading. This is the practice of buying and selling large amouints of foreign and domestic currencies and speculating on changes in the price. I have spent much of the last several weeks reading books on the subject of technical analysis and accumulating a small hoard of resources to help me understand how to trade. I have opened a demo account with Alpari, a russian forex site (don't worry Mom, they don't have my money yet) and am attempting to learn how to play with the big boys.

I have started a blog to track my trading progress. You can watch my progress as I battle my way through a $1.5 trillion market.

An email address for every site

I wrote this advice to a friend on the topic of alternate email addresses used to sign up at various sites:

By the way, a cool trick that I picked up somewhere is to register an account at http://www.cjb.net. You pick a subdomain of cjb.net (like joeblow.cjb.net) and then they redirect any email that goes to @joeblow.cjb.net to an email address of your choosing. So what I do is I have ilowe.cjb.net and every time I sign up somewhere, I give an address that encodes the name of the site I am registering on (for example yahoo-groups@ilowe.cjb.net). This has two benefits: first, I don't have to give out my real email address; second, if somebody starts spamming me using one of the addresses, I just add a filter to my mail client to kill that particular address and continue along my merry way. That way I don't have dozens of sites pointing to one "junk" email address that I need to log into every once in a while to clean out.

Subversion with Python 2.4

Buried deep in the comments on one of Simon Brunner's posts I found a reference to Eugene Lazutkin's setup where he hacked the Python SVN bindings to work with Python 2.4. I am pleased to report that his hack worked beautifully for me and I am now the proud owner of a Trac installation running on Subversion v1.2.3 and Python 2.4.

The trick is to download the Python 2.3 bindings in ZIP format, then unzip them and drop them into your site-packages folder. Once this is done, fire up a hex editor and edit each of the DLLs that are installed in the libsvn package and replace all instances of the string python23.dll with the string python24.dll. That's all there is to it.

YAXL v0.0.16 released

I just released version 0.0.16 of YAXL, tightening up QName manipulation and overall documentation. Check out the website.

Update: the source in the website looks horrible. I'm working on fixing it.
Update (20051219): the source in the website is fixed.

typecheck v0.2.0 released

I just released version 0.2.0 of the typecheck module to the Python Package Index. The module's website has been updated to reflect the new version number and link to the newest build but has not yet been modified to document the changed interface after the merge with Collin's code. He tagged the 0.2.0 version yesterday so we still need to catch up on a couple of minor documentation points.

The new version of the module maintains 90% compatibility with my old API - the only change is that you can no longer elect to not typecheck a parameter; a feature that I hope we will be able to bring back in a subsequent release.

YAXL v0.0.15 released

I just released a maintenance version of YAXL. This release contains a couple of passing test cases that had been waiting for release for a little while. I also fixed an embarrassing bug in the Python Egg distro that screwed up the builtin help.

YAXL's Subversion repository is now public. You can browse version 0.0.15's tag or just snoop around in the trunk to find the bleeding-edge stuff.

New code updates for typecheck

I will be merging the code in my typechecking module with Collin Winter's codebase. The new subversion repository for the continued development of this module is at http://www.ilowe.net/software/typecheck/svn.

We should be releasing a new version shortly with a whole whack of other/new features that Collin has implemented such as return-type checking and logical operators on types (and and or). Look for version 0.2.0 in your local Cheeseshop RSS Feed.

Email tacks

Rob Bray suggests a different tack to take for spam prevention/reduction.

Rob blogged about his resistance to the penny tax for emails that Tim Bray suggested in a recent blog post. In this installment, however, he changes his mind and instead comes up with a brilliant solution: only un-answered emails cost you anything. Most of us write emails to people and receive answers from those same people; spammers on the other hand send emails to thousands of people and only a few (idiots) reply. If we could make it really costly to send un-answered emails and cheap to send answered emails we might have the beginnings of a really nice, low-tech solution.

This suggestion of Rob's reminds me of a socio-economic system called Stone Society described by Peter Merel. The system involves creating an artificial resource that is then exchanged and manipulated by participants in order to allow decision-making to proceed.

In the world that Rob describes, you would exchange tokens freely with people with whom you have a back-and-forth. Spammers would simply send you tokens which you could accumulate. In other words, spam would be beneficial to you even if you didn't want to receive it.

If the tokens here were indeed pennies you would actually get paid to receive spam. You could still have email filters to make sure you don't have to read it. This plan is all about raising the barrier to entry for spammers.

Hacking the iPod nano

I hacked together a little Python script that scrapes Atom feeds and transforms them to iPod "contacts" allowing me to read blog posts from my nano. It uses html2text right now but a lot of entries with embedded HTML come out looking pretty weird. I'll post the script once I clean up the text generation.

I managed to get a couple of the vCalendar features working and found out that even though Apple says you need iSync or iCal to use the nano to add TODO items, you can just add standard vTodo objects to a vCalendar stream and it works just fine.

Also, the rdate property is what controls repeating tasks/events - not any of the other ones that purportedly serve the same purpose.

Now it's time to go to bed...

String/XML representations in YAXL

Damian Cugley asks

Could you eliminate the need for the text member variable by using str(elt) instead? Or would that interfere with using str to return the XML representation for the whole element?

In YAXL v0.0.14 I've split the representation of an element between str and repr so that the former would be a shortcut to return the text property and the latter would return the XML representation of the element.

I'm pleased with the separation since now if you evaluate an element at the interactive prompt, it returns the XML for the element but if you print or interpolate the element its string value will be used. There is also a nice parallel with the way nodes are handled in XPath.

YAXL elements retain the text property in order to continue to support the oodles of screaming fans who have already shipped billion-dollar projects based on my little library.

Thanks for the suggestion Damian!

Fall cleaning

Justin and I cleaned out some of the more cobwebby parts of mystic yesterday. We wanted to install Python 2.4 with subversion, trac and apache 2. It took a fair bit of wrangling (i.e. building from source) to get everything working together after which Justin realised that we hadn't upgraded to Sarge (duh!). He rebuilt everything using the latest packages and it looks OK so far.

The transition from Apache 1.3 to 2.x was pretty smooth with the SSL stuff at the high end weighing in at about 20-30 minutes to figure out. Although the new sites-available/sites-enabled organization is nicely architected, it does make for some pretty painful contortions when performing wholesale changes on a group of virtual hosts. For example, we started off testing the new installation of Apache 2.x on port 81 (which required that we split out the vhost declarations into files of their own) and then proceeded to switch it to port 80 (which ended up requiring a quick cat/sed/re-direct combination). Without a good knowledge of shell scripts or some scripting language this would have been really painful.

The other rough item on our plate was permissions when using the Berkeley DB version of a subversion repository. My advice: don't go there. Just stick to FSFS and you should be OK. This being said, I had originally created the repository in FSFS but had subsequently dumped it and loaded it onto mystic. It seems that I did something wrong because it loaded in Berkeley DB mode. All that is fixed now and taken care of.

We also tossed in Apache subversion support for good measure so now we can setup anonymous browsing and reading of our SVN repositories.

iPod nano in da house

Vem and I finally both got our iPod nanos! What a cool gadget. These things are too slick to be real!

My one small gripe is that we ordered a bunch of stuff from Apple and they shipped it all separately with the predictable result that the iPod armband for Vem's gym sessions arrived first (wow, look - an armband) followed by her iPod, followed several days and a trip to the local Fedex detention centre for un-claimed packages later by mine own.

Rumors about the nano being scratch-prone seem to have some truth to them as I can already (after a fay and a half) detect small scoring on the exterior plastic. Of course, we ordered protective "skins" but since Apple hasn't shipped them yet (and nobody in their right mind would leave an un-opened nano sitting on the shelf for three weeks) we may find that our nanos are not as pristine as they could be. Ah well...

Still, the photo feature is pretty sweet (useless but sweet) although it would be nice to be able to set a photo that displays automatically when you power on (sort of like Vem's digital camera), the rating feature is very cool (I was just ranting to Vem the other day about needing one of these). iTunes is not all it could be but I haven't started playing around with the iPod libraries out there to see what I can hack up.

Anyways, overall we love our nanos and would buy them again: 9.8/10

YAXL v0.0.12 released

Over the weekend I've managed to work in a number of bugfixes, add some more XPath support, add sequence-style access for children and implement a whole slew of namespace features. The latest source version includes 78 unit-tests that cover all this fancy functionality.

I'm still debating the question of whether or not to allow access to children via property names (like xmltramp). The problem is that you end up having to mangle either child names or "real" property names. Also, you need to have a mangled way for dealing with node-sets. YAXL curretly supports XPath even in XML fragments so theoretically you should use that, but it's hard not to like root.title as opposed to root('title'). More complex queries, however, reveal the power of XPath: compare [x for x in root.head.children if x.localname == 'style'] to root('head//style') to retrieve all style elements of an HTML document. Or even root('//p') to retrieve all the paragraphs.

Ideally, YAXL would support both methods but I still need to come to terms with all the mangling required.

Expat doesn't support namespace prefixes

This via "Crest" who sent in some nice stack traces generated by YAXL. I wrote in a fix yesterday to handle a missing QName but I didn't realise that expat would throw a SAXException. As a result, nobody using expat can use the parse function until I add a try/except for that exception.

SAX Parser Funny Stuff

It turns out that the startElementNS method on a SAX ContentHandler is not necessarily passed the QName of the element. The name is passed as a URI/element name tuple, but the QName is not mandatory unless the feature_namespace_prefixes feature is enabled. The following code shows how to do this:

from xml.sax.handler import feature_namespaces, feature_namespace_prefixes
from xml.sax import make_parser

contentHandler = MyContentHandler()

parser = make_parser()

# Perform namespace processing
parser.setFeature(feature_namespaces, True)

# Report original prefixed names (i.e. qname is passed to startElementNS)
parser.setFeature(feature_namespace_prefixes, True)

parser.setContentHandler(contentHandler)

parser.parse(inputSource)

I was made aware of this issue when YAXL failed to parse XML source properly on somebody else's machine; the elements were being named "None". That tip led to this fix. Merci Marc!

YAXL Reviewed

Jeremy Jones blogs about YAXL (already!). I agree with Jeremy that ElementTree is pretty much the best XML tool there is out there for Python. I also agree that the children attribute should not contain descendants. It never really did but since YAXL Elements are represented as XML fragments, it may have seemed that way.

Here's what an interactive session looks like:

>>> import yaxl
>>> x = yaxl.Element('x')
>>> x.append('y').append('z').append('w')
<w />
>>> x.children
[<y><z><w /></z></y>]

It may seem from the output that the z and w elements are returned as part of children but they are just part of the XML fragment that y outputs.

In the latest release (0.0.6) you can also "call" the element with an XPath of child::* and get back a list of immediate children.

YAXL gets XPath

I added some basic XPath support to YAXL this morning. It now supports both abbreviated and unabbreviated XPath queries on Element objects. Currently only attribute-value and node/node-set selections are supported. Also, only the following axes are allowed in queries: self, parent, ancestor, ancestor-or-self, child, descendant, descendant-or-self

Especially cool (IMO) is the fact that YAXL supports two methods of performing XPath queries: you can call select on an instance of Element or you can call an instance of Element and supply the XPath as the only parameter.

Something new, something old

I just launched a new site at http://www.ilowe.net. I will be retiring the schmeez.org site (the permanent redirects are already in place for the 3 people that visit regularly) for many reasons (not least of which is having to spell it every time I tell somebody where my "homepage" is).

I also released the first version of YAXL, a Python module for reading, writing and manipulating XML. It will form the basis for an Atom library I am currently writing.

New version of typecheck module

I've released version 0.1.4 of my typechecking module for python. This version contains a fix that allows you to use doctest tests within typechecked functions.

This doesn't work out of the box since the doctest module looks for tests by recursing through a set of objects and looking for docstrings. If it finds a function, it checks that the functions func_globals object is the same object as the function's module's __dict__ property. This is not true of any decorated function where the decorator is defined outside the module the function is defined in.

Anyway, the new typecheck.doctest module allows you to bypass the check that doctest usually uses and replaces it with a much more liberal one that just verifies that a function's __module__ property is the same as the name of the module in which the function is defined.