Ketil Bjørnstad

I finished reading To Music by Ketil Bjørnstad the other day.

It's mixing a realistic presentation of an young aspiring pianist and his chaotic life before his carrier starts with a sneaky, subtle weirdiness, a bit like Twin Peaks; very unnerving but at the same time funny. Of course, once I started on it, it was impossible to put down. For any interested in their general health condition, I recommend staying away. Seriously.

It makes me wonder what I look for in a book when I go to the library. My previous book was by Alistair MacLean, which today requires an humorous attitude towards his anachronistic a-man-is-man world views to read, at least for me.

Beautiful code

Anders pointed me to a talk by some Ruby guy (Marcel Molina) speaking about beautiful code. I must admit that I was a bit sceptical, but Anders was very convincing.

The verdict?

When it comes to ruminating on software design, I think there's a big nasty trap which he unfortunately walked straight into (as foreseen) with few extenuating circumstances (not expected).

The problem is that the field is hidden in a fog of mysteries, buried in what's governed by intuition and tacit knowledge rather than explainable ration. Good programming, hah, that's an art, nobody can tell you how to do that!

What we need here is to be able to talk about the thing. More ration, less intuition. Trying to explain things as beautiful or not is a step in the wrong direction. It's romantic self-indulgence, like when you look at people younger than you and think, people these days... An operational set of values for evaluating code is an essential thing for an aspiring programmer. How can beauty be operational if you don't even know how to argue about it with a fellow programmer?


Beauty? Digital graphic art from YayArt.

In any case, I think his main point can be summed up to: ensure that the code is as small as possible, as clear as possible and does what it's supposed to do. As I mentioned last time I wrote about software design, I think this can be simplified to make it easy to understand.

Save the interesting but tricky beauty discussions for things like this.

The repair shop

One of the annoying things about modern home electronics is that they are black boxes. Like a microwaver, cold food in, push buttons, warm food out. Who knows what happened inside?

Most people are probably happy they don't know. But a life without curiosity is a life with less passion. I was fortunate enough to get an excuse to take apart two home appliances recently because they broke. And it wasn't even my fault.

The first successful repair was Janne's younger sister's laptop. The power chord had been loose for some time, at some point the laptop simply stopped working with symptoms of no power.

The anatomy of the repair is not too far from debugging software you don't know (I've previously talked about debugging software you've written yourself). Take the system apart, examine the individual components, collect information, reason. Identify the faulty component and apply the easiest fix you can think of.


Taking apart the laptop


The faulty component, the switchboard that the power chord plugs into (visible at the finger tip)

I had an edge here. I'd heard about this problem before. So instead of giving up beforehand, it got me thinking that if it was a common problem, the remedy would probably be well-known among electro-hobbyists. A bit of googling revealed that some people had success with soldering off the small house the power jack is inserted into, and replacing it. When we examined the house, something was in fact wrong with it, as witnessed by a simple currency test with a multimeter.

So we set out to solder it off. Unfortunately, that didn't work out. The soldering metal wouldn't melt properly. Instead we ended up replacing the whole component, i.e. the small part of the mother board that the ports were sitting on. The web is extremely handy here. Just jot down the spare part number and search for it, or parts of it.



We ordered a spare part which ended up costing less than 1/10 of the price of a new laptop, and put it in.

In reality, this was a bit harder than it sounds, because the first two places that turned up on the web didn't actually have the part when we tried ordering it. Also because taking apart a laptop is a bit complicated because of all the tiny screws and chords and plastics that have to be bent in awkward positions, sometimes more violently than you'd like to think about.

Last week, I had a much nicer experience changing the BIOS battery on my trusty old Pentium III laptop. Ugly looks, but nice internals.

Continuing on this saga, I've also fixed our microwave oven. Microwave ovens are a bit more complicated in the feature set than I hinted above. They also make the food turn around, slowly. But our oven stopped doing that. So I took it apart, hoping that it would be a bad connection.



One non-rotating Samsung microwave oven


The faulty component in the microwaver

However, there was nothing wrong with the connections inside the oven. The funny thing about hardware is that as soon as you take off the shell, it looks complicated and futuristic, but in reality it's just a set of interconnected smaller components. In this case, the sealed turntable motor was broken. Again, with the component number it wasn't hard to find a spare part on the web.

However, this presented me with an interesting real-life dilemma. Is it worth 22 £ to me to be able to see the food inside the oven carouseling past? My first answer was no. I already got to see the oven inside, it's actually pretty simple, and identify the problem. I didn't have to actually fix it.

A couple of weeks later, the oven fried a very useful corn bag we're using to loosen stiffened neck muscles, a useful cure for some kinds of head-aches. The oven had burnt a hole at one particular spot in the non-rotating bag.

So today I installed the spare part. Works like a charm.

This may sound silly, but fixing supposedly unfixable things is really rewarding. You feel powerful and virtuous.

A couple of extra garden pictures:


Enjoying the spring sun at noon


Next day it's snowing

Spring!

Spring is upon us here in Denmark! The signs are subtle but sure.

My snowdrops are flowering.




Galanthus, snowdrops

The garden looks as if it is almost dead, but upon closer inspection small green leaves are shooting up. And the ground is soft and wet, not hard and frosty.


My mostly dead, but not quite!, garden

It's getting warmer, my trusty winter coat and wide scarf is getting too warm for driving up the hill on bike.


Me and a tame deer in Arden in December

Yesterday a pair of ducks had occupied the pond in the park. This morning two coots were grassing when I passed downhill. And in the evening blackbirds are singing.

Yay!

PS: If you're quick, there's still time for our special offer for new art on paper and canvas on YayArt.

Ideology and pragmatism

Perhaps I failed to mention the most important point in the thing I posted about capitalism the other day. Actually, I've been thinking about analogies between distributed software architecture (big systems) and society (a big system) for long, but I regret having spent most of the post about it because that kind of ego-centered pattern-matching is usually in itself pretty tiresome to read about.

A more interesting point is that of ideology versus pragmatism.

Interestingly, there's a similarity to them. When you believe in an ideology, you'll defend your choices with the end justifying the means. We may leave the poor bastards in the mud or we may kill half a million Iraqis or we may suppress people who think different from us and destroy the historical monuments of our past, but in the end we're defending freedom or equality and that's more important than anything else. It will prevail, and everything will be better than it once were.

When you believe in pragmatism, you'll defend your choices with the end justifying the means. We may spend ten times as much fixing the minds of violent criminals as the traumatized minds of their victims, we may have to see half of our help to poor nations disappearing in corruption, we may have to accept that people with big salaries get the lion's share of a tax cut. But in the end, we can prove that everyone is generally better off than with the alternative.

Religious pragmatism (from Flickr)

Pragmatism is discomforting. A true pragmatist will defend actions that are against the values that person believes in. Of course, ideology is discomforting too, as it requires you to suppress the truth.

This is all fine in theory. Most people will probably agree that we should base political decisions on ration and facts rather than beliefs.

However, in practice nobody knows for sure what the future outcome of a decision is. This is worse in some fields than in others. There's also the aside that without strong personal beliefs, lots of the valuable work being done in this world would probably cease to happen. Ideologically founded people will put lots of energy to an unselfish end in satisfying their drive.



I'm personally ideological when it comes to environmental issues. I would like to think, self-indulgently, that it's because I'm thinking further ahead than people in general but the truth is that I don't know why. As the windmill production in Denmark has grown to an international leadership, this has become more of a pragmatic decision. To my dismay, the right-wing in Denmark then blocked the whole windmill programme for most of the 00s out of what I see as ideological issues (money rests better in people's own pockets).

However, in other areas I find it easier to be pragmatic. Dismissal of known facts is something not to be taken lightly. This is especially true when it comes to issues where the ideological argument is persuit of justice or equality. The world is not just (sometimes children are molested), and we're not born equal, and although there's nothing wrong with building a system that tries to ameliorate these things, it's important to keep focus on reality, not the ideas.

Actually, that leads me software design again, but I think I'll save that for another blog post. :-)

Caching on the web

When you build a web application intended for a large scale, there are two separate performance issues when it comes to generating the HTML the server spits out.

One is server load. How many visitors can the server cope with?

The other thing is response time as seen by each visitor. How long must I wait for the page to load?

When using a relatively slow dynamic web framework (as Python + Django), the issue with server load is mostly related to CPU time. If you read the Django docs, the recommended solution is caching, i.e. instead of generating the HTML on each hit, we generate it once, store it for a while, handing out the stored snippet until it times out. It's a simple idea that works incredibly well.

First question is where we do the caching. To reduce the workload as much as possible, we want it to happen as close to the browser as possible. For a low-cost solution that works for everyone, we need to stick to the same physical location as the web server, however.

With Django on Apache, you've got some relatively fast C code serializing the requests to separate Python processes running the Django code. The process model limits the concurrency level possible because each process gobs a fair amount of memory, and Python is just generally an order of magnitude slower than C. So while putting the cache in the web framework is arguably very convenient, it's also pretty slow. From my experiments, one or two orders of magnitude.

Ideally, we'd just run a simple process written with speed in mind in front of Apache to do the caching. Afterall, the idea is relatively straightforward. Enter Varnish, a web cache written in C.

With Varnish in place, you know that the server only has to generate a page once per timeout. Some of the pages on YayArt take about 1 second to generate because they need to do multiple complex database queries. Let's say we have a beefy server that can do 10 of these in parallel. Then we could support 10 requests/s. With caching in Varnish, we can scale up to whatever it takes Varnish to retrieve a string from its cache and serve it. You can easily reach 6000 requests/s. For comparison, 100 requests/s is 8.6 million hits/day.

However, as it turns out, while simple whole page caching can solve a large part of the server load problem with one blow without giving up the convenience of a dynamic web framework, it's not necessarily the solution to the response time. If the page is in the cache, the response time is close to perfect. But what if it's a slow day with few visitors and it's not?

The general solution is to precompute the answer. Unfortunately, how to do this is more application specific.

You might be wondering how to combine caching with personalized pages. On YayArt I use Javascript to integrate the per-user data into the page through AJAX. So the per-user server load is reduced to processing the actual per-user data instead of a whole page.

Market economy

For Christmas, my parents bought me 3 months of subscription to Information, a small Danish newspaper with a focus on analysis and criticism of the daily politics. The audience is mostly intellectual (and also mostly left-wing).

Information is interesting for several reasons, one of them being that major politicians in Denmark, the people who actually make the rules for the rest of us, regularly post in the newspaper, and thus presumably also read it. For an old internet-addict, that's a whole new experience.

There's been a lot of talk lately about capitalism in the face of the current financial crisis. The thing is that over the past 20 years, almost everyone in the political landscape has been pursuing a the-more-market-economy-and-deregulation-the-better strategy to a some degree. And now it turns out that too few rules for the gamblers in the financial sector have put us into a global recession. So people are, again, beginning to question whether capitalism is such a hot idea after all.

When I went to high school, I always thought it was possible to do better. Because there is such an obvious waste in our market economy. For an example, go to your nearest supermarket and look at the shelves with shampoo and hair products. Or the shelves with soft drinks. Or the shelves with morning cereals for children. And ask yourself, how much value does all these colourful and overly expensive things bring to our society? Or the really classy ads that never mention any factual qualities in their products, but instead try to install in us an irrational idea of their products, because of the packaging so to speak, bringing improvements to our lives.

I ask you, how can drinking water with added sugar and various brown chemicals make you cool? Because a large company has invested enormous efforts in convincing everyone that it is so.

Surely ration can do better than that.

However, at university I spent a couple of years working with distributed systems. The most important lesson I learned is that centralized systems are bad, unless the scale is very small. Decentralized (peer-to-peer) systems are more adaptive to change, more robust and much more efficient - several orders of magnitude as the system scales. Yes, there's waste, redundancy, suboptimal behaviour.

But it's self-organizing. If I want something from a peer, I just ask - I don't need to contact a central authority which then has to decide how to respond to the change. Overall it just works incredibly better.

I think that the same is true of society. With a market economy, decision-making is decentralized, in spite of the tendency for old industries to advance towards monopolies. In the large scale, this decentralization is unbelievably effective compared to a centralized control because of the complexity involved.

It's also a lot more free than a democracy. In a democracy, 55% can decide for the remaining 45% (that's why we haven't built any new windmills in Denmark lately). With a market economy, everyone decides for himself. Within the limits of the market, of course.

However, not all is good. One of the more peculiar problems of market economy is the drive towards monopolies, i.e. a collapse of the market (read a treatise of Karl Marx's works to understand why). Like any game we set up, it needs rules.

Karl Marx

And the market cannot do long-term thinking. As we have just seen, it can happily drive over the cliff edge because of a phenomenon called the tragedy of the commons. It doesn't hurt me a lot that I exploit the system: I get the whole benefit, the downside is shared between everyone.

The tragedy is that even if some market players want to stop, they face the competition from the others. If price is the only factor, a short-term thinking competitor can drive the others off the market. So they are forced to follow, unless a force beyond the market sets down rules that cannot be ignored. The same reasoning goes for unethical behaviour.

So the issue here is how we change the rules of the game to ameliorate the bad things without throwing the basic idea out of the window. For in spite of the waste, it's working better than the alternatives. Ration is bounded, it's not enough to deal satisfactorily with the needs of millions of humans.