Friday, October 25, 2013

The disproportionality factor - why do people get angry?

It has recently occurred to me that many of the conflicts and negative feelings I see from time to time at work and elsewhere are caused by two parties having very different levels of interest in something they are cooperating on. For one party, the party that wants something from the other party, the effort is extremely important, for the other party, the delivering party, it doesn't really matter.

For instance, think of a situation where you're going to a foreign country on the other side of the world. You have done countless preparations, booked the flights, scheduled everything around it at work, and now just need to obtain the visa from the embassy - which is late.

So you send reminders, press for it to go through, and finally get through by phone in the one service hour per day to a person who kindly explains that yes, he has your papers, and yes they look fine, but unfortunately he will not be able to process them because he's going to lunch and after that he has two weeks off so the case will have to wait on his desk until well into your travels.

I think most people would get pretty angry in that situation.

For better or worse, we have to rely on other people which means they sometimes have power over us. I am not talking about formal power relations, I am talking about the fact that when someone is doing something for you, you have effectively lost control over what happens. A manager who assigns a task to one of this workers is dependent on the worker to do his job. Otherwise everyone, not just the worker but also the manager although he's formally the one in power, gets in trouble.

And when you have lost control over something that is very important to you, and the other party has very little if any inherent interest in the matter, chances are something will go wrong.

Oops, you need to walk here? (by Gabriel)
I think of this as the disproportionality factor.

More examples: the bus is a bit late. For the driver, it's only 3 minutes, but for you it's the difference between being able to catch the train or having to wait two hours for the next one. Are you cursing at the transportation company? Yes, you are.

Or in reverse: you spend 10 minutes extra at work with the phone turned off to finish up an important project, get home in high spirits, and find an angry and hurt husband or wife who wasn't able to go to a concert because you were supposed to get home half an hour early to take care of the children. Which you had of course forgotten - because you weren't the one buying a ticket three months in advance in anticipation.

In a similar way at work: when we call the hosting company because our server is down, it's critical for us. But for the operator in the other end, it's just pressing a button to restart one server among thousands.

It happens with minute details too: why are people not aligning their vehicles properly at the parking lot - now I can't park properly! Why are people not formatting their code properly when they know I hate camelNotation! Why has this error in the software I'm using not yet been fixed when it annoys me everyday!

I think there are two things that are important in dealing with the disproportionality factor.

The first is that we need to understand what's important to other people. This goes both ways - you need to think about what's important for people you are dealing with, and you need to communicate effectively what you think is important.

The second is that we need to remember that on average, we can't expect people to really care that much about anything that isn't important to them. That's just human nature.

So we need to help them, not just by explaining what we think is important but also by helping them out, instead of just crossing our fingers and hoping for the best, and then getting ticked off when we are inevitably disappointed.

And sometimes, when in the land of the minute details, we perhaps need to help ourselves to just get some perspective.

Friday, June 29, 2012

Exim with a whitelist of recipients

At IOLA, we have a couple of servers that we use for a lot of different things - just not running live code.

Now, with a standard install of Linux with something like monit installed to respawn dead processes, whenever there's a problem a local mail server is used to send an email about it. That's a good thing because it means you never hear from the server unless it's got a problem, so you don't have to monitor it at such, except for reachability.

If you're on Debian, you'll get Exim installed as the default mail server, together with a semi-scary, semi-friendly series of debconf questions to configure it. Unless you configure one of the questions, it's locked down so it won't accept connections from the outside.

So if you remember to configure your real email address as an alias for the local users (e.g. with a root: my-email-adresse@example.com in /etc/aliases/) things actually work quite well.

The only problem is when you actually run a web site project on the server, either for development or as a test instance before putting it live. It's not uncommon for a web site to have some corners where it's sending email. Which you definitely don't want to be able to happen by accident during testing.

So in the past, we've configured Exim to not deliver any mail remotely. No accidents possible. But this means we never got any problem reports from the server either.

What we really want is to be able to whitelist a number of adresses that Exim can freely deliver email to, the rest should be discarded.

One way to do that on Debian is to select the split configuration, then put these lines in the file /etc/exim4/conf.d/rewrite/01_rewrite-everything-but-whitelisted_LOCALCHANGE:
my-email@example.com * T
some-other-email@example.com * T
*@* thisaddressdoesnotexist@unrouteabledomain T
This means: skip all rewriting of a To envelope address of my-email@example.com and some-other-email@example.com but rewrite all other To envelope addresses to thisaddressdoesnotexist@unrouteabledomain.

The latter is of course not routeable so will fail and end up in the queue until Exim gives up. This is a bit ugly, so you can add the following lines to a new router in /etc/exim4/conf.d/router/01_drop-unrouteable_LOCALCHANGE:
drop_unrouteabledomain:
  driver = redirect
  domains = unrouteabledomain
  data = :blackhole:
This will match the domain unrouteabledomain and discard everything sent to that right away.

If you're not on Debian, the rewrite rule needs to go in the begin rewrite section of exim.conf and the router rule in the begin routers section.

I'm quite happy with this setup, it's actually a problem that has been bugging me for years. At some point I made a hacky Python script called from cron that would check the local mail and forward it through SSH to another machine with a mail server allowed to send remote email, but that turned out to be a error-prone and high-latency idea.

Thursday, May 10, 2012

Arab spring

National radio in Denmark has been broadcasting several documentaries on the recent change of regimes in the arabic countries.

Last week it was about Yemen with The Reluctant Revolutionary. One smooth tourist guide, one half-exploitative journalist and a sit-down revolution camping in the streets.

Yemen demonstration - by FreedomHouse2

It is amazing to watch.

There's a lot of talk about freedom in libertarian ideology. But here we see how it's like to be beaten by hired thugs or even just the police for requesting a change of regime. The martyr syndrome unfolds - each beating makes people more angry, each casualty is a proof of how callous the rulers are. It is brutality, weapons and bullets against the sheer force of the people.

I'm happy I'm donating to Amnesty, although they were surprisingly quiet during the whole thing. Maybe revolutions just aren't their thing.

Wednesday, March 28, 2012

Monday, March 19, 2012

When automated tests make sense, and when they don't

Some years ago, I stumbled upon an article about automated testing, "When should a test be automated?" (I'm linking to Google since my previous link is now broken). I found it quite interesting because it, unaffected by all the hype surrounding automated tests, provided some rational guidelines for when to write automated tests and when not to.

A main principle was that an automated test is a waste of time if it doesn't find a real bug upon being run again.

This is a good thing to keep in mind, but since we generally don't know where the bugs are or will be in the future, it's not a very practical rule. So on and off, while working on various web applications, I've been thinking about this.

Why automated tests are good
Praises of automated tests are not hard to find. But briefly: the main practical advantage is that if you change the software, you can easily rerun the tests to see if you have broken something, compared to having to retest everything manually (or just crossing your fingers). If you test the same things over and over again, having it automated can be a real time saver.

There is also a psychological advantage; writing and running the tests make you feel better. Plus you're in, which is always a good thing if you're a little low on the hype factor.

Why automated tests are bad
However, there are also some downsides.

First of all, automating tests takes time.

And unfortunately, it's usually a pretty boring task. It doesn't help that you often have to test things manually too while experimenting or as a sanity check, so you pay the cost of testing twice.

And the tests, while radiating nice comforting feelings, also have a tendency to give a false sense of security. The tests are running, so it works, right?

So you always have to keep in mind that automated tests only test part of what makes the code correct. You could argue that this is a sign of a too lazy test writer, I think that's the main driver in test-driven development, but really, you can't foresee everything (pesky user clicks the wrong button), and if you try to, you end up with so detailed tests that you can't change the program without having to the rewrite the tests completely. And then they have lost their main automated advantage.

Besides, an automated test only confirms what you already suspect, it doesn't tell you if the software is creating real value for its users. So you still need human testing.

Furthermore, even if you don't test everything, in my experience you still end up with a large amount of test code, as much or more as the actual program. There's a hidden cost in this; many changes to the program will also require adapting the tests. In other words, code maintenance take longer. People don't think about this, but imagine you show me 100 lines of your code, and I told you I could reduce it to 40 lines with no ill effects on readability or performance? Would you take that? What if those 60 lines I take away are your tests?

The trade-off
With the understanding that automated tests have good and bad sides, if you value your time, you should evaluate the situation on a scale from should-automate to should-not-waste-time-automating.

Circumstances pro automation
Hard to setup tests. If you have to go through ten steps to be able to test a small change, testing quickly becomes a tedious chore. Usual issues in web applications are getting test data prepared, or being able to intercept email sent by the system, or testing error scenarios.

Important corner cases. Testing manually that a new button does the right thing is not tedious. But it is tedious if you have to try 20 different combinations of inputs to see they all work.

Team of disparate people touching the code. Most code have some implicit assumptions that cause bugs if violated. When multiple persons are modifying it, there's a greater chance of bugs, and smaller chance that each remembers all the corner cases that need to work. Note that you don't necessarily need more than one person on the development team to fall into this category. Add a gap of a year or two in the development, and most people will have happily forgotten the pecularities of the project in the meantime.

Complex code. If the code is complex, it's likely to be needing bug fixes that don't change the expected output, which is good because then we don't have to change the tests, and it's also likely that it's harder to see what the expected output is supposed to be, which can be conveniently solved by writing it down in a test.

The code is seeing changing input data. Modifications to complex code is one source of bugs, but another is changes in the input data. You test it today, and it works fine, but tomorrow your data source decides life is too quiet and changes something that violates your assumptions about the input. Again, this requires bug fixes that usually don't change the expected output.

The code is an API with no UI. If there's no UI, you need to write code to test it, and then you might as well automate it anyway. With Python or any other language with a REPL, this is not entirely true, though. I often test small snippets I'm not yet confident are bug-free in the interpreter.

Circumstances tipping the scales toward manual tests
Simple, easy-to-follow code. If it's simple, there are fewer opportunities for bugs and thus a greater chance automation will be a waste of time. For instance, code using templates to spit out HTML is in most cases trivial, a div here, a heading there. You could add a test to find out whether the div and the heading are there, but it's trivial to see that by mere inspection, and if you've checked the way they look in the browser, there's little chance they're magically going to disappear later.

Localized changes. If changes only affect things in their near vicinity and have no impact on the rest of the software, it's easy to test them manually. For example, consider a web application made up of several independent pages. When you're working on one page, you can just reload it to check it's fine and ignore the rest of the program.

Manual tests are simple to set up. If all it takes to run the test is a reload in the browser, the relative cost of automation is high.

Hard-to-quantify issues are important. If look and feel are important, there's currently no substitute for a human. Imagine setting up a test for an animation - how do you test should look good?

The functionality sees lots of changes and experiments. If the functionality of the software keeps changing, maintaining a set of automated tests is a burden and is more likely to turn you into the grumpy nay-saying change-resistant bastard it's important not to be.

The software sees no changes. If nothing changes, there's little opportunity for bugs and little opportunity for rerunning a set of automated tests. This situation sounds strange, but actually in my experience there's a certain polarity in software maintenance; some things change all the time, others are truly write-once. Of course, this observation doesn't help a lot when you're starting on a project and don't know where it's going to end.

Only one person. Again, this is an important special case. If there's only one person working on a given piece of software (or perhaps module), that person is likely to know exactly what needs to be tested, diminishing the value of a broad test suite.

Minor errors are not critical. This sounds sloppy, but in reality with many web sites, it creates more value for the users and customers if you focus on having the major paths hit the target than being 100% correct because errors outside the major paths can be fixed quickly when discovered.

Of course, most of the above points aren't independent. For example, say you're writing a little script to convert your music collection into a standard format. In that case, you're the only developer, you're going to experiment and change the script until you're happy, an error is not critical since you're around to supervise the process, and you don't expect the script to run more than once when it's finished. Would you write automated tests for that script?

Unit testing versus acceptance testing
I have an extra remark regarding the level at which the tests are performed. There's been a lot of talk about unit testing, in fact most automated test frameworks are called unit testing frameworks, but the reality is that unit tests represent little any end-user value in themselves because they are just providing evidence that some component that nobody will ever run by itself is working if it is being run by itself in the lab.

A more appropriate level for end users is acceptance test where you are testing requirements to the system itself. For instance, the site should enable authorized users to add new products which should then show on this page or when a customer pays for some goods, an order should show up in the system. These are the kind of things that when failing will be sure to cause trouble for the users and thus you.

Of course, unit tests can add value indirectly by highlighting bugs early in the process where they are easier and thus cheaper to debug.

So where are we?
In my experience, when it comes to web development, there are some instances where unit tests make sense for a complicated component, and also some examples of vital processes where acceptance tests make sense because testing those over and over gets annoying. One example could be a sign up feature. Nobody tests that by themselves, because you only ever sign up once.

But otherwise a lot of web development fit the above criteria against automation quite well. Hence, contrary to what the hype says, I do believe automation in many cases would be a waste of effort.

Note that this is not an argument against testing. It's an argument against automating the testing.

It's also not a universal observation. For instance, if you're writing library code all day long, your world is likely looking different. Or if you're Google or another big web site, then the operating conditions are different - as a part of a roll out it's important to have easy reassurance everything is behaving as intended because even small mistakes can be painful if you have a million users. But most web sites don't need to scale.

Also, don't forget that the purpose of testing, whether it being manually or automated, is to find and fix bugs. Thus preventing the bugs from happening in the first place is perhaps the best place to focus your attention. Usually the key is easy-to-understand code with few implicit assumptions and attention to detail, making sure corner cases are handled well.

Thursday, October 6, 2011

When two becomes three

The evening before, when Janne and I were going to bed, at around 11, she complained about a pain in the stomach and got up to take a bath. I was mildly suspicious, but the last couple of weeks she'd had some painful practice contractions and trouble with the back. I was exhausted, having slept poorly the previous week, told myself that if true labor contractions had started, she would tell me, and continued to try to sleep poorly.

At around 6:00 in the morning, I finally wake up to a meak yell from the bath room - Ole, I think we need to hurry up. As I later discover, the light of my life has been in labour all night long, not thinking it was a big deal until she can feel the water break and the baby's head coming down. She gets out of the bath room, calls the hospital, they agree to see her at 7:00. As we later learns, she should have made clearer how far she was and they would have sent an ambulance.

Instead, on her request, I then call my parents who live nearby and get an awake and triumphant mother in the other end at the first ring; she had a bet with my sister on the birth being this very day. My father will come to pick up us. At this point, noone but Janne has any idea how far she is. Yet, my father hurries through the dark empty Sunday morning streets of Aalborg and we reach the hospital safely.

There we're greeted by first an assistant and then a midwife whose shift is ending. She decides to have a look at J. anyway and finds out the cervix is fully open, declares her the most tough and calm woman in labor she's seen so far and immediately walks her to the nearest delivery room.

Lots of pressing, sweating, leg-pulling (the midwife recommends a birth on the side so one leg has to be pulled up during contractions by J. and finally me) and about an hour later at 8:45, our son is fully born. Near the end, when the head is just out, there's a funny episode where the three non-labouring women in the room, the mid-wife, assistant and midwife student are rejoicing over the fine baby head and trying to persuade it into not breathing, which isn't possible as the torso is still not out and compressed by J.

But it is over. Janne gets the child, I cut the umbilical cord, we get to see the placenta and Janne gets some stitches while the midwife try to distract her by talking about breastfeeding.

The midwife assistant was kind enough to give us a good pull at the foot so we got all the mucus screamed out of the throat

One happy new mother a couple of hours after the birth (and some breakfast) - if the little guy looks somewhat baked, that's because somebody forgot to tell the new father who was ordered to dress him that he was supposed to turn off the heating lamp in the box after giving the boy outdoor clothes on, oops - this was after the father nearly managed to burn his own hair by sticking it into the heating lamp

Apple, Google, beware - they had a magic tablet, when we got in they wrote our names on it, and by magic, when the birth was over there were happy flags and everything on it, I swear I never saw anyone get near it (the text is "first child, fine boy :), pregnant for 41+0 [weeks/days], congratulations with him")

Afterwards, we were kindly allowed to stay for four days at a patient hotel as a special service for new parents. Four days of getting to know the little guy, the three of us alone, apart from occasional visitors, getting help from midwifes to get breastfeeding going (whoever said the nipple is the only intuitive interface obviously wasn't a mother), and lots of good food. It was fantastic!

One happy, and somewhat tired, new father

The little guy in a transportable cradle we used to get him to the hotel restaurant - the book of hypnosis is used to pacify him temporarily

It's 12 days ago today. We've already learned a lot, all three of us. He's now looking around, taking in the world, folding his little fingers, trying to use his limbs, practicing looking angry, smiling, grinning.

One thing that really surprised me, in addition to the enormous wave of happiness which so far has steered us through sleep deprivation, occasional moments of angry, nerve-wrecking screaming; what really surprised me is how funny such a little guy can be. For instance, at one critical moment in the middle of the night we both hold our breath at a pause in the crying, looking at him, his eye brows pulled together, mouth turned downwards, will he accept our attempt at comforting him? And then he farts loudly, gives us a quick happy grin and falls asleep.

Closeup of the little guy 6 days later, the red marks above the eyes are now fading, they probably come from J. who gave him a good squeeze when the head was half-way out (the midwife ordered a pause)

One of these days we're going to give him his first bath. And I can't wait to see what he will do.

Sunday, July 31, 2011

Cells and biology

I recently borrowed a book from my sister who has a Master's in biology, Zoology by Dorit, Walker and Barnes, about 900 pages, intended audience is college biology students I believe. I've been reading it over the summer, and I must say it's the most interesting book I've read in years. I can't recall the last time I've learned so much in such a short time span.

The book goes through what stuff cells are made of, how respiration and cell division works, how protein compounds are made, how it is thought various solutions to problems evolved, how various features of species stem from physical limits and constraints in the environment. For instance, tiny animals don't have lungs. They don't need them because no part of their internal body is ever far from air or water. The complicated blood and lung system of humans is really an extraneous necessity required by our huge size.

The book then covers the various animal groups one by one. As it turns out, most of the stuff I at least tended to associate with humans and higher animals is really nothing new. Sex was not invented with the monkeys. Also air is a pretty inhospitable environment, as the book puts it, every animal not living in water is constantly living at the risk of desiccation.

One of the things I've learned is that the world of cells is an extremely complicated world in its own. A big animal, like a human, is really a big blob of cooperating cells, each cell being a living being in its own right, although of course dependent on the others. The cells in your foot are dependent on the cells in your mouth to swallow food and the cells in your intestine for taking in nutrients from the soup that passes through. And vice versa, no feet means no catching food, at least in nature.

It's a purpose of life, feeding your cells, and a good one I believe. But each cell is actually a pretty complex thing, with small bacteria-like sub-components called organelles like the mithocondria that take in oxygen and various other stuff and output cell fuel, ATP, not to mention the complex chemical machinery that replicates and executes the DNA code. Cells have a life of their own. For instance some cells in the body feed on the bacteria and other stuff that enters the tissue by engulfing and digesting it; that's an important aspect of the immune system.

Structure of a typical animal cell (from Wikipedia)

Likewise, the interaction between the cells is highly developed, not the least keeping in mind that the chemical cell machinery must at all stages be able to self-repair. It is also grown, unlike a human-made machine in the macro world assembled from ready-made pieces, all animals and all their parts must be made from tiny single cells that collect nutrients and out of them build bigger structures.

And everything is controlled by the laws of physics and long self-replicating double-helix strands of 4-character code that specifies what happens when.

It is simply amazing.