Wednesday, October 29, 2014

Confident idiots

This article about the problem that ignorance makes people more confident and how that affects society was really, really interesting - e.g. is it safer to educate people on how to drive on icy roads or not? It even has some advice on how to not fall into the ignorance trap.

Sunday, May 11, 2014

When three becomes four

It's now a little over three months ago that Janne gave birth to Nor.

Nor, Storm and I a few hours after the birth

He was born almost on time, and slightly disappointingly, he wasn't quite as done as Storm - more wrinkles and odd baby skin.

The birth itself went fast. Despite being warned last time by the midwifes, Janne didn't think the birth was actually starting until quite late, so in the end an ambulance came to pick up us and went through the morning traffic with blinking lights and sirens on. A young female apprentice ambulance driver sailed us through the red traffic lights as if nothing was on.

After delivering us at the hospital, everything went pretty smooth and calm, though. Janne is good a giving birth, and in many ways the perfect mother too. I don't think most people think of that when they pick their partner. But I guess some of us are lucky.

I managed to ask the assistant midwife for some help so didn't faint this time either. I swear they turn up the heating every time we're there - this time a technician even came to try to fix it afterwards.

Nor is turning out well and is day-by-day developing the muscles and skills that make us those human beings we are. Keeping the head up and smiling.

Friday, October 25, 2013

The disproportionality factor - why do people get angry?

It has recently occurred to me that many of the conflicts and negative feelings I see from time to time at work and elsewhere are caused by two parties having very different levels of interest in something they are cooperating on. For one party, the party that wants something from the other party, the effort is extremely important, for the other party, the delivering party, it doesn't really matter.

For instance, think of a situation where you're going to a foreign country on the other side of the world. You have done countless preparations, booked the flights, scheduled everything around it at work, and now just need to obtain the visa from the embassy - which is late.

So you send reminders, press for it to go through, and finally get through by phone in the one service hour per day to a person who kindly explains that yes, he has your papers, and yes they look fine, but unfortunately he will not be able to process them because he's going to lunch and after that he has two weeks off so the case will have to wait on his desk until well into your travels.

I think most people would get pretty angry in that situation.

For better or worse, we have to rely on other people which means they sometimes have power over us. I am not talking about formal power relations, I am talking about the fact that when someone is doing something for you, you have effectively lost control over what happens. A manager who assigns a task to one of this workers is dependent on the worker to do his job. Otherwise everyone, not just the worker but also the manager although he's formally the one in power, gets in trouble.

And when you have lost control over something that is very important to you, and the other party has very little if any inherent interest in the matter, chances are something will go wrong.

Oops, you need to walk here? (by Gabriel)
I think of this as the disproportionality factor.

More examples: the bus is a bit late. For the driver, it's only 3 minutes, but for you it's the difference between being able to catch the train or having to wait two hours for the next one. Are you cursing at the transportation company? Yes, you are.

Or in reverse: you spend 10 minutes extra at work with the phone turned off to finish up an important project, get home in high spirits, and find an angry and hurt husband or wife who wasn't able to go to a concert because you were supposed to get home half an hour early to take care of the children. Which you had of course forgotten - because you weren't the one buying a ticket three months in advance in anticipation.

In a similar way at work: when we call the hosting company because our server is down, it's critical for us. But for the operator in the other end, it's just pressing a button to restart one server among thousands.

It happens with minute details too: why are people not aligning their vehicles properly at the parking lot - now I can't park properly! Why are people not formatting their code properly when they know I hate camelNotation! Why has this error in the software I'm using not yet been fixed when it annoys me everyday!

I think there are two things that are important in dealing with the disproportionality factor.

The first is that we need to understand what's important to other people. This goes both ways - you need to think about what's important for people you are dealing with, and you need to communicate effectively what you think is important.

The second is that we need to remember that on average, we can't expect people to really care that much about anything that isn't important to them. That's just human nature.

So we need to help them, not just by explaining what we think is important but also by helping them out, instead of just crossing our fingers and hoping for the best, and then getting ticked off when we are inevitably disappointed.

And sometimes, when in the land of the minute details, we perhaps need to help ourselves to just get some perspective.

Friday, June 29, 2012

Exim with a whitelist of recipients

At IOLA, we have a couple of servers that we use for a lot of different things - just not running live code.

Now, with a standard install of Linux with something like monit installed to respawn dead processes, whenever there's a problem a local mail server is used to send an email about it. That's a good thing because it means you never hear from the server unless it's got a problem, so you don't have to monitor it at such, except for reachability.

If you're on Debian, you'll get Exim installed as the default mail server, together with a semi-scary, semi-friendly series of debconf questions to configure it. Unless you configure one of the questions, it's locked down so it won't accept connections from the outside.

So if you remember to configure your real email address as an alias for the local users (e.g. with a root: in /etc/aliases/) things actually work quite well.

The only problem is when you actually run a web site project on the server, either for development or as a test instance before putting it live. It's not uncommon for a web site to have some corners where it's sending email. Which you definitely don't want to be able to happen by accident during testing.

So in the past, we've configured Exim to not deliver any mail remotely. No accidents possible. But this means we never got any problem reports from the server either.

What we really want is to be able to whitelist a number of adresses that Exim can freely deliver email to, the rest should be discarded.

One way to do that on Debian is to select the split configuration, then put these lines in the file /etc/exim4/conf.d/rewrite/01_rewrite-everything-but-whitelisted_LOCALCHANGE: * T * T
*@* thisaddressdoesnotexist@unrouteabledomain T
This means: skip all rewriting of a To envelope address of and but rewrite all other To envelope addresses to thisaddressdoesnotexist@unrouteabledomain.

The latter is of course not routeable so will fail and end up in the queue until Exim gives up. This is a bit ugly, so you can add the following lines to a new router in /etc/exim4/conf.d/router/01_drop-unrouteable_LOCALCHANGE:
  driver = redirect
  domains = unrouteabledomain
  data = :blackhole:
This will match the domain unrouteabledomain and discard everything sent to that right away.

If you're not on Debian, the rewrite rule needs to go in the begin rewrite section of exim.conf and the router rule in the begin routers section.

I'm quite happy with this setup, it's actually a problem that has been bugging me for years. At some point I made a hacky Python script called from cron that would check the local mail and forward it through SSH to another machine with a mail server allowed to send remote email, but that turned out to be a error-prone and high-latency idea.

Thursday, May 10, 2012

Arab spring

National radio in Denmark has been broadcasting several documentaries on the recent change of regimes in the arabic countries.

Last week it was about Yemen with The Reluctant Revolutionary. One smooth tourist guide, one half-exploitative journalist and a sit-down revolution camping in the streets.

Yemen demonstration - by FreedomHouse2

It is amazing to watch.

There's a lot of talk about freedom in libertarian ideology. But here we see how it's like to be beaten by hired thugs or even just the police for requesting a change of regime. The martyr syndrome unfolds - each beating makes people more angry, each casualty is a proof of how callous the rulers are. It is brutality, weapons and bullets against the sheer force of the people.

I'm happy I'm donating to Amnesty, although they were surprisingly quiet during the whole thing. Maybe revolutions just aren't their thing.

Wednesday, March 28, 2012

Monday, March 19, 2012

When automated tests make sense, and when they don't

Some years ago, I stumbled upon an article about automated testing, "When should a test be automated?" (I'm linking to Google since my previous link is now broken). I found it quite interesting because it, unaffected by all the hype surrounding automated tests, provided some rational guidelines for when to write automated tests and when not to.

A main principle was that an automated test is a waste of time if it doesn't find a real bug upon being run again.

This is a good thing to keep in mind, but since we generally don't know where the bugs are or will be in the future, it's not a very practical rule. So on and off, while working on various web applications, I've been thinking about this.

Why automated tests are good
Praises of automated tests are not hard to find. But briefly: the main practical advantage is that if you change the software, you can easily rerun the tests to see if you have broken something, compared to having to retest everything manually (or just crossing your fingers). If you test the same things over and over again, having it automated can be a real time saver.

There is also a psychological advantage; writing and running the tests make you feel better. Plus you're in, which is always a good thing if you're a little low on the hype factor.

Why automated tests are bad
However, there are also some downsides.

First of all, automating tests takes time.

And unfortunately, it's usually a pretty boring task. It doesn't help that you often have to test things manually too while experimenting or as a sanity check, so you pay the cost of testing twice.

And the tests, while radiating nice comforting feelings, also have a tendency to give a false sense of security. The tests are running, so it works, right?

So you always have to keep in mind that automated tests only test part of what makes the code correct. You could argue that this is a sign of a too lazy test writer, I think that's the main driver in test-driven development, but really, you can't foresee everything (pesky user clicks the wrong button), and if you try to, you end up with so detailed tests that you can't change the program without having to the rewrite the tests completely. And then they have lost their main automated advantage.

Besides, an automated test only confirms what you already suspect, it doesn't tell you if the software is creating real value for its users. So you still need human testing.

Furthermore, even if you don't test everything, in my experience you still end up with a large amount of test code, as much or more as the actual program. There's a hidden cost in this; many changes to the program will also require adapting the tests. In other words, code maintenance take longer. People don't think about this, but imagine you show me 100 lines of your code, and I told you I could reduce it to 40 lines with no ill effects on readability or performance? Would you take that? What if those 60 lines I take away are your tests?

The trade-off
With the understanding that automated tests have good and bad sides, if you value your time, you should evaluate the situation on a scale from should-automate to should-not-waste-time-automating.

Circumstances pro automation
Hard to setup tests. If you have to go through ten steps to be able to test a small change, testing quickly becomes a tedious chore. Usual issues in web applications are getting test data prepared, or being able to intercept email sent by the system, or testing error scenarios.

Important corner cases. Testing manually that a new button does the right thing is not tedious. But it is tedious if you have to try 20 different combinations of inputs to see they all work.

Team of disparate people touching the code. Most code have some implicit assumptions that cause bugs if violated. When multiple persons are modifying it, there's a greater chance of bugs, and smaller chance that each remembers all the corner cases that need to work. Note that you don't necessarily need more than one person on the development team to fall into this category. Add a gap of a year or two in the development, and most people will have happily forgotten the pecularities of the project in the meantime.

Complex code. If the code is complex, it's likely to be needing bug fixes that don't change the expected output, which is good because then we don't have to change the tests, and it's also likely that it's harder to see what the expected output is supposed to be, which can be conveniently solved by writing it down in a test.

The code is seeing changing input data. Modifications to complex code is one source of bugs, but another is changes in the input data. You test it today, and it works fine, but tomorrow your data source decides life is too quiet and changes something that violates your assumptions about the input. Again, this requires bug fixes that usually don't change the expected output.

The code is an API with no UI. If there's no UI, you need to write code to test it, and then you might as well automate it anyway. With Python or any other language with a REPL, this is not entirely true, though. I often test small snippets I'm not yet confident are bug-free in the interpreter.

Circumstances tipping the scales toward manual tests
Simple, easy-to-follow code. If it's simple, there are fewer opportunities for bugs and thus a greater chance automation will be a waste of time. For instance, code using templates to spit out HTML is in most cases trivial, a div here, a heading there. You could add a test to find out whether the div and the heading are there, but it's trivial to see that by mere inspection, and if you've checked the way they look in the browser, there's little chance they're magically going to disappear later.

Localized changes. If changes only affect things in their near vicinity and have no impact on the rest of the software, it's easy to test them manually. For example, consider a web application made up of several independent pages. When you're working on one page, you can just reload it to check it's fine and ignore the rest of the program.

Manual tests are simple to set up. If all it takes to run the test is a reload in the browser, the relative cost of automation is high.

Hard-to-quantify issues are important. If look and feel are important, there's currently no substitute for a human. Imagine setting up a test for an animation - how do you test should look good?

The functionality sees lots of changes and experiments. If the functionality of the software keeps changing, maintaining a set of automated tests is a burden and is more likely to turn you into the grumpy nay-saying change-resistant bastard it's important not to be.

The software sees no changes. If nothing changes, there's little opportunity for bugs and little opportunity for rerunning a set of automated tests. This situation sounds strange, but actually in my experience there's a certain polarity in software maintenance; some things change all the time, others are truly write-once. Of course, this observation doesn't help a lot when you're starting on a project and don't know where it's going to end.

Only one person. Again, this is an important special case. If there's only one person working on a given piece of software (or perhaps module), that person is likely to know exactly what needs to be tested, diminishing the value of a broad test suite.

Minor errors are not critical. This sounds sloppy, but in reality with many web sites, it creates more value for the users and customers if you focus on having the major paths hit the target than being 100% correct because errors outside the major paths can be fixed quickly when discovered.

Of course, most of the above points aren't independent. For example, say you're writing a little script to convert your music collection into a standard format. In that case, you're the only developer, you're going to experiment and change the script until you're happy, an error is not critical since you're around to supervise the process, and you don't expect the script to run more than once when it's finished. Would you write automated tests for that script?

Unit testing versus acceptance testing
I have an extra remark regarding the level at which the tests are performed. There's been a lot of talk about unit testing, in fact most automated test frameworks are called unit testing frameworks, but the reality is that unit tests represent little any end-user value in themselves because they are just providing evidence that some component that nobody will ever run by itself is working if it is being run by itself in the lab.

A more appropriate level for end users is acceptance test where you are testing requirements to the system itself. For instance, the site should enable authorized users to add new products which should then show on this page or when a customer pays for some goods, an order should show up in the system. These are the kind of things that when failing will be sure to cause trouble for the users and thus you.

Of course, unit tests can add value indirectly by highlighting bugs early in the process where they are easier and thus cheaper to debug.

So where are we?
In my experience, when it comes to web development, there are some instances where unit tests make sense for a complicated component, and also some examples of vital processes where acceptance tests make sense because testing those over and over gets annoying. One example could be a sign up feature. Nobody tests that by themselves, because you only ever sign up once.

But otherwise a lot of web development fit the above criteria against automation quite well. Hence, contrary to what the hype says, I do believe automation in many cases would be a waste of effort.

Note that this is not an argument against testing. It's an argument against automating the testing.

It's also not a universal observation. For instance, if you're writing library code all day long, your world is likely looking different. Or if you're Google or another big web site, then the operating conditions are different - as a part of a roll out it's important to have easy reassurance everything is behaving as intended because even small mistakes can be painful if you have a million users. But most web sites don't need to scale.

Also, don't forget that the purpose of testing, whether it being manually or automated, is to find and fix bugs. Thus preventing the bugs from happening in the first place is perhaps the best place to focus your attention. Usually the key is easy-to-understand code with few implicit assumptions and attention to detail, making sure corner cases are handled well.