Waiting For The Great Leap Sideways…

Having been a developer for 20+ years, I occasionally look back and wonder about the great advances that the industry has made in this time. These, of course, are:-

1) Syntax highlighting
2) Test Driven Development

Ok, maybe, a bit facecious, but sadly, not much. It seems to me that the technology that we use today is not much changed from when I started writing RTL-2 on PDP-11s back in the 80’s and when I try to think of things that have had a real impact on my day-to-day work, these are the two that spring to mind ;^)

In particular, I am constantly surprised by the fact that (for the most part) we still represent code in text files. I understand, that developers have a deep-rooted fear of losing control of their beloved text, but this particular phobia has a number of consequences, in particular in the tools that we use to create code.

Text editors have no notion of the intent (or even the structure) of the code and hence, all they actually do is help you type a bit faster… Whoopee do ;^)

IDE’s try to go a step further (err, sideways) but tend to fall into the same trap of thinking that what we need is to be able to write code faster and hence become little more than glorified text editors. What we actually need is to be able to read and determine the intent of code faster.

As an example take the popular Eclipse IDE (I’m not picking on Eclipse – substitute pretty much any IDE here). When editing code the main focus of the developer’s attention is directed at the “editor area” which shows the textual representation of the code in one or more files. The task of editing is (optionally) supported by an “outline view” that shows the actual structure of the currently selected file (what class(es), attributes, methods etc). In a text-based world-view that makes perfect sense, but what I would like is to flip that around and have multiple “outline views” (as the focus of my work), supported by a single editor view that shows me the textual representation of the currently selected outline. To understand the intent of the code it is the structure and relationships expressed in the code that I need to discover – so why not have the tool help me do it!

As another example, take the ever-present “formatting wars”. Developers like to layout their code in a myriad of ways and all of them are right! The problem is that reading code written in a myriad of ways is neither a) efficient nor b) pleasant, and so many teams introduce coding standards to produce an acceptable team ‘style’ (often after lengthy and tedious debates about where a curly-brace should go, and occasionaly after a code “Tzar” stuffs in a code formatter onto the front of the code repository – mentioning no names, Duncan ;^) When code is parsed an Abstract Syntax Tree (AST) is produced – why not use that as the storage format and allow each developer to specify how the code is textually formatted for them so that they can read it unhindered by somebody elses weird and wonderful layout.

And these issues are just the tip of the very cold thing. Tools for testing, refactoring, metrics etc – could all benefit from a more integrated approach – at least that is my hunch…

Old Punks, Young Crusties

In many technical professions, young practitioners (possibly straight out of university) come into the workplace full of enthusiasm and with the desire to use the latest and greatest developments in the field. Quite often, these ‘young punks’ encounter resistance from the ‘old crusties’ who have been in the game for years and years (and years!) and who stick to their way of doing things simply because they’ve always done it that way.

Recently however, I’ve noticed that in software development the situation seems to be reversed. For example, at one company, a number of the more experienced software engineers (‘old crusties’ like myself) were keen to try out some of the recent developments in the agile development world (pair-programming etc), and in particular, test driven development (TDD). Having tried TDD on a reasonably sized body of code, I was convinced that it was a fantastic way to work and that it improved the quality of, and my confidence in the code, whilst simultaneously reducing the stress involved in producing it. I was also struck by how subtle the TDD mantra of 1) Write a failing test 2) Make it run 3) Remove duplication really is, and how it basically describes the process of development in an eerily accurate way. Anyway, I digress. As it turned out a number of meetings were held to discuss the introduction of TDD, but to my surprise, the younger, less experienced developers (one or two years out of college) were extremely reluctant to even try it out, and at one point the discussion disolved to the point where people were even questioning the idea of unit testing itself (I know, I know, don’t get me started). Now, on occasion I have been known to be a tad cynical myself (and then some), and so I welcome healthy questioning, but I was still shocked to see the next generation basically arguing that we should continue to write code the same way that we always have, with all of its attendant weaknesses and failures.

Thinking about it later, I realised why their position was not so shocking after all. The field of software engineering is all about trying to find better ways to develop code and to help us live with it over a long period of time. There are now many well-documented techniques to help us reduce the defect count, reduce the time taken to fix bugs and add new features, to refactor etc, but if you’ve never worked on a large body of code, and if you’ve never had to live with your code (or, more to the point, other peoples code) for more than say, a semester, then obviously, the techniques to achieve these goals seem superfluous. To put it simply, if you’ve never had the illness, then you won’t have any interest in taking the medicine!

This problem can be attacked on a number of fronts. First, universities need to make sure that their computer science courses include a large software engineering component. Maybe students could be given a project that spans the entire length of their degree, allowing professors to change requirements, add new features and generally throw some real-life spanners in the works. Students could even have to swap bodies of code and documentation etc each semester, and they could be ranked by their peers about how easy (or not) it is to get started working on their code. Secondly, development organisations need to make sure that they have a clear and coherent approach to software engineering, and that every developer joining the organisation is fully exposed to it (for less experienced developers this would probably include close mentoring ). Whether or not you agree with specific techniques such as TDD, you must have a story about *how* you write software that goes beyond giving each of your developers a machine to code on ;^)

They say that the study of history is vital if we want to avoid repeating past mistakes, and the field of software engineering is really just an accumulation of wisdom from the short, but often painful history of software development. It might be an idea to learn from it ;^)

Good Test, Bad Test…

Developers shouldn’t think of writing tests as like writing code – they should think of it as *exactly* the same as writing code. IMHO, the “code” is has 2 parts, the implementation and the tests, and each needs as much care and attention as the other… Anyhoo…

A test remarkably similar to the following cropped up recently:-

def a_test(self):
   ...
   self.assertRaise(SomeError, self.foo(x, y, z).blargle)
   ...

Now, to paraphrase the legendary Mr Clough – “It’s not the worst test I’ve ever seen, but it’s in the top 1” ;^)

How can such a small test be so bad? Well, it manages to pack a couple of critical errors into a single line (and that’s some going I have to say ;^):-

1) It is not clear whether it is the method call or the attribute access (or both) that is expected to raise the exception.

2) The implied API is clunky at best. If the reason for calling the method is to get hold of the “blargle” then just return the “blargle”! The only thing that this test makes clear is that the API is not clear!

If I was a betting man, I would bet that this test was written after the implementation, which might excuse the API weirdness (it could be the start of refactoring a legacy API), but not the lack of clarity…

Do I always need to write tests?

And so it was that the student asked the guru, and the guru said:-

“Of course not. You only need to write tests if you need to refactor your code.”

“Ah, I see”, replied the student, “and will I always need to refactor my code?”

“Of course not. You only need to refactor your code if you need to add features or fix bugs.”