Are smart coders the enemy?

JavaWorld readers are keenly interested in extreme programming, and that interest has been taken up in our forums. Recently, three members of JavaWorld’s Programming Theory & Practice discussion have voiced some doubts about the viability of this methodology.

Are smart coders the enemy?

There is an interesting discussion going on in topcoder.com’s discussion groups about what qualities define a good coder. The theme of this site is that coders who can solve complex problems quickly are inherently better and deserve more points. I believe this philosophy is fundamentally flawed, and I would like to launch a discussion here about XP and what it means to be a good coder.

There is the general notion in the software industry that the ability to absorb large quantities of information and grasp high levels of complexity are the most important traits in a good software coder. Although I think these coders are very effective and useful, I do not believe that the rest of us should idealize and worship them. Coders with a high level of tolerance for complexity may also tend to produce complex code; this also increases the intellectual burden on the rest of the team, who must interface with and manage that same code.

The fundamental bottleneck in software development is our limited ability to grasp complexity. We can solve this limitation in one of two ways: we can hire coders who are smarter at figuring out existing complexity, or we can hire coders who are better at writing less complex code to begin with.

Unfortunately, when software gets out of control, many organizations feel forced to hire coders with a high tolerance for complexity in order to maintain productivity. However, these organizations fail to recognize that good software design reduces this burden significantly. But what constitutes good design? A good design is one that everyone can understand without having to wrack their brains. This aspect is far more important than the traditional goals of reusability and extensibility. If code needs to be rewritten, so what? Typing is not the bottleneck. Our limitation lies with grasping complexity. So, rewriting is fine provided we have a clear understanding of what needs rewriting and how.

Many supposedly sophisticated organizations tend hire an elite group of architects who put in place an infrastructure that embraces good design so that the less-skilled “bricklayers” can fill in the holes. However, I believe that this approach is also flawed. For one thing, there is no clear conceptual difference between design and implementation. Design exists at all levels of code down to the granularity of algorithms. It is important that high-quality design principles underlie all levels of code.

Furthermore, this role distinction lulls us into the naive assumption that we can get away with hiring relatively uneducated and untrained coders to act as bricklayers. This becomes more problematic when organizations hire masses of uneducated/untrained people under the assumption that a few architects can manage all the significant design complexities. Not only do the bricklayers need a high level of skill and training to be effective at a more granular design level, it is highly unlikely that an elite group of architects can have adequate intellectual resources to manage complexity effectively by themselves.

Perhaps a more “distributed” form of intellectual processing is in order. Every coder should be trained and educated in good design, and the complexity load should be distributed more evenly across the development team. I believe that the key to improving the software development process is to focus a lot more attention on simplifying designs and making them accessible and understandable to the team. Less time should be spent worrying about code reuse, extensibility, and even testing. To me, testing is simply a means of identifying a problem that made it through the design process unnoticed. If we emphasize design more, then the burden of testing diminishes and it becomes necessary to unit test only those aspects of design that are not self-evident. And it is the goal of good design to make all aspects of behavior as self-evident as possible.

Many of my ideas come from extreme programming. However, XP does not emphasize design enough and it overemphasizes testing. Yes, testing is important; but good design is even more important. In the same way that unit testing is much cheaper than fixing bugs after production is under way, good design fixes bugs even more cheaply by making them so self-evident that we recognize them and fix them even earlier.

I think XP is a great leap forward. But I believe we have a long way to go in refining XP. Rather than jump on the XP bandwagon in a cult-like way, perhaps we ought to think more critically and try to refine it into something more robust. Another aspect of XP in need of refinement is the pair programming idea. Pair programming is great, but it is a means to an end. It facilitates communication and sharing of design ideas, which is essential to coding activity. Pair programming isn’t absolutely necessary at all times to accomplish this goal. It is most important when programmers aren’t used to working with one another and need to learn how to communicate design ideas effectively in code.

It is certainly conceivable that some developers are effective at writing expressive code that others can understand without the crutch of pair programming. Furthermore, what’s wrong with three or four working together on a particularly hard problem? Pair programming is an excellent suggestion, but the XP literature should focus more on the underlying philosophy and recognize that there are different mechanisms for executing that philosophy.

–josuna

Am I a ‘coder’?

You make some good points here, josuna, thanks. I too do not like some of the ideas floating around in businesspeople’s heads. And I do not like the word coder applied to myself: it sounds like I might end up as the dumb sidekick doing the brainless work for some “architect” or “designer” who deserves all the laurels. If, at my age, I am not able to think of some good software design myself, I’m not a good coder. And that’s that.

Now, I’ve seen fairly experienced people rewriting code from scratch when they were supposed to enhance it. They just did not understand the code of their predecessor who made use of the C language syntax more generously. If that was enough to scare them out of their wits, they might have use for some extra lessons on C. On the other hand, writing code that uses such simple syntax that it can be more quickly rewritten by others (sort of brute-force refactoring) is not a sign of a smart coder either. And with more complex languages, such as Java and (argh!) C++, the danger for project maintenance increases all the more. There is one thing I detest especially strongly: this businesshead idea that the fastest coder is the best coder. Sure, mangers want to get rich (a) with as little effort as they can, and (b) in the shortest possible time. And when businesspeople want money, they want it now! All that an average merchant can measure is the time to complete the first version of the software. Even IT professionals have problems measuring the extra time that is spent later, fixing all the stupid design bugs that were supposed to speed up development. And who is making the fixes anyway? Not the person who produced the mess: others get blamed for his or her lack of foresight. Is there any software metric that measures how much of a problem occurring later is attributable to the people that laid the foundation? How satisfying is it for coders to do mainly such cleanup work for others who left the project while it still looked like a big success? The cleanup was absolutely necessary, but does not look that good on their résumés … Probably the smart coder is not so much the enemy; the fast coder is. Is it smart to spend 3 minutes coding a single line? Psychologically, no — they won’t give you medals for that. Design-wise, it might be smart to spend 10 seconds on typing the line and 2 minutes, 50 seconds suppressing all those lines that the fast coder would have mindlessly jammed out during this time to the dismay of a good coder and architect. If XP gets accepted by the managers, maybe I can earn a medal some day by writing many lines fast, and then clean it up in a “thorough refactoring session,” eliminating most of the stuff I wrote before …

–martinval

Smart coders are your friends!

I tend to believe that code is inherently complex and I think one of XP’s main points is that code is also inherently volatile. Of all the things we have control over in our design, I believe minimizing these hurdles provides the greatest rewards in terms of overall cost for the life of an application. Therefore, I think the trick of good design is making that complexity and volatility manageable: conceptually through our design and concretely through our code. In other words, it should be easy to grasp and it should be easy to modify and extend. Manipulating such inherent complexity and volatility to this state of simplicity and robustness requires a person to be rather “smart” and talented. Furthermore, once such an elegant system is created, you must have the intelligence to understand how the structure is managing the complexity and volatility of the software so that you

don’t mess it up.

I find education, experience, and personality are traits that are just as essential to being a good programmer; however you still must have intelligence to put all the elements together. Having said all this, I’m beginning to believe this is not realistic. Finding such people is pretty rare and expensive, considering how much work there is. I don’t have a solution. Steve McConnell addresses this issue in his book After the Gold Rush. He supports a more formalized certification path akin to other engineering fields. However, the bottom line is that there are not enough good programmers for today’s demand.

Pair programming sounds nice, but I found it tough to realize. Again, you need the right type of people and culture.

The biggest problem I’ve found with XP (and other methodologies) is that you need to be somewhat fanatical and religious about XP to make it work. The impetus must come not from a few individuals, but from everyone — including management. I believe this is acknowledged in the book, and I think they illustrate how to do this in the follow-up books, but I can’t get my manager to read two pages of any text on methodology, let alone an entire set of books. Is there any hope? Am I too pessimistic?

–plee

If you’d like to add your thoughts, head for JavaWorld’s Programming Theory & Practice discussion.

Source: www.infoworld.com