Thinking Inside a Bigger Box

I read an extremely intesting book last week. Cliff Stoll’s “The Cuckoo’s Egg” is a true story about how the author was tracking a hacker in the mid-eighties. It reads like a spy novel, but is appearently all true. I picked the book up at 11 at night, and was unable to put it down until I had completed the whole thing!

The book gives a pretty good understanding of computer crime, crimefighting, and the basic methods of the typical script kiddie. The exploits that the bad guy uses are all real vulnerabilities, but happily, they are all (mostly) corrected today.

Despite its technical detail and accuracy, the book should be accessible to the general public. I highly recommend it!

I bought User Stories Applied to get help with practical problems with writing good user stories and requirements in general, but it ended up changing the way I think about requirements and tracking them.

The book first fullfills one very important mission. It answers “what is a good user story” with a mnemonic rule: Independent, Negotiable, Valuable, Estimatable, Small, and Testable (INVEST). Cohn refers to William Wake as the source of this mnemonic, but expands further upon how to achieve it. For me, the most useful parts of the book was the explainations of some of the more difficult points of extreme programming’s Planning Game, such as how to split up large stories, how to estimate the effort, where to get the stories from (Cohn’s suggests using the technique User Role Modeling), and how to deal with a lack of a “real customer.” Towards the end, Cohn compares User Stories to Use Cases. For me, the discussion revealed some of the weak points of Use Cases. Based on User Stories Applied, I feel confident that the Planning Game from extreme programming addresses requirement management sufficiently for very many projects.

Reading the book made me realise the different faces of requirements and where user stories fit in. There are requirement types for which user stories may not be appropriate, but they will probably still help. More on this in later.

If you are managing requirements, especially in an agile context, this book is a must-read.

Yes!

The first thing that strikes me about Eric Evans “Domain-Driven Design: Tackling Complexity in the Heart of Software” is how it seems to bring together the ideas that have resounded the best with me during the last few years. The central thesis of the book is that that enterprise software should to be built around the model that (non-software) experts in the field has of the problem domain. The shared understanding between the software developers and the domain experts and users facilitates communication. And also: In many domains, there are advanced models that have stood the test of time (for example double-entry bookkeeping). Reinventing such models is not the best thing software developers could spend their time doing.

The model driven design effort is based around the shared discovery of a ubiquitous language that is shared by the development team and the customers.

The part of the book that was most useful to me was Part III: Refactoring Toward Deeper Insight. Unique in the books I have read about modeling, Evans claims that a model that does not evolve is dead (or at least fossilized), and is a sign of failure for the project to be useful. This is in congruence with Robert Glass’s observation in “Facts and Fallacies of Software Engineering”: The most valuable programs are changed, as the users discover new needs. Evans describes how to keep a design supple by strategies that build on and extend the recommendations of Extreme Programming.

I only have two practical problems with the Domain-Driven Design approach.

• The software development projects I have been involved with lately has used technology that constains the design. The technologies are also so technically challenging that the development team don’t have the time to focus on the domain model.


• No software project is an island. Practicing domain driven design is hard in the face of legacy systems that the team must maintain or integrate with.

Evans addresses both of these topics, but I feel only the last is answered to my satisfaction.

The question of domain-driven design with the current technological trends is an interesting one. I feel a lot of vendors (but not all) are trying to sell technology that constrains the design further. The worst example IMHO is Enterprise JavaBeans. This is an example of a framework that is invasive, inflexible, and constraining. However, from the open-source scene, the current trend seems to be more in congruence with domain-driven design. IoC/Dependency Injection, AOP, and Naked-Objects are my favorite examples of paradigms where the goal of the frameworks is to get the technology out of the hair of the development team. “Domain-Driven Development” spells out the argument for such tools, even though Evans does not address this aspect directly.

I think this is the only failing of “Domain-Driven Development” as a book. Evans explains in detail a host of useful techniques for working in a domain-driven world. I feel the ideas are important enough to warrant more of a manifesto-like book. The book is great for implementing Domain-Driven Design once you have sold management and the team on the idea. There is a need for something something shorter and more persuasive as an introduction to the subject. Subjects like technology interaction with the domain-driven design should be explored further.

All in all, this book is not for everyone. I feel domain-driven development as a focus for developing software in the future is an essensial paradigm. However, Evans goes into too much details on the techniques, and the book ends up being a heavy read for anyone who just want to get an idea of what domain-driven development is about.

I was recommended “The Art of UNIX Programming” by Eric Raymond (esr) from Joel Spolsky’s “Joel-On-Software” blog. The recommendation in it self warrants some comments, as Joel Spolsky normally is very much a Windows-kind of guy. Despite the title, the book definately has much to offer people who never touch UNIX as well.

esr focuses on the cultural aspects as much as the technical aspects. For organisations that are migrating into Unix infrastructure the ramifications of the Unix culture can be hard to grasp. In my experience, no other technical platform has as strong of a culture. esr does an excellent job of explaining the Unix mindset and how it influences the software on the platform. The heavy use of case studies from big open-source Unix projects helped me understand the factors very well. I was amused by the examples esr choose in the case-study of the cookie-jar format. I generally disagree with the political views of esr, but he is considerate enough to only preach Unix-philosophy in this book. (esr’s political writings define as extreme left-wing what in Norway would be considered the most right-wing parties. Oh well)

What I loved most about this book was probably its treatment of open-source. The subject of open-source is of course not specific to the Unix world. esr covers as diverse subjects as the economic justifications, the schism between the Free Software Foundation and the Open Source Initiative, practical management of open-source projects, and legal implications of open-source licenses. If you are considering using open-source in your organisation (and you really should be, if you are not already), you need to read the Licensing Issues of chapter 16. It is available online. Go ahead, read it now!

This section is directed to commercial developers considering incorporating software that falls under one of these standard licenses into closed-source products.

Having gone through all this legal verbiage, the expected thing for us to do at this point is to utter a somber disclaimer to the effect that we are not lawyers, and that if you have any doubts about the legality of something you want to do with open-source software, you should immediately consult a lawyer.

With all due respect to the legal profession, this would be fearful nonsense. The language of these licenses is as clear as legalese gets — they were written to be clear — and should not be at all hard to understand if you read it carefully. The lawyers and courts are actually more confused than you are. The law of software rights is murky, and case law on open-source licenses is (as of mid-2003) nonexistent; no one has ever been sued under them.

Amen.

Three things I took with me from this book:

• 
  
  A lot of program design heuristics and ideas


• 
  
  Better understanding of the open-source model


• 
  
  An appreciation for the legacy and future of Unix

If a beginning programmer was to read just one book, this would definately rank high on a list of candidates. (But then again, why should a beginning programmer only read one book)

“Agile Software Development: Practices, Principles, and Patterns” is in many ways Robert C. Martin’s magnum opus. After having read much of his papers on Dependency Inversion Principe, the Open-Closed Principle and other object-oriented methods, as well as Extreme Programming, Agile Software Development puts it all together. More than any other book I have read, possibly including Beck’s excellent “Test-Driven Development”, Agile Software Development makes a very strong case for writing the test before the code. Most of the examples in the book, even when Martin does not talk about testing, start out by writing a test to explain a concept. The book goes a long way to normalize the sensibilities of Extreme Programming.

As the full title implies, Agile Software Development spans a wide range of subjects. The book starts off explaining Agile software development and Extreme Programming in some depth before covering principles of program design. While using a lot of examples and good case studies to clearify his points, Martin then goes on to explain, each in it’s own natural time, 14 of the most crucial design patterns from Gang of Four, as well as a few from other sources. Agile Software Development does not study each pattern in as much depth as the gang of four, but the author provides a good trade-off between giving the motivation for each pattern, explaining the mechanics of the pattern and putting it into context.

All in all, the book tours a great number of subjects that are critical to the design and implementation of software projects. I would be much relieved about the competency of any colleague if I knew this book was on the list of books he or she had read. More than explaining the basics, Martin argues for why and when they are important and just as importantly: when they should be ignored.

But the Satire of Two Companies in the appendix is just lame. ;-)

Update: Cleaned up mess made by “WYSIWYG” tool)

Rating: Must-buy

The agile movement has started to gain speed and become more mainstream, and the Poppendiecks’s “Lean Software Development” added an important part of the puzzle for me.

Like so many manifestos before it, Lean Software Development compares software development to other industries (lean thinking takes its roots in the Toyota manifacturing system). However, the authors reach a very different set of conclusions. The crux is to not compare the practices that were useful in other industries, but the principles underlying these practices. The ideas developed in the book are very much like the agile manifesto.

Lean Thinking uses the following principles (each described in a separate chapter in the book):

1. Eliminate waste
2. Amplify learning
3. Decide as late as possible
4. Deliver as fast as possible
5. Empower the team
6. Build integrity in
7. See the whole

As you might notice, the list is not specific to software development. Each chapter is rich in great examples, both from the software industry and other industries of how to do and not do lean thinking. One of the greatest parts of the book is the table comparing the wastes in manifacturing to the wastes in software development:

As the table shows, “Lean Software Development” very much supports the position of Agile software methods, but adds a new facet to the discussion: Why does agile work, and where can we see similar ideas bringing success in the non-software world. In conclusion: Why agile makes business sense.

If you are addressing software from a business point of view, or if you are trying to convince people why Agile development is a good idea from a business point of view, you need this book.

Beyond Software Architecture (Luke Hohmann) is an invaluable read for any aspiring project manager or program manager. There is so much more to getting successful programs out the door than just the classical analysis, design, development, test.

The most important feature of this book is how it helps frame software development in a larger picture. The point of software development is to create value for someone. It is important not to lose sight of that fact. There is more to value than just features, and Hohmann gives a good overview. My only problem with the book is that I found it to be a little encyclopedic at times. This does not detract from the value, but it makes it a little harder to read.

My favorite part of the book is the introduction of the concepts of marchitecture and tarchitecture, and Hohmann’s descriptions of the very important marchitectural considerations that must be present for a development effort to be successful.

I am sure there is much more to this subject than is covered by the book. I am also sure there is much more to know about each subject. For someone like me, who likes to get a full understanding of the activity that is software development, it provides a good introduction to an important part.

I recommend this book if you are currently or in the future planning on being project manager, program manager or (t)architect on a software project. It is a fairly quick read, and covers a lot of ground.

Infectious Greed: How Deceit and Risk Corrupted the Financial Markets
by Frank Partnoy is a fascinating book. Partnoy describes how the ever raising performance-related bonuses for brokerage bankers in the 1980s lead the brokers to create a succession of schemes to inflate bonuses.

The early waves of this phenomenon was in the dervatives business. Derivatives are financial instruments the payoff of which is determined by other factors. The stated purpose of a derivative is to sell the risk related to something to someone who is better prepared to handle it.

Dervatives can be linked to everything like from the price of a stock (options), the future price of a commodity (futures), the relative exchange rates of two currencies, or interest rates. Derivatives can even be linked to things like the weather. Now, weather derivatives may sound like nothing more than gambling, but in fact, industries like hydro power plants, farming, and travel depend critically upon the weather.

The real problem comes from the fact that when you sell risk, it is very hard to find someone who is really in a better position to handle it. As all derivatives are null-sum games, this means that it is in practice a gamble that big financial insitutions are allowed to make with the money they hold. Examples include mutual fonds, pension fonds, insurance fonds, and municipal authorities.

Around 1990, derivatives were made more and more complex. Buyers of derivatives were making money, but no one could predict how changes in for example the federal interest rate would affect people. Around 1992, the tables turned, and many investors learned the hard lesson of gambling: The house always wins. The house was the brokerage banks. The losers included for example Orange County, California, which went bankrupt from its investments in derivatives linked to the federal interest rate.

So, in the mid-90s, investors were scared off of derivates, and the brokers needed a new scheme to get their multi-million bonuses. What had made derivatives good was this: It was extremely difficult to assess their risk and value. Investors considered the only safe bets to be in stocks. So, the brokers wanted stocks whose value and risk was very hard to assess. Enter .com.

Partnoy provides a convincing explaination of the .com-phenomenon, which is very sobering to technologists: No-one never believed we could produce the values we thought they believed in. No-one never believed in us. We were just pieces in the game to deceive investors with financial instruments whose risk and value were hard to assess.

The book is depressing, sobering, and upsetting. The attitudes Partnoy describes are disturbing and sickening. The book is very well written, and reads easily. I guess the moral of the story is: There is no such thing as a free lunch. You always pay in the end.

Ullman’s book describes the lives of two people related to a large software development project in the early 80s. Ethan Levin is the programmer who is judged responsible for the bug. As it proves to be impossible to reproduce reliably his life seems to spiral down into dispair, loneliness, and depression.

Ullman is a master at describing the almost hypnotizing urge to “just fix this last problem before” when programming. The dysfunctional team and people in the novel are more dysfunctional than anyone I have ever met, but they are perfect carricatures (I hope!) of a software team gone really, really bad.

“I’m leaving!”, [Ethan] said.

“I mean now. Are you leaving now?”

“I’m leaving, I’m leaving,” he said again, but vacantly, automatically, because despite himself, his eyes had been drawn back to the screen, to the irregular pulse of the messages as they appeared: Warning. Warning.

“Hello? Are you there?”

“Yeah, yeah. I’m here,” he said, just as the compiler suddenly displayed the message “Fatal error.: MAXWINSIZE not defined,” and came to a stop.

“Shit!” Ethan Levin muttered under his breath.

“Ethan! You’re compiling! I know it!”

I really enjoyed this book. In fact, I often burst out laughing audibly as Ullman describes some of the absurdities of programming I know so well. The technology in the book is realistic and recognizable, and the bug, in the end, turns out to be something that very well could have evaded a team after a year of searching.

If you are involved in software development, you will enjoy recognizing the themes and character, at the same time as you will thank god they are carricatures.

If you are not involved in software development, this book may bring you some understanding of the addictive captivation of the computer and the source of the weirdness that seems to be so much a part of your programming friends’ personality.

A is for Apple You may have already known that A is for Apple. But did you know that O is for O’Reilly? Check out what Google reports for single character queries. (From Ward Cunningham’s Weblog)