Thinking Inside a Bigger Box

At the last meeting in Oslo Lean Meetup Geoff Watts talked about BTs transition to agility. The most memorable part to me was when BT transformed a huge, waterfall type project with a delivery schedule measured in years into an agile project. The project set out to convert all BT customers to a new network with a brand new set of services. The new objective was simpler: Deliver one service to one customer in eight weeks.

The common approach when organizations undertake a huge project is to try and deliver everything in one big bang. That trick never works, and BT was exceptional for realizing it. The eight weeks goal was indeed reached, not with one, but with four customers. This might not sound like a lot, but after several years with nothing to show for it, this is a huge win for any project.

How did BT facilitate the transition to agile? The order from up high was “everyone needs to deliver something every 90 days.” In order to achieve this, all projects start a ninety day delivery cycle with a three day off site meeting with all stakeholders to find out how the project can deliver something of value within the required time. Surprisingly, according to Watts, it almost always works.

An interesting lesson is that a shorter delivery time means that the projects have to focus on doing less at a time. Yet projects report a greater sense of progress. In essence, they slow down to speed up.

BT is a huge organization with lots of cultural legacy. If they can deliver huge infrastructure projects in increments of no more than ninety days, why can’t you?

People always talk about how learning is something that happens in groups. Last week, I got reminded of the point as a task I had previously struggled with alone became trivial in a pair programming episode.

The first time I tried coding “a bowling scoring program” was in 2001. I’ve practices the exercise many times later. In 2004, I tried to write a more challenging version: Write a program that prints the numbers that should be displayed on a score board for a partially played game of bowling. I got bogged down and sidetracked, and didn’t come up with a very good solution at all.

Last week, my company invited Ron Jeffries and Chet Hendrickson to give a short course in pair programming and test-driven development for 15 of our developers/testers. Ron and Chet has coded this particular program a number of times, but never with the score board variation I mentioned.

In the second half of the course, I sat down with two others of our developers who had never worked on this exercise before. We quickly finished the simple game, and then someone proposed the score board. I said “oh, that’s way too hard, we’ll never finish it in time for the course”.

To make a long story short: I was wrong. Together, we solved the problem in no time flat.

Three years ago, I was asked by one of our teams to give advice on how they should write a parser for a structured file format. Just having read up on SAX again, I recommended that they looked into designing it as a push parser. A push parser works by the design that the parser generates events each times it reads parts of the input. These events are sent to an event handler, which then builds up the internal object structure or whatever the program needs. A push parser is a very object-oriented approach to parsing.

About a year ago, I took over the reigns, as it were, on this project. The strategy I had recommended had resulted in a horrible design. After having spent a total of a few weeks to fix issues with it, I finally gave up and rewrote it as a simple procedural parser. The rewrite took me three days plus about a day of fixing some edge case bugs. The resulting code was a tenth the size of the push parser.

This leads to my dilemma: As a senior professional, what should I do when I’m asked to give advice on software design?

Read the rest of this entry »

Warning: This article requires a lot of editing love before it is very useful. It might be somewhat incoherent. Read at your own risk. ;-)

Silo (software): A silo system cannot easily integrate with any other system.

In software, the term “silo” is used to refer to a system that is constructed as one unit from the front end to the data storage. Everything is tied together to work with the rest of the silo, but not with other elements.

This is considered a bad thing.

The problem comes when we want to integrate the system with other systems or reuse parts of the system.

Many of the new ideas in software development has as one of their big goals to try and rectify the silo problem. In general this is achieved by splitting up the system into services that may or may not be distributed across different computers.

But the badness of the silo hinges on two claims: First, that the applications built as an integrated stack cannot be integrated with, and second, that a system of distributed services can be integrated with.

Read the rest of this entry »

This article is a Norwegian-language version of my article Four bold claims about SOA.

Dette er et utkast til en artikkel jeg ønsker å få publisert. Jeg setter stor pris på tilbakemeldinger om uklare tanker og formuleringer.

To av de vanskeligste problemene vi møter innen programvareutvikling er integrasjon og det som gjerne kalles “business-IT alignment” eller forretningsorientering, altså: Hele organisasjonen arbeider for de samme målene.

Tjenesteorientert arkitektur, eller Service Oriented Architecture (SOA), hevder å kunne løse begge disse problemene. I det siste har jeg forsøkt å lytte mer aktivt til påstander fra evangelister av SOA, og jeg begynner å få en forståelse for hva det er SOA foreslår som løsningen på problemene vi står overfor. Dette er min oppfatning av påstandene om hvordan SOA skal løse problemene med integrasjon og forretningsorientering:

• Web service-standarder vil løse de tekniske integrasjonsproblemene (”WS-*-påstanden”)
• Å sentralisere integrasjon via ett knutepunkt vil løse de organisasjonsmessige utfordringene rundt integrasjon (”ESB-påstanden”)
• Å modellere funksjonalitet som en arbeidsflyt mellom tjenester vil gi oss bedre forretningsorientering (”BPM-påstanden, del 1″)
• Å kunne restrukturere arbeidsflyten mellom tjenestene vil gi oss en en smidig forretningslogikk (”BPM-påstanden, del 2″)

Read the rest of this entry »

Three challenges for agile projects

When I join projects now, I want to challenge all the stakeholders to make three commitments:

• Simulate production at least monthly: The software should be run in an environment that is comparable with the target production environment with loads and data variations similar to that of production. Thus, the technical stability of the project is proved.
• Demo with the business side at least monthly: The results of the project should be showed to the product owner, or perhaps even the end users. Thus, our understanding of the requirements is established.
• Real return before 10 % of business case spent: The project should have an estimated business case. Before 10 % of the potential earnings have been spend, the project should start to generate some income. Thus, the economic expectations of the project can be assessed.

Currently, much project is doing very well on simulated environment and pretty well when it comes to demos. We’re still struggling with delivery within 10 % of the business case.

How are your projects doing on this scale?

As my previous Norwegian language article turned out to be one of the all time top hit articles in my blog, I will continue to write a few articles in Norwegian. This one is on an idea on how to do reseach on the success of agile projects. Next week, I will return to another popular topic: Testing.

Under paneldebatten på Geilo-seminaret til Dataforeningen, satt Magne Jørgensen meg, Lars Arne Skår og Niklas Bjørnerstedt litt til veggs med et ganske enkelt spørsmål: Hvordan kan vi vite om smidige metoder virker? Perspektivet til Magne som forsker på utviklingsprosjekter fikk meg til å tenke på spørsmålet, men før jeg fikk kommet med et forslag til svar. Derfor skriver jeg det her i stedet.

Å forske på et spørsmål som “fungerer smidige metoder” er veldig vanskelig. Mitt håp er at noen i stedet vil ta et skritt tilbake og se om man kan få svart på det mer interessant spørsmålet: Er det en sammenheng mellom hyppig feedback og lønnsomme prosjekter?

Read the rest of this entry »

What is the secret to happiness? Surprisingly, this question can be answered more and more definitively. I want my work to be conductive to the happiness of myself and others, and I believe agile methods can help me do that.

Read the rest of this entry »

Two of the hardest problems of software development are integration and what we could call business-IT alignment: The whole organization working towards the same goal.

SOA claims to address both of these problems. After listening harder than I’ve ever done before to SOA evangelists, I think I understand what mechanisms SOA proposes to solve these problems. I think the idea of SOA is based on a set of rather bold claims:

• Web Services standards will solve the technical integration problems (”the WS-* claim”)
• Centralizing integration will solve the governance issues surrounding integration (”the ESB claim”)
• Modeling use cases as workflows of services will solve the business alignment problem and promotes reuse (”the BPM claim, part I”)
• The flexibility to restructure the workflow between services will enable business agility (”the BPM claim, part II”)

I hope that an SOA evangelist will agree that these are important claims in SOA. Even if he will disagree with my evaluation of them.

Read the rest of this entry »

When I give talks on agile, people often ask the inevitable question: “When is Agile not appropriate?” My response is that I don’t really care what people call what they do. What I am concerned about is the quality and frequency of the feedback that informs the control and decisions regarding project management and technical decisions on my project. I find the idea that you might not want to improve your feedback to be very odd.

But I think I understand what the real question may be now. Feedback comes at a cost. You have to actually release and deploy your system. You have to write and maintain test cases. You have to conduct meetings with the team and the stakeholders.

What cost should you be willing to pay for better information?

I can only speak from my own experience. Currently, we have two-week long iteration. The release, demo, planning and retrospective activities take about one and a half day out of each iteration. So the “feedback cost” is about 15%. I don’t think I would like my project to devote less than 15% of its effort towards implementing a feedback cycle. If you spend days integrating your project every iteration, a one week iteration won’t work for you. I don’t recommend doing fake iterations where the iteration degrades into a planning window, either. So if it takes me several days to do a proper iteration, I would rather have longer iterations.

Here is where I depart from the spirit of the question, though: If it takes you several days to integrate your project, how do you plan to act when you have to fix problems in production? A long feedback cycle is a high risk. If you can’t iterate frequently, this is a sign you need to fix your process, not that you should adopt a waterfall approach.