Thinking Inside a Bigger Box

Barcelona

I just returned from The ServerSide JavaSymposium Europe. Great conference, with interesting tracks and good opportunities to get to know people. The conference was in Barcelona, which was interesting, because hardly anyone (taxi drivers and waiters included) understand English here. It’s the first time where I’ve been a place where I am totally unable to communicate verbally with people around me. So it as a bit of an adventure. My only gripe about the location is the price of WiFi access in Spanish hotels.

Themes

In the tracks I attended, there seemed to be two recurring themes: Grid-technology and EJB 3. JPA looks like it is coming along nicely, and after discussing with spec lead Mike Keith some issues I’ve had with JPA, I am fairly convinced we will attempt a migration soon.

The Grid-based talks were Terracotta, JGroups, GigaSpaces and Coherence (which I did not attend). I especially liked Nati Shalom’s view of the tierless architecture. GigaSpaces has Spring-like DAO support, which makes it as natural-feeling to use as Jdbc. This lowers the barrier of entry quite a bit, but it brings forth a problem with spaces-based technologies: The level of abstraction is currently that of a DAO, not that of an ORM. This means: As far as I could see, you will not have a rich domain model with references between your domain objects if you use spaces. You will also have to implement referencial integrity yourself. (I hope to write up a little piece on creating lazy loaded abstractions on top of tuplespaces in the future, this might help address the issue).

A second problem with grid based data is that of the fragmented cluster/split brain. For those who do not know what Split Brain means, consider the following: A large grid usually has a failover that elects a new master if the old master is unavailable. Now, if the network is split in the middle, there will be two nodes that both think they are the master. The split brains will each update the data, and when they reconnect again, the data will have to be merged. Even with databases, this is not a painless operation, but I expect that with a tuplespace, it will potentially be unrecoverable.
Bela Ban was the only speaker I attended who covered fragmentation satisfactory. I wish that he had explored merging in more detail, though. Jonas Bonér seemed quite unprepared for the same question.
I don’t need split brain remerging to be perfect, but I need to be confident that the vendor has understood the issues and that the whole space won’t be corrupted when it gets back up again.

Outstanding sessions

• John Davies: It was very interesting to hear John Davies speak. C24 has some outstanding technology within the banking sector, and I am really looking forward to seeing if my company can make use of this technology
• Kirk Pepperdine: The performance anti-patterns talk was very interactive and fun. Kirk gave out chocolate to attendees, which is always a good way to win me over.
• Heinz Kabutz: I went to both of Heinz’ talks. He is a very entertaining speaker. Sadly, I knew most of what he talked about already.

Ideas of my own

With the help of people I talked to during the conference, a few new ideas have started germinating in my head. I hope to be writing more about this in the future:

• The SOA battle: More voices who are critical to SOA should be heard. I may reengadge myself in this
• Object-based queries: I hope to be working on a portable version of TopLink Expressions/Hibernate Queries. The query language is the last thing preventing persistence technology portability
• Myopic Development: I had good results focus testing the term “myopic development” instead of “agile development”. I will write more on this later
• Lazy loading: Lazy loading, like dynamic proxies, is not hard, but people are afraid of it. I’d like to see custom lazy-loading become more mainstream

Stay tuned for updates.

Roger Sessions has just published “A Better Path to Enterprise Architectures“. His main point is that large, centralized, big-bang enterprise architecture efforts fail. I could not agree more. Sessions gives some good arguments for why you would want to deliver incrementally. He calls this approach SOA.

This is a fairly common way of defining SOA - basically SOA is another name for incremental deliveries. If this is SOA, I don’t hate SOA at all. However, I find that “SOA as incremental deliveries” fails to shine any new light on the subject. To paraphrase Sessions: if you have a $100 million system, you should at least split it up in 10 deliveries. Now, this is neither hard, unusual, or especially helpful. The problem is of course, a $10M project is still very large. How can you split that up further. Finding good services without too high coupling is harder as you move down into single-project scales. And so far, I have seen no description of SOA that sheds any light on this particular issue.

I have discovered that SOA services (at least XML-based protocols in Java) tend to be used as horizontal integration. That is, you insert a SOA layer as the interface to your data access logic. This is by definition going to be a very wide interface, and any changes to the UI, the data services or the database are likely to require a coordinated change through the whole stack. SOA is not necessarily a good idea for horizontal integration. (There is one valid use: Rich clients beyond the firewall). SOA for vertical integration (for example: a webshop that integrates with a payment system) seem to be benefitial fairly frequently. However, I have not seen this as often in the wild as I’ve seen harmful horizontal integration.

I’d like to finish this post with a jab at a small comment in Sessions’ article: “Code Sharing—Many organizations believe that reuse is achieved through code sharing. It is somewhat amazing that this belief persists, despite decades of failure to achieve this result. The best way to reduce the amount of code that a given project needs is through delegation of functionality (as in the case of Web services), not through code sharing.” This is one of the most common myths about SOA. I would like to submit that code sharing has succeeded wildly. Look at reuse of Java open-source projects, especially after Maven entered the picture. Reuse through services, on the other hand, couples your whole service network together, making it hard to change anything without risking breaking something totally unrelated. With horizontal integration being the most common way to use SOA, the information schema is larger, and the coupling is even tighter. Code sharing is not without problems, but SOA does nothing to improve upon these problems.

Blessed synchronizity:

From Jason Yip’s blog:

Vendor’s Law: Organise people into roles corresponding to the tools that you want to sell to them.

And today’s Dilbert:

In my previous post, I talked about how I feel SOA encourages rigid design. Of course, in some situations, you may not really have a choice. When creating Business-to-Business (B2B) integration, interfaces will naturally be much more rigid. There is no way around it, SOA or SOA-not.

Ian Robinson recently published an article on Martin Fowler’s webpage titled Consumer-Driven Contracts: A Service Evolution Pattern. The article gives some very clever ideas of how to make the consumers drive the service definition instead of doing this on the provider side (this is related to Lean Software Development, by the way). The article also goes through some of the challenges of a contract-oriented interface.

As I said, for B2B integration, contracts may be essential. However, for internal integration in an organisation, it seldom is. When I have the choice, I’d rather not have to incur the cost of developing and maintaining. The cost in time spend on contractual interface is relatively simple, but the cost in lost agility is substantial.

On JavaZone 2005, I talked about “Why I hate SOA”. I found it hard then, and I’ve still found it hard for a while to express this sentiment concisely. I think I’ve finally got it!

One of the most common inefficiencies I discover in organizations is poorly designed boundaries. I find that people suffer when a boundary is too ridig, not when it is too loosely defined. Contractual interfaces create a “mine versus yours” mentality, where every problem beyond the boundary is not corrected, but instead wrapped in even more code. Almost all such boundaries lead to wrappers, lots of extra crufty code, duplicated decisions and thus a rigid architecture.

When used for a stable interface, a contract-driven approach may be appropriate. However, when I create an interface contract, it takes me many attempts before I get it right. In our profession, a big problem is our tendency to focus too narrowly on your small, and poorly divided, slice of the problem. SOA-thinking for in-house integration encourages this approach, instead of encouraging cooperation and “thinking inside a bigger box”.