Tuesday, May 08, 2007

How To Think About The Agile Enterprise

It seems that the technology marketing buzzword factories have been hard at work, because a bit of web research reveals a whole slurry of terms:

Adaptive Enterprise
Agile Enterprise
Business On-Demand
Real-Time Enterprise
Zero-Latency Enterprise

The contrarian in me smells snake oil. It seems that whenever the IT industry gets it's hands on a concept, there ends up being way too many solutions to solve very few problems. When consultants sell methodologies and vendors sell products that claim to provide an agile enterprise, that's a pretty good indicator of the difficulty of the goal. If it's truly that easy, how come more companies aren't agile?

Let's do ourselves a big favor and completely forget about technology, methodologies and processes. Frankly, I think these three things are actually impediments to agility as they are most commonly understood and applied. They are actually all related.

The biggest impediment to agility is inertia. Most enterprises are about as agile as a supertanker. Actually, that's not quite accurate. Ships have freedom of movement, it's just that the movement can be very slow. It's possible to be slow and agile. Most enterprises are as agile as freight trains. That's more accurate. Trains have lots of inertia and they travel along defined paths; there are no steering wheels in locomotives. However, large companies can be steered in different directions, albeit slowly. Most enterprises are as agile as a cow on skates. Not only does it not want to be on skates, once moving, it does not want to change direction for fear of crashing. Aside from the comic value, that's a pretty useful description.

The second is data-centric thinking. For those of you who have never worked in a large organization, making a change to a database or data model is almost impossible. There are legions of people dedicated to protecting the sanctity of the schema and are human-shields to change. We love static, permanent structures. This mindset is a major part of the problem; agile database is an oxymoron. The fact is that data is useless without interpretation. Just look at any database table with several columns of foreign keys to see what I mean; without interpretation (joins), the data in the table is meaningless. The whole goal of normal-form analysis is to maximize the use of joins, thereby rendering raw data even more meaningless. In most cases, agility requires changes in use and interpretation, not the data structure.

The third is our fondness of ceteris paribus models. We like nice, sequential, step-by-step models where only one variable changes at a time. Those models are great for teaching/learning but are useless for operations. It's impossible to run even a lemonade stand with ceteris paribus models. We need to have models that help manage dynamic, multi-variable systems that are often non-sequential.

We build systems using the second and third concepts even though we know they are no longer working well and we consistently apply the first concept in the face of external forces. Going forward, we talk about loose-coupling, re-use and process management, etc. Aren't all of those just more, simple models re-cast into agile roles (and just as likely to fail)? Perhaps, that's all the vendors are selling because that's all they can build?

Instead of loose-coupling (which still implies assembly), perhaps we need to think in terms of mixtures, aggregates and ad-hoc federations?

Instead of processes, maybe we should focus on events and variability management?

Instead of re-use and general purpose, one-size-fits-all components, possibly the focus should be on more multi-purpose components that can be easily adapted within a more limited domain?

Maybe the core problem is that we think of an agile enterprise in terms of what we already have built or know how to build? Maybe the answer is to truly understand what needs to be built and not recapitulate solutions?

5 comments:

Alastair Bathgate said...

Bill
That's a great post but I am going to stick my neck out and stand up for the vendor community.
Vendors can only work with
a) what exists; and
b) what people will buy

There needs to be stepping stones towards your nirvana and unless you are building a brand new enterprise overnight, there are undoubtedly legacy issues to contend with (note I said issues not systems).
Most enterprises need to take one step at a time and loose-coupling is an easily understood way of connecting existing infrastructures in a pseudo-agile way.
It's not perfect but it's better than what exists. It's intuitively achievable and therefore enterprises might buy it.

In the longer term, the problem with mixtures and aggregates is that they ultimately get set into concrete - I much prefer your ad-hoc federations analogy which I think is the long term nirvana, or perhaps you might consider bacteria as a suitable analogy ;-)

Alastair Bathgate said...

Oops - typo in my URL on last comment.

wpbarr said...

Alastair, in the tech business, I still think that to a large degree, vendors create markets. They see a need, come up with a partial solution and then figure out how to sell the product/idea. The sales pitch that generates the most buzz in the trade press, wins. SOAP comes to mind.

Regarding bacteria as an analogy, I have been thinking of distributed systems more in terms of biology and ecosystems than networks and state machines for the past few years.

Richard Nicholson said...

Folks,

Wish such discussions were more common place in the IT world :(

If you are interested in the fundamental characteristics of complex adaptive systems - check out John Holland - "Hidden Order: How Adaptation Builds Complexity" - primarily concerned with designing genetic algorithms - but the fundamentals are of much wider applicability.

There are some additional interesting references at codeCauldron Research

Whilst not exposed to the developer - the ideas of ad-hoc federations and stigmergy underpin the design philosophy behind Newton and our other activities.

Regards

Richard

P.S. Found these entries after finding your comments on Spring/OSGi etc.

wpbarr said...

Richard,

I'm familiar with the Newton project. We were investigating various incarnations of service grids, last year. We evaluated Infiniflow last fall but, we found it didn't quite meet our needs. I see that it has evolved tremendously over the past 6 months or so. I'll look at it again.