Controlling complexity is the essence of computer programming. Brian Kernighan, 1976
We live in a world where the systems that we use or come across in our day to day lives seem to be ever more complicated. The flight software system that controls a modern aircraft like the new Boeing 787 "Dreamliner" are almost unimaginatively complicated, simply because of their sheer size. According to NASA the software that runs the 787 is almost seven million lines of Ada code, triple that of the 777. The F35 Joint Strike Fighter has 19 million lines of C and C++ code! Does all this mean that these systems are also complex however and if not what's the difference?
Inevitably there are a large number of definitions of exactly what a complex system is however they all seem to agree on a few common things:
They are made up of a collection of components.
The components interact with each other.
These interactions can result in emergent behavior.
Emergent behavior refers to the property that a collection of simple components may exhibit when the interactions between them result in new, and sometimes unpredictable, behavior that none of the components exhibit individually. So whilst complicated systems may be complex (and exhibit emergent properties) it does not follow that complex systems have to be complicated. In fact relatively simple systems may exhibit emergent properties.
In his book Emergence Steven Johnson gives several examples of the emergent properties of all sorts of systems from colonies of ants through to software systems. Whilst some software systems clearly thrive on being complex systems where emergent behavior is a positive benefit (I'm thinking Facebook, World of Warcraft and SecondLife for example) we might be a bit more dubious about getting on an aircraft whose flight software system exhibits emergent behavior! My guess is that most of us would prefer it if that software showed entirely predictable behavior.
Here's the thing however. Should it not be possible to build some of our business systems where emergent behavior could be allowed and for the systems themselves to be self adjusting to take advantage of that behavior? How might we do that and what software delivery lifecycles (SDLC) might we adopt to allow that to happen? The interesting thing about SDLCs of course is that almost by definition they build predictability into the systems that are developed. We want those systems to behave in the way the users want (i.e. we want them to "meet the requirements"), not in unpredictable ways. However the thing about systems like Facebook is that the users are an integral part of that system and they drive its behavior in new and interesting ways. The developers of Facebook are able to observe this and respond to new ways in which their system is being used, adapting it accordingly. Facebook has clearly been architected in a way that allows this to happen (Nice one Zuck). The problem with many of our business systems is that they are too rigid and inflexible and do not allow emergent behavior. Even the promise of Service Oriented Architecture (SOA) where we were supposed to be able to reconfigure our business processes almost at will by combining services together in new and interesting ways has not really delivered on that promise. I think that this is for two reasons:
We have failed to adapt our (SDLC) processes to take into account that property and are instead building the same old rigid systems out of slightly new sets of moveable parts.
We fail to recognise that the best complex systems have people in them as an integral part of their makeup and that it is often the combination of people and technology that drive new ways of using systems and therefore emergent properties.
Building and managing complex systems needs to reognise the fact that the same old processes (SOPs)
may no longer work and what new processes we new develop need to better account for people being an integral part of the system as it is used and evolves. The potential for emergent behavior needs not only to be allowed for but to be positively encouraged in certain types of system.
You can have the greatest idea in the world but if you can't present it effectively, aiming it at the interest level and time your audience has, then it's not going to fly. Here's a three-pronged approach to getting your ideas across I have borrowed from Dan Pink's book Drive: The Surprising Truth About What Motivates Us (if you want a quick, animated summary of the book check this out).
You need to be prepared at all times to explain your idea. The amount of time you have to explain it will depend on a number of factors, not least of which is the amount of 'face-time' your stakeholder will give you. Here are three formats you should have prepared for selling your idea depending on how much time you can get:
The Tweet version: A tweet (as delivered via twitter) can be a maximum of 140 characters. The challenge is can you describe your idea in 140 characters or less. Samuel Johnson (or Mark Twain or Winston Churchill depending on who you believe) said "I did not have time to write you a short letter, so I wrote you a long one instead". This version is the most challenging of all. You really need to be brutal and pare your idea down to just the key facts.
The Cocktail Party version: This is a variant on the elevator pitch. Can you describe you idea in 100 - 150 words or a maximum of one minute of talking (talking fast doesn't count). Again you need to focus on the bare essentials but here you have a bit more leeway to focus on the business benefits.
The Real-Deal version (with supporting takeaway): So you twittered your idea, you met some guy at a cocktail party (or in the elevator) to entice him a bit more and you finally got invited to present your idea. The presentation is the real-deal because this is really your chance to stand up and sell (and hopefully clinch the deal). Don't, therefore, screw-up by preparing an overly busy presentation with slides full of tightly packed text (remember PowerPoint bullets kill interest like real bullets kill people). Suppose you have "an hour" to present. Aim for a presentation that can be done in 30 minutes allowing for 15 minutes of questions and five minutes or so either side for people to be late or have to leave early. No one can retain an interest for more than 50 minutes anyway so 30 is good. For some thoughts on presenting see here. I prefer not to follow rules like "one slide every two minutes". The important thing is to structure the presentation first (probably before opening up your favourite presentation software) then write it, then practice until it fits into 30 minutes. For an interesting alternative view on how big a presentation should be see here (a slide every 12 seconds maybe!). Finally, because you will inevitably have had to leave out some detail prepare a short (two to four pages) takeaway which explains your idea that you can leave behind for your audience to take-away. Make sure you include the tweet as the "management summary". You never know, your stakeholder may tweet it herself giving you a bit more publicity!
...they're the ones with no skin in the game. I came across this whilst reading about Hugh MacLeod's new book on his blog. It set me thinking about how true this can sometimes be from an architecture perspective. Steve Jobs famously said “real artists ship”. Whilst there is a point to building good architectural models, defining principles that should be followed and road maps that describe the directions architects will be travelling in there comes a point that we must say “enough is enough, it's time to ship”. The fact of the matter is that software development is, and always will be, a complicated activity and there is often no clearly defined answer to the wicked problems we need our systems to address. In fact, two of the attributes of a wicked problem namely, wicked problems have no stopping rule (instead the problem solving process ends when you run out of resources rather than when some 'correct' solution emerges) and solutions to wicked problems are not right or wrong (they are simply better, worse, good enough or not good enough) would seem to encourage the kind of purist behaviour where nothing ever ships. The danger with architecture (especially Enterprise Architecture) is that the artefacts being created are intangible things like models, road maps and principles and these by themselves do not make working systems. The people that produce such artefacts could often be accused of having “no skin in the game”. So how should you ensure that the architects in your organisation do indeed have skin in the game and knows that shipping is actually all that matters? Here is my SKIN (Specify Kill Integrate Negate) rule of thumb for ensuring delivery takes protracted (and unnecessary) analysis.
Specify to the right level of abstraction and detail, no more no less. The models, road maps, architectural principles that are defined by architects should be understood and usable by as many stakeholders as possible. This may mean having different views (levels of abstraction) of the same model but ensure if this is the case, those views are consistent. Think who your stakeholder is meant to be before building the artefacts and don't build it if you can't explain it clearly to that stakeholder.
Kill unwanted artefacts. Sometimes with the best will in the world you will build an artefact that does not have a purpose, is not read by anyone or has expired. Kill it, don't let it linger on causing confusion and giving you an unnecessary maintenance overhead. Look at something like TOGAF for a good set of useful artefacts.
Integrate the work of the architect(s) with real downstream activities. Don't let them work in ivory towers which have no grounding in reality. When planning a project and considering what artefacts need to be planned it is useful to think of what the inputs are to the task that is to create the artefact. If you cannot see any of the artefacts that are output from architecture tasks being used than maybe it's time to consider killing off those artefacts (see above).
Negate unnecessary activities and tasks. Every task should have an output. This is usually in the form of a work product (a model, a piece of code, a plan or whatever). If you find you have some architecture tasks or activities (groups of relates tasks) that are not producing useful artefacts then its probably time stop performing those tasks.
gap·ol·o·gy n. The systematic study of, and the method used to identify and close, gaps within and between organisational units (individuals, teams, departments) or social structures (gender gaps, race gaps). Becoming an expert in using gapology or studying gapology are the behaviors of a gapologist.
Okay, this is a definition I made up to make a point and create a reason for this post! You'll find a couple more definitions here. My observation is that many of the problems we face in systems development are due to the presence of gaps of various sorts. Until we fill such gaps we will not be able to build systems that are as effective or efficient as they might be. Here are some of the worst.
The Knowing-Doing Gap. An article in Fast Company discusses this book which asks: Why is it that, at the end of so many books and seminars, leaders report being enlightened and wiser, but not much happens in their organizations?
The Business-IT Gap. The best known gap in IT. This is the inability IT have in understanding what business people really want (or the inability the business have in saying what it is they want depending on which side of the gap you are sitting).
The IT-IT Gap. This is the gap between what IT develops and what operations/maintenance think they are getting and need to run/maintain.
The Gender Gap. As I've discussed here there is still unfortunately a real problem getting women into IT which I think is detrimental to the systems we build.
Many of these gaps occur because of the lack of effective stakeholder management that takes place when building systems. A report published back in 2004 (The Challenges of Complex IT Projects) by The Royal Academy of Engineering and and The British Computer Society identified the lack of effective stakeholder management as one of the key reasons for project failure. The key learning point I believe is to understand who your stakeholders are, engage with them early and often and make sure you have good communication plans in place that keep them well informed.
Tim Brown (CEO of IDEO) has written a great book on design thinking called Change by Design. Check out the link to see his mind-map of what the book is about.
The basic premise of the book, why its about design thinking rather just design, is that design thinkers take a far more holistic approach to solving design problems. They use an interdisciplinary approach, think around the problem, including viewing the constraints as enablers rather than what should be fought against, and come up with ideas that would otherwise not have been thought of if 'ordinary' design had been applied. One of the case studies Tim uses in the book is the setting up of a live laboratory in the Mayo Clinic in Rochester, Minnesota to develop new approaches to patient care which involved designers, health care professionals, business strategists and patients to develop ideas for the "patient-provider experience". A methodology called SPARC (See-Plan-Refine-Communicate) was adopted (which is I suspect based on the Deming Cycle) to show how design thinking could be applied not only to product design but also service design.
Returning to the mind-map that is on the inside front-cover of Change by Design its two starting points are 'What' (is it we are trying to do/solve) and 'How' (are we going to approach the design). This fits nicely with my own concept of what we term 'architectural thinking' where we add an additional 'node' which is 'Why' (are we doing it this way). I prefer to illustrate this as a Venn diagram as shown below. The intersection of the three sets is what we consider when 'thinking architecturally'.
What - The requirements we are trying to address. Architectural thinking focuses is on those requirements (functional, qualities and constraints) we think are architecturally significant in some way.
Why - Captures the key decisions we are making. Architectural thinking focuses on the architectural aspects which lead to major structuring, placement or procurement decisions. Could be explicit (as a fully documented decision with options looked at and rational for making the decision we did) or implicit in a diagram or model.
How - The design and implementation of the system. Architectural thinking focuses on those elements of the design that are significant to the architecture (maybe patterns applied, key principles adopted etc).
The key thing in all this is that the thinking applies to the significant elements not everything. The key skill of the architect is to understand which things are important and which can be left to someone else to fret over.