The edge of chaos: Where complexity science meets organisational performance
One of the things I particularly like about viewing organisations through the lens of complexity science is that it does not just attempt to describe organisations, but it also prescribes how they should operate in order to maximise performance. There are two key concepts in this: fitness landscapes and the edge of chaos.
Imagine a mountain range with many jagged peaks. Some are perhaps a few hundred meters tall, while others stretch up thousands of meters, reaching high above the clouds. This landscape represents the competitive environment of a specific industry. The individual peaks stand for different ways an organisation can chase performance. The higher the peak, the better the performance. There are three conditions:
A company may occupy only one peak at a time;
The height of the peaks cannot be seen very well when standing on the ground. Instead, one needs to occupy a peak in order to better survey the landscape;
The landscape itself changes. Peaks may rise and fall as time passes.
A fitness landscape? Or perhaps just a regular landscape.
Let's take this metaphor further by using Nokia as an example. When they decided to enter the mobile phone industry, they effectively entered into an entirely new mountainous landscape. And as they were one of the first companies there, it was not possible to see which of the performance peaks one should attempt to climb because no one knew where the highest peaks were. One had to engage in exploration. Nokia did this well, managing to find and scale one of the highest peaks, setting up shop there and becoming the world's largest mobile phone manufacturer.
Gradually they became better and better at exploiting the opportunities found in their performance peak, increasing efficiency and operational excellence. All was well and good until the landscape started changing. The peak they had found and successfully exploited began to fall in the shadow of other peaks, which were discovered and exploited by companies such as Apple and Google.
The more turbulent the industry, the faster the fitness landscape changes. Old performance peaks become obsolete and new ones arise.
This is a story that has repeated itself countless of times across different industries: The established corporations become stagnant, focused on exploiting their current competences and position in the competitive environment, and failing to see that the environment is not anymore the same it was when they became dominant. Or even if they see that change is inevitable, the efforts to resist it tend to be stronger than the efforts to innovate.
They become incapable of dismantling all the structures built for successfully exploiting the performance peak they have settled on. Without this conscious removal of structure and open engagement in a period of exploration it is not possible to find new and more promising peaks. Think, for example, the inability of recording industry to take advantage of digitalisation, or the lackluster response of American car industry to increasing competitive pressures. Established companies have a tendency to become so focused on the position built on a given performance peak that moving to a new one, and learning again how to exploit it, will never take place. Or if it will, it is only in the face of absolute necessity and involving considerable pain.
Startups are another great example. They are unencumbered by existing structures, built to exploit a particular peak in their industry, which means that they are free to explore the landscape to discover new larger peaks. The concept of fitness landscapes also explains why many initially successful startups become vulnerable later in their life: after finding and settling on a newly discovered performance peak, their focus shifts from exploration to exploitation, optimisation and operational excellence, which in turn makes them forget how to scan the landscape for changes and engage in exploration.
It is an easy thing to say that companies need to cultivate the ability to simultaneously exploit performance peaks in their competitive environment, while also engaging in exploration to discover new higher peaks. To actually do it is a different story, and this is where the edge of chaos comes in.
Organisations like other Complex Adaptive Systems perform best when they are at the edge of chaos. It is a place where the organisation has just enough structure to stay cohesive, preventing it from being pulled apart by competing internal and external forces. Emphasis on just enough. Because the openness to chaos, the ability to dismantle existing structures, to change shape rapidly, they all precede the ability to innovate and to explore new performance peaks in the ever-changing competitive environment. As stated by Shona Brown and Kathleen Eisenhardt:
...the edge of chaos lies in an intermediate zone where organisations never quite settle into a stable equilibrium but never quite fall apart, either. This intermediate zone is where systems of all types—biological, physical, economic, and social—are at their most vibrant, surprising, and flexible. The power of a few simple structures to generate enormously complex, adaptive behaviour—whether flock behaviour among birds, resilient government (as in democracy), or simply successful performance by major corporations — is at the heart of the edge of chaos. The edge of chaos captures the complicated, uncontrolled, unpredictable but yet adaptive (in technical terms, self-organised) behaviour that occurs when there is some structure but not very much. The critical managerial issue at the edge of chaos is to figure out what to structure, and as essential, what not to structure.
- Competing on the Edge: Strategy as Structured Chaos (Brown & Eisenhardt, 1998)
To stay at the edge of chaos requires energy and conscious effort. Complex Systems have a natural tendency to swing either to the chaotic (i.e. to fall apart), or the static (to become stagnant, controlled, and bureaucratic). Organisations need to maintain activities that are purposefully designed to disrupt the existing behaviour. Activities such as experimentation, or the continuous effort to test what works and what does not work, resulting in learning, and the discovery of new opportunities and new performance peaks. Another method is time-pacing, or the systemic effort to generate periodic change from within (e.g. 3M's self-imposed rule that each year 30 % of revenues must come from new products, forcing the company to stay innovative).
My personal experience from different organisations is that there typically is way too much structure and way too little openness for chaos. Yes, some structure will be necessary in order to achieve anything, but it quickly becomes a hindrance to the long-term survivability and performance of the organisation. Here are some questions to get you started in figuring out where and how much structure is needed:
How is the competitive environment of the organisation? The more uncertain, complex, and turbulent, the more the organisation needs to lean towards chaos.
What is the organisations business? An organisation dealing with one-time products and services (e.g. creative agencies, consultancies), or innovative new technologies needs to be more open to chaos and exploration than an organisation such as the IRS. In fact, I don't think anyone would enjoy seeing much experimentation and exploration in the way the government handles our taxes...
Where in the organisation is structure needed, and where it isn't? There are certain activities in every organisation that should go according to specification every single time, such as accounting, hiring, firing and others where e.g. laws need to be followed to the letter. Yet it is good to understand what kind of structure is really needed, and what is superfluous. For example, whether or not employees in accounting wear business attire has nothing to do with how well they are able to do their jobs, yet some companies have created a structure that regulates clothing. On the other hand, there are also activities–namely those that require creativity, agility and innovation–that are particularly sensitive to the harm caused by unnecessary structure.
Think broad guidelines instead of specific instructions. Self-organisation is a property of Complex Adaptive Systems, meaning that the system is capable of figuring out on its own how things should be done. In this sense the purpose of Organisation Design is not to provide specific rules for handling different situations, but to provide boundaries that ensure that the self-organising processes (unhindered by unnecessary structure) are aimed in the right direction. These boundaries include, for example, the organisations mission (why the organisation exists), vision (where it is headed), strategy (how it is going to get there), and values (what kind of behaviour it expects). When these are clearly understood by everyone in the organisation, it creates limits to the self-organising behaviour.
As a summary, the more there is structure in an organisation, the worse it tends to be at innovating, reacting to rapid changes in the business environment, discovering new opportunities, and taking advantage of them. Therefore it becomes an absolutely essential question of Organisation Design where, how much, and what kind of structures are needed, and what kind of mechanisms should be used to periodically challenge those structures, so that their value can be re-estimated in the light of the changing environment. For this, complexity science provides a framework, vocabulary, and way of thinking that is a clear departure from the century old model of industrial organisation.
References and further reading:
Brown, S. L., & Eisenhardt, K. M. (1998). Competing on the Edge: Strategy as Structured Chaos. Boston: Harvard Business School Press.
Carroll, T., & Burton, R. M. (2000). Organizations and Complexity: Searching for the Edge of Chaos. Computational & Mathematical Organization Theory, Vol. 6, Iss. 4, 319-337.
Lansing, Stephen J. (2003). Complex Adaptive Systems. Annual Review of Anthropology, Vol. 32, 183-204.
March, James G. (1991). Exploration and Exploitation in Organizational Learning. Organization Science, Vol. 2, No. 1, 71-87.
Starbuck, W. H., Barnett, M. L., & Baumard, P. (2008). Payoffs and pitfalls of strategic learning. Journal of Economic Behavior & Organization, Vol. 66, Iss. 1, 7-21.