The Hard Truth about Unlocking Innovation

The Hard Truth about Unlocking Innovation

Another phrase for “innovation” is “learning what works.”

But learning stems from failure.

And failure means breakage of customer trust.

Global 2,000 enterprises have an allergic reaction to breakage of customer trust.

Ergo, an aversion for failure leads to a lack of innovation.

The way out of this dilemma? Make experimentation "safe to try" at the edges of your organization, and learn to manage risk in more sophisticated ways.

That’s what this blog post is all about.

I recently gave a keynote on software delivery to Global 2,000 technology leaders — mostly Fortune 500 CIOs — at an Innovation Roundtable event in NYC, which was wrapped around the Formula E race.

The number one thing on their minds was unlocking innovation inside their organizations. Many are starting to feel real competitive pressure from faster-moving, mission-based software companies:

In the past, these cash-rich G2K enterprises would just acquire the innovation  by executing an M&A strategy. But in the past decade, many of the startups have either eclipsed the size where larger enterprises can afford to buy them, or the economics aren’t penciling out from a valuation perspective, often due to their short time horizon payback period thinking.

The net effect is that for the first time these Industrial Age companies, which are rich in cash but short on innovation, are having to think about retooling themselves in some way to stay relevant over the coming decade — and this task often falls to the CIO.

Since Formula E was a main topic, I began my talk with news that the CEO of BMW had just stepped down the weekend before citing enormous changes, which have brought about more transformation than in the previous 30 years. I expect we’ll continue to see leaders choosing or being forced by boards to step down as they grapple with disruptive competitors.

People sometimes say “Innovate or Die” but that’s never resonated with me. I don’t believe BMW will die. But every leader in a Global 2,000 company does need to decide which box they want their company in:

And it’s very unclear if BMW will be in the green “Thrive” box over the coming decade. It might end up just Surviving vs. Thriving.

Enter Conway’s Law:

Industrial Age companies are largely built to provide goods or services with high-trust outcomes. When a company builds a car, it must have high quality built in. When a bank is entrusted with money, it can’t lose that money. This means that a primary driver of the company’s focus is ensuring that customer trust is not broken, and the Command and Control organizational structure reflects that. Conway's Law states that:

A key way to minimize breakage of customer trust is to carefully measure and optimize repeatable patterns for accomplishing a goal. These typically show up as mandates, enforced processes, compliance and audit practices. I am very appreciative of the sophistication with which large global enterprises are able to ensure our cars and planes don’t crash and our money doesn’t disappear.

The problem is that these organizational structures do not lend themselves to building software experiences. The big difference in software is the non-obviousness of successful product market fit. Would one have thought that crashing on a stranger’s futon would be a $30Bn business? (Airbnb.) Or that people would want to trust their life savings to a computer robo advisor with no human in the loop? (Wealthfront & Betterment.)

I call this “The Innovation Cost of the Stoplight Culture”

One can measure the most effective way to move a car down an assembly line, and train workers on the most efficient way to build that car. The worker doesn’t need to think; she can just clock in and clock out, and be told what to do, and how best to do it.

That’s Stoplight thinking. Tell employees when to stop, go, turn left. It worked exceedingly well in the manufacturing assembly line, which was the Second Industrial Revolution. But it breaks down in the digital assembly line, which is the Third Industrial Revolution that's powered by software, and we’re in right now.

I call this thinking the Innovation Cost of the Stoplight Culture. Executives and leaders are focused on protecting the brand equity of their companies and minimizing breakage of customer trust, and so their organizational structures are built around Stoplights, which means ideas and instruction — and answers — come from the top of the organization.

Here’s the problem: Creating experiences through software isn’t like creating goods or providing services. In fact, several key things are in direct conflict to the expertise that Global 2000s have amassed:

Conflict #1 The Jikoda human anti-pattern

In manufacturing there’s a best-practice to empower any employee to pull the “Jikoda” cord when they see a quality issue, which stops the assembly line. This is a practice perfected by high quality Japanese auto manufacturing, and it enables employees on the assembly line to call out quality issues without fear of repercussion. This is very important when the finished good must roll off the assembly line without defects.

However, with software delivery, it’s an anti-pattern. Human nature — especially in big companies with big brands to protect — is to pull the Jikoda line and stop the digital assembly line when a software release causes a problem in production. Companies then put in more safeguards to ensure it never happens again. More integration testing. More unit testing. More manual and automated QA. Often, a VP will say “I must personally approve the update before it goes out to production” because he feels like his job is on the line if the company ships an update which nukes the entire customer base again.

There’s a cruel irony to this human pattern, which is that with software development, the slower you go, the less safe you become. This is a terrifying cycle for companies because of their insistence of 100% quality in production, which was summed up well by this tweet:

Conflict #2: The product features that will drive strong product market fit via software are non-obvious, even to domain experts.

Low performing Fortune 500 companies ship software seven times per year – about once every other month. Netflix, on the other hand, ships software 7,000 times per day (not year). I call this the "Innovation Gap."


It's not that Netflix knows what software features to build per-se – it's that Netflix is taking many more shots on goal than its competitors, which gives it many more opportunities to learn what works vs. doesn't. An analogy would be that a feature with strong product market fit is like a nugget of gold. Netflix is churning through ideas like this strip mining machine while others are panning for gold manually:

Which approach will churn through more ideas to find more nuggets of gold, faster?

All of this means that in order to move faster, companies have to start to accept that they will have failure in production. Building software experiences is not like building physical goods. The mastery comes from a) empowering the edges of the organization and so it is safe for them to try (and fail) with new ideas, and then  b) becoming very sophisticated at managing that risk in production.

Empowering the edges of the organization means moving from a Stoplight to a Roundabout culture, which I write about here. This matters when a business is dealing in ideas instead of physical goods.

Managing risk with sophistication, when it comes to software delivery, means adopting a modern cloud-native platform like Armory's Spinnaker which enables things like:

  • Blue/Green deployments
  • Automated canary deployments
  • One-click rollbacks

To multiple infrastructure targets with safety and velocity.

This helps ensure that when there is an issue in production, you don't nuke your entire user base, but rather, you have a controlled blast radius.

As a company starts to create trust and autonomy amongst its workforce to empower the edges of the organization to experiment and learn through rapid iterations, and becomes willing to allow learning through failure in more sophisticated ways, it will have an opportunity to truly start innovating through code.