When it comes to Cloud, change isn’t coming; it’s already here. Software is eating the world and the only thing between firms dropping out of markets or folding completely is the speed they can change their customer-facing apps.  Beating the competition is in many cases now a simple ‘foot race’ on who can deliver the best software experience to their customers. As a result it innovations in software delivery and IT infrastructure will increasingly determine the success or failure of industry players, rather than the quality of the product or service they provide.

Start-ups are taking ground from Enterprises in part due to their ability to better harness seemingly endless resources of the cloud, and yet many enterprises just don’t seem to be able to make the leap to ‘full cloud’ adoption.

Using my experience working with UK Enterprises I apply Disruptive Innovation (DI) theory to Enterprise IT departments & cloud adoption

TL;DR

• Attempts to move to a cloud-like infrastructure model, with infrastructure managed by the same IT department as has run their ‘traditional IT’, are unlikely to realise the benefits of cloud

• IT departments will face stiff competition from cloud ‘integrators’

• An entirely separate ‘cloud services’ team, targeted at resolving the requirements of the security, compliance teams sooner, should enable business to consolidate their cloud platforms

• Recruit developers with experience of delivering & operating cloud-ready apps on public cloud & form them into teams separate from ‘traditional IT’ delivery teams

Applying DI theory to Enterprises and Cloud

Enterprises find cloud really, really hard. In search of answers to this anomaly I came across The Innovator’s Dilemma, a book which introduced Disruptive Innovation (DI) theory one of the most influential management concepts in recent years. It has been labelled ‘the gospel of innovation’ even by its harshest critics, and has subsequently been used to explain the rise and fall of businesses in a variety of industries.

DI theory categorises new technology as either Sustaining Innovation (SI) or Disruptive Innovation (DI). Described loosely, SIs provide a straight line improvement of existing technology. DIs originate as an inferior product with a different set of characteristics that appeal to a secondary market audience. They eventually accede to the primary market when it outperforms in the characteristics most valued by the customers of that market. Example: the 2.5in disk-drives could store less data than their 3.5in predecessors used in desktops; however they were more rugged and consumed less power, so were more suited to laptops. Eventually desktops also used the smaller disk size at the point when they could store data sufficient data for the same price.

Cloud fits the DI description perfectly: it provides a simpler subset of the functionality Enterprises can get from running their own infrastructure; is less reliable & has a less certain performance level. It has appealed to successful start-ups who have no datacentres or expensive infrastructure, and who value simplicity, speed of provision & flexibility.

According to DI theory, companies are classified as ‘established firms’, present when a DI emerges and ‘entrant firms’, which rise to success on the back of the DI. IT departments at Enterprises are technically established firms since they have existed before the advent of cloud. There have always been competitors for provision of the service they provide, yet the cost barrier to outsourcing the whole operation to a technology behemoth, for example, has been high enough to make it seem as though that were not the case.

IT departments have over the years been providing more efficient infrastructure. Technologies such as hypervisor virtualization have allowed Virtual Machines to be owned and managed using many of the same processes that were used for physical infrastructure. This was a classic SI in that almost all the change was made by the established firms and the ‘customers’ perceived very little change other than being able to get hold of ‘a server’ much faster. Enterprise IT departments have generally applied all the rigmarole that they learnt for physical tin to VMs: patching, inventory, standardisation, compliance, audit, security, etc.

Companies usually start by turning to their established IT departments to provide cloud their infrastructure. The tendency has been to start with private cloud and apply all pre-existing internal rules that applied for traditional IT: role separation, separate prod/non-prod infrastructure, etc.  At least then you want have to get the ‘cloud policy’ rewritten, right?

The trouble with this is: one of the main reasons start-ups are taking the market by storm is that they don’t have these barriers, these preconceptions that infrastructure has to be delivered, managed and secured a certain way.

An easier route to success would be start a new team to break ground on providing cloud infrastructure, with separate reporting lines and a separate location from the existing IT department.

Getting the right customer

Clayton Christenson gives specific examples where engineers at established firms have found opportunities for entry into an emerging market, but were buried by management. He shows that if the innovation is not what the current customers want right now, then it will tend to get dropped or modified in favour of the more pressing requirement to satisfy today’s customers & that this is all a feature of good management. The larger the firm and the more customers they have, the less able they are to satisfy anyone else; no wonder enterprises struggle.

So who are an IT department’s customers? It’s the application teams who have developed applications to run traditional IT infrastructure and those that manage the apps. The infrastructure qualities these customers have most highly valued are reliability, stability and a fixed performance level, which may partly also explain the delays at which enterprises have taken up cloud: the current customer does not value its benefits as highly as it values stability.

Unfortunately too much stability has the effect of getting developers used to relying on systems being available. If you can make assumptions that are true the vast majority of the time (e.g. the messaging platform will always be up, the downstream REST service will always be available) application architecture can be made much simpler. This has a natural tendency to cause tight coupling and hence fragility in systems to proliferate.

In contrast there is a breed of developers working at start-ups, writing native cloud apps, who naturally expect that ‘everything fails all the time’ and do not value reliability in the same way. These are the customers that will drive cloud adoption at enterprises. They might be specifically recruited by a forward-thinking manager who has already seen the benefits of cloud elsewhere, or will naturally tend towards enterprise as they look for career stability. Either way they will want to rekindle the best practices from their earlier work. Learning a cloud ethos from these developers is paramount for enterprises, so isolating them away from existing development teams and practices is a good idea.

Once sufficient cloud native developers are gathered, enterprises will start seeing requirements along the lines of “the business wants to get on cloud now” and without a cloud offering that matches their expectations (e.g. the private cloud isn’t really a cloud, doesn’t have enough self-service, runs out of capacity), the delivery team may turn to an ‘entrant firm’: a new company borne to success on the back of cloud adoption.

Who are the new boys

With the advent of Cloud many off-the-shelf products that previously required significant infrastructure in the datacentre are now being delivered via SaaS (e.g. the move from on-premise CRM to SalesForce).

This makes good business sense, building CRM configuration teams within IT departments is not a core competency for most enterprises and should be avoided if possible. CRM systems were large unwieldy beasts to manage and IT departments won’t be sorry to see these disappear as the trend continues. Such a change would leave IT departments with the business-differentiating apps to manage – the ideal situation. However not all apps are created equal and the more complex ‘traditional IT’ apps will continue to drive SIs on the infrastructure.

DI theory describes the way in which established firms will tend to become focused on the higher-end requirements of a small set of high-paying customers; those who shout the loudest. Those customers will request more features in relation to the qualities they value e.g. HA, in preference to the features the low-end customers value e.g. self-service, flexibility. This in turn increases cost, clearing the way for the cheaper, simpler DI technology vendors to move in and take the lower-end customer ground as their products mature. With cloud as the DI the same model sees delivery units bypassing IT departments altogether as they become too expensive, unreactive, etc., and going straight to cloud themselves.

There are raft of companies vying for business in on-boarding companies to cloud. It doesn’t seem that they’ve cracked Enterprise yet, but they’re definitely coming. I recently read that DualSpark, an AWS consulting firm founded by an ex-Amazon employee, now employs about 800 people. That’s some growth rate.

The trouble with allowing each business team to choose their own integrator is that they will likely recommend a wide variety of cloud platforms and solutions, and make attempts to satisfy the compliance and security team requirements in a variety of ways.

A better approach would be to address the concerns of these internal teams early on & produce a defined set of cloud features that meets their demands in a reduced number of ways. This could be the first task of your enterprise’s ‘cloud services’ team recommended above, right alongside forming the strategy for cloud.

IT Departments

The inevitable effect of cloud on IT departments that cannot change their way of thinking (and I doubt many can) will be shrinkage as they are left with only the most complex traditional IT systems to look after. They will increasingly be run by ageing IT operatives using legacy technologies. As such systems tend to reach end-of-life, and are re-designed to run on cloud, IT departments will shrink further. They will be left with just a small number of central systems, however experience dictates these will probably include the most important systems upon which the core business functions are run, for these are generally the hardest to modify.

Clay Christenson describes an interesting feature of firms that survive disruptive innovation: they usually do so by starting a new company that satisfies the needs of new customers, and the usually absorb the spin-off company back into parent at the point where the new market exceeds the previous. Upon reabsorption the management of the parent company could well be completely replaced with the management structure of the spin-off.

I see no reason why such an approach would not work at an enterprise, with the ‘cloud services’ team absorbing work from incumbent IT department by enabling on-boarding to cloud for those delivery that have the right ethos & developers.

Perhaps then there is a rocky road ahead for enterprise IT department middle management.