2017 Predictions: Clouds, Thunder and Fog

The IT industry is still in flux and in 2016 we saw some tectonic shifts. Setting aside noise and hype, lets analyze the trends and predict what will likely happen in 2017.

Rapid Growth of Enterprise Cloud Adoption

It doesn’t take a prophet to notice the public cloud is growing and Enterprise IT is shrinking. In yesterday’s world we had terminals and mainframes and locality mattered, but today between a mobile work force, mobile clients and globalization, there isn’t much incentive to have an on-prem infrastructure. Most companies prefer using always-connected services on the internet’s backbone.

This move of enterprise organizations to the cloud will accelerate in 2017, driven by two main factors:

The greatest growth is in SaaS. Small and large enterprises are moving to cloud-based email, collaboration, content management, ERP, CRM, HR, etc. Seventy percent of companies already use SaaS and IT infrastructure is taking a hit. This trend will likely accelerate, as there will be even less need for Exchange servers, Oracle, SAP, etc. and all their related hardware.

Another key trend is the move to digital businesses. Companies must now align technology with their core business and maintain constant engagement with customers, otherwise they face disruption by new entrants and competitors. This requires agility and adoption of new technologies like cloud-native, containers, server-less functions and real-time analytics. Business units turn to the cloud because the complexity of DIY hurts productivity and it’s just easier to avoid IT getting in the way.

The Cloud of Musical Chairs

It is very easy and addictive for SaaS companies and startups to run on AWS - no CapEx and you get a rich set of services without the ramp-up time or complexity of traditional IT. But things aren’t so great once you scale or need high performance. Dropbox moved to its own cloud due to costs, GitLab moved out due to performance and Spotify moved to the cheaper Google Cloud. I’m sure the fact that AWS competes over time with more software companies played a part in all these decisions, but the key challenge for public cloud refugees remains the need to build modern AWS-like services and APIs by themselves. Salesforce didn’t manage to do it and ended up surrendering to the cloud in a $400M deal.

Another emerging space is IoT (Internet of things). It’s still pretty early and no one has really experienced the forecasted challenges of scale, so the cloud is the best place to play, using all those IoT kits and Raspberry Pi with cloud-based streaming, functions, databases, etc. However, once companies grow from a few thousand to millions of subscribers, or once they need to push large volumes of content, they will realize it’s quite impractical. IoT analytics will be distributed and work more like Netflix does, which has streaming gear in about 250 ISP and hosted locations worldwide alongside centralized control in the cloud. It is the only way to avoid unbearable communication costs and latency (as covered recently by Wikibon research).

Based on recent studies and my discussions with iguazio’s customers, the biggest concern today is cloud API and data lock-in. Customers want to move faster and use pre-integrated and proprietary services like databases and API gateways. It’s harder to migrate to a different provider once you use those across your apps. Obtaining privacy in a shared infrastructure requires full encryption and has a negative impact on performance, overhead and complexity so other key concerns are data security and privacy, as not everyone likes to operate 100% encrypted.

These issues, coupled with the fact that cloud data services are expensive over time and the need for locality in some cases, lead companies which are large enough to use co-located hosting facilities with direct cloud connections or hybrid cloud deployments. Equinix, the world's largest hosting provider, is on a growth path and recently acquired 29 additional data centers (now 175 world-wide). This is a clear indication that a hybrid of hosting and cloud is the future.

Growth of Public Cloud Providers

We have five main cloud providers: Amazon controls the market, followed far behind by Microsoft Azure and then Google, IBM and Oracle. AWS will keep growing, but my take is that Azure and Google will start closing the gap, as AWS has managed to alienate many users and developers. At the same time, Google has played the open card (open source Kubernetes, Angular, Go, TensorFlow) allowing you to use your laptop or the cloud, and it’s cheaper. IBM will gain share with the bet on intelligence (Watson) and the strength of its services organization. If you are an institutional organization without developers and need to deliver a custom service from A to Z, who else would build it for you?

Oracle has the potential, given that it’s missing some more modern technology, but its key challenge is the amount of negative sentiment people feel towards it. After years of lock-ins and draconian contracts, users want to be free. Why choose Oracle Cloud? Its recent Java license troll doesn’t help. Larry, if you are listening, take lessons from Satya Nadella who transformed Microsoft into a company we can sympathize with.

Order in the House of Containers

One of the ways to build modern, scalable and cloud-native applications is by using container technology, which mitigates the cloud's lock-in with an ability to move containers between clouds. The problem is that containers require quite a bit of infrastructure and orchestration and there are many different container platforms: Kubernetes, DC/OS and Docker Swarm to name a few. It is quite a challenge for users, vendors, and tools which want to integrate with these platforms, which all have different models and APIs.

Looking back at history, in the early days there were many IaaS orchestration platforms. Open Stack created gravity when it managed to stand-out among many other projects. From that point on, most players chose to integrate with it. The same phenomena is now observed with Kubernetes - it is in the driver’s seat. Google quickly formed a foundation around it (CNCF) and it will probably become the de facto standard.

The role of CNCF in defining standards for things like container networking, storage and service binding will benefit all container platforms (if they choose to participate). Commercial or more opinionated variations like Docker (inc), DC/OS and CloudFoundry can co-exist and leverage the same APIs.

Big Data beyond Hadoop

Companies want to become data driven and engage customers with cloud based services. In most cases, they have so far first invested in building complex big data and Hadoop plumbing which then failed with the ROI. In 2017 we will see more and more big data and analytics driven by business units and data scientists, so that they can focus on actual biz problems. We will also see a variety of integrated solutions, from cloud to software providers, some of which may not even be based on Hadoop components anymore. They may use Spark with an external data source like S3 or Cassandra, have an abstract UI to run data processing and analytics queries, and maybe even have specialized data science, AI and stream processing libraries. Server-less functions may be used for hassle free data integration and stream processing.

Summary

We are transitioning to a more digital world, with a sense of survival mode for many companies. Agility and innovation have become critical and cloud companies can address that, which is why we’re seeing this big transition to the cloud. At the same time, on-prem or hosting may be better options if you are big enough or have data locality requirements. Regardless, eventually all will transition to cloud native architectures based on elastic scaling, continuous integration and smarter data processing technologies.

Among cloud providers, the ones that will focus on usability and fast time to value will win over those that focus on technology or generate friction. Lock-in should be avoided as much as possible.