Avoiding an Organizational Cold Start on Serverless

Three Things for Engineering Managers to Look Out for When Migrating Enterprise Applications to Serverless Architecture

While developers have been quick to see the benefits in adopting serverless over the past two years, larger enterprises have lagged behind in adoption.

However, where the developers go, the enterprise follows.

True to form, serverless computing sits right at the top of Gartner’s “Top 10 Trends Impacting Infrastructure and Operations for 2019.”

For the uninitiated, serverless architecture (also referred to as “FaaS,” or Function as a Service) enable organizations to build and deploy software and services without maintaining or provisioning any physical or virtual servers.

Citing “appealing agility,” Gartner Senior Research Director Ross Winser estimates that 20% of global enterprises will have deployed serverless computing technologies by 2020, an increase from less than 5% currently.

Brands like Netflix, Fender Guitars, Major League Baseball, iRobot already get the appeal. So do Bustle, Vogue, and Autodesk. Don’t forget Thomson Reuters, Matson, Deloitte, and, you get the idea.

According to the IDC, 85% of enterprise decision-makers say they have a timeframe of two years to make significant inroads into digital transformation or they will fall behind their competitors.

In other words, enterprise FOMO is real.

Why serverless for the enterprise?

Serverless provides several key benefits for enterprises, primarily focused around costs, speed to deployment, and scalability. While saving costs are certainly an important metric, particularly for operations, both legacy companies and technology-first brands are dealing with an evolving landscape where the ability to deploy applications that improve their customer’s experience and scale quickly is worth its weight in hardware.

When deploying on serverless, companies only pay for the compute time used. When a function is not being used, the company pays nothing, saving considerable costs when compared to traditional virtual machine instances like EC2s. Typically, VMs are subject to almost 90% under-utilization. With Lambda, this approaches zero.

In addition, since the cloud provider is taking care of the heavy lifting of scaling your application and making it highly available, developers can spend more time writing code. Many companies embracing event-driven computing are now able to move faster than their competitors, since they are worrying less about infrastructure.

Autodesk, for example, was able to develop and launch a serverless solution, “Tailor,” to solve a critical bottleneck that was slowing down developers. They reduced account creation time by 99%, increasing business velocity while decreasing costs.

From the AWS case study:

“Prior to implementing Tailor, our cost to provision each account was about $500 in employee time,” says Alan Williams, Enterprise Architect at Auitodesk. “With Tailor, it’s just shy of $6 per account, which means we can create about 10 times as many accounts for the same cost. It’s now much faster and cheaper for my team to give the business what it needs.”

However, as macro-builders or engineering leaders look to leverage FaaS providers like AWS Lambda, there are several things to look out for when migrating applications to serverless.

Cold starts in serverless are essentially an invocation that takes longer to start up due to the initialization of a new container.

While actual cold starts impact less than .2% of function invocations, according to AWS Principal Developer Advocate Chris Munns, an organizational cold start could slow down your ability to derive business value from serverless adoption.

For companies and engineering teams migrating to serverless, here are three red flags that could cause an organizational cold start:

1. Spending too much time on proof-of-concepts

With emerging technologies, it’s always easier to relegate adoption to a far-flung corner of your architecture. Let the innovation group work on a proof-of-concept that won’t impact your core business much one way or the other, and let everyone else continue business as usual.

However, with technology-first companies continually eating market share from established brands and c-suite tenures lasting less than the average NFL running back, business as usual has become a less attractive option.

While starting with a proof-of-concept and low-risk business processes is wise, don’t waste two years avoiding critical workload deployments on serverless. The advantages for both your engineering teams and your bottom line are too important — particularly if you’re in an industry that deals with seasonal spikes to web properties like retail and media.

If you can identify areas in your business that will benefit most from a serverless architecture, start there.Common early use cases where enterprises see the most value from serverless include ingesting and processing of data, customer-facing websites and applications, digital asset processing, API integrations, and more.

For example, Comic Relief, a major UK-based nonprofit, migrated its entire donation platform to serverless architecture in 2019 and saw a 94% decrease in costs.

2. Keeping a monolith mindset

If your organization is migrating to serverless architecture from one cohesive unit of code for your applications, keeping a monolithic mindset can result in a situation where you’ve diluted the performance advantages of serverless, and yet, kept the worst parts of a monolithic architecture.

It’s tempting to just break a monolithic app into microservices based on logical components versus looking at the way data and users flow through your system and using that to determine your serverless architecture. In the Capital One Tech article, “4 Serverless Myths to Understand Before Getting Started with AWS,” Architect Tanusree McCabe cautions against viewing serverless as the monolithic silver bullet:

“The challenges of a traditional client/server architecture like this include:

Synchronous calls resulting in highly coupled systems.

State management required.

Not resilient to failure.

Not able to scale effectively.

Simply putting the server logic inside a lambda function does not solve these challenges for client/server architectures.”

Instead, focusing on building event-based architectures where business logic serves as the glue between services enables scalability, more agile development, and better performance. This isn’t necessarily the right fit for every application.

In Ben Kehoe’s beautifully written article, “Serverless is a State of Mind,” he outlines an example from Lyft where time spent rearchitecting everything to Lambda could impact their ability to deliver value to their customers. Aligning the serverless mindset with a focus on business value, Kehoe points out that they’re actually making a “serverless choice” by not switching from their current applications and processes.

If you’re in charge of migrating applications to serverless, understanding the advantages and limitations of serverless and preparing your team’s mindset for a different approach will make or break those initial projects.

3. Not adapting your monitoring and observability for statelessness

Serverless platforms enforce ephemerality and immutability in a way that demands what we commonly think of as a stateless architecture. While these are not new, the lack of visibility into underlying infrastructure reinforces the need for greater application observability.

As infrastructure is abstracted away from your organization (less to manage, yay!), real-time visibility into the performance of your code becomes critical. With hundreds of functions making up a single application, understanding when and why something didn’t behave as expected is a tall task.

For organizations that see a ton of spiky user traffic patterns due to seasonal retail campaigns, fundraising events, media cycles, etc, the problem is compounded due to any user disruption creating significant impact to business KPIs. Seconds matter when processing millions of events per minute on Cyber Monday or Super Bowl Sunday.

Comic Relief — who raised more than $80 million USD in one day on national television through their serverless donations platform — needed the reassurance of real-time visibility going into production.

“Due to the nature of Red Nose Day, you don’t get many chances to test the system under peak load,” said Adam Clark, Sr. Software Developer at Comic Relief.

The ability to trace applications end-to-end, profile each individual function, debug quickly, and be alerted when a function starts to misbehave gave their team confidence going into their national fundraising event.

Like everything else, not adapting your monitoring solutions and processes for serverless architectures will quickly erode confidence in the benefits of switch to serverless as your team spends days digging through log files trying to figure out what went wrong.

As enterprise adoption increases this year, the complexities of migrating to serverless will compound. Companies that align their teams and processes to avoid an organizational cold start in their serverless journey are much better positioned to deliver business value and, ultimately, provide a better user and developer experience.

To get instant observability into the most granular behavior of your serverless applications at the enterprise level, try IOpipe for free today or schedule a demo for your team.