The Current State of Infrastructure From Code

This post has been a long time coming.

Back in September, I teased about my first impressions of Infrastructure from Code (IfC). Around that time I began an experiment. I wanted to see how people were approaching the Infrastructure from code problem.

Infrastructure from Code is a relatively new way to build out your cloud architecture. A framework will take your code and convert it to cloud resources for you. This omits the declarative part of serverless development (or cloud development in general) which ultimately lets you move faster. Focusing on your business logic instead of building SAM templates or CDK constructs really lets you hone in on actually solving your business problem.

Taking out the infrastructure decisions when building in the cloud is an interesting thought that not everyone is going to appreciate. Some people will fight tooth and nail to keep a sense of control over what gets pushed into their environments. But as Jeremy Daly says, control is an illusion. The desire to control your infrastructure really boils down to the desire to optimize (usually). In the future, that might be handled for us too!

Putting aside the future for a second, I want to talk about what's available now. Let's take a look at what you can explore today to start getting your infrastructure managed for you.

The Experiment

As an experiment, I built out the same API four times. Once as a control in the way I normally would - via a SAM template, and with three different IfC vendors. The vendors I used were:

• Nitric
• Klotho
• Serverless Cloud

There are others in this space I did not assess like Encore, Shuttle, Modal, and Dark. These were not assessed for the sake of time. If you're interested in IfC, I encourage you to take a look at these others.

To compare these frameworks, I assessed them on several areas throughout the lifecycle of development process. The assessment areas are:

• Time to build - how quickly was I able to get something up and running
• Approach - what was the methodology used to abstract or infer the infrastructure from the code
• Deployment - how easy was it to put what I had written in the cloud
• Performance - was the API latency within an acceptable threshold
• Visibility - how much can I see, alter, or manage the infrastructure that was generated

My intention is not to declare a "winner", but rather explore the different approaches and infrastructure management techniques to build an idea of what the future holds.

The API

For a little bit of fun, the assessment is going to build an API for a chicken auction. It has endpoints for adding, updating, and viewing both chickens and auctions. Users can subscribe for updates to a chicken when it goes up for auction and they can subsequently bid on the chicken while the auction is live. To get a view for the architecture built in a SAM template, refer to the diagram below. You can also view the API documentation here.

My goal is to see what Nitric, Klotho, and Serverless Cloud do when I try to build this API in each of their frameworks. Each one of these implementations will be written in JavaScript.

You can view the GitHub repository here.

The Assessment

Time to Build

Time to build is how quickly I can get from nothing to a functional chicken auction in the sky. This has to do with documentation, tooling, quick starts, and if there's an option for local testing. The faster I can get to something real, the faster I can start iterating on my design.

Nitric - This framework felt like I was installing prerequisites forever. Prior to this experiment, I didn't have any "generic deployment" tools on my machine. I had to create an account for and install Pulumi, install Scoop, download and install the Nitric CLI, and make sure I had Docker up and running. Once all the pre-requisites were installed, I could get started - which was quite a nice experience. Nitric bootstraps your application for you when you run the nitric new command. It sets up the scaffolding and is immediately runnable locally with Docker.

However, it was not very intuitive on how to build my API. Nitric expects you to put your compute code in a functions folder, but it wasn't clear to me how the API was registered and how to get to the endpoints. The documentation explained things well, but it felt like there was some magic I didn't understand. Developers don't like magic. Overall it was a little difficult to feel comfortable when starting up despite it being fast to test locally.

Klotho - I immediately ran into speed bumps with Klotho. I run on a Windows machine (I know, I know), and quickly found that Klotho has untested support for it. I downloaded the executable and tried to run it, but no luck. So I decided to proceed without the executable and follow the quick start guides on their website.

Klotho recommends building your application as an express app and plugging in the framework quickly after it is built. So I followed that approach and had an express app running in just under an hour. I could run the app locally with a simple node index.js command and test as I went. Overall, very easy to build and familiar to me as a JavaScript developer.

Serverless Cloud - This was a whimsical experience for me. It was the last framework I built with and by far the easiest. As a JavaScript dev, I was borderline giddy when all I had to do to get rolling was globally install @serverless/cloud via npm and type in npm init cloud. Their Cloud Shell took over my terminal and guided me through all the steps to bootstrap my application.

After the template was up, I was able to to test it by running the cloud start command in my terminal and was actually testing it in the cloud! This framework felt like I was building an express app, which came natural and resulted in a faster build time. Overall an extremely good development experience.

Summary - Overall nothing was bad. These three companies have done a great job on documentation for getting started. The tooling was the differentiator for me. I also liked Klotho's mindset of just build and instrument later. This is great for existing applications that want to migrate to cloud-native architectures.

Approach

This category identifies and calls out each framework's method of how they satisfy the IfC problem area. There is no right or wrong way to solve the problem, but it is interesting to see the decisions made and the different schools of thought.

Nitric - This framework uses an SDK and a configuration yaml file for setup. The SDK abstracts cloud concepts like APIs, databases, queues, and events, allowing you to build with their versions of them. This provides them with the ability to deploy your code across cloud vendors. The config file tells Nitric which cloud vendor and region to use as well as configuring detail like function handlers.

Something I wasn't too keen on was their template adding a "functions" folder. To me, this gives a mental model that I'm writing Lambda functions and structuring the code the way I would via a SAM template. That defeats the whole purpose of IfC. That should be abstracted away by the framework.

Klotho - This is a very unique approach to IfC. The mentality here is to create a web service and annotate key pieces of infrastructure like API, database, and events. The annotations tell Klotho how to treat the resources in the repository when it generates a Pulumi template. This is a low barrier to entry for existing applications because refactoring is not necessary. Add annotations above key infrastructure and your resources will be created automatically.

After you run Klotho, the result is a compiled Pulumi template and an editable Klotho template. The Klotho template is a configuration file that tells the Klotho framework how to interpret annotations in your code. For new annotations it uses the default configuration. For existing annotations it uses the defined resources that map to your code. This template file lets you override any infrastructure decisions the framework made, which brings a little bit more control back to the hands of the developer. Is that a good thing? Maybe to some.

Serverless Cloud - The simplest approach, this framework uses the serverless SDK. There are no templates or configuration files, just the SDK that abstracts away databases and APIs. The SDK has a similar feel to Nitric's, but with more abstractions.

You can tell a lot of thought has gone into the data abstraction of this framework. It offers a single-table design feel for data and also provides events on data changes. It takes a lot of the complexity out of the development process.

Summary - There's a great mix of approaches here. Nitric and Serverless Cloud offer an SDK to write your code which offer high level abstractions of core cloud concepts. On the other hand, Klotho provides a non-invasive way to code that allows you to enter the cloud with your existing apps via annotations. Both are simple approaches that take a significant amount of complexity off the hands of developers.

Deployment

The next checklist item for a complete solution is how easy it is to take your code and get it into the cloud. This involves both sandbox testing and CI deployment for production.

Nitric - This one was a little lackluster for me. At the time I was assessing Nitric, there was no documented way to push my local changes to a sandbox account. I firmly believe you should bring your development environment to the cloud and not vice versa, so it was a big hit against Nitric for me here. However, since I have done my assessment Nitric has extended their CLI to deploy your local version to the cloud. Great feedback loop here.

For CI, I also was feeling a little underwhelmed. Nitric recommends their Nitric Deploy solution for deployments to the cloud. This connects to your GitHub repo and automatically deploys on push to a configured branch. This sounds nice in theory, but leaves something to be desired when you try to factor in tests and rollbacks.

Klotho - Not much to assess here. Klotho does not attempt to deploy anything for you. It is responsible for generating Pulumi code and you're on your own for deployment. The instructions on their site recommend to use Pulumi directly to get the app into the cloud.

Serverless Cloud - As I mentioned in the Time to Build section, Serverless Cloud caught me a little by surprise. When I went to test locally for the first time, I was pleasantly surprised I had actually deployed into my sandbox environment in the cloud! After getting my bearings, I learned how easy it was to get my API up and running from my local machine and just how quickly it deploys. 5 gold stars for local dev!

For CI deployments, Serverless Cloud has an answer for both testing and deployment with their CLI. It's just a couple of commands, which is the same thing I'd use with a SAM template.

Summary - This was a very disparate category. Ranging from no attempt, to a "it works for now...kinda" attempt, to full blown support, deployment is a huge area that needs some love and intentional focus. If you provide a framework that attempts to abstract infrastructure away from consumers, your deployment game better be on point.

Performance

Something on the forefront of every serverless developer's mind is the latency of the deployed resources. After all, serverless is pay-for-what-you-use, so the faster your app runs the less you pay.

Nitric - The first time I hit an endpoint in my deployed Nitric API, I was horrified. The cold start time on my Lambda function was over 20 seconds! So I spoke with the team over at Nitric and they did some investigating. After their research, they pushed out an optimization update and published their findings. The issue had to do with the way container-based Lambda functions cold start. Now cold starts run 3-6 seconds and warm invocations are less than 120ms!

Klotho - Similar to Nitric, Klotho creates container-based Lambda functions, so I had similar expectations. Oddly enough, I had inverse results. Cold start times were significantly faster at around 2 seconds but warm invocations averaged around 400ms. This must be a result of the container configuration, but since I'm not really a container person my investigation stops at speculation. Klotho does provide the option to configure the memory size of Lambda functions, so fine tuning performance is an option here.

Serverless Cloud - When I got to Serverless Cloud, I started to see a trend. Cold start times were between 2-3 seconds while warm invocations were typically right around 200ms. I noticed that not all endpoints had cold starts though. I found that one endpoint in the API would take on the cold start while the others would immediately start responding with the ~200ms response times. This leads me to believe Serverless Cloud does container-based functions as well, which makes sense given how the code is written.

Summary - There is a little variance when it comes to the performance of the API produced by these frameworks. Cold starts are a thing across all three and average latency is pretty close. Klotho allows you to tune the functions, but the others do not. It makes you wonder if that's something that should be managed by the framework itself.

Visibility

When I talk about visibility in regard to IfC, I'm looking for a simple thing: do I know what I have deployed in the cloud and can I support it?

Nitric - Since Nitric uses Pulumi under the covers, they have a great visualization of the resources that were deployed into your AWS account, how they relate to each other, and a link straight to every resource in the AWS console. They have both a list and a graph representation of the deployed infrastructure.

With regard to supporting your app long term, the graph view will probably be your best bet. It shows relationships between deployed resources and each node is a link directly to it in the console. There are no logs or aggregate views of the infrastructure from within Nitric, so it's up to you to extend the capabilities with an observability tool or head over to your cloud vendor console.

Klotho - One of the big upsides to using Klotho is that is automatically generates an architecture diagram when it compiles your code. You can see at a glance which resources will be deployed to your account and how they interact with each other. Again, Pulumi is used here so if you navigate to the Pulumi dashboard, you get a view similar to Nitric's for your resources.

While the diagram is nice, it's a long stretch from relying on the framework for supportability. Since Klotho does not attempt to manage your infrastructure or deployments I don't give it assessment points beyond the cool architecture diagram.

Serverless Cloud - This one really threw me for a loop. The other frameworks compile code and deploy it into your account, but not Serverless Cloud. It abstracts resources completely from you and hosts it in an account they own. You are presented with a management console that shows logs and API endpoints.

So from a "what infrastructure am I running" perspective, it's impossible to tell. But it does surface the values and metrics that are important. This approach to visibility forces you to change your mindset around IfC. It makes you focus on the code, not the infrastructure behind it. This is the only framework that takes infrastructure management as a service seriously.

Summary - Again, we have three very different layers of abstraction here - one for each comfort level you may have. If you want to keep the feeling of control of your infrastructure, Klotho might be a good choice. If you want to bridge the gap, Nitric is a good segue into handing over the keys. If you're ready to dive into a service managing your infrastructure for you, Serverless Cloud is your best bet.

Overall Thoughts

This was not a cut and dry experiment. These frameworks are so different from each other it is impossible to say which one is "the best". Here is a quick summary of what we covered above.

This process has taught me some valuable insights that build on my first impressions.

• You need a mindset change - This is an entirely new level of abstraction
• IfC is about managing infrastructure - It is not simply compiling code into IaC and handing it back to you
• There is a tool for every comfort level - Those who aren't ready to let go of control of infrastructure can still participate
• Cloud constraints don't go away - You still have cold starts, service limits, and retries to consider
• We have a long way to go - What is out there is a great start, but we are a long way away before self-optimizing infrastructure and general acceptance of the premise

Personally, I'm sold on the concept. I think it's the way cloud development will be done in the future. We're still in the infancy stage of this whole concept, so there are many decisions to be made, end users to talk to, abstractions to be made, and frameworks to be built. But the concept is solid and holds a great promise.

In case you missed it, it was recently announced that Serverless Cloud is spinning off of Serverless, Inc and becoming Ampt. This is a huge development for the IfC space, allowing the brilliant engineers behind Serverless Cloud to build without competing with the other priorities of Serverless, Inc.

The future, and the present for that matter, is ripe with opportunity to change the way we build in the cloud. Now is the right time to get involved and try it out yourself. Every framework I tested was more than happy to work with me, receive feedback, and make changes quickly. They are out there to help you build.

Happy coding!