Skip to main content

Infrastructure as code with Pulumi

What youโ€™ll learn
  • what is Pulumi
  • how is Webiny utilizing Pulumi with your project applications
  • what are state files
  • what are backends

Why Pulumi?#

For defining and deploying necessary cloud infrastructure for your project applications, by default, Webiny relies on Pulumi, a modern infrastructure as code (IaC) solution.

Webiny chose Pulumi as its default IaC solution for a couple of reasons:

  • cloud infrastructure is defined via code (TypeScript)
  • it supports multiple cloud providers (AWS, Azure, Google Cloud Platform, ...)
  • it provides multiple solutions for storing infrastructure state
  • a vibrant and rising community
info

Note that Pulumi is the default solution Webiny relies on, which means it doesn't have to be the only one. If you have a need to bring your own IaC solution (like CloudFormation or Terraform), you certainly can. Feel free to message us if that's the case, we'd be glad to assist.

A Closer Look#

In the following folder structure, we are looking at three applications that are set up by default in a new Webiny project. These are the API (./api), Admin Area (./apps/admin), and Website (./apps/website), which are shown in the following directory structure (code folder omitted for brevity):

.
โ”œโ”€โ”€ api
โ”‚ย ย  โ”œโ”€โ”€ pulumi
โ”‚ย ย  โ”œโ”€โ”€ Pulumi.yaml
โ”‚ย ย  โ”œโ”€โ”€ Pulumi.dev.yaml
โ”‚ย ย  โ””โ”€โ”€ index.ts
โ”œโ”€โ”€ apps
โ”‚ย ย  โ”œโ”€โ”€ admin
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ pulumi
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ Pulumi.yaml
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ Pulumi.dev.yaml
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ index.ts
โ”‚ย ย  โ””โ”€โ”€ website
โ”‚ย ย  ย ย  โ”œโ”€โ”€ pulumi
โ”‚ย ย  ย ย  โ”œโ”€โ”€ Pulumi.dev.yaml
โ”‚ย ย  ย ย  โ”œโ”€โ”€ Pulumi.yaml
โ”‚ย ย  ย ย  โ””โ”€โ”€ index.ts
โ”‚ย ย 
โ””โ”€โ”€ (...)

Let's examine how these applications are utilizing Pulumi in order to define and deploy needed cloud infrastructure resources.

info

Three key Pulumi related concepts, that we're using in the following text, are: Pulumi project, program, and stack. Check out their Programming Model article to learn more.

Each of these applications are standalone Pulumi projects, which essentially means two things:

  • every application contains its own infrastructure-related code (Pulumi program), configuration, meta, and state files
  • every application is deployed separately (you cannot deploy two applications in a single operation)
info

Note that, by default, Webiny CLI does provide a convenient deploy method which deploys multiple project applications for you. And although it may seem you are completely deploying your Webiny project with a single command, internally, the command is just deploying project applications, one by one.

Read more about the deploy command in the Deploy your project guide.

It's also important to know you can have multiple instances of your application's cloud infrastructure up and running at the same time. This is achieved via Pulumi stacks:

Every Pulumi program is deployed to a stack. A stack is an isolated, independently configurable instance of a Pulumi program. Stacks are commonly used to denote different phases of development (such as development, staging and production) or feature branches (such as feature-x-dev, jane-feature-x-dev).

In the following section, we briefly cover the key files that form a Pulumi project, that can be found in your application folders. Note that if you want to dive deeper, you're certainly encouraged to visit the official documentation which explains these concepts in more detail.

Project Files#

The following are the key files of every Pulumi project.

Pulumi.yaml#

The Pulumi.yaml project file specifies metadata about the Pulumi project. There can only be one project file per Pulumi project.

A Pulumi.yaml file might look something like the following:

name: api
runtime: nodejs
description: Cloud infrastructure needed by your project's (GraphQL) API.
backend:
url: file://

One of the more important properties in this file is the backend property, which we cover in the following section.

Pulumi.{stack-name}.yaml#

The Pulumi.{stack-name}.yaml file is a Pulumi stack settings file. There can be more than one stack settings files, for example, for a couple of different environments. For example, it's possible to have Pulumi.dev.yaml, Pulumi.staging.yaml, Pulumi.production.yaml, and so on.

info

Environments are explained in more detail in the following Environments key topic.

index.ts#

This is the entry point file in which you start to define your infrastructure (the Pulumi program). You can organize the code in any way you like. For example, in the default API, Admin Area, and Public Website applications, the index.ts file just imports different classes, which encapsulate different cloud infrastructure segments. These classes can be found in the pulumi folder.

State Files#

Except for the files we mentioned in the previous section, there is also one other group of files the Pulumi solution produces. These are Pulumi project's state files.

These files describe at what state your cloud infrastructure is currently at. For example, it contains a list of all deployed resources, various metadata, configuration, and so on. State files are stored per Pulumi program stack, and by default, they are stored in the .pulumi folder, located in your project root.

.pulumi (folder located in your project root)
.
โ”œโ”€โ”€ api
โ”‚ย ย  โ”œโ”€โ”€ .pulumi
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ backups
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ history
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ stacks
โ”œโ”€โ”€ apps
โ”‚ย ย  โ”œโ”€โ”€ admin
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ .pulumi
โ”‚ย ย  โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ backups
โ”‚ย ย  โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ history
โ”‚ย ย  โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ stacks
โ”‚ย ย  โ””โ”€โ”€ website
โ”‚ย ย  ย ย  โ”œโ”€โ”€ .pulumi
โ”‚ย ย  ย ย  โ”‚ย ย  โ”œโ”€โ”€ backups
โ”‚ย ย  ย ย  โ”‚ย ย  โ”œโ”€โ”€ history
โ”‚ย ย  ย ย  โ”‚ย ย  โ””โ”€โ”€ stacks
โ”‚ย ย 
โ””โ”€โ”€ (...)

Notice how state files are organized per project application:

  • api - API project application's cloud infrastructure state files
  • apps/admin - Admin Area project application's cloud infrastructure state files
  • apps/website - Website project application's cloud infrastructure state files
tip

Learn more about how to store cloud in state in CI/CD in the Cloud Infrastructure State Files key topic.

Different Backends#

As mentioned, by default, Pulumi state files are stored in your project application folder, inside the .pulumi folder. While in some cases, storing them locally may work for you, in others, you may want to store these in a controlled, centralized, remote storage.

This is where the concept of Pulumi backends comes into play, which represent different storage services, that you can use for storing your state files.

For example, you can create a simple S3 bucket, and choose to store your files in it. You can also use the Pulumi Service, which, in addition to storing state files, also provides a couple of other cool features.

Read more about different backends in the following official documentation article.

FAQ#

Are you using Pulumi's Automation API?

Currently, Webiny is not using the Automation API. It's actually using its own Pulumi SDK package, which is just a tiny wrapper over Pulumi CLI. Essentially, it enables programmatic use of the Pulumi CLI, which is similar to what the Automation API is also doing.

Is it possible to use a different infrastructure-as-code solution?

Yes, you can certainly use a different solution (for example CloudFormation or Terraform). Feel free to message us if you have this type of need. We'd be glad to assist in that regard.

What is stack output?

Stack output represents values that are exported from your Pulumi program. For example, at the end of your index.ts file (located in root of your project application folder), you might have something like the following:

export const cognitoUserPoolId = cognito.userPool.id;
export const cognitoAppClientId = cognito.userPoolClient.id;
export const updatePbSettingsFunction = pageBuilder.functions.updateSettings.arn;

Doing this can be useful if, for example, one, or more applications depend on these values.

Is it possible to read stack output from another stack?

It's possible to read output from another stack. This is definitely useful if one or more of your current stack's infrastructure resources are depending on one or more resources from another stack. To learn more, check out the Reading stack output article.

Last updated on by Pavel Denisjuk