How to Automate Creating and Destroying Pull Request Review Phoenix Applications on Fly.io

Blog Post Image
This guide explains how to automate the process of creating and destroying Phoenix applications for pull request reviews on Fly.io.

Introduction

As developers, we understand the importance of code review in ensuring the quality of our code. However, when we create new pull requests, reviewers sometimes need to run the app locally to see the changes. This makes it impossible for non-developers to review the work.

One solution is to have each developer manually create an app on Fly.io for each pull request they make. However, this process takes time and developers often forget to remove the apps when they finish working on the pull request.

Fly.io is a platform that allows developers to easily create and destroy review applications for each pull request. The platform has a GitHub action that makes it easy to automate the whole process. It can be found here: https://github.com/superfly/fly-pr-review-apps. In this post, we are going to learn how to automate this process.

While you can use the action as-is, we forked and made some improvements on it, which I will discuss in this post.

Optimum BH’s GitHub Action

To better suit our use case, we made several improvements to our fork of Fly.io's GitHub action:
  • We now create databases and volumes only for apps that require them.
  • When an app is destroyed, we also destroy any associated resources (databases and volumes).
  • We can import any runtime secrets that our Phoenix app requires by using the secrets keyword. Read along to learn how to do this.

To determine if an app requires a database, we wrote a script that searches for the migrate script in the app's source code. This script is typically found in the rel/overlays/bin directory. If the migrate script is found, the action will create and attach a database to the app. The APP and APP_DB variables, which represent the app's name and database name respectively, are declared elsewhere in the GitHub action’s source code.
if [ -e "rel/overlays/bin/migrate" ]; then
  # only create db if the app launched successfully
  if flyctl status --app "$APP"; then
    if flyctl status --app "$APP_DB"; then
      echo "$APP_DB DB already exists"
    else
      flyctl postgres create --name "$APP_DB" --org "$ORG" --region "$REGION" --vm-size shared-cpu-1x --initial-cluster-size 4 --volume-size 1
    fi
    # attach db to the app if it was created successfully
    if flyctl postgres attach "$APP_DB" --app "$APP" -y; then
      echo "$APP_DB DB attached"
    else
      echo "Error attaching $APP_DB to $APP, attachments exist"
    fi
  fi
fi

To determine if the app requires volumes, the script below looks for [mounts] in the config file.
if grep -q "\[mounts\]" fly.toml; then
  # create volume only if none exists
  if ! flyctl volumes list --app "$APP" | grep -oh "\w*vol_\w*"; then
    flyctl volumes create "$VOLUME" --app "$APP" --region "$REGION" --size 1 -y
  fi
  # modify config file to have the volume name specified above.
  sed -i -e 's/source =.*/source = '\"$VOLUME\"'/' "$CONFIG"
fi

First, we need to check if the app already has a volume. If we do not perform this check, multiple volumes will be created. While this is not necessarily problematic, it is wasteful. Also, we need to modify the config file to include the new volume name. If we neglect this step, the deployment will fail.

Automating the Creation of Review Applications

Now, here's an example of how we can set up a workflow to automatically create a review application for each pull request.

For the workflow to work, you need to put FLY_API_TOKEN , generated by running flyctl auth token under GitHub repository or organization secrets.
name: Review App
on:
  pull_request:
    types: [opened, reopened, synchronize]
env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
  FLY_ORG: Personal
  FLY_REGION: jn
  REPO_NAME: sample-app
jobs:
  deploy_review_app:
    name: Create & Deploy Review App
    runs-on: ubuntu-latest
    # Only run one deployment at a time per PR.
    concurrency:
      group: pr-${{ github.event.number }}
    # Create a GitHub deployment environment per review app
    # so it shows up in the pull request UI.
    environment:
      name: pr-${{ github.event.number }}
      url: https://pr-${{ github.event.number }}.${{ env.REPO_NAME }}.fly.dev
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Create & Deploy Review App
        id: deploy
        uses: optimumBA/fly-preview-apps@main
        with:
          name: pr-${{ github.event.number }}-${{ env.REPO_NAME }}

You can import any secrets that your Phoenix application requires to run. After adding the secrets to your application's GitHub repository, you can access them in your workflow using secrets keyword.
- name: Create & Deploy Review App
  id: deploy
  uses: optimumBA/fly-preview-apps@main
  with:
    name: pr-${{ github.event.number }}
    secrets: 'SECRET_1=${{ secrets.YOUR_SECRET_1 }} SECRET_2=${{ secrets.SECRET_2 }}\nSECRET_n=${{ secrets.SECRET_n }}'

For every successful deployment, GitHub actions will set environment and deployment variable variables, pointing to the name and url of the deployed review application. You can find them under Environments tab in your application’s GitHub repository.

Automating the Destruction of Review Applications

After the pull request has been merged or closed, the review application is no longer needed. Here is an example of a workflow that automatically destroys the review application:
name: Delete Review App
on:
  pull_request:
    types:
      - closed
env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
  REPO_NAME: sample-app
jobs:
  delete_review_app:
    runs-on: ubuntu-latest
    name: Delete Review App
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Delete Review Deployment
        uses: optimumBA/fly-preview-apps@main
        with:
          name: pr-${{ github.event.number }}-${{ env.REPO_NAME }}

Bonus

For every deployment, the workflow also creates environments on GitHub. These environments display the name and live link of the deployed review app. It is important to note that the workflow we created to delete the deployments once the pull request is closed only destroys resources on Fly.io, and does not remove the GitHub environments.

To remove these GitHub environments, we will extend our workflow using third-party GitHub actions. These actions require an auth token in order to delete the environments. The available GitHub token (available as `secrets.GITHUB_TOKEN` in the workflow) does not have enough permissions to delete GitHub environments. To proceed, we need to create a GitHub app and grant it the following permissions, under repository permissions:
  • Actions: Read
  • Administration: Read & Write
  • Deployments: Read & Write
  • Environments: Read & Write
  • Metadata: Read
Read more on these permissions on https://docs.github.com/en/rest/overview/permissions-required-for-github-apps?apiVersion=2022-11-28, and steps to create a GitHub app on https://docs.github.com/en/apps/creating-github-apps/creating-github-apps/creating-a-github-app.

Please refer to the respective documentation of these GitHub actions for more information on additional setup steps:
Here is a complete workflow that deletes both deployments and GitHub environments:
name: Delete Review App
on:
  pull_request:
    types:
      - closed
env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
  REPO_NAME: sample-app
jobs:
  delete_review_app:
    runs-on: ubuntu-latest
    name: Delete Review App
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Delete Review Deployment
        uses: optimumBA/fly-preview-apps@main
        with:
          name: pr-${{ github.event.number }}.${{ env.REPO_NAME }}
      - name: Get Token
        uses: navikt/github-app-token-generator@v1.1.1
        id: get-token
        with:
          app-id: ${{ secrets.GH_APP_ID }}
          private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
      - name: Delete GitHub Environments
        uses: strumwolf/delete-deployment-environment@v2.2.3
        with:
          token: ${{ steps.get-token.outputs.token }}
          environment: pr-${{ github.event.number }}
          ref: ${{ github.head_ref }}

Conclusion

Using pull request review applications can greatly improve the efficiency of the code review process. By automating the creation and destruction of these applications, we can save time and ensure that our code is thoroughly reviewed before it is merged into our codebase. This approach also saves on resources. 
With the help of GitHub Actions and Fly.io, automating this process is easy and straightforward.
Post Image

Maintaining GitHub Actions workflows

Whenever we have a choice to make for a CI system to use on a project, we pick GitHub Actions, mainly for convenience. Our code is already hosted on GitHub, and it doesn’t make sense to introduce other tools unnecessarily, so some time ago we started using GitHub Actions as our CI provider. Over the years we've been constantly improving our development workflows thereby adding more complexity to our CI. There were many steps in our pipeline for various code checks, Elixir tests, etc., each increasing the time we needed to wait to make sure our code was good to go. So we’d wait 5 to 10 minutes or so just to find out the code wasn’t formatted properly, there was some compiler warning or something trivial as that. We knew there were better ways to set up our CI, but we felt separating the workflows into separate jobs was going to make for a harder-to-maintain code because GitHub Actions does not support full YAML syntax. I came to Elixir from the Ruby on Rails community where YAML is a default for any kind of configuration, so I was excited to see GitHub Actions using YAML for the workflow definitions. I quickly came to realize it’s not the same YAML I was used to (You’ve changed, bro). Specifically, I couldn’t use anchors which provide the ability to write reusable code in .yml files.  Our way of working around this is writing workflow definitions in Elixir and translating them to YAML, letting us benefit from the beautiful Elixir syntax in sharing variables, steps, and jobs between workflows while still, as a result, having workflow files in the YAML format GitHub Actions supports. To convert the workflow definitions from Elixir to YAML, we wrote a CLI script that uses fast_yaml library with a small amount of code wrapping it up in an easy-to-use package. We used this script internally for years, but now we’ve decided to share it with the community. I’ve had some troubles distributing the script. Usually, we’d execute .exs script to convert the workflow, so I wanted to build an escript, but fast_yaml library contains NIFs that don’t work with it. I liked the way Phoenix is installed so I tried adopting that approach, creating a mix project containing a task, then archiving the project into a .ez file, only to find out that when it gets installed, it doesn’t contain any dependencies. This can be alleviated using burrito or bakeware, but they introduce more complexity, and I didn’t like the way error messages were displayed in the Terminal, so I ended up with a hex package that’s added to an Elixir project in a usual way. Ultimately, I didn’t plan to use the script outside of Elixir projects, so that’s a compromise I was willing to make. If at a later point I feel the need, I’ll distribute it some other way, which will deserve another blog post. Anyway, here’s how you can use this mix task. Add the github_workflows_generator package as a dependency to your mix.exs file:defp deps do [ {:github_workflows_generator, "~> 0.1"} ] end You most likely don’t want to use it in runtime and environments other than dev, so you might find this more appropriate:defp deps do [ {:github_workflows_generator, "~> 0.1", only: :dev, runtime: false} ] end That will let you execute mix github_workflows.generate command that given a .github/github_workflows.ex file like this one: defmodule GithubWorkflows do def get do %{ "main.yml" => main_workflow(), "pr.yml" => pr_workflow() } end defp main_workflow do [ [ name: "Main", on: [ push: [ branches: ["main"] ] ], jobs: [ test: test_job(), deploy: [ name: "Deploy", needs: :test, steps: [ checkout_step(), [ name: "Deploy", run: "make deploy" ] ] ] ] ] ] end defp pr_workflow do [ [ name: "PR", on: [ pull_request: [ branches: ["main"] ] ], jobs: [ test: test_job() ] ] ] end defp test_job do [ name: "Test", steps: [ checkout_step(), [ name: "Run tests", run: "make test" ] ] ] end defp checkout_step do [ name: "Checkout", uses: "actions/checkout@v4" ] end endcreates multiple files in the .github/workflows directory.  main.ymlname: Main on: push: branches: - main jobs: test: name: Test steps: - name: Checkout uses: actions/checkout@v4 - name: Run tests run: make test deploy: name: Deploy needs: test steps: - name: Checkout uses: actions/checkout@v4 - name: Deploy run: make deploy pr.yml name: PR on: pull_request: branches: - main jobs: test: name: Test steps: - name: Checkout uses: actions/checkout@v4 - name: Run tests run: make test Path to the source file and the output directory can be customized. To see available options, run mix help github_workflows.generate You might also want to read the documentation or check out the source code.  The generator’s repo contains its own CI workflows (something something-ception) that show how useful the command is in complex scenarios.defmodule GithubWorkflows do @moduledoc false def get do %{ "ci.yml" => ci_workflow() } end defp ci_workflow do [ [ name: "CI", on: [ pull_request: [], push: [ branches: ["main"] ] ], jobs: [ compile: compile_job(), credo: credo_job(), deps_audit: deps_audit_job(), dialyzer: dialyzer_job(), format: format_job(), hex_audit: hex_audit_job(), prettier: prettier_job(), test: test_job(), unused_deps: unused_deps_job() ] ] ] end defp compile_job do elixir_job("Install deps and compile", steps: [ [ name: "Install Elixir dependencies", env: [MIX_ENV: "test"], run: "mix deps.get" ], [ name: "Compile", env: [MIX_ENV: "test"], run: "mix compile" ] ] ) end defp credo_job do elixir_job("Credo", needs: :compile, steps: [ [ name: "Check code style", env: [MIX_ENV: "test"], run: "mix credo --strict" ] ] ) end # Removed for brevity # ... defp elixir_job(name, opts) do needs = Keyword.get(opts, :needs) steps = Keyword.get(opts, :steps, []) job = [ name: name, "runs-on": "${{ matrix.versions.runner-image }}", strategy: [ "fail-fast": false, matrix: [ versions: [ %{ elixir: "1.11", otp: "21.3", "runner-image": "ubuntu-20.04" }, %{ elixir: "1.16", otp: "26.2", "runner-image": "ubuntu-latest" } ] ] ], steps: [ checkout_step(), [ name: "Set up Elixir", uses: "erlef/setup-beam@v1", with: [ "elixir-version": "${{ matrix.versions.elixir }}", "otp-version": "${{ matrix.versions.otp }}" ] ], [ uses: "actions/cache@v3", with: [ path: ~S""" _build deps """ ] ++ cache_opts(prefix: "mix-${{ matrix.versions.runner-image }}") ] ] ++ steps ] if needs do Keyword.put(job, :needs, needs) else job end end # Removed for brevity # ... end That creates a YAML file I wouldn’t want to look at, much less maintain it, but enables us to have this CI pipeline CI pipeline with jobs running in parallel Our phx.tools project has an even better example with 3 different workflows. Workflow executed on push to the main branch Workflow executed when PR gets created and synchronized Cleanup workflow when PR gets merged or closed Let’s step back to see how the script works. The only rule that we enforce is that the source file must contain a GithubWorkflows module with a get/0 function that returns a map of workflows in which keys are filenames and values are workflow definitions.  defmodule GithubWorkflows do def get do %{ "ci.yml" => [[ name: "Main", on: [ push: [] ], jobs: [] ]] } end end Everything else is up to you.  When you look at the generated .yml files, they might not look exactly the same as if you wrote them by hand. For example, if you were to add caching for Elixir dependencies as in actions/cache code samples, you’d want to have this YAML code:- uses: actions/cache@v3 with: path: | deps _build key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }} restore-keys: | ${{ runner.os }}-mix-with two paths passed without quotes.  I haven’t found a way to tell the YAML encoder to format it that way, so my workaround is to use a sigil that preserves the newline, so that[ uses: "actions/cache@v3", with: [ path: ~S""" _build deps """ ], key: "${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}" restore-keys: ~S""" ${{ runner.os }}-mix- """ ]gets converted to- uses: actions/cache@v3 with: path: "_build\ndeps" key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }} restore-keys: ${{ runner.os }}-mix-In our workflows, you may notice some new ideas not seen elsewhere, so be sure to look out for more posts on our blog in a new series where we’ll unpack our unique DevOps practices. If you have any questions, you can contact us at blog@optimum.ba and we’ll try to answer them in our future posts. If our approach to software development resonates with you and you're ready to kickstart your project, drop us an email at projects@optimum.ba. Share your project requirements and budget, and we'll promptly conduct a review. We'll then schedule a call to dive deeper into your needs. Let's bring your vision to life!

Almir Sarajčić

Post Image

phx.tools: Complete Development Environment for Elixir and Phoenix

Elixir is a powerful functional programming language that has been attracting the attention of developers from different backgrounds since its release. Many of its new users already have experience with tools such as Homebrew and asdf, which makes the installation process smoother. However, setting up the development environment for Phoenix applications can still be a challenge, especially for new developers.  Past few years, the Elixir ecosystem has become more approachable to new developers. The learning curve is flattening every year with the introduction of tools like Livebook. It’s a great way to start with Elixir, as the installation is straightforward. What’s still missing is a complete setup for the development of Phoenix apps.  At Optimum BH, we've seen the potential of the Phoenix and Elixir stack, and have been working with it for some time now. Our team has had several interns who were new to both Elixir and programming, and we've noticed that the process of setting up the development environment can be demotivating for these newcomers.  The Ruby on Rails community has rails.new. It’s a complete development environment containing everything you need to start a new Rails application. We believe that Phoenix and Elixir ecosystem can benefit from something similar.  So, let me introduce you to phx.tools. It’s a shell script for platforms Linux and macOS (sorry, Windows users) that configures the development environment for you in a few easy steps. Once you finish running the script, you'll be able to start the database server, create a new Phoenix application, and launch the server.    To get started, visit phx.tools and follow the instructions for your platform. Happy coding!

Almir Sarajčić