Search…
End-to-End Testing
How to integrate third-party CI platforms with Release to do end-to-end testing.
If you have end-to-end tests set up on a platform like GitHub actions or CircleCI, you can configure these tests to run in an ephemeral Release environment. The flow will be:
  1. 1.
    Create a new Release environment
  2. 2.
    Run the E2E tests in this new environment
  3. 3.
    Destroy the environment
This ensures that your E2E tests run on a 'clean slate', which gives you confidence that they do not rely on any unknown state that might exist in a long-running environment, and also reduces costs (because the environment is destroyed as soon as it's no longer needed).

Configuring GitHub actions for E2E testing with Release

To integrate Release with GitHub actions and run end-to-end tests, you'll probably want to structure your GitHub Action main.yml file into four jobs.
  1. 1.
    Build and upload the image
  2. 2.
    Create Release environment
  3. 3.
    Run E2E tests
  4. 4.
    Delete Release environment
We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file as part of our example React, Express, and MongoDB application on GitHub.

Building and uploading your tests as a Docker image

Our first job, build-and-upload-image, should build the image of our application to test, or (as in the example below), fetch these images from a registry. In our example, we have a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use the latest Ubuntu image from GitHub for this.
We also have some standard GitHub Actions boilerplate to define the credentials we need and define the trigger.
1
name: GitHub Actions E2E Testing Demo
2
3
on:
4
push:
5
branches:
6
- demo/github-actions
7
8
env:
9
RELEASE_ACCOUNT_ID: ${{ secrets.RELEASE_ACCOUNT_ID }}
10
RELEASE_APP_ID: ${{ secrets.RELEASE_APP_ID }}
11
RELEASE_LOGIN: ${{ secrets.RELEASE_LOGIN }}
12
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
13
14
jobs:
15
build_and_upload:
16
name: Build and upload docker image
17
runs-on: ubuntu-latest
18
steps:
19
- shell: bash
20
run: |
21
echo "building docker image..."
22
echo "uploading docker image..."
23
mkdir tmp
24
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
25
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
26
- name: upload image name
27
uses: actions/upload-[email protected]
28
with:
29
name: images
30
path: tmp/
Copied!

Creating the Release Environment

The next job we define is create_environment, and this is where most of the Release-specific configuration needs to be. We pull a Docker image that has the Release CLI and use it to create a new Release Environment. We use the Release Account ID and Release Application ID defined earlier, pulled from our GitHub Secrets.
1
create_environment:
2
name: Create release environment
3
needs: build_and_upload
4
runs-on: ubuntu-latest
5
container: public.ecr.aws/b4g8c3s2/release-cli
6
steps:
7
- name: download artifacts
8
uses: actions/download-[email protected]
9
with:
10
name: images
11
- name: create release environment
12
shell: bash
13
run: |
14
BRANCH=e2e-testing
15
FRONTEND_IMAGE=$(cat frontend_image.txt)
16
BACKEND_IMAGE=$(cat backend_image.txt)
17
FRONTEND=frontend
18
BACKEND=backend
19
release environments create \
20
--account "$RELEASE_ACCOUNT_ID" \
21
--app "$RELEASE_APP_ID" \
22
--branch "$BRANCH" \
23
--image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
24
--image-overrides "$BACKEND=$BACKEND_IMAGE" \
25
--output json \
26
--wait > res.json
27
- name: upload image name
28
uses: actions/upload-[email protected]
29
with:
30
name: json
31
path: res.json
Copied!

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run_e2e_tests. It should look as follows.
1
run_e2e_tests:
2
name: run e2e tests
3
needs: create_environment
4
runs-on: ubuntu-latest
5
steps:
6
- name: checkout repository
7
uses: actions/[email protected]
8
- name: download artifacts
9
uses: actions/download-[email protected]
10
with:
11
name: json
12
- name: create release environment
13
shell: bash
14
run: |
15
echo "Install dependencies and run E2E tests..."
16
FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' res.json)
17
BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' res.json)
18
jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
19
jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
20
sudo apt-get update
21
sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
22
npm install
23
npm run cy:run
Copied!
Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the Release Environment

Our last job is to clean up the Release Environment once the tests have run. We'll call the job delete_environment, and once again it'll use the Release CLI image and need access to your Release Account ID, App ID, and the Environment ID that was created in the previous step.
1
delete_environment:
2
name: Delete release environment
3
needs: run_e2e_tests
4
runs-on: ubuntu-latest
5
container: public.ecr.aws/b4g8c3s2/release-cli
6
steps:
7
- name: checkout repository
8
uses: actions/[email protected]
9
- name: download artifacts
10
uses: actions/download-[email protected]
11
with:
12
name: json
13
- name: delete release environment
14
shell: bash
15
run: |
16
echo "Delete the Release environment..."
17
ENVIRONMENT_ID=$(jq -r '.environment.id' res.json)
18
release environments delete "$ENVIRONMENT_ID" \
19
--account "$RELEASE_ACCOUNT_ID" \
20
--app "$RELEASE_APP_ID"
Copied!

Configuring CircleCI for E2E testing with Release

To integrate Release with CircleCI and run end-to-end tests, you'll probably want to structure your CircleCI config.yml file into four jobs.
  1. 1.
    Build and upload the image
  2. 2.
    Create Release environment
  3. 3.
    Run E2E tests
  4. 4.
    Delete Release environment
We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file as part of our example React, Express, and MongoDB application on GitHub.

Building and uploading your tests as a Docker image

Our first job, build-and-upload-image, should build the image of our application to test, or (as in the example below), fetch these images from a registry. In our example, we have a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use a Node image from CircleCI to run this.
We also have some standard CircleCI boilerplate to specify the target version.
1
version: 2.1
2
3
jobs:
4
# In this example, we are not actually building and pushing docker image
5
build-and-upload-image:
6
docker:
7
- image: circleci/node:13.8.0
8
steps:
9
- run:
10
name: Build and push docker image
11
command: |
12
mkdir tmp
13
echo "Building docker image..."
14
echo "Push docker image..."
15
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
16
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
17
- persist_to_workspace:
18
root: .
19
paths:
20
- tmp/*
Copied!

Creating the Release Environment

The next job we define is create-release-environment, and this is where most of the Release-specific configuration needs to be. We pull a Docker image that has the Release CLI and use it to create a new Release Environment. For this to work, you'll need to make sure your Release Account ID and Release Application ID are in the appropriate environment variables.
1
create-release-environment:
2
docker:
3
- image: public.ecr.aws/b4g8c3s2/release-cli
4
steps:
5
- attach_workspace:
6
at: ./
7
- run:
8
name: Create a new Release environment
9
command: |
10
BRANCH=demo/circleci
11
FRONTEND_IMAGE=$(cat tmp/frontend_image.txt)
12
BACKEND_IMAGE=$(cat tmp/backend_image.txt)
13
FRONTEND=frontend
14
BACKEND=backend
15
release environments create \
16
--account "$RELEASE_ACCOUNT_ID" \
17
--app "$RELEASE_APP_ID" \
18
--branch "$BRANCH" \
19
--image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
20
--image-overrides "$BACKEND=$BACKEND_IMAGE" \
21
--output json \
22
--wait > tmp/res.json
23
- persist_to_workspace:
24
root: .
25
paths:
26
- tmp/*
Copied!

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run-e2e-tests. It should look as follows.
1
run-e2e-tests:
2
docker:
3
- image: circleci/node:13.8.0
4
steps:
5
- checkout
6
- attach_workspace:
7
at: ./
8
- run:
9
name: Run E2E Tests
10
command: |
11
echo "Install dependencies and run E2E tests..."
12
FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' tmp/res.json)
13
BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' tmp/res.json)
14
jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
15
jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
16
sudo apt-get update
17
sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
18
npm install
19
npm run cy:run
Copied!
Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the Release Environment

Our last job is to clean up the Release Environment once the tests have run. We'll call the job delete-release-environment, and once again it'll use the Release CLI image and need access to your Release Account ID, App ID, and the Environment ID that was created in the previous step.
1
delete-release-environment:
2
docker:
3
- image: public.ecr.aws/b4g8c3s2/release-cli
4
steps:
5
- attach_workspace:
6
at: ./
7
- run:
8
name: Delete Release Environment
9
command: |
10
ENVIRONMENT_ID=$(jq -r '.environment.id' tmp/res.json)
11
release environments delete "$ENVIRONMENT_ID" \
12
--account "$RELEASE_ACCOUNT_ID" \
13
--app "$RELEASE_APP_ID"
Copied!
Finally, we connect all of the jobs into a workflow as follows.
1
workflows:
2
my_workflow:
3
jobs:
4
- build-and-upload-image:
5
filters:
6
branches:
7
only: demo/circleci
8
- create-release-environment:
9
requires: [build-and-upload-image]
10
filters:
11
branches:
12
only: demo/circleci
13
- run-e2e-tests:
14
requires: [create-release-environment]
15
filters:
16
branches:
17
only: demo/circleci
18
- delete-release-environment:
19
requires: [run-e2e-tests]
20
filters:
21
branches:
22
only: demo/circleci
Copied!