Search…
End-to-End testing
How to integrate third-party CI platforms with ReleaseHub to do end-to-end testing.
If you have end-to-end tests set up on a platform like GitHub actions or CircleCI, you can configure these tests to run in an ephemeral ReleaseHub environment. The flow will be:
  1. 1.
    Create a new ReleaseHub environment
  2. 2.
    Run the E2E tests in this new environment
  3. 3.
    Destroy the environment
This ensures that your E2E tests run on a 'clean slate', which gives you confidence that they do not rely on any unknown state that might exist in a long-running environment, and also reduces costs (because the environment is destroyed as soon as it's no longer needed).

Configuring GitHub actions for E2E testing with ReleaseHub

To integrate ReleaseHub with GitHub actions and run end-to-end tests, you'll probably want to structure your GitHub Action main.yml file into four jobs.
  1. 1.
    Build and upload the image
  2. 2.
    Create ReleaseHub environment
  3. 3.
    Run E2E tests
  4. 4.
    Delete ReleaseHub environment
We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file as part of our example React, Express, and MongoDB application on GitHub.

Building and uploading your tests as a Docker image

Our first job, build-and-upload-image, should build the image of our application to test, or (as in the example below), fetch these images from a registry. In our example, we have a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use the latest Ubuntu image from GitHub for this.
We also have some standard GitHub Actions boilerplate to define the credentials we need and define the trigger.
name: GitHub Actions E2E Testing Demo
on:
push:
branches:
- demo/github-actions
env:
RELEASE_ACCOUNT_ID: ${{ secrets.RELEASE_ACCOUNT_ID }}
RELEASE_APP_ID: ${{ secrets.RELEASE_APP_ID }}
RELEASE_LOGIN: ${{ secrets.RELEASE_LOGIN }}
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
jobs:
build_and_upload:
name: Build and upload docker image
runs-on: ubuntu-latest
steps:
- shell: bash
run: |
echo "building docker image..."
echo "uploading docker image..."
mkdir tmp
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
- name: upload image name
uses: actions/upload-[email protected]
with:
name: images
path: tmp/

Creating the ReleaseHub Environment

The next job we define is create_environment, and this is where most of the ReleaseHub-specific configuration needs to be. We pull a Docker image that has the ReleaseHub CLI and use it to create a new ReleaseHub Environment. We use the ReleaseHub Account ID and ReleaseHub Application ID defined earlier, pulled from our GitHub Secrets.
create_environment:
name: Create release environment
needs: build_and_upload
runs-on: ubuntu-latest
container: public.ecr.aws/b4g8c3s2/release-cli
steps:
- name: download artifacts
uses: actions/download-[email protected]
with:
name: images
- name: create release environment
shell: bash
run: |
BRANCH=e2e-testing
FRONTEND_IMAGE=$(cat frontend_image.txt)
BACKEND_IMAGE=$(cat backend_image.txt)
FRONTEND=frontend
BACKEND=backend
release environments create \
--account "$RELEASE_ACCOUNT_ID" \
--app "$RELEASE_APP_ID" \
--branch "$BRANCH" \
--image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
--image-overrides "$BACKEND=$BACKEND_IMAGE" \
--output json \
--wait > res.json
- name: upload image name
uses: actions/upload-[email protected]
with:
name: json
path: res.json

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run_e2e_tests. It should look as follows.
run_e2e_tests:
name: run e2e tests
needs: create_environment
runs-on: ubuntu-latest
steps:
- name: checkout repository
uses: actions/[email protected]
- name: download artifacts
uses: actions/download-[email protected]
with:
name: json
- name: create release environment
shell: bash
run: |
echo "Install dependencies and run E2E tests..."
FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' res.json)
BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' res.json)
jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
sudo apt-get update
sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
npm install
npm run cy:run
Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the ReleaseHub Environment

Our last job is to clean up the ReleaseHub Environment once the tests have run. We'll call the job delete_environment, and once again it'll use the ReleaseHub CLI image and need access to your ReleaseHub Account ID, App ID, and the Environment ID that was created in the previous step.
delete_environment:
name: Delete release environment
needs: run_e2e_tests
runs-on: ubuntu-latest
container: public.ecr.aws/b4g8c3s2/release-cli
steps:
- name: checkout repository
uses: actions/[email protected]
- name: download artifacts
uses: actions/download-[email protected]
with:
name: json
- name: delete release environment
shell: bash
run: |
echo "Delete the ReleaseHub environment..."
ENVIRONMENT_ID=$(jq -r '.environment.id' res.json)
release environments delete "$ENVIRONMENT_ID" \
--account "$RELEASE_ACCOUNT_ID" \
--app "$RELEASE_APP_ID"

Configuring CircleCI for E2E testing with ReleaseHub

To integrate ReleaseHub with CircleCI and run end-to-end tests, you'll probably want to structure your CircleCI config.yml file into four jobs.
  1. 1.
    Build and upload the image
  2. 2.
    Create ReleaseHub environment
  3. 3.
    Run E2E tests
  4. 4.
    Delete ReleaseHub environment
We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file as part of our example React, Express, and MongoDB application on GitHub.

Building and uploading your tests as a Docker image

Our first job, build-and-upload-image, should build the image of our application to test, or (as in the example below), fetch these images from a registry. In our example, we have a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use a Node image from CircleCI to run this.
We also have some standard CircleCI boilerplate to specify the target version.
version: 2.1
jobs:
# In this example, we are not actually building and pushing docker image
build-and-upload-image:
docker:
- image: circleci/node:13.8.0
steps:
- run:
name: Build and push docker image
command: |
mkdir tmp
echo "Building docker image..."
echo "Push docker image..."
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
- persist_to_workspace:
root: .
paths:
- tmp/*

Creating the ReleaseHub Environment

The next job we define is create-release-environment, and this is where most of the ReleaseHub-specific configuration needs to be. We pull a Docker image that has the ReleaseHub CLI and use it to create a new ReleaseHub Environment. For this to work, you'll need to make sure your ReleaseHub Account ID and ReleaseHub Application ID are in the appropriate environment variables.
create-release-environment:
docker:
- image: public.ecr.aws/b4g8c3s2/release-cli
steps:
- attach_workspace:
at: ./
- run:
name: Create a new ReleaseHub environment
command: |
BRANCH=demo/circleci
FRONTEND_IMAGE=$(cat tmp/frontend_image.txt)
BACKEND_IMAGE=$(cat tmp/backend_image.txt)
FRONTEND=frontend
BACKEND=backend
release environments create \
--account "$RELEASE_ACCOUNT_ID" \
--app "$RELEASE_APP_ID" \
--branch "$BRANCH" \
--image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
--image-overrides "$BACKEND=$BACKEND_IMAGE" \
--output json \
--wait > tmp/res.json
- persist_to_workspace:
root: .
paths:
- tmp/*

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run-e2e-tests. It should look as follows.
run-e2e-tests:
docker:
- image: circleci/node:13.8.0
steps:
- checkout
- attach_workspace:
at: ./
- run:
name: Run E2E Tests
command: |
echo "Install dependencies and run E2E tests..."
FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' tmp/res.json)
BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' tmp/res.json)
jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
sudo apt-get update
sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
npm install
npm run cy:run
Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the ReleaseHub Environment

Our last job is to clean up the ReleaseHub Environment once the tests have run. We'll call the job delete-release-environment, and once again it'll use the ReleaseHub CLI image and need access to your ReleaseHub Account ID, App ID, and the Environment ID that was created in the previous step.
delete-release-environment:
docker:
- image: public.ecr.aws/b4g8c3s2/release-cli
steps:
- attach_workspace:
at: ./
- run:
name: Delete ReleaseHub Environment
command: |
ENVIRONMENT_ID=$(jq -r '.environment.id' tmp/res.json)
release environments delete "$ENVIRONMENT_ID" \
--account "$RELEASE_ACCOUNT_ID" \
--app "$RELEASE_APP_ID"
Finally, we connect all of the jobs into a workflow as follows.
workflows:
my_workflow:
jobs:
- build-and-upload-image:
filters:
branches:
only: demo/circleci
- create-release-environment:
requires: [build-and-upload-image]
filters:
branches:
only: demo/circleci
- run-e2e-tests:
requires: [create-release-environment]
filters:
branches:
only: demo/circleci
- delete-release-environment:
requires: [run-e2e-tests]
filters:
branches:
only: demo/circleci
Copy link
On this page
Configuring GitHub actions for E2E testing with ReleaseHub
Building and uploading your tests as a Docker image
Creating the ReleaseHub Environment
Running the E2E tests
Destroying the ReleaseHub Environment
Configuring CircleCI for E2E testing with ReleaseHub
Building and uploading your tests as a Docker image
Creating the ReleaseHub Environment
Running the E2E tests
Destroying the ReleaseHub Environment