Stability is one of the key ingredients of Mondata’s software products. We achive our high level of reliability through extensive automated software tests. One particular area that we are very fond of, is our testing facility for our word press plugins.

Classic server side code usually runs in only a few configurations, making it comparatively easy to test. Word press plugins on the other hand run in a complex and ever changing ecosystem. There is simply not the one word press configuration to test against. Instead, we have to make sure that our plugins work in combination with various themes, plugins and word press and php versions.

The only reproducible way to deliver a stable product for such an ecosystem, is through extensive testing. We at Mondata prepared word press instances for many real world combinations of themes, plugins and word press versions. Those instances can be reset at any time in order to provide a reproducible testing environment.

Requirements

When designing our “word press test farm” we built for the following requirements:

1. Flexible: Our system should enable us to build instances with whatever configuration necessary.
2. Reproducible: Tests might manipulate the state of the instances. To make subsequent runs deterministic, it must be possible, to reset the instances to a specific state.
3. Easy: Adding a new instance or state must be easy. Otherwise people won’t be motivated to add test cases.
4. Scriptable: All actions should be scriptable, so our automated testing tools can reset the whole system at any desired time.
5. Versioning: All instances should have some kind of versioning that allows to track changes made to their states.

Tooling

In order to have maximum flexibility, we decided to build a containerized infrastructure. By using docker containers, we can ensure that each instance is isolated from the others and is configured exactly the way we want it to.

To make states reproducible, we use docker mounts into local folders. We backup those folders in tar.gz archives. During reset, we restore the state of those folders from the archives and reset all the docker containers.

All the docker configuration is laid out in terraform. A reset is then as easy as typing terraform destroy && terraform apply. Furthermore, we implemented scripts for all actions (e.g. backing up the state, updating the wordpress instances, etc.).

Our automated tests can kick off the reset via a web interface, making the whole test suite automateable.

Finally, we use git to provide versioning of the configuration and state archives.

Implementation Overview

You can find all code at https://github.com/mondata-dev/testfarm. However, in this blog article, we won’t go into the details of the implementation. Instead, we just give a brief overview of the overall structure.

The following diagram shows the topology of a setup with three word press instances.

It includes a total of 5 docker containers: a mysql instance, an nginx reverse proxy and three word press instances. The word press wwwdata and the mysql data directory are mounted from local folders which can be restored from backup archives. Everything is connected to the outside world via an nginx reverse proxy that distributes requests based on their subdomain.

Usage

Prerequisites

Before you can use our testfarm implementation, you have to install the required tools:

1. docker
2. terraform

Also, in this article, we assume you run a linux system. Running the whole system on windows is possible but requires adjusting the scripts and commands that we use here.

Running the system

To start, clone the git repository:

git clone https://github.com/mondata-dev/testfarm
cd testfarm

The repository is set up to already contain a vanilla word press instance with mysql database and reverse proxy. You can start everything by running the following commands:

cd terraform
terraform init
terraform apply

Terraform will show you the changes it wants to make. Mainly, three docker containers will be set up (nginx, mysql, wordpress) with their dependencies. Accept the changes by typing yes.

Terraform will boot the system and create one word press instance with the name wp-vanilla. By default, we configured the instances of the testfarm to be available at testfarm-<instance-name>.example.com (you can change this behavior by adjusting the values in data/testfarm-nginx-conf/conf.d/sites-enabled/testfarm-<instance-name>.conf). As we don’t own the example.com domain, we simply hijack the DNS by adding the domain testfarm-wp-vanilla.example.com in your /etc/hosts file. To do so, change your /etc/hosts file to contain the following:

127.0.0.1   localhost localhost.localdomain [...] testfarm-wp-vanilla.example.com

Now, you should be able to go to http://testfarm-wp-vanilla.local in your browser. You should be at the beginning of the word press set up wizard which looks as follows:

You can complete the word press installation and do any changes you desire.

Backing up the instance state

After you have done some changes to the word press instance (e.g. created some pages, installed some plugins, etc.), you may want to create a snapshot of this state so you can reproduce it at any time. To do this, we provide the docker-backup.sh script in the repository.

A successful run of the script will look as follows:

$ ./docker-backup.sh 
testfarm-mysql
testfarm-wp-vanilla

Note: the script might ask for your sudo password because of the directory rights on the docker mounts.

Now, if you run git status, you will see that the archive files for the mysql data and word press wwwdata have changed:

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   data/testfarm-mysql-store.tar.gz
	modified:   data/testfarm-wp-vanilla-wwwroot.tar.gz

You can use normal git tooling to commit a new version of your state to the repository.

Finally, note that the backup shuts down all instances. You can get them back up and running via

cd terraform
terraform apply

Resetting the instances to the initial state

Let’s say you made some changes to the instance during your plugin tests and want to go back to the initial state. This is as easy as rebooting the system:

cd terraform
terraform destroy -auto-approve && terraform apply -auto-approve

Note: terraform destroy might ask for your sudo password to remove the local mounts from /tmp.

Adding another instance

All word press instances are configured in a terraform variable. To add another one, open the terraform/variables.tf file and find the wordpress_instances section. By default, this looks as follows:

variable "wordpress_instances" {
  description = "Create wordpress instances with these names"
  type        = list(object({
    name = string,
    image = string,
  }))
  default = [
    {
      name = "wp-vanilla"
      image = "wordpress:5.3.2-php7.3"
    },
  ]
}

Add another instance in the default array. For example, we add a wordpress 4.9 instance with php 5.6:

variable "wordpress_instances" {
  // [...]
  default = [
    {
      name = "wp-vanilla"
      image = "wordpress:5.3.2-php7.3"
    },
    {
      name = "wp49-vanilla"
      image = "wordpress:4.9.8-php5.6"
    },
  ]
}

Now, we only needs to create an initial wwwdata archive and the configuration for the nginx reverse proxy. Luckily, we provide a script for that. Call it as follows:

cd data
./create-instance-backup.sh wp49-vanilla

Note: the script might ask for your sudo password in order to set the correct access rights for the wwwdata mounts.

The new instance will be configured for the domain testfarm-wp49-vanilla.example.com. Again, we have to add it to the /etc/hosts file:

127.0.0.1   localhost localhost.localdomain [...] testfarm-wp-vanilla.example.com testfarm-wp49-vanilla.example.com

Now, after resetting the system…

cd terraform
terraform destroy -auto-approve && terraform apply -auto-approve

…you should be able to access the new instance at http://testfarm-wp49-vanilla.example.com.

Wrap up

Using standard cloud development tools such as containers, terraform, docker and git in combination with scripting makes it fairly easy to create reproducible instance states of word press instances. In fact, it is quite easy to extend the system to other applications or CMS. We at Mondata use this infrastructure every day to run our automated end to end tests against various CMS configurations.

Maybe you have similar solutions or feedback on our approach? We’re always glad to hear from you! Drop us a comment or reach us at dev@mondata.de. We’re happy to discuss.

Picture credits:
Adobe Stock