Fabric tasks

Once you have the development environment setup, you can use various fabric tasks that have been defined to make your life easier.

Index of tasks

  • bandit - runs security checks
  • build_wheels - build a wheels for Mataara and its dependencies
  • flake8 - run linting checks on python code
  • jslint - running linting checks on the JS code.
  • make_docs - use Sphinx to compile up an updated set of HTML documentation
  • migrate - runs the Django database migrations
  • new_tag - create and push a new git tag
  • pip_update
  • reload_server - reloads server and runs migrations.
  • run_checks - short had or running all the linting, security checks and tests
  • runjs - runs npm to server the frontend React code for development
  • setup_dev_env - setup your local development environment
  • setup_feeds - configure the default security advisory feeds
  • show_logs - displays the output from the Mataara logs.
  • test - run the built-in unit tests

Bandit Security Checks

Bandit is a security linter for Python source code. You can run it manually using:

fab bandit

Build Mataara Wheels

The server installation instructions are designed around the use of wheels for installation of Mataara. Sometimes it may be possible to install all necessary wheels using pip from a standard Python repository.

To install a custom version of Mataara from source, this command can be used to build the Mataara wheel and those for all its dependencies (assuming that the target installation server is the same as the build server).

The built wheels are placed in the directory wheels/.

To build the wheels run:

fab build_wheels

Creating new git tags

We use tags natively in git to manage deployments. They are created using the format:

%Y%m%d-%N

Tags will be created for the current date followed by -1 if it’s the first tag of the day. For subsequent tags the number will be incremented. There is an assumption that a maximum of 9 tags will be needed in any one day.

To create the tag and upload it to your git server, run:

fab new_tag

Flake8 Linting Checks

Flake8 is a generic linter for Python source code. It is a wrapper around other tools such as PyFlakes.

The configuration file for flake8 is setup.cfg.

Source code lines that need to be skipped because they are producing false positives need the following comment to be appended:

# noqa

Running the Server

This command is being deprecated as bringing up the development environment starts a backend server through supervisor.

There is a short-cut for running the server using the integrated Django web service. (Note this interface is only suitable for development environments. In production it is important to use a full web server)

To run the server:

fab run

This will default to running the web site using the URL: http://192.168.33.5:8000/

The IP address can be configured in the Vagrantfile if you want to choose something else.

Setting up Initial Security Advisory Feeds

To initialise the default security advisory feeds run:

fab setup_feeds

Setting up the Development Environment

Setup your local development environment - installs a git pre-commit hook to run check, and install locally on the host the required python packages:

fab setup_dev_env

Running the tests

To run all both the Django Python server tests and the front end React JS tests:

fab test

The following options can be added: * fast - skip vagrant environment system checks * server_tests=’’ - to skip the server tests * frontend_tests=’’ - to skip the frontend tests

Updating Pip Dependencies

All of the Python packages installed by Pip and used on a production server have pinned versions. (We don’t bother to pin the versions of packages that are specific to development or the build process)

By pinning packages, we can have confidence in the specific versions of all components on a server and we can control when we do the upgrades. Using this approach does require doing regular upgrades. This task takes the core requirements defined in requirements/requirements.in and updates requirements/common.txt to the latest pinned versions.

To run the task:

fab pip_update