Pytest using pyproject.toml (#29)

# New Features
* Allow configuration of the unit test directory (default: `tests/unit`).
* Allow configuration of a `pyproject.toml` or `.coveragerc` file.
* Extract values from `pyproject.toml` or `.coveragerc`.

# Changes
* Jobs deriving from template job `CoverageCollection` need to specify their `.coveragerc` content and `pytest.ini` content in a `pyproject.toml` file or provide the job parameter `coverage_config` with the path to the `.coveragerc` file.

# Bug Fixes
*None*

-------------
* Closes #2.

.github/workflows/CoverageCollection.yml

@@ -35,6 +35,16 @@ on:
         required: false
         default: '-r tests/requirements.txt'
         type: string
+      unittest_directory:
+        description: 'Path to the directory containing unit tests.'
+        required: false
+        default: 'tests/unit'
+        type: string
+      coverage_config:
+        description: 'Path to the .coveragerc file. Use pyproject.toml by default.'
+        required: false
+        default: 'pyproject.toml'
+        type: string
       artifact:
         description: 'Name of the coverage artifact.'
         required: true
@@ -62,27 +72,68 @@ jobs:
       - name: 🗂 Install dependencies
         run: |
           python -m pip install -U pip
+          python -m pip install tomli
           python -m pip install ${{ inputs.requirements }}
 
+      - name: 🔁 Extract configurations from pyproject.toml
+        id: getVariables
+        shell: python
+        run: |
+          from pathlib import Path
+          from tomli   import load as tomli_load
+
+          htmlDirectory = 'htmlcov'
+          xmlFile = './coverage.xml'
+          coverageRC = "${{ inputs.coverage_config }}".strip()
+
+          # Read output paths from 'pyproject.toml' file
+          if coverageRC == "pyproject.toml":
+              pyProjectFile =  Path("pyproject.toml")
+              if pyProjectFile.exists():
+                  with pyProjectFile.open("rb") as file:
+                      pyProjectSettings = tomli_load(file)
+
+                  htmlDirectory = pyProjectSettings["tool"]["coverage"]["html"]["directory"]
+                  xmlFile       = pyProjectSettings["tool"]["coverage"]["xml"]["output"]
+              else:
+                  print(f"File '{pyProjectFile}' not found and no ' .coveragerc' file specified.")
+
+          # Read output paths from '.coveragerc' file
+          elif len(coverageRC) > 0:
+              coverageRCFile = Path(coverageRC)
+              if coverageRCFile.exists():
+                  with coverageRCFile.open("rb") as file:
+                      coverageRCSettings = tomli_load(file)
+
+                  htmlDirectory = coverageRCSettings["html"]["directory"]
+                  xmlFile       = coverageRCSettings["xml"]["output"]
+              else:
+                  print(f"File '{coverageRCFile}' not found.")
+
+          print(f"::set-output name=coverage_report_html_directory::{htmlDirectory}")
+          print(f"::set-output name=coverage_report_xml::{xmlFile}")
+          print(f"DEBUG:\n  html={htmlDirectory}\n  xml={xmlFile}")
+
       - name: Collect coverage
         continue-on-error: true
         run: |
-          python -m pytest -rA --cov=.. --cov-config=tests/.coveragerc tests/unit --color=yes
+          [ 'x${{ inputs.coverage_config }}' != 'x' ] && PYCOV_ARGS='--cov-config=${{ inputs.coverage_config }}' || unset PYCOV_ARGS
+          python -m pytest -rA --cov=. $PYCOV_ARGS ${{ inputs.unittest_directory }} --color=yes
 
       - name: Convert to cobertura format
         run: coverage xml
 
       - name: Convert to HTML format
         run: |
-          coverage html
+          coverage html -d ${{ steps.getVariables.outputs.coverage_report_html_directory }}
-          rm htmlcov/.gitignore
+          rm ${{ steps.getVariables.outputs.coverage_report_html_directory }}/.gitignore
 
       - name: 📤 Upload 'Coverage Report' artifact
         continue-on-error: true
         uses: actions/upload-artifact@v2
         with:
           name: ${{ inputs.artifact }}
-          path: htmlcov
+          path: ${{ steps.getVariables.outputs.coverage_report_html_directory }}
           if-no-files-found: error
           retention-days: 1
 
@@ -90,7 +141,7 @@ jobs:
         continue-on-error: true
         uses: codecov/codecov-action@v1
         with:
-          file: ./coverage.xml
+          file: ${{ steps.getVariables.outputs.coverage_report_xml }}
           flags: unittests
           env_vars: PYTHON
 
@@ -99,4 +150,4 @@ jobs:
         uses: codacy/codacy-coverage-reporter-action@master
         with:
           project-token: ${{ secrets.codacy_token }}
-          coverage-reports: ./coverage.xml
+          coverage-reports: ${{ steps.getVariables.outputs.coverage_report_xml }}

.github/workflows/UnitTesting.yml

@@ -34,6 +34,11 @@ on:
         required: false
         default: '-r tests/requirements.txt'
         type: string
+      unittest_directory:
+        description: 'Path to the directory containing unit tests.'
+        required: false
+        default: 'tests/unit'
+        type: string
       artifact:
         description: "Generate unit test report with junitxml and upload results as an artifact."
         required: false
@@ -68,7 +73,7 @@ jobs:
       - name: ☑ Run unit tests
         run: |
           [ 'x${{ inputs.artifact }}' != 'x' ] && PYTEST_ARGS='--junitxml=TestReport.xml' || unset PYTEST_ARGS
-          python -m pytest -rA tests/unit $PYTEST_ARGS --color=yes
+          python -m pytest -rA ${{ inputs.unittest_directory }} $PYTEST_ARGS --color=yes
 
       - name: 📤 Upload 'TestReport.xml' artifact
         if: inputs.artifact != ''

README.md

@@ -95,13 +95,16 @@ As shown in the screenshot above, the expected order is:
 - Global:
   - [Parameters](.github/workflows/Parameters.yml): a workaround for the limitations to handle global variables in
     GitHub Actions workflows (see [actions/runner#480](https://github.com/actions/runner/issues/480)).
-    It generates outputs with artifact names and job matrices to be used in other jobs.
+    It generates outputs with artifact names and job matrices to be used in later running jobs.
 - Code testing/analysis:
   - [UnitTesting](.github/workflows/UnitTesting.yml): run unit test with `pytest` using multiple versions of Python, and
-    optionally upload results as XML reports.
+    optionally upload results as XML reports. Configuration options to `pytest` should be given via section
-  - [CoverageCollection](.github/workflows/CoverageCollection.yml): collect coverage data with `pytest` using a single
+   `[tool.pytest.ini_options]` in a `pyproject.toml` file.
-    version of Python, generate HTML and Cobertura (XML) reports, upload the HTML report as an artifact, and upload the
+  - [CoverageCollection](.github/workflows/CoverageCollection.yml): collect code coverage data (incl. branch coverage)
-    results to Codecov and Codacy.
+    with `pytest`/`pytest-cov`/`coverage.py` using a single version of Python (latest). It generates HTML and Cobertura
+    (XML)reports, upload the HTML report as an artifact, and upload the test results to Codecov and Codacy. Configuration
+    options to `pytest` and `coverage.py` should be given via section `[tool.pytest.ini_options]` and `[tool.coverage.*]`
+    in a `pyproject.toml` file.
   - [StaticTypeCheck](.github/workflows/StaticTypeCheck.yml): collect static type check result with `mypy`, and
     optionally upload results as an HTML report.
     Example `commands`:
@@ -128,7 +131,7 @@ As shown in the screenshot above, the expected order is:
         mypy --html-report ../htmlmypy -p ToolName
       ```
 
-  - [VerifyDocs](.github/workflows/VerifyDocs.yml): extract code examples from the README and test.
+  - [VerifyDocs](.github/workflows/VerifyDocs.yml): extract code examples from the README and test these code snippets.
 - Packaging and releasing:
   - [Release](.github/workflows/Release.yml): publish GitHub Release.
   - [Package](.github/workflows/Package.yml): generate source and wheel packages, and upload them as an artifact.