Reworked PublishTestResults.

doc/JobTemplate/AllInOne/CompletePipeline.rst

@@ -72,7 +72,7 @@ It can be used for simple Python packages as well as namespace packages.
          * Automatic tagging of merge commits on main branch to trigger a tagged pipeline (release pipeline).
          * Create a release page with text derived from pull request description.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    .. todo:: CompletePipeline:Behavior needs documentation.
 

doc/JobTemplate/Documentation/LaTeXDocumentation.rst

@@ -11,7 +11,7 @@ The ``LatexDocumentation`` job template ................
 
    * Translate a LaTeX document to PDF.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    .. todo:: LatexDocumentation:Behavior needs documentation.
 

doc/JobTemplate/Documentation/SphinxDocumentation.rst

@@ -18,7 +18,7 @@ The ``SphinxDocumentation`` job template ..........
    * Optionally: download code coverage artifact (JSON format) given by :ref:`JOBTMPL/SphinxDocumentation/Input/coverage_json_artifact`.
    * Optionally: download unit test report artifact (XML format) given by :ref:`JOBTMPL/SphinxDocumentation/Input/unittest_xml_artifact`.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    .. todo:: SphinxDocumentation:Behavior needs documentation.
 

doc/JobTemplate/Package/Package.rst

@@ -11,7 +11,7 @@ as an artifact.
    * Package source code as wheel and source distribution.
    * Support packaging using :pypi:`build` (recommended) or :pypi:`setuptools`.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    1. Checkout repository.
    2. Setup Python and install dependencies.

doc/JobTemplate/Package/PublishOnPyPI.rst

@@ -9,14 +9,14 @@ Publish a wheel (``*.whl``) packages and/or source (``*.tar.gz``) package to `Py
 
    * Publish a Python package to `PyPI <https://pypi.org/>`__.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    1. Download package artifact
    2. Publish source package(s) (``*.tar.gz``)
    3. Publish wheel package(s) (``*.whl``)
    4. Delete the artifact
 
-.. topic:: Preconditions:
+.. topic:: Preconditions
 
    1. A PyPI account was created and the package name is either not occupied or the user has access rights for that
       package.

doc/JobTemplate/Publish/PublishTestResults.rst

@@ -3,26 +3,71 @@
 PublishTestResults
 ##################
 
-This job downloads all artifacts and uploads jUnit XML reports as a Markdown page to GitHub Actions to visualize the
+This job downloads all matching JUnit report artifacts, merged the reports into one single report and then publishes the
-results a an item in the job list. For publishing, :gh:`dorny/test-reporter` is used.
+JUnit test summary reports to multiple services for visualization and longterm statistical tracking.
 
-**Behavior:**
+Supported services are:
+
+* GitHub Actions - Job results using :gh:`dorny/test-reporter`
+* `Codecov <https://about.codecov.io/>`__
+
+.. topic:: Features
+
+   * Package source code as wheel and source distribution.
+   * Support packaging using :pypi:`build` (recommended) or :pypi:`setuptools`.
+
+.. topic:: Behavior
 
    1. Checkout repository
-2. Download (all) artifacts
+   2. Download multiple artifacts containing test report summaries in JUnit XML format conforming to an artifact name
-3. Publish test results as a markdown report page to GitHub Actions.
+      pattern (see :ref:`JOBTMPL/PublishTestResults/Input/unittest_artifacts_pattern`) for limiting the number of
+      downloaded artifacts and the herby generated traffic.
+   3. Rename the found JUnit XML files.
+   4. Merge all found JUnit XML files using :pypi:`pyEDAA.Reports` into a new JUnit XML file. |br|
+      Optionally, apply certain transformation and cleanup operations to the JUnit report structure.
+   5. Publish test results as a markdown report page to GitHub Actions using :gh:`dorny/test-reporter`.
+   6. Publish test results to `Codecov <https://about.codecov.io/>`__ using :gh:`codecov/test-results-action`.
 
-.. note::
+.. topic:: Job Execution
 
-   The :gh:`actions/download-artifact` does not support wildcards to specify a subset of artifacts for downloading.
+   .. grid:: 2
-   Thus, all artifacts need to be downloaded.
 
-**Dependencies:**
+      .. grid-item::
+         :columns: 4
+
+         .. rubric:: Job Steps
+
+         .. image:: ../../_static/pyTooling-Actions-PublishTestResults.png
+            :width: 500px
+
+      .. grid-item::
+         :columns: 4
+
+         .. rubric:: Generated Dorny Report below Pipeline Graph
+
+         .. image:: ../../_static/pyTooling-Actions-PublishTestResults-Dorny.png
+            :width: 500px
+
+.. topic:: Dependencies
 
    * :gh:`actions/checkout`
-* :gh:`actions/download-artifact`
+   * :gh:`pyTooling/download-artifact`
-* :gh:`dorny/test-reporter`
 
+     * :gh:`actions/download-artifact`
+
+   * pip
+
+     * :pypi:`pyEDAA.Reports`
+
+   * :gh:`dorny/test-reporter`
+   * :gh:`codecov/test-results-action`
+   * :gh:`pyTooling/upload-artifact`
+
+     * :gh:`actions/upload-artifact`
+
+
+
+.. _JOBTMPL/PublishTestResults/Instantiation:
 
 Instantiation
 *************
@@ -54,35 +99,263 @@ Complex Example
          - CodeCoverage
          - UnitTesting
 
+.. seealso::
 
-Parameters
+   :ref:`JOBTMPL/UnitTesting`
-**********
+     tbd
+   :ref:`JOBTMPL/PublishCoverageResults`
+     tbd
 
-report_files
+
+.. _JOBTMPL/PublishTestResults/Parameters:
+
+Parameter Summary
+*****************
+
+.. rubric:: Goto :ref:`input parameters <JOBTMPL/PublishTestResults/Inputs>`
+
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| Parameter Name                                                      | Required | Type     | Default                                                             |
++=====================================================================+==========+==========+=====================================================================+
+| :ref:`JOBTMPL/PublishTestResults/Input/ubuntu_image_version`        | no       | string   | ``'24.04'``                                                         |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/unittest_artifacts_pattern`  | no       | string   | ``'*-UnitTestReportSummary-XML-*'``                                 |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/merged_junit_filename`       | no       | string   | ``'Unittesting.xml'``                                               |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/merged_junit_artifact`       | no       | string   | ``''``                                                              |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/merge-input-dialect`         | no       | string   | ``'pyTest-JUnit'``                                                  |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/merge-output-dialect`        | no       | string   | ``'pyTest-JUnit'``                                                  |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/additional_merge_args`       | no       | string   | ``'"--pytest=rewrite-dunder-init;reduce-depth:pytest.tests.unit"'`` |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/testsuite-summary-name`      | no       | string   | ``''``                                                              |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/publish`                     | no       | string   | ``'true'``                                                          |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/report_title`                | no       | string   | ``'Unit Test Results'``                                             |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/dorny`                       | no       | string   | ``'true'``                                                          |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/codecov`                     | no       | string   | ``'false'``                                                         |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+| :ref:`JOBTMPL/PublishTestResults/Input/codecov_flags`               | no       | string   | ``'unittest'``                                                      |
++---------------------------------------------------------------------+----------+----------+---------------------------------------------------------------------+
+
+.. rubric:: Goto :ref:`secrets <JOBTMPL/PublishTestResults/Secrets>`
+
++-----------------------------------------------------------+----------+----------+--------------+
+| Token Name                                                | Required | Type     | Default      |
++===========================================================+==========+==========+==============+
+| :ref:`JOBTMPL/PublishTestResults/Secret/CODECOV_TOKEN`    | no       | string   | — — — —      |
++-----------------------------------------------------------+----------+----------+--------------+
+
+.. rubric:: Goto :ref:`output parameters <JOBTMPL/PublishTestResults/Outputs>`
+
+This job template has no output parameters.
+
+
+.. _JOBTMPL/PublishTestResults/Inputs:
+
+Input Parameters
+****************
+
+.. _JOBTMPL/PublishTestResults/Input/ubuntu_image_version:
+
+ubuntu_image_version
+====================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'24.04'``
+:Possible Values: See `actions/runner-images - Available Images <https://github.com/actions/runner-images?tab=readme-ov-file#available-images>`__
+                  for available Ubuntu image versions.
+:Description:     Version of the Ubuntu image used to run this job.
+
+                  .. note::
+
+                     Unfortunately, GitHub Actions has only a `limited set of functions <https://docs.github.com/en/actions/reference/workflows-and-actions/expressions#functions>`__,
+                     thus, the usual Ubuntu image name like ``'ubuntu-24.04'`` can't be split into image name and image
+                     version.
+
+
+.. _JOBTMPL/PublishTestResults/Input/unittest_artifacts_pattern:
+
+unittest_artifacts_pattern
+==========================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'*-UnitTestReportSummary-XML-*'``
+:Possible Values: Any valid artifact matching pattern using fixed text and ``*`` characters.
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/merged_junit_filename:
+
+merged_junit_filename
+=====================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'Unittesting.xml'``
+:Possible Values: Any valid filename suitable for a JUnit XML report.
+:Description:     The filename for the merged JUnit report in XML format. |br|
+                  See :ref:`JOBTMPL/PublishTestResults/Input/merge-output-dialect` for the used JUnit dialect in the
+                  merged report file.
+
+
+.. _JOBTMPL/PublishTestResults/Input/merged_junit_artifact:
+
+merged_junit_artifact
+=====================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``''``
+:Possible Values: Any valid artifact name.
+:Description:
+
+
+.. _JOBTMPL/PublishTestResults/Input/merge-input-dialect:
+
+merge-input-dialect
+===================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'pyTest-JUnit'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/merge-output-dialect:
+
+merge-output-dialect
+====================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'pyTest-JUnit'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/additional_merge_args:
+
+additional_merge_args
+=====================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'"--pytest=rewrite-dunder-init;reduce-depth:pytest.tests.unit"'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/testsuite-summary-name:
+
+testsuite-summary-name
+======================
+
+:Type:            string
+:Required:        no
+:Default Value:   ``''``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/publish:
+
+publish
+=======
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'true'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/report_title:
+
+report_title
 ============
 
-+----------------+----------+----------+---------------------------------+
+:Type:            string
-| Parameter Name | Required | Type     | Default                         |
+:Required:        no
-+================+==========+==========+=================================+
+:Default Value:   ``'Unit Test Results'``
-| report_files   | optional | string   | ``artifacts/**/*.xml``          |
+:Possible Values: tbd
-+----------------+----------+----------+---------------------------------+
+:Description:     tbd
 
-Pattern of jUnit report files to publish as Markdown.
 
-The parameter can be a comma separated list. Wildcards are supported.
+.. _JOBTMPL/PublishTestResults/Input/dorny:
 
-.. hint::
+dorny
+=====
 
-   All artifacts are downloaded into directory ``artifacts``, thus the pattern should include this directory as a
+:Type:            string
-   prefix.
+:Required:        no
+:Default Value:   ``'true'``
+:Possible Values: tbd
+:Description:     tbd
 
 
+.. _JOBTMPL/PublishTestResults/Input/codecov:
+
+codecov
+=======
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'false'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Input/codecov_flags:
+
+codecov_flags
+=============
+
+:Type:            string
+:Required:        no
+:Default Value:   ``'unittest'``
+:Possible Values: tbd
+:Description:     tbd
+
+
+.. _JOBTMPL/PublishTestResults/Secrets:
+
 Secrets
 *******
 
-This job template needs no secrets.
+.. _JOBTMPL/PublishTestResults/Secret/CODECOV_TOKEN:
 
-Results
+CODECOV_TOKEN
+=============
+
+:Type:            string
+:Required:        no
+:Default Value:   — — — —
+:Description:     The token to publish unit test results on  `Codecov <https://about.codecov.io/>`__.
+
+
+.. _JOBTMPL/PublishTestResults/Outputs:
+
+Outputs
 *******
 
 This job template has no output parameters.
+
+
+.. _JOBTMPL/PublishTestResults/Optimizations:
+
+Optimizations
+*************
+
+This template offers no optimizations (reduced job runtime).
+
+merged_junit_artifact

doc/JobTemplate/Setup/ExtractConfiguration.rst

@@ -19,7 +19,7 @@ The ``ExtractConfiguration`` job template is a .....
    * Extract code coverage XML report file as directory name, filename and full path from :file:`pyproject.toml`.
    * Extract code coverage JSON report file as directory name, filename and full path from :file:`pyproject.toml`.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    .. todo:: ExtractConfiguration:Behavior needs documentation.
 

doc/JobTemplate/Testing/UnitTesting.rst

@@ -22,7 +22,7 @@ file.
      * Provide code coverage results as pytest JSON file.
      * Provide code coverage results as HTML report.
 
-.. topic:: Behavior:
+.. topic:: Behavior
 
    .. todo:: UnitTesting:Behavior needs documentation.