From 4f49964e57317b52eb16dcc5ec33545fd49c69df Mon Sep 17 00:00:00 2001 From: Patrick Lehmann Date: Fri, 22 Aug 2025 19:28:42 +0200 Subject: [PATCH] Documented PrepareJob. --- doc/JobTemplate/AllInOne/CompletePipeline.rst | 3 + doc/JobTemplate/Global/PrepareJob.rst | 53 --- .../ExtractConfiguration.rst | 0 .../{Global => Setup}/Parameters.rst | 0 doc/JobTemplate/Setup/PrepareJob.rst | 429 ++++++++++++++++++ doc/JobTemplate/{Global => Setup}/index.rst | 8 +- doc/JobTemplate/Templates.rst | 6 +- doc/index.rst | 2 +- 8 files changed, 440 insertions(+), 61 deletions(-) delete mode 100644 doc/JobTemplate/Global/PrepareJob.rst rename doc/JobTemplate/{Global => Setup}/ExtractConfiguration.rst (100%) rename doc/JobTemplate/{Global => Setup}/Parameters.rst (100%) create mode 100644 doc/JobTemplate/Setup/PrepareJob.rst rename doc/JobTemplate/{Global => Setup}/index.rst (78%) diff --git a/doc/JobTemplate/AllInOne/CompletePipeline.rst b/doc/JobTemplate/AllInOne/CompletePipeline.rst index 7a0a26a..4e305ca 100644 --- a/doc/JobTemplate/AllInOne/CompletePipeline.rst +++ b/doc/JobTemplate/AllInOne/CompletePipeline.rst @@ -153,6 +153,9 @@ It can be used for simple Python packages as well as namespace packages. .. topic:: Dependencies * :ref:`pyTooling/Actions/.github/workflows/PrepareJob.yml ` + + * :gh:`actions/checkout` + * :ref:`pyTooling/Actions/.github/workflows/Parameters.yml ` * :ref:`pyTooling/Actions/.github/workflows/ExtractConfiguration.yml ` * :ref:`pyTooling/Actions/.github/workflows/UnitTesting.yml ` diff --git a/doc/JobTemplate/Global/PrepareJob.rst b/doc/JobTemplate/Global/PrepareJob.rst deleted file mode 100644 index e49e5f2..0000000 --- a/doc/JobTemplate/Global/PrepareJob.rst +++ /dev/null @@ -1,53 +0,0 @@ -.. _JOBTMPL/PrepareJob: - -PrepareJob -########## - -The ``PrepareJob`` job template is a workaround for the limitations of GitHub Actions to handle global variables in -GitHub Actions workflows (see `actions/runner#480 `__. - -It generates output parameters with artifact names and a job matrix to be used in later running jobs. - -**Behavior:** - -.. todo:: Parameters:Behavior Needs documentation. - -**Dependencies:** - -*None* - -Instantiation -************* - -Simple Example -============== - -The following instantiation example creates a job `Params` derived from job template `Parameters` version `r0`. It only -requires a `name` parameter to create the artifact names. - -.. code-block:: yaml - - name: Pipeline - - on: - push: - workflow_dispatch: - - jobs: - Params: - uses: pyTooling/Actions/.github/workflows/PrepareJob.yml@r5 - with: - name: pyTooling - -Parameters -********** - - -Secrets -******* - - -Results -******* - - diff --git a/doc/JobTemplate/Global/ExtractConfiguration.rst b/doc/JobTemplate/Setup/ExtractConfiguration.rst similarity index 100% rename from doc/JobTemplate/Global/ExtractConfiguration.rst rename to doc/JobTemplate/Setup/ExtractConfiguration.rst diff --git a/doc/JobTemplate/Global/Parameters.rst b/doc/JobTemplate/Setup/Parameters.rst similarity index 100% rename from doc/JobTemplate/Global/Parameters.rst rename to doc/JobTemplate/Setup/Parameters.rst diff --git a/doc/JobTemplate/Setup/PrepareJob.rst b/doc/JobTemplate/Setup/PrepareJob.rst new file mode 100644 index 0000000..026a989 --- /dev/null +++ b/doc/JobTemplate/Setup/PrepareJob.rst @@ -0,0 +1,429 @@ +.. _JOBTMPL/PrepareJob: + +PrepareJob +########## + +The ``PrepareJob`` template is a workaround for the limitations of GitHub Actions to handle global variables in GitHub +Actions workflows (see `actions/runner#480 `__) as well as providing basic +string operations (see GitHub Action's `limited set of functions `__). + +The job template generates various output parameters derived from +`${{ github }} context `__. + +.. topic:: Features + + * Provide branch name or tag name from ``${{ github.ref }}`` variable. + * Check if pipeline is a branch or tag pipeline. + * Check if a commit is a merge commit. + * Check if a commit is a release commit. + * Check if a tag is a nightly tag (rolling release tag). + * Check if a tag is a release tag. + * Find associated pull-request (PR) for a merge commit/release commit. + * Provide a version from tag name or pull-request name. + +.. note:: + + Due to GitHub Action's broken type system and missing implicit type conversions in YAML files, *boolean* values need + to be returned as *string* values otherwise type compatibility and comparison are broken. This also requires all + inputs to be *string* parameters, otherwise an step's, job's or template's output cannot be assigned to a template's + input. + + **Problems:** + + 1. Scripts (Bash, Python, ...) return results as strings. There is no option or operation provided by GitHub Actions + to convert outputs of ``printf`` to a ``boolean`` in the YAML file. + 2. Unlike job template inputs, outputs have no type information. These are all considered *string* values. Again no + implicit or explicit type conversion is provided. + 3. While inputs might be defined as ``boolean`` and a matching default can be set as a *boolean* value (e.g., + ``false``), a connected variable from ``${{ needs }}`` context will either cause a typing error or a later + comparison will not work as expected. Either the comparison works with ``inputs.param == false`` for the default + value, **or** it works with a value from ``${{ needs }}`` context, which is a string ``inputs.param == 'false'``. + +.. topic:: Dependencies + + * :gh:`actions/checkout` + * :gh:`GitHub command line tool 'gh' ` + + +.. _JOBTMPL/PrepareJob/Instantiation: + +Instantiation +************* + +Simple Example +============== + +The following instantiation example creates a job `Params` derived from job template `PrepareJob` version `r5`. + +.. code-block:: yaml + + name: Pipeline + + on: + push: + workflow_dispatch: + + jobs: + Prepare: + uses: pyTooling/Actions/.github/workflows/PrepareJob.yml@r5 + + : + needs: + - Prepare + if: needs.Prepare.outputs.is_release_tag == 'true' + ... + with: + version: ${{ needs.Prepare.outputs.version }} + + +.. _JOBTMPL/PrepareJob/Parameters: + +Parameters +********** + +.. topic:: Parameter Summary + + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | Parameter Name | Required | Type | Default | + +=====================================================================+==========+==========+===================================================================+ + | :ref:`JOBTMPL/PrepareJob/Param/ubuntu_image` | no | string | ``'ubuntu-24.04'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Param/main_branch` | no | string | ``'main'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Param/development_branch` | no | string | ``'dev'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Param/release_branch` | no | string | ``'main'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Param/nightly_tag_pattern` | no | string | ``'nightly'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern` | no | string | ``'(v|r)?[0-9]+(\.[0-9]+){0,2}(-(dev|alpha|beta|rc)([0-9]*))?'`` | + +---------------------------------------------------------------------+----------+----------+-------------------------------------------------------------------+ + +.. topic:: :ref:`Secret Summary ` + + This job template needs no secrets. + +.. topic:: :ref:`Output Summary ` + + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | Result Name | Type | Description | + +=====================================================================+==========+===================================================================+ + | :ref:`JOBTMPL/PrepareJob/Result/on_main_branch` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/on_dev_branch` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/on_release_branch` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/is_regular_commit` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/is_merge_commit` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/is_release_commit` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/is_nightly_tag` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/is_release_tag` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/ref_kind` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/branch` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/tag` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/version` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/pr_title` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + | :ref:`JOBTMPL/PrepareJob/Result/pr_number` | string | | + +---------------------------------------------------------------------+----------+-------------------------------------------------------------------+ + +.. _JOBTMPL/PrepareJob/Param/ubuntu_image: + +ubuntu_image +============ + +:Type: string +:Required: no +:Default Value: ``'ubuntu-24.04'`` +:Possible Values: See `actions/runner-images - Available Images `__ +:Description: Name of the Ubuntu image used to run this job. + + +.. _JOBTMPL/PrepareJob/Param/main_branch: + +main_branch +=========== + +:Type: string +:Required: no +:Default Value: ``'main'`` +:Possible Values: Any valid branch name. +:Description: Name of the main branch. + + +.. _JOBTMPL/PrepareJob/Param/development_branch: + +development_branch +================== + +:Type: string +:Required: no +:Default Value: ``'dev'`` +:Possible Values: Any valid branch name. +:Description: Name of the development branch. + + +.. _JOBTMPL/PrepareJob/Param/release_branch: + +release_branch +============== + +:Type: string +:Required: no +:Default Value: ``'main'`` +:Possible Values: Any valid branch name. +:Description: Name of the branch containing releases. + + +.. _JOBTMPL/PrepareJob/Param/nightly_tag_pattern: + +nightly_tag_pattern +=================== + +:Type: string +:Required: no +:Default Value: ``'nightly'`` +:Possible Values: Any valid regular expression. |br| + Suggested alternative values: ``latest``, ``rolling`` +:Description: Name of the tag used for rolling releases, a.k.a nightly builds. + + + +.. _JOBTMPL/PrepareJob/Param/release_tag_pattern: + +release_tag_pattern +=================== + +:Type: string +:Required: no +:Default Value: ``'(v|r)?[0-9]+(\.[0-9]+){0,2}(-(dev|alpha|beta|rc)([0-9]*))?'`` +:Possible Values: Any valid regular expression. +:Description: A regular expression describing a pattern for identifying a release tag. + + The default pattern matches on a `semantic version number `__ separated by dots. + It supports up to 3 digit groups. It accepts an optional ``v`` or ``r`` prefix. Optionally, a postfix + of ``dev``, ``alpha``, ``beta`` or ``rc`` separated by a hyphen can be appended. If needed, the + postfix can have a digit group. + + **Matching tag names as releases:** + + * ``v1``, ``r1`` + * ``1``, ``1.1``, ``1.1.1`` + * ``v1.2.8-dev`` + * ``v3.13.5-alpha2`` + * ``v4.7.22-beta3`` + * ``v10.2-rc1`` + +.. _JOBTMPL/PrepareJob/Secrets: + +Secrets +******* + +This job template needs no secrets. + + +.. _JOBTMPL/PrepareJob/Results: + +Results +******* + +.. _JOBTMPL/PrepareJob/Result/on_main_branch: + +on_main_branch +============== + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is on :ref:`main branch `, + otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/on_dev_branch: + +on_dev_branch +============= + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is on :ref:`development branch `, + otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/on_release_branch: + +on_release_branch +================= + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is on :ref:`release branch `, + otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/is_regular_commit: + +is_regular_commit +================= + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is not a :ref:`merge commit ` + nor :ref:`release commit `, otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/is_merge_commit: + +is_merge_commit +=============== + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is on :ref:`main branch ` + or :ref:`development branch ` and has more than one + parent (merge commit), otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/is_release_commit: + +is_release_commit +================= + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline's commit is on :ref:`release branch ` + and has more than one parent (merge commit), otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/is_nightly_tag: + +is_nightly_tag +============== + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline is a tag pipeline for a commit on :ref:`release branch ` + and the tag's name matches the :ref:`nightly tag pattern `, + otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/is_release_tag: + +is_release_tag +============== + +:Type: string +:Default Value: ``'false'`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns ``'true'`` if the pipeline is a tag pipeline for a commit on :ref:`release branch ` + and the tag's name matches the :ref:`release tag pattern `, + otherwise return ``'false'``. + + +.. _JOBTMPL/PrepareJob/Result/ref_kind: + +ref_kind +======== + +:Type: string +:Default Value: ``'unknown'`` +:Possible Values: ``'branch'``, ``'tag'``, ``'unknown'`` +:Description: Returns ``'branch'`` if pipeline's commit is on a branch or returns ``'tag'`` if the pipeline runs for + a tagged commit, otherwise returns ``'unknown'`` in case of an internal error. + + If the kind is a branch, the branch name is available in the job's :ref:`JOBTMPL/PrepareJob/Result/branch` + result. |br| + If the kind is a tag, the tags name is available in the job's :ref:`JOBTMPL/PrepareJob/Result/tag` + result. |br| + Moreover, if the tag matches the :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`, the extracted + version is available in the job's :ref:`JOBTMPL/PrepareJob/Result/version` result. + +.. note:: + + GitHub doesn't provide standalone branch or tag information, but provides the variable ``${{ github.ref }}`` + specifying the currently active reference (branch, tag, pull, ...). This job template parses the context's variable + and derives if a pipeline runs for a commit on a branch or a tagged commit. + + +.. _JOBTMPL/PrepareJob/Result/branch: + +branch +====== + +:Type: string +:Default Value: ``''`` +:Possible Values: Any valid branch name. +:Description: Returns the branch's name the pipeline's commit is associated to, if :ref:`JOBTMPL/PrepareJob/Result/ref_kind` + is ``'branch'``, otherwise returns an empty string ``''``. + +.. _JOBTMPL/PrepareJob/Result/tag: + +tag +=== + +:Type: string +:Default Value: ``''`` +:Possible Values: Any valid tag name. +:Description: Returns the tag's name the pipeline's commit is associated to, if :ref:`JOBTMPL/PrepareJob/Result/ref_kind` + is ``'tag'``, otherwise returns an empty string ``''``. + + +.. _JOBTMPL/PrepareJob/Result/version: + +version +======= + +:Type: string +:Default Value: ``''`` +:Possible Values: Any valid version matching :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`. +:Description: In case the pipeline runs for a tag, it returns the tag's name, if the name matches + :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`, otherwise returns an empty string ``''``. |br| + In case the pipeline runs for a branch, then the commit is checked if it's a + :ref:`merge commit ` and corresponding pull-request (PR) is + searched. When a matching PR can be located and it's title matches + :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`, then this title is returned as a version, + otherwise it returns an empty string ``''``. + + +.. _JOBTMPL/PrepareJob/Result/pr_title: + +pr_title +======== + +:Type: string +:Default Value: ``''`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns the associated pull-request's title, if the pipeline's commit is a + :ref:`merge commit ` and the located pull-request's title + for this commit matches :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`, otherwise returns an + empty string ``''``. + + +.. _JOBTMPL/PrepareJob/Result/pr_number: + +pr_number +========= + +:Type: string +:Default Value: ``''`` +:Possible Values: ``'true'``, ``'false'`` +:Description: Returns the associated pull-request's number, if the pipeline's commit is a + :ref:`merge commit ` and the located pull-request's title + for this commit matches :ref:`JOBTMPL/PrepareJob/Param/release_tag_pattern`, otherwise returns an + empty string ``''``. diff --git a/doc/JobTemplate/Global/index.rst b/doc/JobTemplate/Setup/index.rst similarity index 78% rename from doc/JobTemplate/Global/index.rst rename to doc/JobTemplate/Setup/index.rst index 076216b..9605040 100644 --- a/doc/JobTemplate/Global/index.rst +++ b/doc/JobTemplate/Setup/index.rst @@ -1,9 +1,9 @@ -.. _JOBTMPL/Global: +.. _JOBTMPL/Setup: -Global -###### +Pipeline Setup +############## -The category *global* provides workflow templates implementing preparation steps suitable for every pipeline. +The category *pipeline setup* provides workflow templates implementing preparation steps suitable for every pipeline. * :ref:`JOBTMPL/Parameters` - Compute a matrix of (operating system |times| Python version |times| environment) combinations for unit testing, platform testing or application testing and provide the result as a JSON string via diff --git a/doc/JobTemplate/Templates.rst b/doc/JobTemplate/Templates.rst index 7798f5a..dafeef3 100644 --- a/doc/JobTemplate/Templates.rst +++ b/doc/JobTemplate/Templates.rst @@ -3,11 +3,11 @@ .. grid-item:: :columns: 2 - .. rubric:: All-In-One Templates + .. rubric:: All-In-One * :ref:`JOBTMPL/CompletePipeline` |br| |br| |br| |br| - .. rubric:: Global Templates + .. rubric:: Pipeline Setup * :ref:`JOBTMPL/Parameters` * :ref:`JOBTMPL/PrepareJob` @@ -61,7 +61,7 @@ .. grid-item:: :columns: 2 - .. rubric:: Cleanup Templates + .. rubric:: Cleanup * :ref:`JOBTMPL/IntermediateCleanup` * :ref:`JOBTMPL/ArtifactCleanup` diff --git a/doc/index.rst b/doc/index.rst index b81eb1e..d0c4146 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -187,7 +187,7 @@ License JobTemplate/index JobTemplate/AllInOne/index - JobTemplate/Global/index + JobTemplate/Setup/index JobTemplate/Documentation/index JobTemplate/Testing/index JobTemplate/Quality/index