mirror of
https://github.com/pyTooling/Actions.git
synced 2026-02-11 18:46:55 +08:00
Reworked more documentation sections.
This commit is contained in:
Patrick Lehmann
@@ -1,22 +0,0 @@
|
||||
# Development
|
||||
|
||||
## Tagging/versioning
|
||||
|
||||
See context in [#5](https://github.com/pyTooling/Actions/issues/5).
|
||||
|
||||
Tag new releases in the `main` branch using a semver compatible value, starting with `v`:
|
||||
|
||||
```sh
|
||||
git checkout main
|
||||
git tag v0.0.0
|
||||
git push upstream v0.0.0
|
||||
```
|
||||
|
||||
Move the corresponding release branch (starting with `r`) forward by creating a merge commit, and using the merged tag
|
||||
as the commit message:
|
||||
|
||||
```sh
|
||||
git checkout r0
|
||||
git merge --no-ff -m 'v0.0.0' v0.0.0
|
||||
git push upstream r0
|
||||
```
|
||||
105
README.md
105
README.md
@@ -7,86 +7,15 @@ language for writing reusable CI code.
|
||||
However, Python being equally popular and capable, usage of JS/TS might be bypassed, with some caveats.
|
||||
This repository gathers reusable CI tooling for testing, packaging and distributing Python projects and documentation.
|
||||
|
||||
|
||||
## Context
|
||||
|
||||
GitHub Actions supports five procedures to reuse code:
|
||||
|
||||
- JavaScript Action:
|
||||
- [docs.github.com: actions/creating-actions/creating-a-javascript-action](https://docs.github.com/en/actions/creating-actions/creating-a-javascript-action)
|
||||
- Container Action:
|
||||
- [docs.github.com: actions/creating-actions/creating-a-docker-container-action](https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action)
|
||||
- Container Step:
|
||||
- [docs.github.com: actions/learn-github-actions/workflow-syntax-for-github-actions#example-using-a-docker-public-registry-action](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-using-a-docker-public-registry-action)
|
||||
- [docs.github.com: actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepswithargs](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepswithargs)
|
||||
- Composite Action:
|
||||
- [docs.github.com: actions/creating-actions/creating-a-composite-action](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action)
|
||||
- [github.blog/changelog: 2020-08-07-github-actions-composite-run-steps](https://github.blog/changelog/2020-08-07-github-actions-composite-run-steps/)
|
||||
- [github.blog/changelog: 2021-08-25-github-actions-reduce-duplication-with-action-compositio](https://github.blog/changelog/2021-08-25-github-actions-reduce-duplication-with-action-composition/)
|
||||
- Reusable Workflow:
|
||||
- [docs.github.com: actions/learn-github-actions/reusing-workflows](https://docs.github.com/en/actions/learn-github-actions/reusing-workflows)
|
||||
- [github.blog/changelog: 2021-10-05-github-actions-dry-your-github-actions-configuration-by-reusing-workflows](https://github.blog/changelog/2021-10-05-github-actions-dry-your-github-actions-configuration-by-reusing-workflows/)
|
||||
|
||||
Container Actions and Container Steps are almost equivalent: Actions use a configuration file (`action.yml`), while
|
||||
Steps do not.
|
||||
Leaving JavaScript and Container Actions and Steps aside, the main differences between Composite Actions and Reusable
|
||||
Workflows are the following:
|
||||
|
||||
- Composite Actions can be executed from a remote/external path or from the checked out branch, and from any location.
|
||||
However, Reusable Workflows can only be used through a remote/external path (`{owner}/{repo}/{path}/{filename}@{ref}`),
|
||||
where `{path}` must be `.github/workflows`, and `@{ref}` is required.
|
||||
See [actions/runner#1493](https://github.com/actions/runner/issues/1493).
|
||||
As a result:
|
||||
- Local Composite Actions cannot be used without a prior repo checkout, but Reusable Workflows can be used without
|
||||
checkout.
|
||||
- Testing development versions of local Reusable Workflows is cumbersome, because PRs do not pick the modifications by
|
||||
default.
|
||||
- Composite Actions can include multiple steps, but not multiple jobs.
|
||||
Conversely, Reusable Workflows can include multiple jobs, and multiple steps in each job.
|
||||
- Composite Actions can include multiple files, so it's possible to use files from the Action or from the user's repository.
|
||||
Conversely, Reusable Workflows are a single YAML file, with no additional files retrieved by default.
|
||||
|
||||
### Callable vs dispatchable workflows
|
||||
|
||||
Reusable Workflows are defined through the `workflow_call` event kind.
|
||||
Similarly, any "regular" Workflow can be triggered through a `workflow_dispatch` event.
|
||||
Both event kinds support `input` options, which are usable within the Workflow.
|
||||
Therefore, one might intuitively try to write a workflow which is both callable and dispatchable.
|
||||
In other words, which can be either reused from another workflow, or triggered through the API.
|
||||
Unfortunately, that is not the case.
|
||||
Although `input` options can be duplicated for both events, GitHub's backend exposes them through different objects.
|
||||
In dispatchable Workflows, the object is `${{ github.event.inputs }}`, while callable workflows receive `${{ inputs }}`.
|
||||
|
||||
As a result, in order to make a reusable workflow dispatchable, a wrapper workflow is required.
|
||||
See, for instance, [hdl/containers: .github/workflows/common.yml](https://github.com/hdl/containers/blob/main/.github/workflows/common.yml) and [hdl/containers: .github/workflows/dispatch.yml](https://github.com/hdl/containers/blob/main/.github/workflows/dispatch.yml).
|
||||
Alternatively, a normalisation job might be used, similar to the `Parameters` in this repo.
|
||||
|
||||
### Call hierarchy
|
||||
|
||||
Reusable Workflows cannot call other Reusable Workflows, however, they can use Composite Actions and Composite Actions
|
||||
can call other Actions.
|
||||
Therefore, in some use cases it is sensible to combine one layer of reusable workflows for orchestrating the jobs, along
|
||||
with multiple layers of composite actions.
|
||||
|
||||
### Script with post step
|
||||
|
||||
JavaScript Actions support defining `pre`, `pre-if`, `post` and `post-if` steps, which allow executing steps at the
|
||||
beginning or the end of a job, regardless of intermediate steps failing.
|
||||
Unfortunately, those are not available for any other Action type.
|
||||
|
||||
Action [with-post-step](with-post-step) is a generic JS Action to execute a main command and to set a command as a post
|
||||
step.
|
||||
It allows using the `post` feature with scripts written in bash, python or any other interpreted language available on
|
||||
the environment.
|
||||
See: [actions/runner#1478](https://github.com/actions/runner/issues/1478).
|
||||
|
||||
See [GitHub Actions and GitHub Reusable Workflows](https://pytooling.github.io/Actions/Background.html) for more
|
||||
background information.
|
||||
|
||||
## Reusable workflows
|
||||
|
||||
This repository provides 10+ Reusable Workflows based on the CI pipelines of the repos in this organisation,
|
||||
[EDA²](https://github.com/edaa-org), [VHDL](https://github.com/vhdl), and others.
|
||||
By combining them, Python packages can be continuously tested and released along with Sphinx documentation sites, to GitHub Releases, GitHub Pages and PyPI.
|
||||
Optionally, coverage and static type check reports can be gathered.
|
||||
This repository provides 10+ *Reusable Workflows* based on the CI pipelines of the repos in this GitHub organisation,
|
||||
[EDA²](https://github.com/edaa-org), [VHDL](https://github.com/vhdl), and others. By combining them, Python packages can
|
||||
be continuously tested and released along with Sphinx documentation sites, to GitHub Releases, GitHub Pages and PyPI.
|
||||
Optionally, coverage and static type check reports can be gathered and integrated into the online documentation.
|
||||
|
||||
[](ExamplePipeline_dark.png)
|
||||
|
||||
@@ -111,28 +40,6 @@ As shown in the screenshots above, the expected order is:
|
||||
optionally upload results as an HTML report.
|
||||
Example `commands`:
|
||||
|
||||
1. Regular package
|
||||
|
||||
```yml
|
||||
commands: mypy --html-report htmlmypy -p ToolName
|
||||
```
|
||||
|
||||
2. Parent namespace package
|
||||
|
||||
```yml
|
||||
commands: |
|
||||
touch Parent/__init__.py
|
||||
mypy --html-report htmlmypy -p ToolName
|
||||
```
|
||||
|
||||
3. Child namespace package
|
||||
|
||||
```yml
|
||||
commands: |
|
||||
cd Parent
|
||||
mypy --html-report ../htmlmypy -p ToolName
|
||||
```
|
||||
|
||||
- [VerifyDocs](.github/workflows/VerifyDocs.yml): extract code examples from the README and test these code snippets.
|
||||
- Packaging and releasing:
|
||||
- [Release](.github/workflows/Release.yml): publish GitHub Release.
|
||||
|
||||
4
doc/Deveopment.rst
Normal file
4
doc/Deveopment.rst
Normal file
@@ -0,0 +1,4 @@
|
||||
Development
|
||||
###########
|
||||
|
||||
.. todo:: Development - Explain how to write new job templates.
|
||||
110
doc/Instantiation.rst
Normal file
110
doc/Instantiation.rst
Normal file
@@ -0,0 +1,110 @@
|
||||
Instantiantion
|
||||
##############
|
||||
|
||||
The job templates (GitHub Action *Reusable Workflows*) need to be stored in the same directory where normal pipelines
|
||||
(GitHub Action *Workflows*) are located: ``.github/workflows/<template>.yml``. These template files are distinguished
|
||||
from a normal pipeline by a ``on:workflow_call:`` section compared to an ``on:push`` section.
|
||||
|
||||
**Job Template Definition:**
|
||||
|
||||
The ``workflow_call`` allows the definition of input and output parameters.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
<Param1>:
|
||||
# ...
|
||||
outputs:
|
||||
# ...
|
||||
|
||||
jobs:
|
||||
<JobName>:
|
||||
# ...
|
||||
|
||||
**Job Template Instantiation:**
|
||||
|
||||
When instantiating a template, a ``jobs:<Name>:uses`` is used to refer to a template file. Unfortunately, besides the
|
||||
GitHub SLUG (*<Organization>/<Repository>*), also the full path to the template needs to be gives, but still it can't be
|
||||
outside of ``.github/workflows`` to create a cleaner repository structure. Finally, the path contains a branch name
|
||||
postfixed by ``@<branch>`` (tags are still not supported by GitHub Actions). A ``jobs:<Name>:with:`` section can be used
|
||||
to handover input parameters to the template.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
on:
|
||||
push:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
<InstanceName>:
|
||||
uses: <GitHubOrganization>/<Repository>/.github/workflows/<Template>.yml@v0
|
||||
with:
|
||||
<Param1>: <Value>
|
||||
|
||||
|
||||
Example Pipelines
|
||||
*****************
|
||||
|
||||
Documentation Only (Sphinx)
|
||||
===========================
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
name: Pipeline
|
||||
|
||||
on:
|
||||
push:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
BuildTheDocs:
|
||||
uses: pyTooling/Actions/.github/workflows/BuildTheDocs.yml@r0
|
||||
with:
|
||||
artifact: Documentation
|
||||
|
||||
PublishToGitHubPages:
|
||||
uses: pyTooling/Actions/.github/workflows/PublishToGitHubPages.yml@r0
|
||||
needs:
|
||||
- BuildTheDocs
|
||||
with:
|
||||
doc: Documentation
|
||||
|
||||
ArtifactCleanUp:
|
||||
name: 🗑️ Artifact Cleanup
|
||||
needs:
|
||||
- BuildTheDocs
|
||||
- PublishToGitHubPages
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: 🗑️ Delete artifacts
|
||||
uses: geekyeggo/delete-artifact@v2
|
||||
with:
|
||||
name: Documentation
|
||||
|
||||
|
||||
Simple Package
|
||||
==============
|
||||
|
||||
|
||||
Package with Unit Tests
|
||||
=======================
|
||||
|
||||
|
||||
Package with Code Coverage
|
||||
==========================
|
||||
|
||||
Complex Pipeline
|
||||
================
|
||||
|
||||
|
||||
Further Reference Examples
|
||||
**************************
|
||||
|
||||
Find further usage cases in the following list of projects:
|
||||
|
||||
- `edaa-org/pyEDAA.ProjectModel <https://github.com/edaa-org/pyEDAA.ProjectModel/tree/main/.github/workflows>`__
|
||||
- `edaa-org/pySVModel <https://github.com/edaa-org/pySVModel/tree/main/.github/workflows>`__
|
||||
- `VHDL/pyVHDLModel <https://github.com/VHDL/pyVHDLModel/tree/main/.github/workflows>`__
|
||||
@@ -368,7 +368,7 @@ The supported artifacts are:
|
||||
needs:
|
||||
- Params
|
||||
with:
|
||||
artifact: ${{ fromJson(needs.UnitTestingParams.outputs.artifact_names).codecoverage_html }}
|
||||
artifact: ${{ fromJson(needs.Params.outputs.artifact_names).codecoverage_html }}
|
||||
|
||||
|
||||
Params
|
||||
|
||||
@@ -48,30 +48,6 @@ Workflow). They can also serve as an example for creating or driving own job tem
|
||||
Instantiation
|
||||
*************
|
||||
|
||||
The job templates (GitHub Action *Reusable Workflows*) need to be stored in the same directory where normal pipelines
|
||||
(GitHub Action *Workflows*) are located: ``.github/workflows/<template>.yml``. These template files are distinguished
|
||||
from a normal pipeline by a ``on:workflow_call:`` section compared to an ``on:push`` section.
|
||||
|
||||
**Job Template Definition:**
|
||||
|
||||
The ``workflow_call`` allows the definition of input and output parameters.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
<Param1>:
|
||||
# ...
|
||||
outputs:
|
||||
# ...
|
||||
|
||||
jobs:
|
||||
<JobName>:
|
||||
# ...
|
||||
|
||||
**Job Template Instantiation:**
|
||||
|
||||
When instantiating a template, a ``jobs:<Name>:uses`` is used to refer to a template file. Unfortunately, besides the
|
||||
GitHub SLUG (*<Organization>/<Repository>*), also the full path to the template needs to be gives, but still it can't be
|
||||
outside of ``.github/workflows`` to create a cleaner repository structure. Finally, the path contains a branch name
|
||||
@@ -89,9 +65,3 @@ to handover input parameters to the template.
|
||||
uses: <GitHubOrganization>/<Repository>/.github/workflows/<Template>.yml@v0
|
||||
with:
|
||||
<Param1>: <Value>
|
||||
|
||||
|
||||
Development
|
||||
***********
|
||||
|
||||
.. todo:: JobTemplate:Development Needs documentation
|
||||
|
||||
@@ -4,6 +4,13 @@ Releases Management
|
||||
Releases
|
||||
********
|
||||
|
||||
r1
|
||||
==
|
||||
|
||||
.. note:: Upcoming next release based in `v1.x.y`.
|
||||
|
||||
.. attention:: This release introduces breaking changes.
|
||||
|
||||
r0
|
||||
==
|
||||
|
||||
@@ -52,7 +59,7 @@ Each merge-commit is tagged with a semantic version.
|
||||
Tagging
|
||||
*******
|
||||
|
||||
See context in `#5 <https://github.com/pyTooling/Actions/issues/5>`__.
|
||||
See context in :ghissue:`#5 Tagging/versioning of this repo <5>`.
|
||||
|
||||
Tag new releases in the ``main`` branch using a semver compatible value, starting with ``v``:
|
||||
|
||||
|
||||
@@ -6,16 +6,28 @@ names in default parameters to job templates, almost all can be overwritten if t
|
||||
structure.
|
||||
|
||||
* Python source code is located in a directory named after the Python package name.
|
||||
* All tests are located in a ``/tests`` directory and separated by testing approach.
|
||||
|
||||
* A ``<package>/__init__.py`` should be provided with global package information like: version number, author,
|
||||
copyrights, license, maintainer, ...
|
||||
|
||||
* All tests are located in a ``/tests`` directory and further divided into subdirectories by testing approach.
|
||||
|
||||
* E.g. unit tests are located in a ``/tests/unit`` directory.
|
||||
|
||||
* The package documentation is located in a ``/doc`` directory.
|
||||
|
||||
* Documentation is written with ReStructured Text (ReST) and translated using Sphinx.
|
||||
* Documentation requirements are listed in a ``/doc/requirements.txt``.
|
||||
|
||||
* Dependencies are listed in a ``/requirements.txt``.
|
||||
|
||||
* If the build process requires separate dependencies, a ``/build/requirements.txt`` is used.
|
||||
* If the publishing/distribution process requires separate dependencies, a ``/dist/requirements.txt`` is used.
|
||||
* To reduce duplication of dependencies, dependency files should recursively call each other with ``-r <path>``.
|
||||
|
||||
* All Python project settings are stored in a ``pyproject.toml``.
|
||||
* The Python package is described in a ``setup.py``.
|
||||
* Packages are build with ``build`` instead of ``setuptools``.
|
||||
* A repository overview is given in a ``README.md``.
|
||||
|
||||
.. code-block::
|
||||
@@ -47,4 +59,3 @@ structure.
|
||||
README.md
|
||||
requirements.txt
|
||||
setup.py
|
||||
|
||||
|
||||
@@ -88,12 +88,6 @@ Minimal required modifications are the following:
|
||||
- Set the ``name`` input of job ``Parameters``.
|
||||
- Specify the ``commands`` input of job ``StaticTypeCheck``.
|
||||
|
||||
Find further usage cases in the following list of projects:
|
||||
|
||||
- `edaa-org/pyEDAA.ProjectModel <https://github.com/edaa-org/pyEDAA.ProjectModel/tree/main/.github/workflows>`__
|
||||
- `edaa-org/pySVModel <https://github.com/edaa-org/pySVModel/tree/main/.github/workflows>`__
|
||||
- `VHDL/pyVHDLModel <https://github.com/VHDL/pyVHDLModel/tree/main/.github/workflows>`__
|
||||
|
||||
|
||||
GitHub Actions
|
||||
**************
|
||||
@@ -137,13 +131,15 @@ License
|
||||
This document was generated on |docdate|.
|
||||
|
||||
.. toctree::
|
||||
:caption: Overview
|
||||
:caption: Introduction
|
||||
:hidden:
|
||||
|
||||
Background
|
||||
RepositoryStructure
|
||||
Releases
|
||||
Instantiation
|
||||
Deveopment
|
||||
Dependency
|
||||
Releases
|
||||
|
||||
.. raw:: latex
|
||||
|
||||
|
||||
Reference in New Issue
Block a user