diff --git a/releaser/DEVELOPMENT.md b/releaser/DEVELOPMENT.md deleted file mode 100644 index 3fdc386..0000000 --- a/releaser/DEVELOPMENT.md +++ /dev/null @@ -1,8 +0,0 @@ -# Releaser Development - -- [pyTooling/pyAttributes](https://github.com/pyTooling/pyAttributes) or - [willmcgugan/rich](https://github.com/willmcgugan/rich) might be used to enhance the UX. - -- It might be desirable to have pyTooling.Version.SemVersion handle the regular expression from - [semver.org](https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string), and use - proper Python classes in **Releaser**. diff --git a/releaser/Dockerfile b/releaser/Dockerfile deleted file mode 100644 index 080b725..0000000 --- a/releaser/Dockerfile +++ /dev/null @@ -1,12 +0,0 @@ -FROM python:3.12-slim-bookworm -COPY releaser.py /releaser.py -RUN pip install PyGithub --progress-bar off \ - && apt update -qq \ - && apt install -y curl \ - && curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | \ - dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \ - && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | \ - tee /etc/apt/sources.list.d/github-cli.list > /dev/null \ - && apt update -qq \ - && apt install -y gh -CMD ["/releaser.py"] diff --git a/releaser/README.md b/releaser/README.md deleted file mode 100644 index b63de11..0000000 --- a/releaser/README.md +++ /dev/null @@ -1,181 +0,0 @@ -# Releaser - -**Releaser** is a Docker GitHub Action written in Python. - -**Releaser** allows to keep a GitHub Release of type pre-release and its artifacts up to date with latest builds. -Combined with a workflow that is executed periodically, **Releaser** allows to provide a fixed release name for users willing -to use daily/nightly artifacts of a project. - -Furthermore, when any [semver](https://semver.org) compilant tagged commit is pushed, **Releaser** can create a release -and upload assets. - -## Context - -GitHub provides official clients for the GitHub API through [github.com/octokit](https://github.com/octokit): - -- [octokit.js](https://github.com/octokit/octokit.js) ([octokit.github.io/rest.js](https://octokit.github.io/rest.js)) -- [octokit.rb](https://github.com/octokit/octokit.rb) ([octokit.github.io/octokit.rb](http://octokit.github.io/octokit.rb)) -- [octokit.net](https://github.com/octokit/octokit.net) ([octokitnet.rtfd.io](https://octokitnet.rtfd.io)) - -When GitHub Actions was released in 2019, two Actions were made available through -[github.com/actions](https://github.com/actions) for dealing with GitHub Releases: - -- [actions/create-release](https://github.com/actions/create-release) -- [actions/upload-release-asset](https://github.com/actions/upload-release-asset) - -However, those Actions were contributed by an employee in spare time, not officially supported by GitHub. -Therefore, they were unmaintained before GitHub Actions was out of the private beta -(see [actions/upload-release-asset#58](https://github.com/actions/upload-release-asset/issues/58)) -and, a year later, archived. -Those Actions are based on [actions/toolkit](https://github.com/actions/toolkit)'s hydrated version of octokit.js. - -From a practical point of view, [actions/github-script](https://github.com/actions/github-script) is the natural replacement to those Actions, since it allows to use a pre-authenticated *octokit.js* client along with the workflow run context. -Still, it requires writing plain JavaScript. - -Alternatively, there are non-official GitHub API libraries available in other languages (see [docs.github.com: rest/overview/libraries](https://docs.github.com/en/rest/overview/libraries)). -**Releaser** is based on [PyGithub/PyGithub](https://github.com/PyGithub/PyGithub), a Python client for the GitHub API. - -**Releaser** was originally created in [eine/tip](https://github.com/eine/tip), as an enhanced alternative to using -`actions/create-release` and `actions/upload-release-asset`, in order to cover certain use cases that were being -migrated from Travis CI to GitHub Actions. -The main limitation of GitHub's Actions was/is verbosity and not being possible to dynamically define the list of assets -to be uploaded. - -On the other hand, GitHub Actions artifacts do require login in order to download them. -Conversely, assets of GitHub Releases can be downloaded without login. -Therefore, in order to make CI results available to the widest audience, some projects prefer having tarballs available -as assets. -In this context, one of the main use cases of **Releaser** is pushing artifacts as release assets. -Thus, the name of the Action. - -GitHub provides an official CLI tool, written in golang: [cli/cli](https://github.com/cli/cli). -When the Python version of **Releaser** was written, `cli` was evaluated as an alternative to *PyGitHub*. -`gh release` was (and still is) not flexible enough to update the reference of a release, without deleting and -recreating it (see [cli.github.com: manual/gh_release_create](https://cli.github.com/manual/gh_release_create)). -Deletion and recreation is unfortunate, because it notifies all the watchers of a repository -(see [eine/tip#111](https://github.com/eine/tip/issues/111)). -However, [cli.github.com: manual/gh_release_upload](https://cli.github.com/manual/gh_release_upload) handles uploading -artifacts as assets faster and with better stability for larger files than *PyGitHub* -(see [msys2/msys2-installer#36](https://github.com/msys2/msys2-installer/pull/36)). -Furthermore, the GitHub CLI is installed on GitHub Actions' default virtual environments. -Although `gh` does not support login through SSH (see [cli/cli#3715](https://github.com/cli/cli/issues/3715)), on GitHub -Actions a token is available `${{ github.token }}`. -Therefore, **Releaser** uses `gh release upload` internally. - -## Usage - -The following block shows a minimal YAML workflow file: - -```yml -name: 'workflow' - -on: - schedule: - - cron: '0 0 * * 5' - -jobs: - mwe: - runs-on: ubuntu-24.04 - steps: - - # Clone repository - - uses: actions/checkout@v5 - - # Build your application, tool, artifacts, etc. - - name: Build - run: | - echo "Build some tool and generate some artifacts" > artifact.txt - - # Update tag and pre-release - # - Update (force-push) tag to the commit that is used in the workflow. - # - Upload artifacts defined by the user. - - uses: pyTooling/Actions/releaser@r0 - with: - token: ${{ secrets.GITHUB_TOKEN }} - files: | - artifact.txt - README.md -``` - -### Composite Action - -The default implementation of **Releaser** is a Container Action. -Therefore, a pre-built container image is pulled before starting the job. -Alternatively, a Composite Action version is available: `uses: pyTooling/Actions/releaser/composite@main`. -The Composite version installs the dependencies on the host (the runner environment), instead of using a container. -Both implementations are functionally equivalent from **Releaser**'s point of view; however, the Composite Action allows -users to tweak the version of Python by using [actions/setup-python](https://github.com/actions/setup-python) before. - -## Options - -All options can be optionally provided as environment variables: `INPUT_TOKEN`, `INPUT_FILES`, `INPUT_TAG`, `INPUT_RM` -and/or `INPUT_SNAPSHOTS`. - -### token (required) - -Token to make authenticated API calls; can be passed in using `{{ secrets.GITHUB_TOKEN }}`. - -### files (required) - -Either a single filename/pattern or a multi-line list can be provided. All the artifacts are uploaded regardless of the -hierarchy. - -For creating/updating a release without uploading assets, set `files: none`. - -### tag - -The default tag name for the tip/nightly pre-release is `tip`, but it can be optionally overriden through option `tag`. - -### rm - -Set option `rm` to `true` for systematically removing previous artifacts (e.g. old versions). -Otherwise (by default), all previours artifacts are preserved or overwritten. - -Note: - If all the assets are removed, or if the release itself is removed, tip/nightly assets won't be available for - users until the workflow is successfully run. - For instance, Action [setup-ghdl-ci](https://github.com/ghdl/setup-ghdl-ci) uses assets from [ghdl/ghdl: releases/tag/nightly](https://github.com/ghdl/ghdl/releases/tag/nightly). - Hence, it is recommended to try removing the conflictive assets only, in order to maximise the availability. - -### snapshots - -Whether to create releases from any tag or to treat some as snapshots. -By default, all the tags with non-empty `prerelease` field (see [semver.org: Is there a suggested regular expression (RegEx) to check a SemVer string?](https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string)) -are considered snapshots; neither a release is created nor assets are uploaded. - -## Advanced/complex use cases - -**Releaser** is essentially a very thin wrapper to use the GitHub Actions context data along with the classes -and methods of PyGithub. - -Similarly to [actions/github-script](https://github.com/actions/github-script), users with advanced/complex requirements -might find it desirable to write their own Python script, instead of using **Releaser**. -In fact, since `shell: python` is supported in GitHub Actions, using Python does *not* require any Action. -For prototyping purposes, the following job might be useful: - -```yml - Release: - name: '馃摝 Release' - runs-on: ubuntu-24.04 - needs: - - ... - if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/master' || contains(github.ref, 'refs/tags/')) - steps: - - - uses: actions/download-artifact@v3 - - - shell: bash - run: pip install PyGithub --progress-bar off - - - name: Set list of files for uploading - id: files - shell: python - run: | - from github import Github - print("路 Get GitHub API handler (authenticate)") - gh = Github('${{ github.token }}') - print("路 Get Repository handler") - gh_repo = gh.get_repo('${{ github.repository }}') -``` - -Find a non-trivial use case at [msys2/msys2-autobuild](https://github.com/msys2/msys2-autobuild). diff --git a/releaser/action.yml b/releaser/action.yml deleted file mode 100644 index 62068a4..0000000 --- a/releaser/action.yml +++ /dev/null @@ -1,45 +0,0 @@ -# ==================================================================================================================== # -# Authors: # -# Unai Martinez-Corral # -# # -# ==================================================================================================================== # -# Copyright 2020-2025 The pyTooling Authors # -# # -# Licensed under the Apache License, Version 2.0 (the "License"); # -# you may not use this file except in compliance with the License. # -# You may obtain a copy of the License at # -# # -# http://www.apache.org/licenses/LICENSE-2.0 # -# # -# Unless required by applicable law or agreed to in writing, software # -# distributed under the License is distributed on an "AS IS" BASIS, # -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # -# See the License for the specific language governing permissions and # -# limitations under the License. # -# # -# SPDX-License-Identifier: Apache-2.0 # -# ==================================================================================================================== # -name: 'Releaser' -description: 'Publish releases, upload assets and update tip/nightly tags' -inputs: - token: - description: 'Token to make authenticated API calls; can be passed in using {{ secrets.GITHUB_TOKEN }}' - required: true - files: - description: 'Multi-line list of glob patterns describing the artifacts to be uploaded' - required: true - tag: - description: 'Name of the tag that corresponds to the tip/nightly pre-release' - required: false - default: tip - rm: - description: 'Whether to delete all the previous artifacts, or only replacing the ones with the same name' - required: false - default: false - snapshots: - description: 'Whether to create releases from any tag or to treat some as snapshots' - required: false - default: true -runs: - using: 'docker' - image: 'docker://ghcr.io/pytooling/releaser' diff --git a/releaser/composite/action.yml b/releaser/composite/action.yml deleted file mode 100644 index 3f4e638..0000000 --- a/releaser/composite/action.yml +++ /dev/null @@ -1,59 +0,0 @@ -# ==================================================================================================================== # -# Authors: # -# Unai Martinez-Corral # -# # -# ==================================================================================================================== # -# Copyright 2020-2025 The pyTooling Authors # -# # -# Licensed under the Apache License, Version 2.0 (the "License"); # -# you may not use this file except in compliance with the License. # -# You may obtain a copy of the License at # -# # -# http://www.apache.org/licenses/LICENSE-2.0 # -# # -# Unless required by applicable law or agreed to in writing, software # -# distributed under the License is distributed on an "AS IS" BASIS, # -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # -# See the License for the specific language governing permissions and # -# limitations under the License. # -# # -# SPDX-License-Identifier: Apache-2.0 # -# ==================================================================================================================== # -name: 'Releaser' -description: 'Publish releases, upload assets and update tip/nightly tags' -inputs: - token: - description: 'Token to make authenticated API calls; can be passed in using {{ secrets.GITHUB_TOKEN }}' - required: true - files: - description: 'Multi-line list of glob patterns describing the artifacts to be uploaded' - required: true - tag: - description: 'Name of the tag that corresponds to the tip/nightly pre-release' - required: false - default: tip - rm: - description: 'Whether to delete all the previous artifacts, or only replacing the ones with the same name' - required: false - default: false - snapshots: - description: 'Whether to create releases from any tag or to treat some as snapshots' - required: false - default: true -runs: - using: 'composite' - steps: - - - shell: bash - run: | - [ "$(source /etc/os-release && echo $VERSION_ID)" == "24.04" ] && UBUNTU_2404_ARGS='--break-system-packages' || unset UBUNTU_2404_ARGS - pip install --disable-pip-version-check --progress-bar off $UBUNTU_2404_ARGS PyGithub - - - shell: bash - run: '''${{ github.action_path }}/../releaser.py''' - env: - INPUT_TOKEN: ${{ inputs.token }} - INPUT_FILES: ${{ inputs.files }} - INPUT_TAG: ${{ inputs.tag }} - INPUT_RM: ${{ inputs.rm }} - INPUT_SNAPSHOTS: ${{ inputs.snapshots }} diff --git a/releaser/pyproject.toml b/releaser/pyproject.toml deleted file mode 100644 index 55ec8d7..0000000 --- a/releaser/pyproject.toml +++ /dev/null @@ -1,2 +0,0 @@ -[tool.black] -line-length = 120 diff --git a/releaser/releaser.py b/releaser/releaser.py deleted file mode 100755 index a75d1ad..0000000 --- a/releaser/releaser.py +++ /dev/null @@ -1,193 +0,0 @@ -#!/usr/bin/env python3 -# ==================================================================================================================== # -# Authors: # -# Patrick Lehmann # -# Unai Martinez-Corral # -# # -# ==================================================================================================================== # -# Copyright 2020-2025 The pyTooling Authors # -# # -# Licensed under the Apache License, Version 2.0 (the "License"); # -# you may not use this file except in compliance with the License. # -# You may obtain a copy of the License at # -# # -# http://www.apache.org/licenses/LICENSE-2.0 # -# # -# Unless required by applicable law or agreed to in writing, software # -# distributed under the License is distributed on an "AS IS" BASIS, # -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # -# See the License for the specific language governing permissions and # -# limitations under the License. # -# # -# SPDX-License-Identifier: Apache-2.0 # -# ==================================================================================================================== # -import re -from sys import argv as sys_argv, stdout, exit as sys_exit -from os import environ, getenv -from glob import glob -from pathlib import Path -from github import Github, GithubException -from subprocess import check_call - - -paramTag = getenv("INPUT_TAG", "tip") -paramFiles = getenv("INPUT_FILES", None).split() -paramRM = getenv("INPUT_RM", "false") == "true" -paramSnapshots = getenv("INPUT_SNAPSHOTS", "true").lower() == "true" -paramToken = ( - environ["GITHUB_TOKEN"] - if "GITHUB_TOKEN" in environ - else environ["INPUT_TOKEN"] - if "INPUT_TOKEN" in environ - else None -) -paramRepo = getenv("GITHUB_REPOSITORY", None) -paramRef = getenv("GITHUB_REF", None) -paramSHA = getenv("GITHUB_SHA", None) - - -def GetListOfArtifacts(argv, files): - print("路 Get list of artifacts to be uploaded") - args = files if files is not None else [] - if len(argv) > 1: - args += argv[1:] - if len(args) == 1 and args[0].lower() == "none": - print("! Skipping 'files' because it's set to 'none'.") - return [] - elif len(args) == 0: - stdout.flush() - raise (Exception("Glob patterns need to be provided as positional arguments or through envvar 'INPUT_FILES'!")) - else: - flist = [] - for item in args: - print(f" glob({item!s}):") - for fname in [fname for fname in glob(item, recursive=True) if not Path(fname).is_dir()]: - if Path(fname).stat().st_size == 0: - print(f" - ! Skipping empty file {fname!s}.") - continue - print(f" - {fname!s}") - flist.append(fname) - if len(flist) < 1: - stdout.flush() - raise (Exception("Empty list of files to upload/update!")) - return sorted(flist) - - -def GetGitHubAPIHandler(token): - print("路 Get GitHub API handler (authenticate)") - if token is not None: - return Github(token) - raise (Exception("Need credentials to authenticate! Please, provide 'GITHUB_TOKEN' or 'INPUT_TOKEN'")) - - -def CheckRefSemVer(gh_ref, tag, snapshots): - print("路 Check SemVer compliance of the reference/tag") - env_tag = None - if gh_ref[0:10] == "refs/tags/": - env_tag = gh_ref[10:] - if env_tag != tag: - rexp = r"^(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)(?:-(?P(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$" - semver = re.search(rexp, env_tag) - if semver == None and env_tag[0] == "v": - semver = re.search(rexp, env_tag[1:]) - tag = env_tag - if semver == None: - print(f"! Could not get semver from {gh_ref!s}") - print(f"! Treat tag '{tag!s}' as a release") - return (tag, env_tag, False) - else: - if semver.group("prerelease") is None: - # is a regular semver compilant tag - return (tag, env_tag, False) - elif snapshots: - # is semver compilant prerelease tag, thus a snapshot (we skip it) - print("! Skipping snapshot prerelease.") - sys_exit() - - return (tag, env_tag, True) - - -def GetRepositoryHandler(gh, repo): - print("路 Get Repository handler") - if repo is None: - stdout.flush() - raise (Exception("Repository name not defined! Please set 'GITHUB_REPOSITORY")) - return gh.get_repo(repo) - - -def GetOrCreateRelease(gh_repo, tag, sha, is_prerelease): - print("路 Get Release handler") - gh_tag = None - try: - gh_tag = gh_repo.get_git_ref(f"tags/{tag!s}") - except Exception: - stdout.flush() - - if gh_tag: - try: - return (gh_repo.get_release(tag), False) - except Exception: - return (gh_repo.create_git_release(tag, tag, "", draft=True, prerelease=is_prerelease), True) - else: - err_msg = f"Tag/release '{tag!s}' does not exist and could not create it!" - if sha is None: - raise (Exception(err_msg)) - try: - return ( - gh_repo.create_git_tag_and_release( - tag, "", tag, "", sha, "commit", draft=True, prerelease=is_prerelease - ), - True, - ) - except Exception: - raise (Exception(err_msg)) - - -def UpdateReference(gh_release, tag, sha, is_prerelease, is_draft): - print("路 Update Release reference (force-push tag)") - - if is_draft: - # Unfortunately, it seems not possible to update fields 'created_at' or 'published_at'. - print(" > Update (pre-)release") - gh_release.update_release( - gh_release.title, - "" if gh_release.body is None else gh_release.body, - draft=False, - prerelease=is_prerelease, - tag_name=gh_release.tag_name, - target_commitish=gh_release.target_commitish, - ) - - if sha is not None: - print(f" > Force-push '{tag!s}' to {sha!s}") - gh_repo.get_git_ref(f"tags/{tag!s}").edit(sha) - - -files = GetListOfArtifacts(sys_argv, paramFiles) -stdout.flush() -[tag, env_tag, is_prerelease] = CheckRefSemVer(paramRef, paramTag, paramSnapshots) -stdout.flush() -gh_repo = GetRepositoryHandler(GetGitHubAPIHandler(paramToken), paramRepo) -stdout.flush() -[gh_release, is_draft] = GetOrCreateRelease(gh_repo, tag, paramSHA, is_prerelease) -stdout.flush() - -if paramRM: - print("路 RM set. All previous assets are being cleared...") - for asset in gh_release.get_assets(): - print(f" - {asset.name}") - asset.delete_asset() -stdout.flush() - -if len(files) > 0: - print("路 Upload assets") - env = environ.copy() - env["GITHUB_TOKEN"] = paramToken - cmd = ["gh", "release", "upload", "--repo", paramRepo, "--clobber", tag] + files - print(f" > {' '.join(cmd)}") - check_call(cmd, env=env) - stdout.flush() -else: - print("! Skipping uploading assets because the file list is empty.") - -UpdateReference(gh_release, tag, paramSHA if env_tag is None else None, is_prerelease, is_draft)