Files
llvm/.github/workflows/release-binaries.yml
Diogo Teles Sant'Anna f3524e9aeb Hashpin sensitive dependencies and configure dependabot to update them automatically (#75859)
Closes #75620 

As I mentioned on the issue, this PR aims to hash-pin the CI
dependencies used on sensitive context -- i.e., they either are called
with write permissions, or are being used to build critical artifacts
like a release. In summary, this PR brings 3 changes:

1. Hash pin GitHub Actions called on sensitive context
2. Hash pin python dependencies used on sensitive context
3. Configure dependabot to automatically update those hashes
I'm further explaining the steps bellow.

The dependencies in format of GitHub Actions, I simply hash-pinned them.
I also made sure to keep the human-readable version as comments at the
same line.

At the
[release-tasks.yml](https://github.com/llvm/llvm-project/blob/main/.github/workflows/release-tasks.yml)
file, I've changed the installation method of some python dependencies
to install them considering their hashpinning. That required the
generation of a requirements file that had all the correct hashes, and
for that I used [pip-tools](https://pypi.org/project/pip-tools/2.0.0/).

While configuring dependabot, I set it to send a monthly PR updating all
the GitHub Actions, and a weekly PR to update any python dependency
required by
[/llvm/docs/requirements.txt](https://github.com/llvm/llvm-project/blob/main/llvm/docs/requirements.txt).

Let me know if you have any questions or concerns, I'd be happy to
clarify and help.

Thanks!

---------

Signed-off-by: Diogo Teles Sant'Anna <diogoteles@google.com>
2024-01-15 12:30:41 -08:00

168 lines
5.5 KiB
YAML

name: Release Binaries
on:
push:
tags:
- 'llvmorg-*'
workflow_dispatch:
inputs:
upload:
description: 'Upload binaries to the release page'
required: true
default: true
type: boolean
tag:
description: 'Tag to build'
required: true
type: string
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 8 1 * *'
permissions:
contents: read # Default everything to read-only
jobs:
prepare:
name: Prepare to build binaries
runs-on: ubuntu-22.04
if: github.repository == 'llvm/llvm-project'
outputs:
release-version: ${{ steps.validate-tag.outputs.release-version }}
flags: ${{ steps.validate-tag.outputs.flags }}
build-dir: ${{ steps.validate-tag.outputs.build-dir }}
rc-flags: ${{ steps.validate-tag.outputs.rc-flags }}
ref: ${{ steps.validate-tag.outputs.ref }}
upload: ${{ steps.validate-tag.outputs.upload }}
steps:
- name: Checkout LLVM
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Validate and parse tag
id: validate-tag
# In order for the test-release.sh script to run correctly, the LLVM
# source needs to be at the following location relative to the build dir:
# | X.Y.Z-rcN | ./rcN/llvm-project
# | X.Y.Z | ./final/llvm-project
#
# We also need to set divergent flags based on the release version:
# | X.Y.Z-rcN | -rc N -test-asserts
# | X.Y.Z | -final
run: |
tag="${{ github.ref_name }}"
trimmed=$(echo ${{ inputs.tag }} | xargs)
[[ "$trimmed" != "" ]] && tag="$trimmed"
if [ "$tag" = "main" ]; then
# If tag is main, then we've been triggered by a scheduled so pass so
# use the head commit as the tag.
tag=`git rev-parse HEAD`
fi
if [ -n "${{ inputs.upload }}" ]; then
upload="${{ inputs.upload }}"
else
upload="true"
fi
bash .github/workflows/set-release-binary-outputs.sh "${{ github.actor }}" "$tag" "$upload"
# Try to get around the 6 hour timeout by first running a job to fill
# the build cache.
fill-cache:
name: "Fill Cache ${{ matrix.os }}"
needs: prepare
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- ubuntu-22.04
steps:
- name: Checkout LLVM
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
ref: ${{ needs.prepare.outputs.ref }}
- name: Install Ninja
uses: llvm/actions/install-ninja@22e9f909d35b50bd1181709564bfe816eaeaae81 # main
- name: Setup sccache
uses: hendrikmuhs/ccache-action@ca3acd2731eef11f1572ccb126356c2f9298d35e # v1.2.9
with:
max-size: 250M
key: sccache-${{ matrix.os }}-release
variant: sccache
- name: Build Clang
run: |
cmake -G Ninja -C clang/cmake/caches/Release.cmake -DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache -DCMAKE_POSITION_INDEPENDENT_CODE=ON -S llvm -B build
ninja -v -C build clang
build-binaries:
name: ${{ matrix.target.triple }}
permissions:
contents: write # To upload assets to release.
needs:
- prepare
- fill-cache
runs-on: ${{ matrix.target.runs-on }}
strategy:
fail-fast: false
matrix:
target:
- triple: x86_64-linux-gnu-ubuntu-22.04
os: ubuntu-22.04
runs-on: ubuntu-22.04-16x64
debian-build-deps: >
chrpath
gcc-multilib
ninja-build
steps:
- name: Checkout LLVM
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
ref: ${{ needs.prepare.outputs.ref }}
path: ${{ needs.prepare.outputs.build-dir }}/llvm-project
- name: Setup sccache
uses: hendrikmuhs/ccache-action@ca3acd2731eef11f1572ccb126356c2f9298d35e # v1.2.9
with:
max-size: 250M
key: sccache-${{ matrix.target.os }}-release
save: false
variant: sccache
- name: Install Brew build dependencies
if: matrix.target.brew-build-deps != ''
run: brew install ${{ matrix.target.brew-build-deps }}
- name: Install Debian build dependencies
if: matrix.target.debian-build-deps != ''
run: sudo apt install ${{ matrix.target.debian-build-deps }}
- name: Set macOS build env variables
if: runner.os == 'macOS'
run: |
echo "MACOSX_DEPLOYMENT_TARGET=10.9" >> "$GITHUB_ENV"
- name: Build and test release
run: |
${{ needs.prepare.outputs.build-dir }}/llvm-project/llvm/utils/release/test-release.sh \
${{ needs.prepare.outputs.flags }} \
-triple ${{ matrix.target.triple }} \
-use-ninja \
-no-checkout \
-use-cmake-cache \
-no-test-suite \
-configure-flags "-DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache"
- name: Upload binaries
if: ${{ always() && needs.prepare.outputs.upload == 'true' }}
run: |
sudo apt install python3-github
${{ needs.prepare.outputs.build-dir }}/llvm-project/llvm/utils/release/github-upload-release.py \
--token ${{ github.token }} \
--release ${{ needs.prepare.outputs.release-version }} \
upload \
--files ${{ needs.prepare.outputs.build-dir }}/clang+llvm-${{ needs.prepare.outputs.release-version }}-${{ matrix.target.triple }}.tar.xz