Skip to content

Commit

Permalink
[FLINK-33914][ci] Introduces a basic GitHub Actions workflow
Browse files Browse the repository at this point in the history
- Adds basic CI workflow
- Adds composite workflow for starting Flink's CI stages
- Updates CI helper scripts to support GitHub Actions
- Adds composite workflow for doing pre-compile checks
- Adds custom action to enable maven-utils.sh in workflows
- Adds custom action for initializing a job
  • Loading branch information
XComp committed Jan 31, 2024
1 parent fff63f4 commit a64e1c9
Show file tree
Hide file tree
Showing 9 changed files with 653 additions and 3 deletions.
72 changes: 72 additions & 0 deletions .github/actions/job_init/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
---
name: "Initializes the job"
description: "Does all the necessary steps to set up the job"
inputs:
jdk_version:
description: "The JDK version that's supposed to be used."
required: true
maven_repo_folder:
description: "The location of the local Maven repository (not setting this parameter will omit the caching of Maven artifacts)."
required: false
default: ""
source_directory:
description: "Specifies the directory from which the code should be moved from (needed for containerized runs; not setting this parameter will omit moving the checkout)."
required: false
default: ""
target_directory:
description: "Specifies the directory to which the code should be moved to (needed for containerized runs; not setting this parameter will omit moving the checkout)."
required: false
default: ""
runs:
using: "composite"
steps:
- name: "Initializes GHA_PIPELINE_START_TIME with the job's start time"
shell: bash
run: |
job_start_time="$(date --rfc-3339=seconds)"
echo "GHA_PIPELINE_START_TIME=${job_start_time}" >> "${GITHUB_ENV}"
echo "The job's start time is set to ${job_start_time}."
- name: "Set JDK version to ${{ inputs.jdk_version }}"
shell: bash
run: |
echo "JAVA_HOME=$JAVA_HOME_${{ inputs.jdk_version }}_X64" >> "${GITHUB_ENV}"
echo "PATH=$JAVA_HOME_${{ inputs.jdk_version }}_X64/bin:$PATH" >> "${GITHUB_ENV}"
- name: "Setup Maven package cache"
if: ${{ inputs.maven_repo_folder != '' }}
uses: actions/cache@v4
with:
path: ${{ inputs.maven_repo_folder }}
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven

- name: "Moves checkout content from ${{ inputs.source_directory }} to ${{ inputs.target_directory }}."
if: ${{ inputs.source_directory != '' && inputs.target_directory != '' }}
shell: bash
run: |
mkdir -p ${{ inputs.target_directory }}
# .scalafmt.conf is needed for Scala format checks
# .mvn is needed to make the Maven wrapper accessible in test runs
mv ${{ inputs.source_directory }}/* \
${{ inputs.source_directory }}/.scalafmt.conf \
${{ inputs.source_directory }}/.mvn \
${{ inputs.target_directory }}
42 changes: 42 additions & 0 deletions .github/actions/run_mvn/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
---
name: "Runs Maven Command"
description: "Executes Flink's Maven wrapper with the passed Maven parameters."
inputs:
working_directory:
description: "The directory under which the Maven command should be executed."
default: "${{ github.workspace }}"
maven-parameters:
description: "Any parameters of the Maven command."
default: ""
env:
description: "Any environment-specifics that are meant to be available in the context of the call."
default: ""
runs:
using: "composite"
steps:
- name: "Runs Maven Command"
working-directory: "${{ inputs.working_directory }}"
shell: bash
run: |
# errexit needs to be disabled explicitly here because maven-utils.sh handles the error if a mirror is not available
set +o errexit
${{ inputs.env }} source "./tools/ci/maven-utils.sh"
run_mvn ${{ inputs.maven-parameters }}
40 changes: 40 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This workflow is meant for as a basic CI run covering the most-important features
# and default Java version. It is meant to run before a PullRequest is merged.

name: "Flink CI (beta)"

on:
push:
workflow_dispatch:

permissions: read-all

jobs:
pre-compile-checks:
name: "Pre-compile Checks"
uses: ./.github/workflows/template.pre-compile-checks.yml
ci:
name: "Default (Java 8)"
uses: ./.github/workflows/template.flink-ci.yml
with:
environment: 'PROFILE="-Dinclude_hadoop_aws"'
jdk_version: 8
secrets:
s3_bucket: ${{ secrets.IT_CASE_S3_BUCKET }}
s3_access_key: ${{ secrets.IT_CASE_S3_ACCESS_KEY }}
s3_secret_key: ${{ secrets.IT_CASE_S3_SECRET_KEY }}
Loading

0 comments on commit a64e1c9

Please sign in to comment.