Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-3091: Add verification guide and .rat-excludes.txt for release #3101

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .rat-excludes.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
.gitignore
.rat-excludes.txt
PULL_REQUEST_TEMPLATE.md
strings-2.parquet$
nested_array.avsc$
map_with_nulls.avsc$
map.avsc$
list_with_nulls.avsc$
fixedToInt96.avsc$
array.avsc$
allFromParquetOldBehavior.avsc$
allFromParquetNewBehavior.avsc$
all.avsc$
stringBehavior.avsc$
logicalType.avsc$
58 changes: 58 additions & 0 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,61 @@ Merge hash: 485658a5
Would you like to pick 485658a5 into another branch? (y/n):
```
For now just say n as we have 1 branch

# Release Verification
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! Is it better to add this to the parquet site: https://github.com/apache/parquet-site/blob/production/content/en/docs/Contribution%20Guidelines/releasing.md?


The Apache Arrow Release Approval process follows the guidelines defined at the
`Apache Software Foundation Release Approval <https://www.apache.org/legal/release-policy.html#release-approval>`_.

For a release vote to pass, a minimum of three positive binding votes and more
positive binding votes than negative binding votes MUST be cast.
Releases may not be vetoed. Votes cast by PMC members are binding, however,
non-binding votes are greatly encouraged and a sign of a healthy project.

In order to cast a vote individuals are expected to follow the following steps.

## Download source package, signature file, hash file and KEYS

The Release candidate will be present at `https://dist.apache.org/repos/dist/dev/parquet/`.
The RC folder will depend on the version and the release candidate id. See the following example files for
Apache Parquet 1.15.0 RC 1:
```
wget https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.15.0-rc1/apache-parquet-1.15.0.tar.gz
wget https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.15.0-rc1/apache-parquet-1.15.0.tar.gz.asc
wget https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.15.0-rc1/apache-parquet-1.15.0.tar.gz.sha512
wget https://dist.apache.org/repos/dist/release/parquet/KEYS
```

## Verify signature and hash

GnuPG is recommended, which can be install by:
- `yum install gnupg`, `apt-get install gnupg` on Linux based environments.
- `brew install gnupg` on macOS environments.


```
gpg --import KEYS
gpg --verify apache-parquet-1.15.0.tar.gz.asc apache-parquet-1.15.0.tar.gz
sha512sum --check apache-parquet-1.15.0.tar.gz.sha512
```

## Verify license header

Apache RAT is recommended to verify the license header, which can be dowload with the following command.

```
wget https://archive.apache.org/dist/creadur/apache-rat-0.16.1/apache-rat-0.16.1-bin.tar.gz
tar zxvf apache-rat-0.16.1-bin.tar.gz
```

You can check with the following command.
It will output a file list which doesn't include ASF license headers.
Please substitute `$PARQUET_SRC_FOLDER` with your `parquet-java` source folder from the following command.

```
java -jar apache-rat-0.16.1/apache-rat-0.16.1.jar -a -d apache-parquet-1.15.0.tar.gz -E $PARQUET_SRC_FOLDER/.rat-excludes.txt
```

## Verify building and tests

Check the [building section](../README.md#building)
1 change: 1 addition & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -498,6 +498,7 @@
<consoleOutput>true</consoleOutput>
<excludes>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pretty sure we can change excludes for excludesFile (https://creadur.apache.org/rat/apache-rat-plugin/rat-mojo.html#excludesFile) but I am unsure why some of the regex on the individual excludes don't seem to work with excludesFile when I ran:
java -jar apache-rat-0.16.1/apache-rat-0.16.1.jar -a -d apache-parquet-1.15.0.tar.gz -E $PARQUET_SRC_FOLDER/.rat-excludes.txt
I'll investigate how to consolidate those two lists

<exclude>.github/PULL_REQUEST_TEMPLATE.md</exclude>
<exclude>.rat-excludes.txt</exclude>
<exclude>**/*.parquet</exclude>
<exclude>**/*.avro</exclude>
<exclude>**/*.json</exclude>
Expand Down
Loading