Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-43104][BUILD] Set
shadeTestJar
of protobuf
module to false
### What changes were proposed in this pull request? This pr aims to set `shadeTestJar` of `protobuf` module to `false` to skip shade `spark-protobuf_**-tests.jar` process. ### Why are the changes needed? When `shadeTestJar` is true, `maven-shade-plugin` always try to find(sometime is downloading) some non-existent jars: ``` Downloading from central: https://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar [WARNING] Could not get tests for org.apache.hadoop:hadoop-client-api:jar:3.3.5:compile Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9-tests.jar [WARNING] Could not get tests for org.tukaani:xz:jar:1.9:compile Downloading from gcs-maven-central-mirror: https://maven-central.storage-download.googleapis.com/maven2/org/slf4j/slf4j-api/2.0.7/slf4j-api-2.0.7-tests.jar Downloaded from gcs-maven-central-mirror: https://maven-central.storage-download.googleapis.com/maven2/org/slf4j/slf4j-api/2.0.7/slf4j-api-2.0.7-tests.jar (0 B at 0 B/s) [WARNING] Could not get tests for com.google.protobuf:protobuf-java:jar:3.22.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar [WARNING] Could not get tests for org.apache.curator:curator-framework:jar:2.13.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/logging/log4j/log4j-slf4j2-impl/2.20.0/log4j-slf4j2-impl-2.20.0-tests.jar [WARNING] Could not get tests for org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.20.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/jetbrains/annotations/17.0.0/annotations-17.0.0-tests.jar [WARNING] Could not get tests for org.jetbrains:annotations:jar:17.0.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/logging/log4j/log4j-1.2-api/2.20.0/log4j-1.2-api-2.20.0-tests.jar [WARNING] Could not get tests for org.apache.logging.log4j:log4j-1.2-api:jar:2.20.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/threeten/threeten-extra/1.7.1/threeten-extra-1.7.1-tests.jar [WARNING] Could not get tests for org.threeten:threeten-extra:jar:1.7.1:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/yetus/audience-annotations/0.13.0/audience-annotations-0.13.0-tests.jar [WARNING] Could not get tests for org.apache.yetus:audience-annotations:jar:0.13.0:compile Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.14.2/jackson-databind-2.14.2-tests.jar [WARNING] Could not get tests for com.fasterxml.jackson.core:jackson-databind:jar:2.14.2:compile Downloading from central: https://repo.maven.apache.org/maven2/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0-tests.jar [WARNING] Could not get tests for com.google.code.findbugs:jsr305:jar:3.0.0:runtime Downloading from central: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.9.1/snappy-java-1.1.9.1-tests.jar [WARNING] Could not get tests for org.xerial.snappy:snappy-java:jar:1.1.9.1:compile Downloading from central: https://repo.maven.apache.org/maven2/javax/activation/activation/1.1.1/activation-1.1.1-tests.jar [WARNING] Could not get tests for javax.activation:activation:jar:1.1.1:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/hive/hive-storage-api/2.8.1/hive-storage-api-2.8.1-tests.jar [WARNING] Could not get tests for org.apache.hive:hive-storage-api:jar:2.8.1:compile Downloading from central: https://repo.maven.apache.org/maven2/org/spark-project/spark/unused/1.0.0/unused-1.0.0-tests.jar [WARNING] Could not get tests for org.spark-project.spark:unused:jar:1.0.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.12.17/scala-library-2.12.17-tests.jar [WARNING] Could not get tests for org.scala-lang:scala-library:jar:2.12.17:compile Downloading from central: https://repo.maven.apache.org/maven2/com/github/luben/zstd-jni/1.5.4-2/zstd-jni-1.5.4-2-tests.jar [WARNING] Could not get tests for com.github.luben:zstd-jni:jar:1.5.4-2:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/logging/log4j/log4j-core/2.20.0/log4j-core-2.20.0-tests.jar [WARNING] Could not get tests for org.apache.logging.log4j:log4j-core:jar:2.20.0:runtime Downloading from central: https://repo.maven.apache.org/maven2/org/apache/curator/curator-recipes/2.13.0/curator-recipes-2.13.0-tests.jar [WARNING] Could not get tests for org.apache.curator:curator-recipes:jar:2.13.0:compile Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.14.2/jackson-annotations-2.14.2-tests.jar [WARNING] Could not get tests for com.fasterxml.jackson.core:jackson-annotations:jar:2.14.2:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-client-runtime/3.3.5/hadoop-client-runtime-3.3.5-tests.jar [WARNING] Could not get tests for org.apache.hadoop:hadoop-client-runtime:jar:3.3.5:compile Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.14.2/jackson-core-2.14.2-tests.jar [WARNING] Could not get tests for com.fasterxml.jackson.core:jackson-core:jar:2.14.2:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/logging/log4j/log4j-api/2.20.0/log4j-api-2.20.0-tests.jar [WARNING] Could not get tests for org.apache.logging.log4j:log4j-api:jar:2.20.0:compile Downloading from central: https://repo.maven.apache.org/maven2/org/apache/curator/curator-client/2.13.0/curator-client-2.13.0-tests.jar [WARNING] Could not get tests for org.apache.curator:curator-client:jar:2.13.0:compile ``` But the shaded `spark-protobuf_**-tests.jar` was not used now and sbt assembly didn't do this. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions - Check ` build/mvn clean install -pl connector/protobuf` and all test passed Closes apache#40753 from LuciferYang/protobuf-shadeTestJar-false. Lead-authored-by: yangjie01 <[email protected]> Co-authored-by: YangJie <[email protected]> Signed-off-by: Chao Sun <[email protected]>
- Loading branch information