Repository navigation
Commit 61399f8
committed
[KYUUBI #7789] Bump Paimon 2.1.0 and enable Paimon tests for Spark 4.1
### Why are the changes needed?
Paimon 1.4.2 has no Spark 4.1 build, so the Spark 4.1 profile had to depend on the Spark 4.0 artifact (`paimon-spark-4.0_2.13`) and keep `PaimonTest` excluded. Paimon 2.1.0 ships `paimon-spark-4.1_2.13`, which unblocks running the Paimon authz suite on Spark 4.1.
- `paimon.version` 1.4.2 -> 2.1.0 and `paimon.artifact` -> `paimon-spark-${spark.binary.version}_${scala.binary.version}` for the Spark 4.1/4.2/4.3 profiles
- Spark 4.1: drop `org.apache.kyuubi.tags.PaimonTest` from `maven.plugin.scalatest.exclude.tags`
- Spark 4.2/4.3: `PaimonTest` stays excluded; both profiles alias `spark.binary.version` to 4.1, so they keep resolving `paimon-spark-4.1_2.13`
- Spark 4.0 keeps Paimon 1.4.2: the Spark 4.0 build of Paimon 2.x declares Spark 4.0.3, but its classes reference the Spark 4.1-only `org.apache.spark.sql.connector.catalog.constraints.Constraint` (`Table.constraints()`), so it cannot run on Spark 4.0
### How was this patch tested?
`PaimonCatalogRangerSparkExtensionSuite`, Java 17 and Scala 2.13:
- Spark 4.1.2 + Paimon 2.1.0: 37/37 passed
- Spark 4.0.3 + Paimon 1.4.2: 37/37 passed
- Spark 4.0.3 + Paimon 2.0.0 and 2.1.0: fails with `NoClassDefFoundError: org.apache.spark.sql.connector.catalog.constraints.Constraint`, raised inside the authz reflection helper - hence the Spark 4.0 profile stays on 1.4.2
`HiveCatalogRangerSparkExtensionSuite` on Spark 4.1.2 + Paimon 2.1.0: 53/53 passed.
```
build/mvn -B test -pl extensions/spark/kyuubi-spark-authz -am -Pspark-4.1 -Pscala-2.13 -Pspark-provided \
-Dtest=none -DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.ranger.PaimonCatalogRangerSparkExtensionSuite
```
### Was this patch assisted by generative AI tooling?
Assisted-by: Codex with gemini-4-argon
Closes #7789 from pan3793/paimon-2.1.
Closes #7789
c84ee87 [Cheng Pan] nit: fix comment wording
25f329c [Cheng Pan] nit
c24af36 [Cheng Pan] Bump Paimon 2.1.0 and enable Paimon tests for Spark 4.1
Authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>1 parent 564627e commit 61399f8
1 file changed
Lines changed: 9 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2080 | 2080 | | |
2081 | 2081 | | |
2082 | 2082 | | |
| 2083 | + | |
| 2084 | + | |
2083 | 2085 | | |
2084 | 2086 | | |
2085 | 2087 | | |
| |||
2103 | 2105 | | |
2104 | 2106 | | |
2105 | 2107 | | |
2106 | | - | |
2107 | | - | |
2108 | | - | |
2109 | | - | |
| 2108 | + | |
| 2109 | + | |
| 2110 | + | |
2110 | 2111 | | |
2111 | 2112 | | |
2112 | 2113 | | |
| |||
2132 | 2133 | | |
2133 | 2134 | | |
2134 | 2135 | | |
2135 | | - | |
2136 | | - | |
| 2136 | + | |
| 2137 | + | |
2137 | 2138 | | |
2138 | 2139 | | |
2139 | 2140 | | |
| |||
2161 | 2162 | | |
2162 | 2163 | | |
2163 | 2164 | | |
2164 | | - | |
2165 | | - | |
| 2165 | + | |
| 2166 | + | |
2166 | 2167 | | |
2167 | 2168 | | |
2168 | 2169 | | |
| |||
0 commit comments