Skip to content

Commit 4e6e564

Browse files
authored
chore: Hide internal public utility APIs (#25963)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #. ## Rationale for this change <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. Please explain the problem you are trying to solve in terms of the user-visible behavior, rather than the implementation. For example, "The code in `foo.rs` doesn't handle nulls" is a symptom of the implementation. "COUNT(DISTINCT) returns wrong results when the column contains nulls" is the user-visible problem. --> See the previous PR for rationale - #25495 This PR adds #[doc(hidden)] and comments for public APIs that is used as internal utility. The major categories are - already includes `#doc(hidden)` - Already explained in the comment that this public API is internal-only - Optimizer utilities, with strict preconditions to use it correctly. ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here, but it is sometimes worth providing a summary of the individual changes in this PR. --> comments only ## What is the testing strategy for this PR? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code Briefly describe how this PR is tested, and point to the specific tests you added. For example: 'This new feature is covered by the `sqllogictest` cases added in `foo.slt`'. If this PR does not add tests, explain why. For example, if the change is already covered by existing tests, please mention it. You should also check the `codecov` bot reply on this PR to confirm the changed code is exercised. --> ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please add the `api change` label. -->
1 parent d137137 commit 4e6e564

16 files changed

Lines changed: 141 additions & 19 deletions

File tree

‎datafusion-examples/.examples-docs-check.E3y1ZS/README.md‎

Whitespace-only changes.

‎datafusion/expr/src/logical_plan/plan.rs‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2854,12 +2854,18 @@ impl Filter {
28542854
/// Create a new filter operator.
28552855
///
28562856
/// Skips the type-checking, window function check and dealiasing done in
2857-
/// [Self::try_new]. For internal use in DataFusion only.
2857+
/// [Self::try_new].
28582858
///
28592859
/// **Preconditions:**
28602860
/// - the `predicate` expression returns a boolean value
28612861
/// - the `predicate` expression is not aliased
28622862
/// - the `predicate` expression contains no window function calls
2863+
///
2864+
/// # Public Only for Internal Use:
2865+
///
2866+
/// This is not a public API and is for internal use only; see [API policy] for details.
2867+
///
2868+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
28632869
#[doc(hidden)]
28642870
pub fn new(predicate: Expr, input: Arc<LogicalPlan>) -> Self {
28652871
Self { predicate, input }

‎datafusion/physical-expr-common/src/regex.rs‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,11 @@ pub fn compile_regex(
106106
/// compiles a single pattern up front, before it reads any value, takes
107107
/// `None`: it compiles that pattern whatever the values are.
108108
///
109-
/// This is `pub` only so that the crates that call the kernels can reach it.
110-
// Not public API.
109+
/// # Public Only for Internal Use:
110+
///
111+
/// This is not a public API and is for internal use only; see [API policy] for details.
112+
///
113+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
111114
#[doc(hidden)]
112115
pub fn explain_regexp_kernel_error(
113116
function_name: &str,

‎datafusion/physical-expr/src/aggregate.rs‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -353,6 +353,11 @@ impl AggregateExprBuilder {
353353
self
354354
}
355355

356+
/// # Public Only for Internal Use:
357+
///
358+
/// This is not a public API and is for internal use only; see [API policy] for details.
359+
///
360+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
356361
#[doc(hidden)]
357362
pub fn human_display_alias(mut self, alias: impl Into<String>) -> Self {
358363
let alias = alias.into();
@@ -702,6 +707,11 @@ impl AggregateFunctionExpr {
702707
.map(AggregateHumanDisplay::expression)
703708
}
704709

710+
/// # Public Only for Internal Use:
711+
///
712+
/// This is not a public API and is for internal use only; see [API policy] for details.
713+
///
714+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
705715
#[doc(hidden)]
706716
pub fn human_display_alias(&self) -> Option<&str> {
707717
self.human_display

‎datafusion/physical-plan/src/aggregates/mod.rs‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -936,8 +936,9 @@ impl AggregateExec {
936936
///
937937
/// # Public Only for Internal Use:
938938
///
939-
/// This is not part of the supported public API, it's made public for internal
940-
/// optimizer usage.
939+
/// This is not a public API and is for internal use only; see [API policy] for details.
940+
///
941+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
941942
///
942943
/// # TopK Optimization Overview
943944
///
@@ -1142,6 +1143,13 @@ impl AggregateExec {
11421143
/// leaves the aggregate in a consistent state. Ineligible aggregates return
11431144
/// `None`. Eligible aggregates are returned unchanged when the limit is zero
11441145
/// or no tighter than the existing limit.
1146+
///
1147+
/// # Public Only for Internal Use:
1148+
///
1149+
/// This is not a public API and is for internal use only; see [API policy] for details.
1150+
///
1151+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
1152+
#[doc(hidden)]
11451153
pub fn try_optimize_distinct_soft_limit(
11461154
mut self,
11471155
limit: usize,
@@ -1167,6 +1175,13 @@ impl AggregateExec {
11671175
///
11681176
/// The caller must preserve the schema, filters, and ordering requirements.
11691177
/// Changing expressions in a TopK aggregate drops its specialization.
1178+
///
1179+
/// # Public Only for Internal Use:
1180+
///
1181+
/// This is not a public API and is for internal use only; see [API policy] for details.
1182+
///
1183+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
1184+
#[doc(hidden)]
11701185
pub fn with_new_aggr_exprs(
11711186
&self,
11721187
aggr_expr: impl Into<Arc<[Arc<AggregateFunctionExpr>]>>,

‎datafusion/physical-plan/src/distribution_requirements.rs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,12 @@ impl InputDistributionRequirements {
188188
/// Independent per-child requirements are intentionally ignored here, use
189189
/// [`Self::child_satisfaction`] for those checks. An empty result means all
190190
/// co-partitioning requirements are satisfied.
191+
///
192+
/// # Public Only for Internal Use:
193+
///
194+
/// This is not a public API and is for internal use only; see [API policy] for details.
195+
///
196+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
191197
#[doc(hidden)]
192198
pub fn unsatisfied_co_partitioned_children(
193199
&self,

‎datafusion/physical-plan/src/joins/cross_join.rs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,19 @@ impl CrossJoinExec {
187187
/// This function should be called BEFORE inserting any repartitioning
188188
/// operators on the join's children. Check [`super::HashJoinExec::swap_inputs`]
189189
/// for more details.
190+
///
191+
/// # Public Only for Internal Use:
192+
///
193+
/// This is not a public API and is for internal use only; see [API policy] for details.
194+
///
195+
/// Direct use of this API in downstream projects is discouraged because
196+
/// correctness depends on strict preconditions. See the notes above for
197+
/// correct usage.
198+
///
199+
/// This API may change frequently as internal join optimizations evolve.
200+
///
201+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
202+
#[doc(hidden)]
190203
pub fn swap_inputs(&self) -> Result<Arc<dyn ExecutionPlan>> {
191204
let new_join =
192205
CrossJoinExec::new(Arc::clone(&self.right), Arc::clone(&self.left));

‎datafusion/physical-plan/src/joins/hash_join/exec.rs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1496,6 +1496,19 @@ impl HashJoinExec {
14961496
/// physical optimizer rule to determine a good join order, which is
14971497
/// executed before the `EnforceDistribution` rule (the rule that may
14981498
/// insert `RepartitionExec` operators).
1499+
///
1500+
/// # Public Only for Internal Use:
1501+
///
1502+
/// This is not a public API and is for internal use only; see [API policy] for details.
1503+
///
1504+
/// Direct use of this API in downstream projects is discouraged because
1505+
/// correctness depends on strict preconditions. See the notes above for
1506+
/// correct usage.
1507+
///
1508+
/// This API may change frequently as internal join optimizations evolve.
1509+
///
1510+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
1511+
#[doc(hidden)]
14991512
pub fn swap_inputs(
15001513
&self,
15011514
partition_mode: PartitionMode,

‎datafusion/physical-plan/src/joins/join_hash_map.rs‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,11 +97,16 @@ use hashbrown::hash_table::Entry::{Occupied, Vacant};
9797
/// At runtime we choose between using `JoinHashMapU32` and `JoinHashMapU64` which oth implement
9898
/// `JoinHashMapType`.
9999
///
100-
/// ## Note on use of this trait as a public API
101-
/// This is currently a public trait but is mainly intended for internal use within DataFusion.
102100
/// For example, we may compare references to `JoinHashMapType` implementations by pointer equality
103101
/// rather than deep equality of contents, as deep equality would be expensive and in our usage
104102
/// patterns it is impossible for two different hash maps to have identical contents in a practical sense.
103+
///
104+
/// # Public Only for Internal Use:
105+
///
106+
/// This is not a public API and is for internal use only; see [API policy] for details.
107+
///
108+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
109+
#[doc(hidden)]
105110
pub trait JoinHashMapType: Send + Sync {
106111
fn extend_zero(&mut self, len: usize);
107112

‎datafusion/physical-plan/src/joins/mod.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,13 @@ use utils::JoinHashMapType;
7979
/// contains rows.
8080
///
8181
/// [`NullEquality::NullEqualsNothing`]: datafusion_common::NullEquality::NullEqualsNothing
82+
///
83+
/// # Public Only for Internal Use:
84+
///
85+
/// This is not a public API and is for internal use only; see [API policy] for details.
86+
///
87+
/// [API policy]: https://datafusion.apache.org/contributor-guide/api-health.html#internal-public-apis
88+
#[doc(hidden)]
8289
pub enum Map {
8390
HashMap(Box<dyn JoinHashMapType>),
8491
ArrayMap(ArrayMap),

0 commit comments

Comments
 (0)