Skip to content

feat: implement expressions, cut, and aliases for RNTuple.arrays via formulate - #1705

Open
KaranSinghDev wants to merge 23 commits into
scikit-hep:mainfrom
KaranSinghDev:feat/rntuple-expressions
Open

feat: implement expressions, cut, and aliases for RNTuple.arrays via formulate #1705
KaranSinghDev wants to merge 23 commits into
scikit-hep:mainfrom
KaranSinghDev:feat/rntuple-expressions

Conversation

@KaranSinghDev

@KaranSinghDev KaranSinghDev commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Description
This PR implements initial support for expressions, cut, and aliases in RNTuple.arrays() by delegating parsing to formulate and evaluation to ak.numexpr, as discussed in #1673.

Key Changes
1.formulate.from_root() parses expressions and extracts ast.variables to ensure minimal I/O (we only fetch required fields).
2.Evaluation happens post-materialization via ak.numexpr.evaluate().
3.cut expressions generate a boolean mask applied to the resulting Awkward array.
4.Added regression test in tests/test_1673_rntuple_expressions.py.

@ariostas Let me know your thoughts on this.

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.17949% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.92%. Comparing base (7916fe6) to head (7c71f32).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/uproot/behaviors/RNTuple.py 87.17% 1 Missing and 4 partials ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/uproot/behaviors/RNTuple.py 75.75% <87.17%> (+0.97%) ⬆️

... and 1 file with indirect coverage changes

Comment thread src/uproot/behaviors/RNTuple.py Outdated
@KaranSinghDev
KaranSinghDev force-pushed the feat/rntuple-expressions branch 2 times, most recently from b8230b3 to dc6e2fc Compare August 20, 2026 08:43
@KaranSinghDev

Copy link
Copy Markdown
Contributor Author

@ariostas I was thinking if we should put the virtual arr args first on function, the invalid ones would fail before loading dependencies and parsing

@ariostas

Copy link
Copy Markdown
Member

I was thinking if we should put the virtual arr args first on function, the invalid ones would fail before loading dependencies and parsing

Yeah, the virtual check should come checks. Expressions won't be supported for virtual arrays (there's already a check there), so it should fail right away.

@ariostas

Copy link
Copy Markdown
Member

Also, I see that the tests are failing because numexpr is not installed. I forgot that numexpr is not a dependency of formulate. So we should also add it as a dependency as well.

What you have so far looks good to me. I think the one thing it might be worth thinking a bit about is if there should be some simple check that decides whether the cut should be applied before of after evaluation. If the cut throws away most of the data then it's worth applying it before, but if it let's most things through, then it's better to apply it at the end.

@KaranSinghDev
KaranSinghDev force-pushed the feat/rntuple-expressions branch from 3f5426c to fa7be54 Compare August 27, 2026 17:14
@ariostas

Copy link
Copy Markdown
Member

Hmm I guess Pyodide might be a problem. I'll see what we can do about that

@KaranSinghDev

Copy link
Copy Markdown
Contributor Author

Hi @ariostas I tried to lok in it, since numexpr does not have a WebAssembly wheel, so making it a strict dependency seems to be breaking the Uproot installation for browser users. I think this can be fixed by using an environment marker in pyproject.toml ,"numexpr; sys_platform != 'emscripten'"

Also working on the codecov coverage. Let me know your thoughts on this.

@ariostas

Copy link
Copy Markdown
Member

Yeah, I think for now we can just add the marker to the pyproject.toml and skip the tests in Pyodide. I'm gonna see how feasible it is to have a numexpr wheel for pyodide.

As for codecov. Don't worry too much about it. If the tests ends up needing to be really convoluted just to hit coverage, then they might not be worth it. We might need to reconsider that threshold since we haven't been hitting it for most PRs.

@ariostas

Copy link
Copy Markdown
Member

I opened pydata/numexpr#568. We'll see what they say.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants