Skip to content

gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants - #148081

Open
corona10 wants to merge 25 commits into
python:mainfrom
corona10:gh-145742-work
Open

gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants#148081
corona10 wants to merge 25 commits into
python:mainfrom
corona10:gh-145742-work

Conversation

@corona10

@corona10 corona10 commented Apr 4, 2026

Copy link
Copy Markdown
Member

@corona10

corona10 commented Apr 5, 2026

Copy link
Copy Markdown
Member Author

@Fidget-Spinner @markshannon PTAL when you have times :)

@corona10

corona10 commented Apr 8, 2026

Copy link
Copy Markdown
Member Author

I’ve just resolved all the conflicts. Gentle ping to @markshannon and @Fidget-Spinner.

@markshannon

Copy link
Copy Markdown
Member

Thanks for doing this.

There is work underway to remove the many variants of _LOAD_CONST_INLINE_BORROW that also pop, swap or rotate.
Once that work is done, this PR should become much simpler.
I'd recommend leaving this for a week or two before attempting to resolve conflicts.

@corona10

corona10 commented Apr 9, 2026

Copy link
Copy Markdown
Member Author

Got it

@corona10

Copy link
Copy Markdown
Member Author

@markshannon I've resolved conflicts with the latest main and simplified the pre-tagging/untagging to be path-based (auto-tagging in add_op() when the opcode is a BORROW variant). PTAL.

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested this with Py_STACKREF_DEBUG set?

Comment thread Include/internal/pycore_stackref.h Outdated
#define PyStackRef_FromPreTagged(ptr) _PyStackRef_FromPreTagged((uintptr_t)(ptr))

/* Tag a PyObject pointer as a borrowed operand for BORROW variants. */
#define PyStackRef_TagBorrow(ptr) \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't it need defining for Py_STACKREF_DEBUG as well?
Can you make this an inline function, for type safety?

Comment thread Python/executor_cases.c.h
PyObject *ptr = (PyObject *)CURRENT_OPERAND0_64();
value = PyStackRef_FromPyObjectBorrow(ptr);
_PyFrame_SetStackPointer(frame, stack_pointer);
value = PyStackRef_FromPreTagged(ptr);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to whitelist PyStackRef_FromPreTagged as non-escaping

@corona10

Copy link
Copy Markdown
Member Author

Have you tested this with Py_STACKREF_DEBUG set?

FYI, it's already broken see: #148718

@corona10
corona10 requested a review from markshannon April 20, 2026 16:35
@corona10

Copy link
Copy Markdown
Member Author

@markshannon gentle ping?

@markshannon

Copy link
Copy Markdown
Member

This looks fine, once updated to main.

The problem I have with it is the lying to the type system and tagged pointers.
The fix would be to differentiate between object pointers, other pointers, stackrefs and 64 bit integers in the DSL. But that's a lot of work for this small change.

So, can you either:

  • Make a new issue make those changes to the DSL in the future, and then we can merge this, or
  • Make the DSL changes in this PR

Comment thread Include/internal/pycore_stackref.h Outdated
Comment thread Python/bytecodes.c

tier2 pure op(_LOAD_CONST_INLINE_BORROW, (ptr/4 -- value)) {
value = PyStackRef_FromPyObjectBorrow(ptr);
value = PyStackRef_FromPreTagged(ptr);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we always cast /4 values to PyObject * even when they aren't pointers.

Comment thread Python/bytecodes.c Outdated
Comment thread Python/optimizer_analysis.c Outdated
@bedevere-app

bedevere-app Bot commented Apr 30, 2026

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@corona10

corona10 commented Apr 30, 2026

Copy link
Copy Markdown
Member Author

Make the DSL changes in this PR

I prefer this one, since this PR is not urgent thing :)

@corona10 corona10 changed the title gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants [WIP] gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants May 1, 2026
@corona10 corona10 changed the title [WIP] gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants May 1, 2026
@corona10
corona10 requested a review from markshannon May 1, 2026 10:04
@corona10

corona10 commented May 1, 2026

Copy link
Copy Markdown
Member Author

@markshannon
I’ve added a ^ marker to annotate the pretagged cache effect.

I’d like to know whether this is the kind of DSL you had in mind, or if you would prefer a different grammar. I would be happy to follow your suggestion if you have one.

PTAL.

    op ( LOAD_CONST_INLINE_BORROW, (ptr/4^ -- value) ) {
        value = PyStackRef_FromPreTagged(ptr);
    }

@corona10

corona10 commented May 1, 2026

Copy link
Copy Markdown
Member Author

FYI, the current Ubuntu CI failure is not related to this change; it's a Canonical infra issue that they are handling at this moment. https://x.com/ubuntu/status/2050112955132297652

@corona10

corona10 commented May 1, 2026

Copy link
Copy Markdown
Member Author

DSL Summary

  • The ^ marker on cache slots (e.g. ptr/4^) tells the DSL the operand is pre-tagged, so consumers declare uintptr_t + read_u64 instead of lying with PyObject *.
  • Producers (add_op, convert_global_to_const) now call a generated _PyUop_PrepareOperand0 dispatch keyed on the same marker, so no more hardcoded _LOAD_CONST_INLINE_BORROW checks, future ^ ops are wired in by DSL alone.

@corona10

corona10 commented May 6, 2026

Copy link
Copy Markdown
Member Author

Gentle ping @markshannon

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 90 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting changes stale Stale PR or inactive for long period of time. topic-JIT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants