Skip to content

Commit 2cecb06

Browse files
Merge pull request #1409 from LalatenduMohanty/issue-1387-hook-standard-resolution
feat(resolver): let hook profiles work with only a download hook
2 parents 2d88bca + fb7b165 commit 2cecb06

6 files changed

Lines changed: 259 additions & 77 deletions

File tree

‎docs/concepts/resolver-architecture.rst‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,15 +107,22 @@ CLI commands interact with providers through a common
107107
Hook profiles pass their ``index_url`` to the legacy
108108
``get_resolver_provider`` hook as ``sdist_server_url``. ``hook-sdist``
109109
defaults to PyPI, while ``hook-prebuilt`` requires an explicit index.
110+
A hook profile without a ``get_resolver_provider`` hook finds versions on
111+
its ``index_url`` and uses the ``download_source`` hook.
110112

111113
Per-package settings in YAML can select which provider to use and
112114
configure its parameters (index URL, tag pattern, etc.). When a package
113115
has a ``source:`` resolver configured, that resolver creates the provider.
114116
The ``hook-sdist`` and ``hook-prebuilt`` profiles delegate provider creation
115-
to the package's required ``get_resolver_provider`` hook and validate that it
116-
returns a :class:`~fromager.resolver.BaseProvider`.
117+
to the package's ``get_resolver_provider`` hook when it exists and validate
118+
that it returns a :class:`~fromager.resolver.BaseProvider`.
117119
The hook may ignore or replace the profile's index URL, but the profile's
118120
value is stable across Fromager commands.
121+
Without that hook, the profiles find versions on ``index_url``:
122+
``hook-sdist`` looks at sdists and wheels for any platform, like
123+
``pypi-git``, and ``hook-prebuilt`` looks at wheels for this platform, like
124+
``pypi-prebuilt``. The package's ``download_source`` hook then gets the
125+
artifact. A package with neither hook is an error.
119126
Otherwise, override plugins can replace the provider for a specific package
120127
via the ``get_resolver_provider`` hook, after which the legacy resolver
121128
settings are used.

‎docs/proposals/new-resolver-config.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,12 +147,17 @@ variants:
147147
[hooks](../reference/hooks.rst). The `resolver_provider` hook receives the
148148
`req` argument for access to the current requirement and
149149
`sdist_server_url=str(index_url)`. `index_url` defaults to PyPI. Hooks can
150-
return any source artifact (sdist, tarball, or git checkout).
150+
return any source artifact (sdist, tarball, or git checkout). Without a
151+
`resolver_provider` hook, Fromager finds versions on `index_url` in sdists
152+
and wheels for any platform, and the `download_source` hook gets the
153+
source. A package needs at least one of the two hooks.
151154

152155
- The `hook-prebuilt` profile is similar to `hook-sdist` but the downloaded
153156
artifact must be a pre-built wheel. Its required `index_url` replaces the
154157
legacy variant `wheel_server_url`; the hook receives it as
155-
`sdist_server_url` and may use, ignore, or replace it.
158+
`sdist_server_url` and may use, ignore, or replace it. Without a
159+
`resolver_provider` hook, Fromager finds versions on `index_url` in
160+
wheels for this platform.
156161

157162
- The `not-available` profile raises an error. It can be used to block a
158163
package and only enable it for a single variant.

‎docs/reference/hooks.rst‎

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -142,8 +142,8 @@ Resolver hooks
142142
``req_type`` and ``ignore_platform`` when those parameters are supported
143143
by its signature.
144144

145-
The ``hook-sdist`` and ``hook-prebuilt`` source profiles require this hook.
146-
They pass the following keyword arguments:
145+
The ``hook-sdist`` and ``hook-prebuilt`` source profiles call this hook
146+
when it exists. They pass the following keyword arguments:
147147

148148
* ``ctx``
149149
* ``req``
@@ -154,9 +154,17 @@ Resolver hooks
154154
* ``ignore_platform``
155155

156156
Older hooks may omit newer arguments; unsupported keyword arguments are
157-
filtered for compatibility. A missing hook or a return value that is not a
158-
:class:`~fromager.resolver.BaseProvider` is an error. Hook profiles never
159-
fall back to the default PyPI provider.
157+
filtered for compatibility. A return value that is not a
158+
:class:`~fromager.resolver.BaseProvider` is an error.
159+
160+
Without this hook, a hook profile finds versions on its ``index_url`` and
161+
the ``download_source`` hook gets the artifact. A package with neither
162+
hook is an error; use ``pypi-sdist`` or ``pypi-prebuilt`` instead.
163+
164+
.. versionchanged:: 0.102.0
165+
166+
``hook-sdist`` and ``hook-prebuilt`` no longer require this hook when
167+
the package has a ``download_source`` hook.
160168

161169
For a configured hook profile, ``sdist_server_url`` is the profile's
162170
``index_url``. ``hook-sdist`` defaults that field to PyPI;
@@ -275,8 +283,16 @@ Source hooks
275283
URL as an sdist into ``ctx.sdists_downloads``. ``hook-prebuilt`` downloads
276284
it as a wheel into ``ctx.wheels_prebuilt``. These defaults use the
277285
standard archive and wheel validation. Custom tarball URLs and git
278-
downloads require a download hook. The ``get_resolver_provider`` hook
279-
remains required for both profiles.
286+
downloads require a download hook.
287+
288+
Without a ``get_resolver_provider`` hook, ``hook-sdist`` finds
289+
versions on its ``index_url`` in sdists and wheels for any platform,
290+
and ``hook-prebuilt`` finds them in wheels for this platform. The
291+
download hook then gets the artifact. ``download_url`` is the URL of
292+
a matching file on the index, which can be a wheel, so the hook
293+
should get the source from its own location, such as a git tag.
294+
If the hook only clones a tag, use ``pypi-git`` instead and drop
295+
the hook.
280296

281297
To migrate a raw-tarball hook to ``hook-sdist``, specify ``build_sdist``
282298
in YAML and return its artifact kind explicitly:

‎src/fromager/packagesettings/_resolver.py‎

Lines changed: 68 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -781,9 +781,30 @@ class AbstractHookResolver(AbstractResolver, CooldownMixin):
781781
supports_override_hooks: typing.ClassVar[bool] = True
782782
"""Hook resolvers support override hooks."""
783783

784+
hookless_provider: typing.ClassVar[str]
785+
"""Profile to suggest for packages without hooks."""
786+
787+
def _find_resolver_hook(self, req: Requirement) -> typing.Callable | None:
788+
"""Return the package resolver hook, or None to resolve on the index.
789+
790+
Resolving on the index needs a ``download_source`` hook to get the
791+
source. Raise when the package has neither hook.
792+
"""
793+
hook = overrides.find_override_method(req.name, "get_resolver_provider")
794+
if hook is None and not overrides.find_override_method(
795+
req.name, "download_source"
796+
):
797+
raise ValueError(
798+
f"{req.name}: source resolver {self.provider!r} requires a "
799+
"get_resolver_provider or download_source override hook; "
800+
f"use {self.hookless_provider!r} for a package without hooks"
801+
)
802+
return hook
803+
784804
def _resolver_provider_from_hook(
785805
self,
786806
*,
807+
hook: typing.Callable,
787808
ctx: context.WorkContext,
788809
req: Requirement,
789810
req_type: requirements_file.RequirementType | None,
@@ -792,14 +813,7 @@ def _resolver_provider_from_hook(
792813
include_wheels: bool,
793814
ignore_platform: bool,
794815
) -> resolver.BaseProvider:
795-
"""Invoke the required package resolver hook and validate its result."""
796-
hook = overrides.find_override_method(req.name, "get_resolver_provider")
797-
if hook is None:
798-
raise ValueError(
799-
f"{req.name}: source resolver {self.provider!r} requires a "
800-
"get_resolver_provider override hook"
801-
)
802-
816+
"""Invoke the package resolver hook and validate its result."""
803817
try:
804818
provider = overrides.invoke(
805819
hook,
@@ -903,9 +917,11 @@ class HookSDistResolver(AbstractHookResolver):
903917
"""Call resolver_provider and download_source hook, build from source
904918
905919
The ``hook-sdist`` provider delegates resolution and download to
906-
plugin hooks. The downloaded artifact can be a source distribution,
907-
a tarball, or a git checkout. Download hooks can return ``DownloadedSource``
908-
with a concrete kind, or a legacy archive ``Path`` treated as an sdist.
920+
plugin hooks. Either hook is optional, but not both. Without a resolver
921+
hook, versions come from ``index_url``. The downloaded artifact can be a
922+
source distribution, a tarball, or a git checkout. Download hooks can
923+
return ``DownloadedSource`` with a concrete kind, or a legacy archive
924+
``Path`` treated as an sdist.
909925
Without a download hook, the resolved URL is downloaded as an sdist.
910926
Specify ``build_sdist`` explicitly: ``tarball`` to repack an existing
911927
sdist, or ``pep517`` to build from a git checkout.
@@ -929,15 +945,32 @@ class HookSDistResolver(AbstractHookResolver):
929945
download_kinds: typing.ClassVar[DownloadKindSet] = frozenset(
930946
{DownloadKind.sdist, DownloadKind.tarball, DownloadKind.git_checkout}
931947
)
948+
hookless_provider: typing.ClassVar[str] = "pypi-sdist"
932949

933950
def resolver_provider(
934951
self,
935952
ctx: context.WorkContext,
936953
req: Requirement,
937954
req_type: requirements_file.RequirementType | None,
938955
) -> resolver.BaseProvider:
939-
"""Return a source provider from the package resolver hook."""
956+
"""Return a source provider from the resolver hook or the index.
957+
958+
Without a ``get_resolver_provider`` hook, find versions on
959+
``index_url`` in sdists and wheels for any platform, like
960+
``pypi-git``. The ``download_source`` hook then gets the source.
961+
"""
962+
hook = self._find_resolver_hook(req)
963+
if hook is None:
964+
return resolver.PyPIProvider(
965+
include_sdists=True,
966+
include_wheels=True,
967+
sdist_server_url=str(self.index_url),
968+
constraints=ctx.constraints,
969+
req_type=req_type,
970+
ignore_platform=True,
971+
)
940972
return self._resolver_provider_from_hook(
973+
hook=hook,
941974
ctx=ctx,
942975
req=req,
943976
req_type=req_type,
@@ -964,10 +997,11 @@ class HookPrebuiltResolver(AbstractHookResolver):
964997
"""Call resolver_provider and download_source hook, use pre-built wheel
965998
966999
The ``hook-prebuilt`` provider delegates resolution and download to
967-
plugin hooks. The downloaded artifact must be a pre-built wheel. Download
968-
hooks can return ``DownloadedSource`` with kind ``prebuilt_wheel`` or a
969-
legacy wheel ``Path``. Without a download hook, the resolved URL is downloaded
970-
into ``ctx.wheels_prebuilt``.
1000+
plugin hooks. Either hook is optional, but not both. Without a resolver
1001+
hook, versions come from ``index_url``. The downloaded artifact must be a
1002+
pre-built wheel. Download hooks can return ``DownloadedSource`` with kind
1003+
``prebuilt_wheel`` or a legacy wheel ``Path``. Without a download hook,
1004+
the resolved URL is downloaded into ``ctx.wheels_prebuilt``.
9711005
9721006
Example::
9731007
@@ -986,15 +1020,32 @@ class HookPrebuiltResolver(AbstractHookResolver):
9861020
{DownloadKind.prebuilt_wheel}
9871021
)
9881022
resolves_prebuilt_wheel: typing.ClassVar[bool] = True
1023+
hookless_provider: typing.ClassVar[str] = "pypi-prebuilt"
9891024

9901025
def resolver_provider(
9911026
self,
9921027
ctx: context.WorkContext,
9931028
req: Requirement,
9941029
req_type: requirements_file.RequirementType | None,
9951030
) -> resolver.BaseProvider:
996-
"""Return a pre-built wheel provider from the package resolver hook."""
1031+
"""Return a pre-built wheel provider from the resolver hook or the index.
1032+
1033+
Without a ``get_resolver_provider`` hook, find versions on
1034+
``index_url`` in wheels for this platform, like ``pypi-prebuilt``.
1035+
The ``download_source`` hook then gets the wheel.
1036+
"""
1037+
hook = self._find_resolver_hook(req)
1038+
if hook is None:
1039+
return resolver.PyPIProvider(
1040+
include_sdists=False,
1041+
include_wheels=True,
1042+
sdist_server_url=str(self.index_url),
1043+
constraints=ctx.constraints,
1044+
req_type=req_type,
1045+
ignore_platform=False,
1046+
)
9971047
return self._resolver_provider_from_hook(
1048+
hook=hook,
9981049
ctx=ctx,
9991050
req=req,
10001051
req_type=req_type,

0 commit comments

Comments
 (0)