use a script to download premade binaries instead of compiling them

[xrstf]

Summary

This resurrects #3431, but better.

The main issue I could not solve in the previous PR was @embik's wish to have checksums being checked when downloading a tool. It would have been trivial to just record the checksum for the host's specific OS/arch combo, because that's the binary we just downloaded, BUT trouble arises when anyone wants to update a tool like golangci-lint: If all I can do is update the checksum for my own OS/arch combo, then everyone on other machines will suddenly run into issues.

To solve this, the download script now doesn't get the full, final URL passed as an arg, but instead a URL pattern (with placeholders like {GOOS}). This enables the script to construct the URLs for any other OS/arch combo, download those, calculate the checksums and update all golangci-lint checksums in one go.

Since I want to use the same approach in my private projects, I worked on this on my own time and therefore created the repo at Codeberg.

Using it

Unless you want to update a tool, nothing changes for you. make lint will still just download golangci-lint and use it.

If you want to update a tool, you have to

• bump the version in the Makefile (as before)
• run the make command to download the tool (make imports would also, for example, work for refreshing golangci-lint) with the env variable UGET_UPDATE=true. This will instruct µget to not install the binary for you, but re-download and re-checksum all variants of the tool that is knows from the checksum file. I.e. UGET_UPDATE=true make lint will update all golangci-lint checksums hack/tools.checksums. Afterwards make lint just works as expected.

What Type of PR Is This?

/kind cleanup

Release Notes

NONE

[kcp-ci-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[xrstf]

/test all

[xrstf]

/test pull-kcp-lint

[xrstf]

/test pull-kcp-lint

[xrstf]

/test pull-kcp-lint

[xrstf]

/test pull-kcp-lint

[xrstf]

/test all

[xrstf]

/test all

[xrstf]

/retest

[xrstf]

/retest

[xrstf]

/test all

[xrstf]

/test all

[mjudeikis]

If you want to update a tool, you have to
bump the version in the Makefile (as before)
run the make command to download the tool (make imports would also, for example, work for refreshing golangci-lint) with the env variable UGET_UPDATE=true. This will instruct µget to not install the binary for you, but re-download and re-checksum all variants of the tool that is knows from the checksum file. I.e. UGET_UPDATE=true make lint will update all golangci-lint checksums hack/tools.checksums. Afterwards make lint just works as expected.

I dont quite get this.

1. Update version in make file
2. run make codegen or whatever you just updated
3. I got new binary but no update
4. I run GET_UPDATE=true make codegen and still no update.

I Im a bit lost on how to update the checksums now?

[mjudeikis]

/retest

[xrstf]

controller-gen accidentally was left in a superposition where it was a Go module (those are not checksummed), but it still had checksums in the checksum file from my previous testing. I have now corrected it and now the checksums will be updated as described.

[xrstf]

I've made a number of changes now based on your feedback:

• UGET_VERSIONED_BINARIES (new option) is now set to true in kcp, leading to binaries like hack/tools/httest-v0.3.3. This means I also removed the .PHONY's again.
• UGET_UPDATE now only warns about Go modules and them not having checksums, rather than aborting. This makes things like UGET_UPDATE=true make lint easier.
• UGET_UPDATE now also installs the appropriate binary instead of just updating the checksums. Again, this makes the previous command much easier to use (no need to run it twice). UGET_UPDATE_ONLY can be used for the old behaviour.

[xrstf]

/retest

[mjudeikis]

/lgtm
/approve

lets try and see

[kcp-ci-bot]

LGTM label has been added.

Details

Git tree hash: 6da73e24187c7d75236d156486052650f7717362

[kcp-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mjudeikis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [mjudeikis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mjudeikis]

/retest