Simplified Release Procedures

The Current Situation

Currently, the Hipcheck project has the following artifacts we release periodically.

• hc: The main Hipcheck binary, produced by the hipcheck package.
• hc-update: The Hipcheck self-updater, generated by cargo-dist.
• hipcheck-macros: A Hipcheck-internal library.
• Docker image: published to Docker Hub.

These are published to the following places:

• hc:
  • Crates.io (as hipcheck)
  • GitHub Releases w/ install script generated by cargo-dist
• hc-update:
  • GitHub Releases w/ install script generated by cargo-dist - hipcheck-macros`:
  • Crates.io
• Docker image:
  • Docker Hub

When we cut a new release of hc, the flow has been:

1. Update the CHANGELOG.md for the project with release notes for the new version, open a PR, and get that merged.
2. Run cargo release locally to make a new release of hc to Crates.io. At the end of that process, cargo release creates and publishes a tag for the new version
3. This triggers cargo-dist's CI release job on GitHub. This job then builds our pre-built binaries and produces the GitHub Release with our install script and the hc-update tool.
4. This release triggers our Docker release build, which publishes our image to Docker Hub.

The Problems

There are quite a few problems with the above setup:

1. Inconsistency: When a user installs hc from the install script built by cargo-dist and included in each GitHub Release, they get the hc-update tool, which allows Hipcheck to self-update through the hc update subcommand. This means that users installing from the install script get a strictly better outcome than users installing from Crates.io.
2. Bad Release Staging: The current setup has, for hc at least, three separate stages of "releasing" a new version, each of which can fail independently:
   1. Publishing to Crates.io with cargo release
   2. Publishing a GitHub Release with cargo-dist
   3. Publishing a container image to Docker Hub

Proposal

We institute a new rule: One Artifact, One Release. This means, for any artifact we're publishing, there's exactly one release event that publishes to one place.

Let's walk through what this means in more detail, looking first at hc, and then at the soon-to-be-released Rust plugin SDK.

Releasing hc

Under this proposal, a release for hc would look like this:

1. Write the CHANGELOG.md update for the new version, make a PR, and get it merged.
2. Tag the new version as hipcheck-v<NEW_VERSION>, with <NEW_VERSION> replaced with the SemVer version we're releasing (X.Y.Z).

That's it! Then, cargo-dist would build our release artifacts, including the pre-built binaries and the install scripts, and make the GitHub Release when it's done.

Once the GitHub Release is made, the Docker build will run, building a Docker container to publish with the version of the container image's curl | sh install script set to the just-released version. Since this image is effectively just a wrapper around this install script, it should run without issue.

The cargo-dist folks at Axo.dev have expressed an interest in handling generation and publication of Docker images themselves in the future, which would fit in nicely with this flow as well.

Also, if we wanted to generate any additional artifacts for a release in the future, like SBOMs or SLSA attestations, they'd fit right into this same process, being attached to the GitHub Release.

Releasing hipcheck-sdk

In the near future we'll start publishing a new hipcheck-sdk crate which provides a Rust SDK for plugin authors to use when writing plugins in Rust. This would be a library, and would be released in its own similar way:

1. Update the CHANGELOG.md for this project to reflect the new version. Make a PR and get this CHANGELOG.md merged.
2. Push a tag for hipcheck-sdk-v<NEW_VERSION> with <NEW_VERSION> reflecting the new version.

We could then have GitHub CI set up to identify the new Git tag pushed, and run cargo publish to publish the new version from CI onto Crates.io.

This has some nice properties, including automatically permitting anyone with access to push tags to cut new releases of the crate, and allowing us to established trusted publication of crates from CI with attestations in the future.

What We'd Lose

The first, most conspicious thing we'd lose here is that we'd stop publishing the hipcheck-macros crate to Crates.io. This is really a piece of logic internal to Hipcheck, but is one we have to publish as a separate crate because it exposes a procedural macro, which needs its own crate; and that crate has to be public because the hipcheck crate itself is published on Crates.io, and public crates can't have dependencies published to other registries or not published to a registry at all.

The second thing we'd lose (though this has substantial benefits) is that we'd no longer publish the hipcheck crate to Crates.io. This means users would no longer be able to run cargo install hipcheck or cargo binstall hipcheck to install Hipcheck locally. This is a reduction in flexibility for users, but does simplify our processes, make releases more reliable, and enable us to have more flexibility in the internal structure of Hipcheck itself.

Which brings me to a big, but as yet un-discussed, benefit…

Vendoring Salsa

The thing that finally tipped me over into wanting to pursue this simplification was the looking challenge of vendoring the salsa crate. salsa is a Rust crate for incremental computation which we rely on heavily within Hipcheck. Unfortunately, the maintainers of salsa stopped publishing releases to Crates.io years ago, and salsa itself has gone through 1 major refactor, with a second currently underway, entirely away from the public versions. If you want to depend on salsa as published to Crates.io, you have to use salsa 1.0, which is no longer supported and has a number of deficits.

Normally, this isn't a problem, as users of salsa can specify it as a git dependency in their Cargo.toml file, and use the package key on the dependency to specify the salsa-2022 package (the 1st majorly-rewritten version, now itself in the process of being replaced by Salsa 3.0). However, because hipcheck has so far been published to Crates.io, this option is not available to us. If we wanted to use the newer versions of salsa ourselves, we'd need to vendor them into some sort of hipcheck-incremental crate, which we'd publish to Crates.io ourselves, and which we'd need to sync with the upstream Salsa project periodically.

We did tinker with some efforts to do this, but quickly found it to be reasonably complicated to do. We do not believe pursuing this or maintaining this vendored version over the long term would be a good use of the team's time and effort.

Finally, pursuing this would raise the additional question of the value of exposing these "should-be-internal-but-have-to-be-external" crates, hipcheck-macros and hipcheck-incremental. Neither are intended for others to use, and while we can certainly document this and tell people not to use them, we can't stop anyone from using them, and exposing these APIs publicly is awkward and creates risk and complexity for the Hipcheck project.