Your installer says success. Was the binary actually installed?

A failed download can leave an installer with nothing to extract. If the script continues and ends with a successful echo, CI can turn green even though the program is missing. Test the installation's result inside the same container, not just the installer's exit code.

Make installation success explicit

After running the installer, check its expected binary and execute a harmless, noninteractive smoke command. Adapt this path and command to your program:

binary="$HOME/.local/bin/mytool"
test -x "$binary" || { echo "Installer did not create $binary" >&2; exit 1; }
"$binary" --version

The binary and installer must be checked in the same container. A separate step on the GitHub runner sees a different filesystem.

Add the four-distribution workflow

Download installer-smoke.yml

  1. Save the workflow as .github/workflows/installer-smoke.yml.
  2. Change install.sh to your standalone installer's path.
  3. Replace $HOME/.local/bin/mytool and --version with your installed binary and a smoke command that fails when it cannot start.
  4. Review packages, timeout and supported images. Add packaging configuration to the workflow's trigger paths when archive layout depends on it.
  5. Open a pull request or run the workflow manually. Inspect the saved JSON report for failed images.

The workflow uses OpsScript Gate v0.9.4 to run Debian 12, Ubuntu 22.04/24.04 and Alpine 3.20. It embeds the checked-out installer in a subshell, preserves its failure status and runs the binary check afterwards. An explicit exit 0 in the installer cannot skip the check.

Downloads require network access. The example provisions Bash, curl, CA certificates and tar. That verifies installation with these prerequisites; it does not prove the installer works on a bare image. Only the target script is mounted, so installers requiring sibling repository files need another setup.

A real false-success case

In a public fork test of Latitude's CLI installer, an Ubuntu 22.04 job downloaded content that was not gzip. Extraction and moving the binary failed, but the installer still printed “Installation finished”. The binary check correctly failed because $HOME/.lsh/lsh was absent.

A later run with installation checks passed all four images. The earlier failure demonstrates why postconditions matter; it does not establish an Ubuntu-specific defect. These are fork validations, not an upstream endorsement.

What a green result proves

The tested artifact installed and its smoke command succeeded in that container during that run. HTTP errors, rate limits or changing release assets can affect repeat runs. Containers share the host kernel; this is not a full-machine test or a test of every application feature.

Read the source guide · Diagnose missing commands and interpreters