This mostly applies to test suites, but could potentially apply to other checks.
We should add a notion of CI mode to scripts and/or our Makefile where checks bail out on first failure.
For pytest, this might take the form of adding the --exitfirst flag only in CI, perhaps controlled by an environment variable.