cmd/link: linkerFlagSupported drops flags required by hermetic toolchains without sysroots

[cerisier]

Go version

go version go1.25.5 darwin/arm64

Output of go env in your module/workspace:

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/corentinkerisit/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/corentinkerisit/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/h0/nqcw70jj7p10x4gccxg3db5h0000gn/T/go-build1435066892=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/Users/corentinkerisit/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/corentinkerisit/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/corentinkerisit/code/github.com/golang/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/corentinkerisit/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/corentinkerisit/code/github.com/golang/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.25.5'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

I cross-compile a Go program that uses cgo with a hermetic clang/LLD toolchain (no implicit sysroot / no default CRT+lib search paths). The toolchain requires passing explicit flags on every linker-driver invocation so that crt objects and runtime libraries can be found.

When cgo is enabled, cmd/link performs external linking and probes whether the external linker supports certain flags, notably disabling PIE (-no-pie/-nopie).

The probe is done via linkerFlagSupported, which filters the external linker argv down to an allowlist before invoking the driver.

In our hermetic toolchain, the filtered argv becomes invalid (because required CRT/lib search flags are removed), so the probe fails for reasons unrelated to -no-pie/-nopie support. cmd/link then concludes the flag is unsupported and omits it, which causes clang (Linux defaulting to PIE) to attempt a PIE link and fail with relocation errors.

Adding some logging in the cmd/link source code, here are the flags that are passed to the go linker:

-target x86_64-linux-gnu --sysroot=/dev/null -nodefaultlibs -fuse-ld=lld
-Wl,-no-as-needed -Wl,-z,relro,-z,now -rtlib=compiler-rt
-Bbazel-out/linux_x86_64-fastbuild/bin/external/toolchains_llvm_bootstrapped+/runtimes/crt_objects_directory_linux
-Lbazel-out/linux_x86_64-fastbuild/bin/external/toolchains_llvm_bootstrapped+/runtimes/libcxx/libcxx_library_search_directory
-Lbazel-out/linux_x86_64-fastbuild/bin/external/toolchains_llvm_bootstrapped+/runtimes/glibc/glibc_library_search_directory
-Lbazel-out/linux_x86_64-fastbuild/bin/external/toolchains_llvm_bootstrapped+/runtimes/libunwind/libunwind_library_search_directory
-L. bazel-out/linux_x86_64-fastbuild-ST-53b361441bc9/bin/external/toolchains_llvm_bootstrapped++http_archive+compiler-rt/clang_rt.builtins.static_/libclang_rt.builtins.static.a
-Wl,--push-state -Wl,--as-needed -lm -lpthread -lc -ldl -lrt -lutil -Wl,--pop-state

Note the --sysroot=/dev/null and other hermetic search paths provided using -B, -L and -l especially since -nodefaultlibs is passed. Similarly, the compiler-rt builtins are passed manually.

What did you see happen?

During linkerFlagSupported check, cmd/link invokes the external linker driver with a trimmed argv that is invalid for hermetic toolchains because required CRT/lib search flags are removed.

Adding logs in the cmd/link, here is the linker invocation performed by linkerFlagSupported:

-m64 -target x86_64-linux-gnu
--sysroot=/dev/null -fuse-ld=lld
-Wl,-no-as-needed -Wl,-z,relro,-z,now -Wl,--push-state -Wl,--as-needed -Wl,--pop-state -pthread 
-o /var/folders/h0/nqcw70jj7p10x4gccxg3db5h0000gn/T/go-link-232801632/a.out 
-no-pie
/var/folders/h0/nqcw70jj7p10x4gccxg3db5h0000gn/T/go-link-232801632/trivial.c

From wrapper logging, the probe invocation fails like:

ld.lld: error: cannot open crt1.o: No such file or directory
ld.lld: error: cannot open crti.o: No such file or directory
ld.lld: error: cannot open crtbegin.o: No such file or directory
ld.lld: error: unable to find library -lstdc++
ld.lld: error: unable to find library -lm
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lpthread
ld.lld: error: unable to find library -lc
ld.lld: error: cannot open crtend.o: No such file or directory
ld.lld: error: cannot open crtn.o: No such file or directory
clang++: error: linker command failed with exit code 1

Because this probe fails, cmd/link decides -no-pie is not supported, even tho the fail is unrelated, and therefore does not pass the flag in the final external link.

On Linux, clang defaults to PIE, so the final link fails with a relocation error consistent with attempting to link non-PIC Go objects as PIE:

ld.lld: error: relocation R_X86_64_64 cannot be used against symbol 'type:.eq.runtime.Frame'; recompile with -fPIC
>>> defined in .../go.o
>>> referenced by go.go
>>>               .../go.o:(.rodata+0x...)

What did you expect to see?

I expected cmd/link to keep required linker flags so that the link could be performed using a fully hermetic sysroot free toolchain.

I am aware that the usecase is niche but this comes from an effort to build a fully hermetic cross-compilation toolchain for Bazel that doesn't require any sysroot nor any user configuration. The toolchain is gaining popularity for its simplicity of configuration and total hermeticity it provides as well as free CGO cross-compilation when using rules_go. (See https://github.com/cerisier/toolchains_llvm_bootstrapped).

As part of this effort, we already provided a number of fixes in foreign build systems or compiler wrappers that make assumptions about a regular compiler toolchain where the driver can do its job fully implicity and everything works.

I'm not entirely sure of what is the best solution here and would love to have your feedback on it.

I fear that allowing the flags required by this invocation will fix it for now but as this is an ongoing effort, we may find additional flags that shouldn't be filtered out from cmd/link.

There is a question of whether our toolchain should provide a transient sysroot, built on the fly, which would remove the need for explicit -L, -l but we would still need to add support for -resource-dir on clang in my case so that the driver can find compiler-rt libs implicitely.

Another idea would be to be able to explicitly avoid specific checks similar to what autoconf premits ?

Really eager to have your thoughts here.

[gabyhelp]

Related Issues

• cmd/link: wrong flag order when checking supported linker flags #69350 (closed)
• cmd/link: inconsistent argument order in hostLink() and linkerFlagSupported() #69954 (closed)
• cmd/link: relative paths are not respected when checking supported linker flags #59952 (closed)
• cmd/link: cgo cross linking using the clang --target flag creates crashing programs #73406
• cmd/go: Can't override linkmode when setting buildmode=pie #44480 (closed)
• cmd/cgo: cannot link (cgo) on linux/arm with `-no-pie` #45940 (closed)
• cmd/link: add internal linking support for pie build mode on arm #50405
• cmd/link: Unable to cross-compile darwin/amd64 PIE on Linux #35831 (closed)
• cmd/cgo: linux-{386,amd64}-clang and linux-arm builders failing #26197 (closed)
• cmd/go: build fails when setting linker to lld using ldflags #27110 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

Can we just add some more flags to trimLinkerArgv? We started trimming arguments in https://go.dev/cl/134057. The essential point of that was to remove -l options. It also removes -L options but I'm not sure that is essential. And I'm not sure whether you need to keep them. You clearly need to keep -B options, and I don't see any problem with modifying trimLinkerArgv to keep -B options. (You also mention -resource-dir though I don't see that in the earlier command line; in any case it should be OK to keep -resource-dir if it is needed.)