Possible tweaks to speed up the compilation process?

Until now, I have just been compiling Syncthing using the default Go settings as follows.

go run build.go

However, it can take a few minutes to compile, especially when starting from scratch. One thing that bothers me a little is that the CPU usage while compiling is very low, e.g. only ~10% with Ryzen 4350G (4 cores + 4 threads). Everything is located on a fast SSD RAID, so storage is not the culprit either.

Are there any possible tweaks to speed up the compilation process, e.g. by increasing the multithreading or such?

The first compilation needs to populate module cache (dependencies), so I guess what you see is a network/disk io being the limit.

I am not aware of any tweaks.

Hmm, I was thinking of something in the vein of make -jX to increase the number of threads when compiling the Linux kernel and such.

It would the network here, as downloading from GitHub is kind of slow where I live.

It’s multithreaded by default.

Yeah, I was just wondering whether it was possible to maybe increase the number of threads, as you can do when compiling the C languages.

I have another question related to the compilation though. It is possible to make the output more verbose when building Syncthing? Right now, especially on successive builds, there is basically no output on the screen if there have been no file changes, and the command prompt just hangs with a blank screen for about 1-2 minutes with no visible progress until the build is ready.

A ./build.sh or go run build.go when there have been no changes takes less than a second for me, on my computer from 2015. A full git clean -fxd && go clean -cache && go run build.go takes about 25s, and that’s with rebuilding everything including the Go standard library. This also produces a lot of output because it’s doing a lot of things. There’s something strange with your setup that has nothing to do with the number of compilation threads. If you’re on Windows the first guess is obvious.

You can see what precisely takes time though, set BUILDDEBUG=1.

1 Like

I know what you mean, but not this time :sweat_smile:. Windows Defender is disabled with GPO and there is no other AV software on this machine.

This is what the actual output looks like with BUILDDEBUG=1.

go run build.go -goos windows -no-upgrade -goarch 386 zip

runError: git describe --always --dirty --abbrev=8
... in 83.0196ms
runError: git describe --always --abbrev=0
... in 60.0133ms
runError: git branch -a --contains
... in 116.0256ms
Notice: Next generation GUI will not be built; see --with-next-gen-gui.
rm -r syncthing.exe
runError: git describe --always --dirty --abbrev=8
... in 65.0155ms
runError: git describe --always --abbrev=0
... in 72.0152ms
runError: git branch -a --contains
... in 107.0247ms
runError: git describe --always --dirty --abbrev=8
... in 93.0199ms
runError: git describe --always --abbrev=0
... in 96.0223ms
runError: git branch -a --contains
... in 125.0302ms
runError: goversioninfo -o X:\syncthing\cmd\syncthing\resource.syso
... in 52.8538259s
runError: git show -s --format=%ct
... in 57.0119ms
runPrint: go build -v -trimpath -tags noupgrade -ldflags -w -X github.com/syncthing/syncthing/lib/build.Version=v1.14.0-rc.1.dev.133.gfb078068 -X github.com/syncthing/syncthing/lib/build.Stamp=1613411453 -X github.com/syncthing/syncthing/lib/build.User=tomasz86 -X github.com/syncthing/syncthing/lib/build.Host=tomasz86 -X github.com/syncthing/syncthing/lib/build.Tags=noupgrade github.com/syncthing/syncthing/cmd/syncthing
... in 2.7686189s
runError: signtool.exe sign /fd sha1 syncthing.exe
... in 5.0008ms
Codesign: signing failed:
syncthing-windows-386-v1.14.0-rc.1.dev.133.gfb078068.zip
... build completed in 57.8219399s

Apparently goversioninfo takes forever to run :slightly_frowning_face:.

Indeed! Now you know where to continue debugging.

Yeah, I will try to see what the problem is about. I am not sure about the git errors either.

At least with this knowledge, I can compile my own test builds without using goversioninfo altogether, which will reduce the process to ~7s.

Edit 1:

Without goversioninfo

... build completed in 861.1932ms

so this is the culprit for sure.

Edit 2:

Just for the record, building the binary for Android also takes 40+ seconds, but there the toolchain from the Android NDK is the culprit.

Interestingly, after updating Go to 1.16, the compilation time has dropped to only 5-10 seconds now (with goversioninfo enabled). If this stands, then I have no more complaints then :slightly_smiling_face:.