Using build.go or build.sh - why? Why not use golang’s own build system or at least makefile? And why doesn’t it put the executable in $GOPATH/bin, but instead creates it in a subdirectory in its own directory? I am pretty sure that’s against golang principles.
Syncthing is fully go-gettable. The wrappers are necessary to do additional magic. Make is not always available, while Go is (per definition).
For that matter, I challenge you to find any nontrivial Go application that does not use a build script of some sort, be it written in Go, bash, make or whatever.
But it’s not go-gettable:
$ go get github.com/syncthing/syncthing
package github.com/syncthing/syncthing
imports github.com/syncthing/syncthing
imports github.com/syncthing/syncthing: no buildable Go source files in /home/username/projects/go/src/github.com/syncthing/syncthingNope
$ go get github.com/syncthing/syncthing/cmd
package github.com/syncthing/syncthing/cmd
imports github.com/syncthing/syncthing/cmd
imports github.com/syncthing/syncthing/cmd: no buildable Go source files in /home/username/projects/go/src/github.com/syncthing/syncthing/cmdOnce more, with emphasis. 
go get github.com/syncthing/syncthing/cmd/...
There are binary releases as well…
Indeed, it works like that. Why not include that one command in build instructions? It puts the executable in $GOPATH/bin and I consider that a big plus.
Because the resulting binary is unsupportable, unfortunately. But if it works, that’s cool.
What do you mean “unsupportable”?
It’s not version tagged so a bug report won’t have any context, and it uses whatever random version of dependencies you happen to have lying around in your gopath so I don’t really know code is in there.
This brings up an interesting issue for me. I’ve created a “port” for Syncthing for FreeBSD, which is just a Makefile and some other stuff that are used to build a binary. This is done in order to have a binary for every version of FreeBSD and on every type of hardware supported (arm for example). Another reason for this port is that we ensure the binary is built from the source (we lock the version of syncthing source and also lock the version of source for all dependencies) and pristine and uncontaminated.
Not that we don’t trust the builders in particular, but you never know, and we have a policy of preferring to build from source. This is one of the reasons I like Syncthing rather than the binary only btsync. You never know what binaries you didn’t build are doing.)
So, this port is used to create binary packages for every version and platform of FreeBSD, and that binary package is distributed and used by users. Would bug reports from users of this binary be rejected?
This is already done in the existing repo - are you locking to some different set of dependencies?
If the answer to the question above is “yes”, or syncthing -version with your build says “unknown-dev”, then yes, because I have absolutely no idea what code that is and troubleshooting it is pointless.
If you use the dependencies that syncthing ought to be built with (i.e. those that are included in the repo) and correctly tag the binary (like the official build.go does), then no, it’s all good.
No, definitely not. I’m using the source from the github release page:
https://github.com/syncthing/syncthing/archive/v0.9.18.tar.gz
And I’m going to extra trouble to make double extra sure that I use the exact same sources that your build uses. (Editing to say this should have been same sources for deps.)
The thing is, our build system prevents fetching source during build. So there is a “fetch” stage of the build which fetches everything ahead of time, and normally I would just list the tar above during that. But since the build system fetches source as part of the build, I also fetch the sources it would fetch and put them in place before doing the build.
The output of syncthing --version with my binary is syncthing v0.9.18 ... (Edit: yep, same versions of deps)
Then there are no issues.
It shouldn’t do that, actually. There’s a step that checks if godep is available in the $PATH; if it isn’t, the script goes ahead and grabs that and a few other things for calculating test coverage etc that are not strictly necessary for the build as such. If godep is available, it should build without any kind of internet access just with the stuff in the repo.
And godep is really just a convenience, the build could be made without it. Perhaps that might be worth fixing, so there are no external dependencies necessary for the build apart from the Go environment.
There, as of latest master (9d81669) there’s no dependency on godep to build. It still wants git to figure out the correct tag, but the build script prints the commands it runs and the environment it sets so you should be able to replicate it if you don’t want to use that.
$ go run build.go clean build
rm -r bin
rm -r Godeps/_workspace/pkg
rm -r Godeps/_workspace/bin
rm -r /Users/jb/pkg/darwin_amd64/github.com/syncthing
rm -r syncthing
rm -r syncthing.exe
GOPATH=/Users/jb/src/github.com/syncthing/syncthing/Godeps/_workspace:/Users/jb
go build -ldflags -w -X main.Version v0.9.18+16-g9d81669 -X main.BuildStamp 1411902595 -X main.BuildUser jb -X main.BuildHost jborg-mbp-2.local -X main.BuildEnv default ./cmd/syncthing
Awesome, that’s great to hear.
Thanks! That’ll be very helpful and make things a lot easier!
And, you helped me realize that I was setting GOPATH wrong, causing it to not use the copy of the deps in the workspace (which I hadn’t seen, somehow), so I had done a lot of extra work having it download them and put them in place (all the while ensuring they were the right ones). So now that I see that, I should be able to simplify even more.
Once 0.9.19 is out, I can remove all that. Though I will probably not have git in the build environment and instead just use the build command you listed, with version set to the same as the tarball I download.
If you could call the “official” build command as go run build.go -version=v0.9.19 and it did not depend on anything further (i.e no git, nothing to download), would that be viable? (Ideally I’d like packages to use the same build structure as I do, to minimize unexpected weirdness.)
Yes, that would be great. Edit: Oh, and by the way, I’ve created ports of the syncthing CLI and the discovery server (with init script) for those who want to use those. If the build for those worked the same too, that’d be awesome too.
I was just trying out building the latest master and it seems like it is finding the deps but not it’s own src, which it for some reason is looking in the wrong place for. It seems to be looking in src/github.com/syncthing/syncthing for the internal stuff (auto config, etc.). Am I doing something wrong?
Maybe, because the build is succeeding on the build server. What are you doing and what is the error, exactly? It could also be I screwed up something in the latest build changes.