Request for 64k binaries


(Audrius Butkevicius) #21

I doubt this is the case here, as it’s failing very very early. It’s probably some alignment issue.


(Jakob Borg) #22

Yeah, probably some interaction with the larger page size somehow, I have no clue how that works internally in Go, so… Maybe some constant isn’t tweaked that should be, so it allocates a 64k page for every 4k page it wanted or whatever.


(Dustin) #23

There is some go-lang discussion of this here from a few months ago: http://grokbase.com/t/gg/golang-nuts/152vdsmx4e/go-nuts-panic-on-arm-cross-compile

Seems like I may need to modify go to set _PhysPageSize to 64k manually and recompile go. Even then though segfaulting may occur.

Perhaps for now, go & syncthing are paused for 64k arm architectures…


(Jakob Borg) #24

Maybe try to revive the subject with Go 1.5, as much of the relevant code seems to have changed there? Or at least so that the patches that were floating around earlier don’t apply any more…


(Dustin) #25

I got it with go 1.5 and bootstraping 1.4. The trick was to manually edit the arch_arm.h to account for the correct _PhysPageSize then do your make.bash or all.bash appropriately.

For anyone interested in (master as of today with 64k page on arm7) binary: https://dl.dropboxusercontent.com/u/15524812/syncthing

Thanks for the help. I am hoping to have an install on the Western Digital EX4100 before too long now. Now I just need to customize syncthing to account for the persist-able directories on the NAS. I guess I should start reading the docs now for how to customize syncthing locations :smile: and interface binding for the webui.


(Jakob Borg) #26

And it runs? Can you provide a writeup of what you needed to change, exactly? If it’s not too onerous, it might be something we can add to our standard build images…


(Dustin) #27

Yes, it runs fine now. I’ll do a write up tonight or tomorrow, especially if it could lead to some regular binaries :smiley:

This is only my second day dealing with Go (coming from Java world), and I’ve got to say Go has its own fair share of complexities.


(Jakob Borg) #28

I guess, especially when hacking the compiler for an unsupported architecture :stuck_out_tongue:


ST build assistance
(Dustin) #29

As promised a writeup on how to get syncthing to run on ARM with 64bit pagesize.

Assumed no existing install of go, instructions from scratch build.

% git clone https://go.googlesource.com/go $HOME/go1.4

% cd $HOME/go1.4

% git checkout release-branch.go1.4

% cd src/runtime

% vi arch_arm.h

Remove any conditional logic so that the variable PhysPageSize is set like the line below:

PhysPageSize = 65536,

Then save and quit with “:wq”

% cd ..

% ./make.bash

Let it build and finish then we will install go 1.5

% git clone https://go.googlesource.com/go $HOME/go

% cd $HOME/go/src/runtime

% vi arch_arm.h

Again, remove any conditional logic so that the variable PhysPageSize is set like the line below:

PhysPageSize = 65536,

Then save and quit with “:wq”

% cd ..

% env GOROOT_BOOTSTRAP=$HOME/go1.4 ./all.bash

Ensure that your paths are setup correctly

export GOROOT=$HOME/go

export GOPATH=$HOME/work

export PATH=$PATH:$GOPATH/bin

Next lets grab syncthing to our go path. Mine is ~/work

% mkdir -p ~/work/src/github.com/syncthing

% cd ~/work src/github.com/syncthing

% git clone https://github.com/syncthing/syncthing

% cd syncthing

Now let’s build!

% go get -d ./cmd/syncthing

% env GOOS=linux GOARCH=arm GOARM=7 GOPATH=$HOME/work/src/github.com/syncthing/syncthing/Godeps/_workspace:$HOME/work go build -ldflags "-R 65536 -w -X main.Version=v0.12.0-beta0+101-ga32b66c -X main.BuildStamp=1442258191 -X main.BuildUser=ddubyat -X main.BuildHost=linux32box -X main.BuildEnv=default" ./cmd/syncthing

Now you should see synthing in the current directory, let’s verify

% ls -al synthing 

Now scp it over to your device and run it (use your own values where obvious)

% scp syncthing user@ip:/home/user

% ssh user@ip

% ./syncthing

If all went well you should see it startup in the foreground. To startup in background, refer to syncthing docs

Hope this helps someone out.

-Dustin


#30

Thank you very much for your detailed how-to!

I followed it but something is missing. The following part (for go 1.5) fails on my site:

 % git clone https://go.googlesource.com/go $HOME/go
 % cd $HOME/go/src/runtime
 % vi arch_arm.h

In my directory i miss the file arch_arm.h which contains the line PhysPageSize.

When I grep the whole directory, I get following file with that content:

src/runtime/arch1_arm64.go:     _PhysPageSize  = 65536
src/runtime/arch1_amd64.go:     _PhysPageSize  = 4096
src/runtime/arch1_amd64p32.go:  _PhysPageSize  = 65536*goos_nacl + 4096*(1-goos_nacl)
src/runtime/mbitmap.go: n = round(n, _PhysPageSize)
src/runtime/arch1_ppc64.go:     _PhysPageSize  = 65536
src/runtime/arch1_ppc64le.go:   _PhysPageSize  = 65536
src/runtime/arch1_386.go:       _PhysPageSize  = goos_nacl*65536 + (1-goos_nacl)*4096 // 4k normally; 64k on NaCl
src/runtime/mbarrier.go:        if src != 0 && (src < _PhysPageSize || src == poisonStack) {
src/runtime/mbarrier.go:        if src != 0 && (src < _PhysPageSize || src == poisonStack) {
src/runtime/arch1_arm.go:       _PhysPageSize  = 65536*goos_nacl + 4096*(1-goos_nacl)
src/runtime/mheap.go:   n = round(n, _PhysPageSize)
src/runtime/mheap.go:   if _PhysPageSize > _PageSize {
src/runtime/mem_linux.go:       _PAGE_SIZE = _PhysPageSize

On the other side a question about go 1.4 change. arm_arch.h has the following content:

// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

enum {
        thechar = '5',
        BigEndian = 0,
        CacheLineSize = 32,
        RuntimeGogoBytes = 60,
#ifdef GOOS_nacl
        PhysPageSize = 65536,
#else
        PhysPageSize = 4096,
#endif
        PCQuantum = 4,
        Int64Align = 4
};

If i understand it right, the change shouldn’t be necessary. I think, with the right compile flag PhysPageSize should be set right or? Respectively with a modification a patch could be commited to go project made the 64k permanent available.


(Audrius Butkevicius) #31

I think the nacl flag might be used in other places which might break stuff.


(Dustin) #32

The right approach will absolutely be to submit a patch to Go to allow for 64k page size that could be controlled by another parameter. This is simply a work around until they do so. The NaCL project from google does support 64k page size for arm but it also seems to limit the go environment, so it would need to be a new parameter.

For the 1.4 change, I removed all of the conditional logic.

enum {
        thechar = '5',
        BigEndian = 0,
        CacheLineSize = 32,
        RuntimeGogoBytes = 60,
        PhysPageSize = 65536,
        PCQuantum = 4,
        Int64Align = 4
};

On the 1.5 change, I just took a look here: https://github.com/golang/go/tree/master/src/runtime and it seems that there is still arch_arm.go available.

I would make a similar change as 1.4 in any 32bit arm arch file.

-Dustin


(Dustin) #33

Also, I went ahead and reverted my syncthing to version 11.25 since it seemed more stable. Binary here for 64k page arm7: https://dl.dropboxusercontent.com/u/15524812/syncthing1125


(Jakob Borg) #34

I don’t think there should be any need to path the 1.4 compiler, as that’s only used to bootstrap the 1.5 compiler?

(We don’t currently build with 1.5, but we will do that starting with 1.5.2)


#35

In between: Can confirm that syncthing1125 from ddubyat is working on my 64k system!


#36

Question between: There are any plan, made 64k version regular available? Would be nice, get current stable as a 64k version :smile:


(Dustin) #37

Sorry to bring back and old thread, but due to the security advisory, I have updated the 64k binary for arm7 to v0.12.23 available here: https://dl.dropboxusercontent.com/u/15524812/syncthing1223


(Jakob Borg) #38

How are you building that nowadays, with what compiler?


(Dustin) #39

go1.5

I actually just used the same instructions as I posted sept 15.

I’ve got a VM setup for quickly building this that I run every so often to update the binaries I need.


(luke) #40

Hey Dustin,

I tried to follow your instructions, but wasn’t successful :frowning: I tried it several times, but everytime, something else wasn’t working - I already failed at compiling Go 1.5.

Is it possible that you share your Go compiler files (source + bin files), so I can use them to compile 64kb libs by myself?

Best luke