Request for 64k binaries

calmh · May 12, 2016, 5:57am

How are you building that nowadays, with what compiler?

ddubyat · May 12, 2016, 2:59pm

go1.5

I actually just used the same instructions as I posted sept 15.

I’ve got a VM setup for quickly building this that I run every so often to update the binaries I need.

luke · July 9, 2016, 11:24am

Hey Dustin,

I tried to follow your instructions, but wasn’t successful I tried it several times, but everytime, something else wasn’t working - I already failed at compiling Go 1.5.

Is it possible that you share your Go compiler files (source + bin files), so I can use them to compile 64kb libs by myself?

Best luke

ddubyat · August 14, 2016, 3:34am

On the latest version 0.14.4 + I had to change it up and ended up editing the build.go file since it was always building the host arch.

Then I compiled using the following command:

go run build.go build syncthing -goos linux -goarch arm -no-upgrade

Here is the modifications to build.go file:

func build(target target, tags []string) {
	lazyRebuildAssets()

	tags = append(target.tags, tags...)

	rmr(target.binaryName)

 	args := []string{"build", "-i", "-v", "-ldflags", "-R 65536 -w -X main.Version=v0.14.4+5-g42849af -X main.BuildStamp=1471144232 -X main.BuildUser=ddubyat -X main.BuildHost=linux32box -X main.BuildEnv=default"}

if len(tags) > 0 {
		args = append(args, "-tags", strings.Join(tags, " "))
	}
	if race {
		args = append(args, "-race")
	}
	args = append(args, target.buildPkg)

 goos="linux"
 goarch="arm"
 var goarm="7"

	os.Setenv("GOOS", goos)
	os.Setenv("GOARCH", goarch)
        os.Setenv("GOARM", goarm)
	runPrint("go", args...)
}

hope it helps you out luke

luke · August 14, 2016, 5:59pm

Thanks for getting back.

Appearently I compiled the file successfuly. Unfortunatenly I get an “can’t execute binary file” error when executing the compiled file on my NAS

Thanks for your update anyway. Would you mind to share the compiled file for v14?

ddubyat · August 15, 2016, 4:17am

Sure,

https://dl.dropboxusercontent.com/u/15524812/syncthing1445

Thats the latest code as of yesterday.

You probably got a good compile but not for arm. you can check by using the command “file filename” and see what architecture the binary is for.

I think there is something funky in the latest buildfile for syncthing that prevents other architectures besides the host. Editing it manually was the only way for me to get around it.

calmh · August 15, 2016, 6:19am

We build the distribution binaries with build.go -goos linux -goarch 386 build and so on so I don’t think that part is broken.

luke · August 15, 2016, 8:30am

Thank you very much. Your file works great

luke · August 15, 2016, 11:04am

When executing my file now I get a “not found” error, but I’m more than sure, that the file exists (e.g. commands chmod & file are working).

I made this file filename thing. This is the given info and there is a difference between my file and your file:

My file:

syncthing: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, not stripped

Your file:

syncthing: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

Any ideas? I’m not very familiar with Go.

ddubyat · August 18, 2016, 6:30pm

This is what I saw as well, even when using “-goos linux -goarch arm” which is why I think the buildfile is not working correctly @calmh

Also with just the args, no way to set arm version or page size.

Mirco · December 17, 2016, 8:28pm

Spent hours to compile v0.14.14 and v0.14.15 for the WD MyCloud but nothing as here described worked for me. Tried the go build command on Ubuntu and on MacOS X. However the resulting binary doesn’t run on the WD and gives the same “runtime: kernel page size (65536) is larger than runtime page size (4096)” error as the original arm binary from the syncthing homepage does.

Is it because of me or because of the more recent synchting or go (1.7.4) code?

Anyone has a 64k binary for v0.14.15? dubyat’s v0.14.4+5 works like a charm - thanks for that

A working binary for those NAS drives on the releases page would be doozy. There’s a arm64 download, but seems to be 64bit?

I did so far:

Install Go
Clone syncthing from git
go get -d ./cmd/syncthing
go run build.go assets (to get rid of “undefined: auto.Assets” error)
GOOS=linux GOARCH=arm GOARM=7 go build -v -ldflags “-R 65536” ./cmd/syncthing

and *

edit build.go like ddubyat described
go run build.go build syncthing -goos linux -goarch arm -no-upgrade
or: go run build.go build syncthing -goos linux -goarch arm -goarm 7 -no-upgrade
or: GOOS=linux GOARCH=arm GOARM=7 go build -v -ldflags “-R 65536” ./cmd/syncthing

Everything gives an executable but none runs on the WD MyCloud.

progers885 · April 10, 2018, 10:57am

Hi All, Just revisiting this as I want to go to V4 firmware on my wdmclousd and need a 32bit arm version of syncting with 64k pages.

On researching this I believe the issue of 64k page sizes has been fixed in go and it now detects the pagesize in use; I refer to this link https://github.com/golang/go/issues/10180.

If this is true would standard arm version of syncthing work on v4 f/w? or does it still require further steps to be taken by syncthing? if so can it be done?

Regards

Paul

calmh · April 10, 2018, 11:09am

Why not try it and see if it works?

progers885 · April 10, 2018, 12:15pm

Well it must be worth a try the worst that can happen is I will have to go through the PITA exercise of downgrading to V3 firmware, I just thought I would ask in case someone said don’t do it

calmh · April 10, 2018, 1:24pm

Oh. I thought you had a machine that required 64k page size and was wondering if Syncthing would work, sorry. Carry on, then.

progers885 · April 10, 2018, 9:08pm

Well I upgraded my WDMyCloud from V3.04.01-230 to V4.05.00.320 went to check syncthing and forgot that the upgrade overwrites the filesystem so installed syncthing-linux-arm-v0.14.46 and it ran up ok.

set up all my sync folders. Upgraded my three other systems to v0.14.46, now just have to wait for them to sync up;

Observation: on the WDMyCloud Syncthing Admin page the CPU Utilization shows low CPU useage 5.3% which I know is vastly out as my terminal session is really sluggish and top shows:-

top - 21:37:58 up 5:59, 1 user, load average: 108.82, 91.95, 61.06

KiB Mem: 232320 total, 179520 used, 52800 free, 1856 buffers

KiB Swap: 500672 total, 289280 used, 211392 free, 7424 cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14563 root 31 11 784m 26m 2688 S 5.9 11.8 14:26.25 syncthing
15068 root 20 0 4544 2176 1408 R 1.6 0.9 0:02.21 top

A scan is running on one folder about 26GB in size

I inadvertently set some traffic on another folder which was in sync and that caused my other systems to show the WDMyCloud as disconnected (tcp-client closed: read timeout) I tried and get around this by pausing the 26GB folder the web page eventually shows a connection error and my ssh session is locked up so ended up power cycling.

Obviously I will have to wait for the scan and sync to complete which will be quite a while before I can run some activity to see how it performs under normal conditions.

lesson to learn is backup .config to a share before upgrading the WD firmware.

progers885 · April 14, 2018, 6:14pm

So far the results are not good basically The wdmycloud takes for ever to boot and when I connect to the syncthing gui the wdmycloud is very unresponsive, so much so that I log in with a terminal session it takes a minute or two before I get a # prompt; I shut down syncthing and the wdmycloud bursts back into life. I am doing nothing different than I did when the mycloud was on V3 firmware. any suggestions before I go back to the V3 firmware?

AudriusButkevicius · April 14, 2018, 7:08pm

You have to understand that syncthing relies on cryptographic primitives to do it’s job, which by nature is costly operation. I don’t think a device with so little power is our target audience.

It’s most likely scanning, and once it’s done with that, I suspect the CPU usage will go down and the device will be more usable, other than that, I don’t think there is much you can do.

progers885 · April 15, 2018, 3:42pm

Hello Andrius your reply go me thinking; I only use my wdmycloud as a server I have 2.5TB of data of which only 50GB is being served by syncthing and I recall that I had disabled the functionality I don’t use in through the wd admin gui but on further investigation I needed to disable stuff that wasn’t available through the gui:-

/etc/init.d/wdmcserverd stop /etc/init.d/wdphotodbmergerd stop

update-rc.d wdphotodbmergerd disable update-rc.d wdmcserverd disable

I will test further.

On another note I have 4 directories being served by syncthing would it be possible to prioritise which directories are scanned first on startup rather than scanning them all at the same time this way on low power devices you can have your most wanted directories syncing sooner?

Regards

Paul

AudriusButkevicius · April 15, 2018, 3:45pm

You can pause and unpause the folders yourself as you see fit.