v0.9.0 beta testing [beta testing is over]

Beta9

v0.9.0-beta9 fixes bugs and contains smaller tweaks. This will be the last beta.

Beta8

v0.9.0-beta8 introduces a faster hasher, slightly reduced memory usage, and more GUI languages (de, el, en, es, fr, it, pt, ru, sv).

Beta7

v0.9.0-beta7 introduces a different, forward compatible packet format, compression can be disabled, UI is event driven (quicker to reflect what happens, uses less CPU). This should be very close to a real v0.9.0 release, unless I hear screaming…

Beta6

v0.9.0-beta6 introduces browser language detection and GUI translation to German, Swedish, French and almost complete (I added some English phrases just prior to releasing) Spanish and Portuguese. If your browser is set to prefer one of these languages, you should see that in the GUI. If it’s not, you can specifically request it by using the URL format http://localhost:8080#?lang=sv (or lang=de, lang=fr, lang=es, lang=pt).

Beta5

v0.9.0-beta5 improves performance, introduces LZ4 compression of the sync protocol and fixes a bug with 100% CPU usage.

Beta4

v0.9.0-beta4 improves performance (matching these benchmarks) and fixes GUI issues.

Beta3

v0.9.0-beta3 passes all the smoke tests that v0.8 does, so should at least not be much buggier. :wink: The notes below still apply, with the following additions:

  • Directory modification times are no longer synced. This was a right pain in the ass to keep in sync, never worked well, and there are situations where we’ll do the wrong thing no matter what. The feature might reemerge if someone finds and elegant way to handle it.
  • The global announce server is up for v0.9 now.

Beta1

v0.9.0-beta1 is a build of the branch that will become v0.9.0. It kinda seems to work for me, most of the time, but there’s a lot left to do for v0.9.0 and a lot of things I haven’t tested. Feedback from the community would be valuable.

Important Notes

  • :bangbang: Don’t deploy this in production or on valuable data! :bangbang:
  • Back up your entire config directory if you want to be able to revert back to a v0.8 build.
  • Not compatible with v0.8 nodes.
  • Global discovery is currently broken.
  • Does not read or understand your old indexes, so best to deploy when a cluster is already in sync. There’ll be a brief period of resynchronization, but not much actual data should be transferred if the files are already in sync.
  • CPU usage is currently/still higher than it needs to be, especially when the GUI is active.
  • Rescan is somewhat inefficient for large repos. You might want to increase the scan interval from the default 60 seconds.

What’s New

1 Like

I just installed beta4 and initially everything seemed to be OK. But after about an hour, the CPU useage went up to 30% and my box’s fans started spinning like crazy. (Win7 version master node and Linux Ubuntu slave nodes. The problem we are talking about is on Win box).

I waited for a few minutes even thought nothing seemed to be happening as nodes were fully in sync before that. So I had to exit the program and restarted it again. It seems to be OK after a few minutes of running it. I hope it stays that way.

What would be the best thing to do in this situation?

Issue Update 1:

The good news is that after I restarted syncthing it no longer eats so much CPU, and it’s been up for several hours now.

Issue Update 2:

Well, unfortunately, after several hours it all repeated. The CPU is overloaded and fans are spinning. Syncthing process eats the CPU for minutes on. We just had to exit the program. This is “no go” for us.

But there is no signs of any activity. Nodes are fully synced. No transfers are in progress. Nor there are any signs of activity of any kind. Nor there are log messages about something being in progress.

At this point, the question is: to what version should we switch back and can we switch back at all?

Also, it’s a bit confusing to see all these betas coming out and, at the same time, .8xx versions are being released.

What’s the idea here?

Right now, I’d rather have a stable system then some theoretical increases in performance or memory utilization.

We have several shares and the main one has 1.4 Gigs and over 10k files.

It would be interesting to know what’s going on here, obviously. On Linux/Mac, sending the process a QUIT signal (pkill -QUIT syncthing) causes it to print a nice trace message that helps. On Windows, I’m not sure how to do that. I can write up some instructions for using the profiler, but given the note below I’m guessing that might not be what you’re looking for.

Well, the v0.8 versions are the stable series. The v0.9 betas are just that, development betas. Hence the “-beta” in the version and the yellow “prerelease” notice - that’s not what you want if you’re going for stable. :wink: The “latest release” link also points to the latest v0.8 release, not to a beta. And there’s this thing with the red exclamation marks and “do not run in production” above…

You can of course switch back to the latest v0.8 release. If you have a backup of the config from that version, that’s best. If not, you might need to reconfigure it from start or hand edit the config.xml (I can give some instructions here, it’s just editing the format of the node IDs a bit).

Then again, depending on exactly how stable you mean by stable, you might not want syncthing at all until it hits 1.0 at least. :wink:

Oh, if you inadvertently ended up running the beta by using syncthing -upgrade, I’m sorry about that and it has been fixed. v0.8.21 and newer won’t upgrade from non-beta to beta releases.

Hi calmh,

Thanx for the feedback. I am not sure about the CPU issue. Somehow, the behavior on that seems to be inconsistent. So, I am not sure how much you can rely on it. I tried to go back to beta3 as I initially thought that that one was not eating the CPU, but then, after some hours it seems to have started doing that as well. Unfortunately, I do not like to let it run overheated for a long time, so, after a few minutes I decided to exit. I am not sure what I am seeing here because nothing seems to be going on.

Then I tried to go back to 0.8.21 and it seems to be better in term of CPU load. As far as configs go, I did not change anything when going back to .8.21. So, I’m not sure if I have a problem that I don’t know exists. If you think there IS a problem, would be great if you tell me what exactly do I have to change in configs. I don’t even know where they are :smile:

Side issue - Logging:

If you recall, I suggested to add some more detailed logging as to dynamic nature of things. Sorry, I did not follow up on that post because I am swamped with extremely important things to be done. Hope you can understand.

I am not sure what makes sense to YOU, but what I did in my last project is to provide logging of different operations. The program I worked on had to do with NNTP functionality. So, there are operations, like downloading or posting. During those operations, you have different things happening, like connecting to server, loading configs, processing archives, actual downloading or posting, etc. And you also have some intermediate results or statuses of completion, error, not critical warnings which mean the system had to take some default value as a result of some wrong or incomplete information provided in the configuration, etc.

So, each module that does some major operation has a status line and history log box in GUI. Status line displays the current thing being done, like connecting, loading, etc., and a rate of success. It is also automatically copied to a scrolling history box in the module’s GUI and also to the main program window. The same history box, but not all the details are displayed there. That one is only for major things program-wise.

The logging process is completely automatic and daily logs are automatically created upon program start as rotating logs, such as log_YYMMDD.txt, which is a regular text file.

So, what happens is no matter what the issue is or where you are in execution process, you have date/time stamped log messages in the log file that tell you which operation is in progress, which exact archive is being loaded, and was it loaded successfully, which exact message is being posted and so on.

As a result, I was able to find some pretty nasty bugs within minutes because logging is pretty descriptive.

In your case, what I see is that you could log different major phases of operation that, for example, tell you that you are trying to connect and upload to remote node such and such and are trying to sync such and such a share. On a more detailed level, you might want to say something like: “YYMMDD HHMMSS: Uploading file such and such”. Once it is done, you display “… completed”. Else, you display … error: detailed reason for error and which incorrect parameter or file or something else caused which exact incorrect behavior and what was expected to happen. Like “>>> ERROR: could not write to file filename,” or “>>> ERROR: file does not exist”, or “>>> ERROR: expected parameter this, but it is all claborred…” and things like that.

Also, any intermediary and possibly minor errors or warnings could be logged with time stamps.

You could have several different logs. One is for major system status things. Like starting to run, loading configs, starting upload and so on. You may not want to provide too much details in that log and use a separate log for file transfers and related status messages.

So, what you have is a system where behavior on different levels and from different aspects is logged in a detailed manner and anyone could simply open that log in his Notepad++ editor and have it automatically updated by syncthing in case there is something funky going on.

Otherwise, I am having a hard time to comprehend what is actually going on in terms of system dynamics. Where is the system right now, is it idling, is it doing some maintenance work, checking something, verifying integrity of something, which files is it transferring right now, which way, from where to where, what is the progress of transfer, what is the percentage of the total file size that have been transferred and so on.

In other words, I’d LOVE to see the system dynamics. Not sure if some of it could go to a GUI into some scroll box and which part would not be desirable to see in the GUI. But I like those scroll boxes with more or less complete logs on each module’s window.

So… How much of it makes sense to you, only you know. If you’d like some clarification, I’d be willing to spend some time on it.

Good luck. I like the program and I do like the GUI. Nice large text and those colored fields and buttons, structured so the display is not cluttered with too much information and things like that. I just wish it would be more informative as to dynamics of things.

Another thing - I’d LOVE to see that issue with automatic public shares joining the appropriate swarm without any user interaction resolved, just the way it is done on BTSync, for example. Basically, it looks to me that if you add a concept of a public SHARE KEY, not just node ID, then you should be able to automatically join those shares without even showing the rest of the shares on remote nodes. It looks like a parallel mechanism. But I am not sure about the desirability of requirement for any user interaction to join some share for PUBLIC shares that should be made available to any node regardless. So, the user could join either the way it is done right now, or by share key, in which case different logic applies. If someone is joining by share key, ALL they see on any node is that share only. You might even decide to have a tab in GUI that displays things differently for public, by share, nodes and regular nodes. So, on share GUI page all you see is the list of nodes currently sitting on that share and you may not even have any idea that the system behaves totally differently if it is accessed by the node ID key and what is displayed to them. On that page you may decide to implement the hold or drop functionality, just like it is done on uTorrent for example where you can put some torrent on hold or stop it completely without deleting it as such.

Yes, you could add the “accept” node functionality, or even exclude some node from some share. But this stuff gets hairy in a hurry and I am really way too busy with things I do to have a good look at syncthing in terms of code. Plus I do not know GO even though from a quick look it reminded me Pascal.

Why did you chose GO, if you don’t mind the question? Is there something special about it that could be described in a sentence or two?

Anyway, I do appreciate your work and the project seems to be going in the right direction and I’ve been using the program for about a month now and so far, it is quite reasonable and well behaved. Sure, there are all sorts of things to be done, like with anything else, but the idea is right as far as I can see without diving into it upto my ears.

Good luck and thanks again for all the work you are tirelessly doing and keeping the project going.

If you had a v0.8 config before and didn’t change it from the GUI in v0.9 you are OK.

The GUI should show what’s going on, and currently it’s a bit lacking there I know. I don’t want to show log messages in the GUI nor get into the whole business of handling log files. How log files should be handled is very system dependent and what you want is likely very different from what someone else wants. Syncthing can produce more log output than likely want though, already, by setting the various STTRACE options. Note that the log messages for that kind of stuff is optimized for the developer though, and likely won’t make much or any sense to a causal user.

This isn’t a use case I have in mind for syncthing. If you have no privacy requirements, there are all kinds of awesome file sharing tools (Dropbox, regular BitTorrent, etc).

This was discussed in another thread a while back, actually.

I moved 5 posts to a new topic: Public shares

Hey, I see we have “Public shares” as a feature?

Zo…

It looks like we are getting somewhere “after all these years”.

I just wish I could read GO as I can read Java and C++!!!

Btw, I just took a look at your sources today and have spend about half an hour.

Well, what can I say, you do look like one of the greatest programmers that has ever lived on this planet. Because there is virtually no comments or explanations in your code.

Can you comprehend how much of this kind of code have I seen in my life?

And do you know where it leads you eventually?

Edit: there is something funky here. You get error messages about problems posting. And if you try to post again, you get dups. Well, the world is not perfect after all…

v0.9.0-beta8 introduces a faster hasher, slightly reduced memory usage, and more GUI languages (de, el, en, es, fr, it, pt, ru, sv).

just wanted to say thx for your hard work, dont listen to such bashing :wink: I will upgrade my 2 machines today, and report any errors.

Edit: found some bug: i have a static ip for my pcs(you know the connection problem that i had). Now after upgrading, i change the ip to dynamic and then a message appears that the file is already opened and my second pc shows up 2 times. I fixed this by deleting the static pc(they had the same id). Im currently waiting for the initial scan and then i will post the log(there is no log of this error because there was none saved(is this intended?), used the cmd message instead) Edit2 Log: http://pastebin.com/xRrqaw28 Edit3: So after 2 hours scanning its finished now… No further errors for now.

Greetings Saladien

Thank you. I think that problem is this bug: https://github.com/calmh/syncthing/issues/450 (fixed, but not in beta8).

k thx for the info, but the pc showed up 2 times, is that a known bug to?

Don’t know, never seen that. Screenshots and config file would be useful, perhaps file that as a bug.

unfortunately i have already fixed it, by deleting the static version. There were 2 exact copys of “MainPC” the first with the static ip and the one under that had the same ID but had dynamic. I think this happened because there was the permission error and so 2 copys ended up in the config. But if i encounter that again i will report it.

the 0.8.x version running on my NAS managed to autoupdate itself to 0.9 beta8 and now I’m stuck as to how to compile beta 8 on my MAC to match the version…

Any docu anywhere?

Bug in older v0.8 builds, sorry latest should not auto-update to the beta. Download v0.8.21 and copy back the files from the backup-of-v0.8 directory inside the config directory and you should be fine.

Or! Do some testing. :slight_smile:

I can’t go back, I don’t know much about freeBSD and that’s what’s running on my NAS hence my question about how to build it on my MAC.

I tried executing ./build.sh but I only get: ./build.sh fatal: Not a git repository (or any of the parent directories): .git fatal: Not a git repository (or any of the parent directories): .git ./build.sh: line 115: godep: command not found

Aha, well, you could download the binary of beta8 (or beta9), but if you want to build it, see [archived] Building Syncthing. Super short version that might work:

export GOPATH=~
go get github.com/calmh/syncthing
cd ~/src/github.com/calmh/syncthing
./build.sh setup
./build.sh
ls -l bin/syncthing

ah :-/ I must be going blind. I couldn’t find the darwin build but it was right there in front of my eyes. Sorry for that.