Hey everyone, I’ve been using syncthing for a bit now, usually to sync things between my Nokia n900 phone and Linux PC. Today I’ve hit upon a snag. For some reason my “Pictures” repository stops (Spitting out the error onto the WebUI in the title). I run syncthing from a terminal (not daemonized) so I can see that it’s showing errors like this:
It does this for what presumably are all the pictures it hasn’t synced (which is strange, as some of them are far older than when this started, so presumably should already have synced just fine!)
I’ve tried clearing out my .syncthing/ folder and starting from scratch, but the problem persists. I have quite a few repositories syncing between these nodes (as well as a 3rd machine that occasionally joins in), and it’s only my ‘Pictures’ repository that has this problem.
That’s the operating system returning “invalid argument” when syncthing tries to open/create the file in the error above. I have no idea why, unless “nameofpicturefile” is in fact something else interesting that you have hidden.
Well, if I wanted to put the name of the picture files not working, the post would’ve been far longer, as I have about 100 picture files that gave that error
Interesting the OS is returning that though… Why would that be? I have read/write permissions in the directories the repo’s are pointed to And like I said, it’s only doing it for some pictures, not all of them!
Well, at a guess it’s because there’s some characters in the name that are not allowed on the filesystem, or that the path is longer than what the filesystem allows, but we’ll never know unless you tell us a filename it’s having trouble with.
Most of the pictures in this directory (whether they send out errors like this or not) follow similar names (IE. just timestamps). Some of them are, for example, “Screenshot from 2014-08-04 20:29:30.png”.
Just a quick edit - I’ve also checked permissions on the Pictures directory just to make sure. It belongs to the user ‘user’ (which is my phone’s default user…) which is the user I’m running syncthing as (I’m just cd’ing into the untarred directory and running ./syncthing, which is how I’ve done it since I started with syncthing!).
Are there plans for a workaround for forbidden filenames?
Or at least a meaningful error message would be helpful in such cases.
I just had the same issue with a filename containing : (syncing to android). It was a single file causing the repo to stop with 11 unsynced files. It took me quite some time to realize that only one of those 11 files was the source of the problem. Especially because the problematic file was the one listed at the bottom of the list of unsynced files (I was expecting it on top).
Maybe the file could be skipped instead of stopping the entire repo. And only if multiple files in a row would fail, the repo would be stopped maybe.
Or just highlight the file, that caused too many errors in the GUI.
Just some ideas
This is what’s supposed to happen. A small amount of failures are tolerated (assumed temporary, although they are not in this case), and only if there’s more than ten files to sync and all failed do we stop the repo.
I changed the Filenames. All the files in the logs are the ones that ended up unsynced and _backup/fileWithColon-20131031-20:23:50.json was the one causing the error.
Looking at the logs I would guess, that the 11 files stayed unsynced, because more blocks were need (pulling loop needs more blocks). And the only file that could provide those blocks caused and error.
Let me know If you need more information.
Should I open an issue on github for that?
Are you running some strange version of syncthing? puller.go:240: DEBUG: queueing "Notes" As far as I can tell, there’s no version I’ve made where that piece of code is on line 240 in that file… O_o
Sorry. I completely forgot to mention that.
This is a log from syncthing for android 0.4.11 (syncthing 0.9.9). I started the instance in an adb root shell using:
cd /data/data/com.nutomic.syncthingandroid/
STTRACE=model ./lib/libsyncthing.so -home=files/
Other than that there should be nothing special about the version.
OK, might you have search-and-replaced the line numbers as well? I can puzzle things together anyway if that’s the case, but it looks very confusing as is.
This time running on Windows 64bit (log is from syncthing 0.9.10, but was reproducible with 0.9.9 as well). 5 Files remained unsynced while only 1 was causing errors.
I realized, that the number of unsynced files would go down when repeatedly restarting syncthing. It seem that the files where synced if they were “fast enough”, i.e. syncing before the repo was stopped.
Yeah, there’s a bug. What I described above, with it needing multiple files to fail, is not the case. One is enough, if it’s the only one. The definition of “only” in this case is what is broken, in that this is what happens:
It queues 11 files (or something) that need to be fetched from the network
For ten of those, creating a temporary file and requesting a block from the network succeeds.
For the 11th, the temporary file can’t be created due to the filename error.
We check if there are more files to queue
There is one, the failed one (the other ten are still being processed in the background)
It fails! All of the newly queued files failed! Abort!
However this is almost the right behavior. The bug here is that we shouldn’t stop until those ten other files are correctly synced. But what do about the last one? We can retry forever every few seconds, but that’s stupid. We, as a human, can see what the problem is but syncthing just sees an IO error with some string from the operating system and doesn’t know the root cause. I.e. you have two different ones in your pastes:
[MRBZP] 2014/09/01 00:28:29.860123 puller.go:497: DEBUG: pull: error: "Notes" / "_backup\\fileWithColon-2014058-02:08:05x.json" has already failed: open E:\notes-test\_backup\~syncthing~.fileWithColon-2014058-02:08:05x.json.tmp: The filename, directory name, or volume label syntax is incorrect.
Although the second one is from Windows where syncthing should flag the filename as invalid and warn about it far earlier, not sure why that didn’t happen here… Hmm… OK, fixed that at least (bd2772e).
It seems to me that I have a similar issue, since I try to sync my home folder in Linux (master) -> Linux (non-master) (Ubuntu), using syncthing v0.9.13.
It stops with 3492 items remaining to be synced (after restart the number gets a little smaller), and doesn’t give a clear answer on which file exactly is failing.
Anyway, is it possible to add an option like “persist” or “ignore-errors”, that would completely ignore errors and continue to sync ?
