Folder isn't making progress 2017


(Giovanni Vagante) #1

First of all: sorry for my poor english.

Second of all: I’ve read all similar topics, tried every solution suggested (rename folder in config.xml with “\?”, resync, stop syncthing, rescan, restart), unistalled synctrayzor (yes, I use it), but nothing to do.

Ok, the problem is completely new: I’m in windows 10, 64 bit, syncthing ver.0.14.39 and never had it before and I use SyncThing with SyncTrayzor from an year now. All was okay a week ago, then an error in SyncThing console, but I haven’t modified nothing in S.O., in configuration or in folder synced, just added some files in that not with a long name I think. Now everytime it gives the error “pausing puller” and I have some syncthing.tmp files in the folder. I don’t know how to uplod the log: should I copy right here?


Capital letters in filenames
(Simon) #2

In the “editor” field when writing a post there is a sign to upload a file. Alternatively, you can directly paste the logs as e.g. preformatted text like this:

```
put your logs here
```

The problem might be ignores (see docs, especially for (?d) prefix) or less likely a race condition: https://github.com/syncthing/syncthing/issues/4475#issuecomment-341662063


(Giovanni Vagante) #3

This is the log. I must explain that I’ve erased some lines of the log wich contains the name of file that has the name of some customers. This thing is driving me crazy. It scans everytime and never ends.

syncthing.log (143.3 KB)


(Simon) #4

Unfortunately, there is not more info in the log (I still don’t understand how, but I think I borked the puller logging up pretty good in 0.14.39, 0.14.40 with Audrius improvements will hopefully be better).

You can run Syncthing with the environment variable STTRACE=model set to get the relevant info for sure, or you double check the following (I forgot some things before): Directories that should be deleted, but aren’t due to ignored file, symlinks (not supported on windows), invalid filenames (weird chars, …) or files/dirs in the same dir whose names only differ by case.


(Giovanni Vagante) #5

Ok, this is the log with STTRACE=model syncthing.log (19.1 MB) The program take a large, really large amount of RAM


(Simon) #6

Relevant are these lines:

[KFH2Y] 2017/11/04 15:35:56.235376 progressemitter.go:225: DEBUG: progress emitter: deregistering qjznm-pqohs Giornale di Diritto Amministrativo\2007_7.pdf
[KFH2Y] 2017/11/04 15:35:56.247387 rwfolder.go:1435: DEBUG: sendReceiveFolder/qjznm-pqohs@0xc0421a2b40 closing Giornale di Diritto Amministrativo\2000_2.pdf
[KFH2Y] 2017/11/04 15:35:56.247888 rwfolder.go:1367: DEBUG: file modified but not rescanned; not finishing: 

The problem is the backslash in the filename, which is not allowed on windows, where it is used as path separator. You need to remove these backslashes in these files on the other device.

I don’t understand at all why this results in the “not finishing” log line, which should happen if there is an unexpected file or a file changed while pulling. However windowsInvalidFilename is broken, as it incorrectly assumes \ is the path separator for the given path.
See also this issue: https://github.com/syncthing/syncthing/issues/4484


(Simon) #7

The above is completely wrong - I don’t seem to be able to edit or delete the post, please ignore it. The log lines are relevant, but all the conclusions are false. Your filenames are just fine.

From your logs I don’t understand why the puller always detects, that these files were changed during pulling. Maybe those files have weird modification times - you could check that. You could also run it with STTRACE=model,scanner to get even more information.


(Giovanni Vagante) #8

This is instead the last log with STTRACE=model, scanner. Thank you for your time and patience.

syncthing - Copia.log (16.2 MB)


(Simon) #9

Well, scanning happens, it doesn’t report any errors and sees all the temporary files corresponding to those that weren’t put into place at the end of pulling. Please pick any of those files and look what their modification times is or if there is anything else that seems somehow “not normal”. Other than any such “anomaly” in the files I can’t think of anything that might cause this.


(Giovanni Vagante) #10

The strange thing is that this file doesn’t get modifications. I saved them months ago and never modified them. I’ve tried another thing: I’ve copied all files from a machine to another so this are exact the same, but the problem is still there.


(Jakob Borg) #11

I don’t know, but this looks like a bug of some sort. The log message

DEBUG: file modified but not rescanned; not finishing:

indicates that there was a file in place when it went to finalize the sync, but we did not expect there to be a file based on a database lookup. (It’s subtle: the empty string after the colon is the name of the file according to the database… The debug message could be better.)

It should then be picked up on the next scan, but apparently it isn’t because we don’t see it.

My spider sense says “odd filesystem shenanigans”: symlinks or NTFS deduplication maybe?


(Giovanni Vagante) #12

Well, I don’t know what’s going on, but there’s a thing I’m sure. Until 10 days ago syncthing was perfect for me, now it has become unusable: not only it scans and rescan always, but it uses all (ALL!) the RAM on my pc, so it has become unstable and so slow. I can say another thing about this problem, it has come up after I’ve synced all with a laptop I don’t use everyday, then I’ve tried to sync again all node on my syncthing, but the problem is always there and it hasn’t gone away too if I’ve reinstalled the program deleting all option, all folders indicators, anything I think (I don’t know if there’s something in the registry). I wanna say too that I use syncthing with SynTrayzor to sync always 2 nodes (a backup folders and two folders that contains pdf files I just read and never modify), all nodes are Win10 64 bit. I use it too to sync photo on my Android phone in a subfolder of photo folders and I sync this folder with another node. Thank you for your attention.


(Simon) #13

Neither do I unfortunately, and as there is no way to reproduce it, neither can I investigate it. The attached file contains a list of all file that cause the problem, maybe that helps.

Also I wouldn’t focus on the installs of Syncthing/Synctrayzor, but on these failing files. You can try to recreate the Syncthing folder or move the directory to a new location. I am just fishing in the dark here.

failed (16.5 KB)


(Audrius Butkevicius) #14

All files have a fishy forward slash that is impossible to create on windows, so I am pretty sure that’s the cause. The error reporting for this is broken however.


(Jakob Borg) #15

Forward slashes are impossible everywhere (afaik), but a backslash could be part of the filename on unixes. That would probably cause unhappiness on the Windows side, yeah. I think we should filter it out before it gets there, but there might be bugs (i.e. it gets interpreted as the path separator… since it is… on Windows…).


(Giovanni Vagante) #16

I’m sorry to say that I’ve all windows files and no backslash in files (windows doesn’t let do it): tmp files are named like this “~syncthing~2017_2.pdf.tmp”; pdf files are like this “2017_2.pdf”. I don’t use linux, MAC or Unix, just Windows10 64 bit.


(Simon) #17

@AudriusButkevicius where do you see forward slashes in filenames?

@calmh We do filter out filenames with backslashes in Windows when receiving indexes. the entire ruckus I made about backslashes is moot.


(Giovanni Vagante) #18

@imsodin I’ve deleted the folder, recreated it, renamed, copyed in another location before I’ve opened this thread.


(Audrius Butkevicius) #19

I see \ in the name, which I am pretty sure is not kosher on windows, hence why I think this happens.


(Simon) #20

That was my initial reaction too, but the “name” is not the filename, but the path - so the backslash is simply a path separator.