I am seeing, often several times a week, messages such as
INFO: Puller (folder “xyz”, dir “uwv”): delete: remove /home/ramon/xyz/uwv: directory not empty
But neither xyz nor uwv match or have anything in .stignore (the file is just a pdf). These are old files (e.g., over three years), and no node has tried (or should have tried) deleting them. These are directories which have been kept in sync for several weeks already. This often happens in medium size directories (~ 7.5 GB, over 6000 files). I am syncing a couple of Linux machines and a couple of Android devices (which are running syncthing for arm under Linux under Android). I have set “Ingore permissions” to true in all machines (because of the Androids).
I am using v0.10.29 (but this has happened to me at least since version 0.10.25). What I often do to stop the problem is move the directory out, sync, and then move it back.
So two things I find strange:
why is anyone complaining about removal of things that no one should remove?
why is this happening after weeks of having synced these folders?
No. I am not using network/bind mounts. And I do not understand why any of the nodes would be losing a folder. The folders are there in the nodes. What kind of information can I look at to see what is happening?
So it has happened twice again in the last 48 hours. In both cases, instead of moving the files and directories out and back again, I’ve done nothing (except click on the “OK” in the warning message that shows in the web gui). And then, after a while, there are no longer any complaints. So I guess this is basically harmless, but still I do not understand it. And I worry about what would have happened if the directories had been empty.
About the “.” (the period) matching any character: I just tried it, and I think it does not match any character. For instance, an .stignore with
.*.swp
will still sync the file
aeiou.swp
but will not sync
.aeiou.swp
Anyway, I am not sure why it would have made a difference, as even with the period matching any character none of the affected directories do match.
And in that same directory I also noticed the following, which I think is related: for many files, I see a pattern of them being deleting and then, shortly after, added back again. So I guess they are deleted on some node, and then put back from another. (I know this is happening because I keep a backup of the directory using seafile from one of the nodes, and I can see the history of additions and deletions). For instance, this happened to a file called “A generalized, likelihood-free method for posterior estimation/likelihood-free-posterior.pdf”: it was deleted on 2015-03-14 19:32:43 and added back on 2015-03-14 19:53:10 (and this file has original time of 2014-10-09). But I do not know which of the nodes triggered the deletion or addition. I’ve looked at ~/.config/syncthing/index but cannot figure it out.
I further suspect this is related to the Androids, since none of this weirdness happens in directories that are shared only between linux computers.
I think that a deletion and readdition actually happens on the same node, as other nodes couldn’t readd them if they were deleted.
It could be that the android’s fuse wrapper just gives up temporarily which causes it to do what it does, but that’s a very wild guess.
It seems the problems are due to the androids (when I keep them off no problems appear).
However, the “Puller (folder …)” problem appears even when I do not see the pattern of deletions followed by an addition shortly after (maybe because they are not the same problem, or maybe because it cannot complete the deletion). I wonder if this is also related to
Since this seems an Android problem (I consistently reproduce it with two android devices) I am opening a question with proper name in the Android forum.
This is the most painful error I get frequently (then gives xxx is not making progress error), I am at a point where I just want to ditch ST ;( I constantly delete remove folders on nodes to stop this errror and it is very tiring ;(
I have 10 nodes mix of WIndows, Linux and Android and you can see how complicated this can be. Because I constantly need to ssh to nodes and clean up shit that does not make sense.
I delete one folder than it starts complaining about another. So there seems to be no end to this.
My android nodes are running the arm version not the Android apk.
Well, that is not possible, because I have so many shares and so many nodes. That is like full time job right there. If I am going to do that constanty what is the point of using a sync program. And why shall I use an ignore pattern while I want folders and files to get sync. The whole point of using a sync program is to automate things. I want things to sync unattanded and so far this has been a nanny job for me.
I constantly delete folders that ST complains about, then it starts complainiung about new ones, it is not like it is bitching about the same files and folders over and over.
I do not want to ignore anything, I want everything in the folders to sync.
Well you might be on to a bug which needs fixing, but instead of providing information which helps us understand and fix the bug you just tell me how hard your life is. This is not going to take us anywhere.
I suggest you download dropbox and do us all a favor, and hopefully somebody which has a bit more patience and less drama will rediscover the bug eventually and help us fix it.
You know that is such a sarcastic recommendation. Seriously, I should move to dropbox and so you can stop hearing people complaining about certain bizarre behaviors displayed by ST. I know developers always complain about broken platforms, broken compilers, broken OS etc. Such recommendation is just silly and childish. At least please complain about how broken arm and android platforms are, that is more viable.
The reason I brought my complains is so that the devs can see how complicated of a scenario such issue can create on the user side.
You first asked me to create a ignore pattern for hundreds oif folders that I had to delete manually, now you are recommending dropbox as an alternative. This is just cynical.
Anyway I get this issue and issues like “dst stat dir no such folder” etc type errors on Linux as well. So it is not just the Android platform that is borked. I also never use ignore patterns in case there is an issue around it.
The other bizarre issue is this, I see that the manually deleted folders (because ST was compalining that it was not able to delete them itself) show up as errors later. To me it sounds like the nodes are not communicating changes properly between eachother.
I did not ask you to create any patterns, I suggested you adjust the existing ones as that’s usually the cause for the errors you are referring to. But let’s continue not describing the exact problem properly in hopes to resolve it.
I hear your frustration. But you’re not telling us anything to help narrow it down, or trying the suggestions to do so. This problem never happens to me, and I use Syncthing quite a lot - though not on Android. We can’t solve it for you unless you help us understand why it happens.
Well, you are the devs, you need to tell me what kind of information is you are looking for. My ST ouputs are flooded by thousands of the same messages on Linux and Android and probably by millions by now, they have been running for a while. And in general the message is like the title of this topic.
The thing is that this issue does not just happen on Android mix nodes. I have a share that are shared only by 2 Linux nodes. I started that one fresh and couple of days later they started throwing errors as well. I have not touch the folders and files after I started the sync. So why would they start complaining and throwing “not making progress” type errors.
Anyway the number or errors and number of interesting situations are very overwhelming for me as a user. Therefor you need to ask for specific information from me, so I can help you out with debugging these issues.
Also one node might have 10 different shares, which means that all kinds of error messages are bundled and jumbled together. So it is confusing, hard to read what is going there.
One thing that struck me when I was doing some test is that, ST is not able to deal with “renames” well especially if I just rename lower case to upper case and this is all under Linux platforms where the letter case mattters. After the rename the nodes complain that no such and such file and folder under the old lowercase folder (it seems like all the nodes involved are effected not just one). Although it is able to create the new folder with the uppercase and delete the lower case one but still compaling about missing some files and folders in lowercase folders.
SInce the complained folder is not there anymore I cant do anything about it.