"directory is not empty" is back


#1

3 linux devices share one folder.

A & B are full sync setup and v0.14.44. C is the workstation, v.14.45-rc3. C partially syncs, with help of an ignore pattern:

!Work
*

[EDIT]: all 3 devices have fsWatcher enabled and Scan Interval=0

Case : on C rename Work/dir to Work/otherdir

On A & B, Work/otherdir is created fine but Work/dir (and content) remains there, endind Out of sync 1 item failed with the detailed message in topic title.

On C we can see this device wishes to push Work/dir (deletion) to A & B for 0 bytes.

On A & B we see they wish to push Work/dir to C for 0 bytes too.


FS Watcher Status
#2

…[EDIT]: …now trying to reproduce

Found more :slight_smile: : whitespace in new name seems the culprit (in fact I used mv dir new\ dir (in file browser))

Once sync is broken this way, doing an other same kind of rename on another subdir with a space in target name may drive in data loss as content of second dir isn’t sync’d : deleting it (empty on remote) will remove (kind of rm -rf) it on local


(Audrius Butkevicius) #3

It’s totally not clear what you are talking about.

If you think this is a bug, produce an exact list of steps that reproduces this in an isolated environment


#4

3 linux devices share one folder with each others.

A & B are full sync setup and v0.14.44. C is the workstation, v.14.45-rc3. C partially syncs, with help of an ignore pattern:

!17
*

All 3 devices have fsWatcher enabled and Scan Interval=0

Start from a clean status on all 3 devices : all stuff well sync’d, i.e. no “Out of sync error” nowhere and global is the same on all 3, local being the same as global on A & B, and smaller on C according to the ignore pattern.

Just to not puzzle ST at all, create out of ST scope this tree : 2016/fold/ with one file in fold. Then move 2016 tree into 17 folder

Let it sync, check and wait everything is stable: all 6 counters must have increased of 3 units (2 more folders plus one file)

Case: on C rename 17/2016/fold to 17/2016/fold nouv

Issue:

On A & B, 17/2016/fold nouv is created fine but 17/2016/fold (and content) remains there, endind Out of sync 1 item failed with the detailed message in topic title.

On C we can see this device wishes to push 17/2016/fold (deletion) to A & B for 0 bytes.

On A & B we see they wish to push 17/2016/fold to C for 0 bytes too.


#5

I can reproduce the issue as many times I want.

I did it once more a bit different: this time, in B, I dropped a test/file directly in shared folder root (that C can’t see because of ignore) => good.

Then still on B, rename test to test two => both test and test two now exist in A.

B Uptodate wants to push test to A.

A OutOfSync 1 failed item details=> test directory is not empty

C UptoDate wants to push 3 items to A (test + test two + file (Mod. Device being B for all 3 items)). It is strange that C is aware of files it never saw. This can’t be a broadcast from A because A really has test test two and twice file (one in each dir test & test two)… or maybe A is not aware of what he wrote on disk, or forgot what he wrote, or thinks he deleted something that he really did not?

Additional info that may be important to know : B & C use simple trashcan style and A uses staggered versioning trashcan.

The issue seems to be related to the way renames are performed.


#6

And now, something completely different. Hammer and 4mm iron… let’s bend this piece :smile:

Good week-end ST devs & users


(Audrius Butkevicius) #7

This is still not a clear report as it’s full of observations that might not have any meaning, it’s totally not obvious why this needs 3 devices rather than 2, where does the 17 folder comes from, etc.

Something like this is easier to follow

Pls make sure you reproduce the issue in a clean folder with no prior history.


#8

Done with new shared folder with no-history-fresly-created: Same setup , same issue

just the real name I use. No matters

Feel free to check with 2 devices only. If it where obvious, there wouldn’t issue ;-). The point is this is my set up. 3 devices may not be required to reproduce. 3 devices are required so that we have 2 from them that do full sync, and the third only partial one. Whatever, the issue can be seen outside (above) !IncludedFolder i.e. root. Maybe this is only an issue with fsWatcher enabled & ScanInterval=0

I mistyped the way my devices B & C are setup to backup deletion : the english name is “Trash Can File Versioning” (not Simple Trash Can as I said before)

Also, take note that fsWatcher is not supported in basic settings, and also I set ScanInterval=0 instead of 60, and also that my devices are not at same version level as stated in OP. This post was only to report to @Imsodin about the use of fsWatcher (unrelated to restartOnWakeup=0, that why I posted a new topic). Also, take note that my decision to set ScanInterval=0 is one of a complete ignorant: just a guess that fsWatcher makes scheduled scanning pointless.

Sorry for the hot potatoe


#9

Solved by disabling FSwatcher and resetting Scan Interval to default (60) : now the original folder name is well removed on remote once st finishes copying with new name.


(Audrius Butkevicius) #11

Ping @imsodin


(Simon) #12

I can’t reproduce this with the following steps. If you did something differently, please correct me.

  1. On device A enable only watching, no regular scans, and create dir foo and make sure it is synced.
  2. Rename foo to foo bar.

Result:
After 10s, foo bar is created on device B. After another 50s, foo is deleted on device B.
This delay of the deletion is intentional, to make sure that we “copy” before “removing”, such that the data doesn’t need to be sent but can be reused locally. The exact time of the delay is an implementation detail.


(Peter Marquardt) #13

Hold the press 8)

I can reproduce. I tried to set it up as described, so there are still too many factors involved. I’ll try to hunt it down.

Setup: Windows 7 / Sync v0.14.44 : Share D:\FOO, Ignore: !BAR\n*

Sync to (2) v0.14.44 linux VMs.

On Windows create folder D:\FOO\BAR, copy file FLUFFYBUTT.gif to D:\FOO\BAR

Everything is fine.

Now rename “FLUFFYBUTT.gif” to “FLUFFY BUTT.gif” on the Windows machine.

On the client the file “FLUFFY BUTT.gif” gets created BUT “FLUFFYBUTT.gif” does not get removed.

Two files on the client as a result of a “\0x20” rename on the master and the database does not know about the “old” one anymore.

I’ll deliver logfiles later, since I have to clean up the test mess 8)

Ok. Doesn’t seem to be an issue with white spaces.

This also happens with FLUFFYBUTT.gif to BUTTERCUP.gif . The new file appears, the old one stays.

Maybe thats exactly the described behaviour: It takes a while until the file gets deleted in the file system.

Lets see, I’ll be more patient now 8)


(Peter Marquardt) #14

Hmmm… OK, I fell for it. The “lost” files get removed after a short time.

Grmpf, I thought I could reproduce it 8)

Need more input 8)


#15

My setup had 3 devices, same setup you stated Simon (1.) on all 3.

B-C full share

Only A has ignores (mostly all):

!include
*

renaming include/dir/foo (foo being a dir) to include/dir/foo bar on A left foo on B & C (with directory is not empty). Maybe I didn’t wait enough although I think I did.

Also, (sometimes) I even saw foo coming back to A like a boomerang (being recreated beside foo bar!!!). Sometimes, foo on B or C even remained populated, in which case the message could eventually make sense.


(Simon) #16

Sorry, but without more specific steps I can’t reproduce this, even with these ignores everything performs as expected for me. You could enable the watchaggregator debug facilities and then look for lines mentioning the paths that make trouble - maybe that can shed light on what’s happening for you.


#17

Maybe the problem is related to my Scan Interval setting from default 60s to 0 (i.e. disabled according to the documentation). I’m not sure where I found this idea that FSWatcher is intended to replace scheduled scans.


(Simon) #18

That’s easy to check: Do your problems go away after hitting the scan button in the web UI? I also have scan interval at 0 on my laptop, as I consider full scans on every start/wake-up as sufficient. If you run Syncthing on a server or any mostly on device you certainly want regular scans as well, that’s why the new default will still be full scans every hour with watcher.


(Marco Trevisan) #19

Same thing happens here…

fsWatch enabled, I remove a folder from my machine, this is what I get on the other nodes and the folder is still there with some contents

EDIT: sorry, I just updated my .stignore accordingly and things go well now :slight_smile:


(Audrius Butkevicius) #20

/slap


(system) #21

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.