Summoning @imsodin. I guess we’ll need a repro.
I get the following issue, maybe because releases mismatch:
0.14.49-rc1:
!/2012/2012 01 foo
*
0.14.48:
no ignore, no !ignore
Subtree exists at both end with 138 files. When added a file at 49-rc1 side, 48 stable showed 139 items Out of Sync
I reproduced observed the out of sync issue and know how it happens - it is due to the new need database stuff in combination with ignored files, but not related to descending to catch include paths. It’s totally possible that what you describe in your first post is unrelated to this (haven’t found anything about that yet).
I can’t reproduce the unintended deletions. Please check my steps and correct me if I did something different:
- Setup share between devices A and B.
- Add the exact ignore patternss of your first post on device A.
- Create folder
2006/200604 07 fooand some random files therein on A and wait for sync to B. - Remove
!200604 07 foofrom the ignore patterns on A.
Intermediate observation: Local state goes down to just 1 folder (2006). - On A delete
200604 07 fooinside of2006(i.e. I don’t delete2006itself).
My result: Nothing happens on B.
Your result (?): The files are also deleted on B.
Hi Simon
Differences from me in your steps:
Everything files/dirs/shares/!ignores yet existed at the moment ST was upgraded on A (the !ignoring device) was upgraded to 0.14.49-rc1. The 2 other devices in the cluster are/was set to autoupgrade to stable releases only. At the time of upgrade, the cluster was UpToDate on the 3 everywhere.
I can’t remember if I deleted 200604 07 foo then the related !ignore line before or after the upgrade.
Like you I didn’t deleted the 2006 directory as I still have the !200604 17-23 bar to work on inside.
Is this is the new best syntax for this goal?:
!/2006/200604 07 foo
!/2006/200604 17-23 bar
!/2014/201402 foo
*
Notice the leading / I added to feel secure
Also, the strange thing, before & after I reset-database on the 3 nodes is that B & C show(resurects) missing OutOfSync items that where deleted several months ago.
See Deleted files remain ‘Syncing (95%, 0B)’ for report of resurrected ‘zombie’ files after upgrade to .49rc. I was experimenting with ignores on separate folder when zombies (deleted files from days ago) appeared as out-of-sync. Reverting to .48 made no difference. Resolved by --reset-deltas on offending machine and deleting ignores. Updated again to .49rc and no further problem but yet to reintroduce ignores. I might sit on my hands for a while.
1 Like
[EDIT] other things: I found fswatcher was enabled on dev C with default max_user_watches (8192) which was roughly 1/4 the number of files in all the shares on this dev. I disabled and set MUW to 131072 for further re-enablement.
Don’t worry about the out of sync stuff, it is most likely https://github.com/syncthing/syncthing/issues/5007 for which a PR with a fix is already in place - there will be an RC 2 soon.
And yes, those ignore patterns should work with 0.14.49.
As to reproducing the deletion issue: I don’t see anything to try/change on my reproducer based on your answer (or I didn’t understand). If you can reproduce it, please post steps the way I did. As it stands there is nothing more that I can investigate about this for me.
Thank you guys. I was so puzzled by these issues that I didn’t notice today the Out Of Sync stuff went away.
Have a good sunday
It would still be nice if you produced steps for the deletion reproduction issue.
I’ll do my best Audrius, but it will be very difficult as I didn’t change the way I usually work with the files in this shared folder. As I said, given the !ignored syntax I used while on v48, maybe the introduction of new behaviour in 49rc did something, maybe plus a bad setting in one 48stable about fsWatcher. Now I changed it on my 49rc device.
Ignores and watcher is not related.
1 Like
Maybe also related to #5007 : device A is filling its own Recent Changes GUI list each minute, as well as the 2 other devices one, with:
| Device | Action | Type | Folder | Path | Time |
|---|---|---|---|---|---|
| DevA | modified | dir | Photos | 2012 | 2018-06-17 15:46:55 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 15:46:08 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 15:45:11 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 15:44:18 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 15:43:12 |
Although all 3 devices show UpToDate, B & C blink each minute showing Synching 0% (intended as fsWatcher is disabled), and sometimes briefly Out Of Sync 99% missing 1 item for 128B.
[EDIT] I commented out all !ignores but one (!/2012/201208 foo … and * of course 
)
I don’t think it it should be related to the rc
Now I uncommented !/2009/200912 26* and the Recent Changes lists went:
| Device | Action | Type | Folder | Path | Time |
|---|---|---|---|---|---|
| DevA | modified | dir | Photos | 2012 | 2018-06-17 18:33:02 |
| … | |||||
| DevA | modified | dir | Photos | 2009 | 2018-06-17 18:24:52 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 18:23:46 |
| DevA | modified | dir | Photos | 2009 | 2018-06-17 18:23:46 |
| DevA | modified | dir | Photos | 2012 | 2018-06-17 18:22:45 |
| DevA | added | dir | Photos | 2009 | 2018-06-17 18:22:45 |
| … |
… and went on my usual work by treating my local .../2009/200912 26 etc end of year. The remaining job was to look at the pictures inside, decide which ones to keep and delete the others. Before I started I checked the Up to Date status on the 3 devices (OK) and I cleaned .stversions/2009 everywhere to track any further possible issue.
I did the job, then locally added added my usual flag inside the subdir (a 0 byte empty file called 0k), then I was ready to remove the !ignore and delete the local subdir (and even the no more needed 2009 dir as there is the single 200912blah dir inside) after I check everything is still UptoDate … but:
Now, devices B & C show syncthing 99% ~265MiB to Device A for 107 items! Local subdir is 123 items ~297MB, remote ones (B & C) are 153 items ~382MB, and remote .stversions/2009 are only 13 files ~20MB when I deleted 43 (153+13-123) on dev A.
To have better/cleaner view I re-commented !2012/... so that only one !ignore remains, the 2009 one.
On A Local State shows 123 files/1 dir (what maybe ok or ko, depending we count 2009 for one, and 200912 26... for an other)
While A is scanning, Global State dir number decreases by one (1) and increases back by 1 in the end, on every scan even no real change happened.
The 107 files B & C would want to send to A either are & were yet on A, or are neither in A nor in B/C’s .stversions .
Either initial Up to Date was wrong (actually it wasn’t, I checked this) or st was puzzled at some point during the deletions on devA.
Sorry, but I have no idea what you are trying to describe here. What we care is a reproducible list of steps on an empty folder like @imsodin has provided, that produces some issue.
Everything else you provide has very little value, as it’s it only has meaning to the state of folders you have, not the state of folders we can reproduce.
@imsodin On .49rc, if I introduce an ignore pattern for a .~lock file on machine A, it works and excludes the lock file as expected on other nodes. A similar ignore pattern on machine B causes machine A to appear Out-of-Sync after the .~lock file is deleted by closing the open file on B.
Removing the ignore pattern on A makes no difference. Remove it on B and Out-of-Sync folder is now Up-to-Date on A.
Is this what you were referring to earlier in this thread when you said rc2 would address this?
Not sure, but likely - I suggest you update to rc.2 and try again.