Deleted files being recreated

Support
1

Syncthing 18.5 on three points – running for about four years and working flawlessly. ST on Windows 10 synchronizing with a Windows 10 PC on the same LAN and to a NAS off-site. For the last month, deleted files and folders are being recreated across connected nodes. (Actually, it’s also happening in a similar way on another Windows to Windows Syncthing setup as well.) What would be the reason that files deleted have suddenly started being recreated?

2

You should try and understand which device recreates them from the recent changes dialog. Most likely a dodgy filesystem.

3

All nodes 1.18.5. The offending one is a Windows 10 PC. Curiously, there’s a metadata.xml file being synced from that node and a notification is thrown up whenever it syncs. What would constitute a “dodgy filesystem?”

4

I am not sure how metadata.xml is related to this and I’m not sure what “notification” you are referring to.

Is the filesystem on the device a real filesystem, (i.e NTFS on windows), not some sort of network mounted storage, SD card, VeraCrypt container or something like that?

I guess if you have a way to reproduce this reliably, we can discuss further debugging steps.

5

Are all the hosts online all the time? I have a simmilar issue with my two laptops. Every time I work with my files and folders while the other laptop is offline, all my deleted files get recreated when it comes online again.

6

Effected nodes are always up and online. But, I haven’t seen that issue with one of the nodes (a laptop) that is offline frequently. Thanks for your reply!

7

First of all, I very much appreciate your time to reply!

I’m not sure how metadata.xml being synced should be an issue either except that it’s just lately been doing that. The notification to which I refer is a Windows 10 notification telling me that sync has completed, syncing metadata.xml. I guess I was thinking that perhaps that file is somehow replicating on other nodes a false state of those now deleted files.

The filesystems on the Windows 10 nodes are NFTS. One node is on a QNAP NAS also running 1.18.5. Here’s the setup: 4 Windows 10 desktops 1 QNAP NAS 1 Windows 10 Laptop All nodes sync their local files to a separate drive on one of the Windows 10 desktops which in turn is synced with a cloud storage service. (Gives me a local copy, an on site copy, an off site NAS copy and a cloud copy of all files) I thought at one point the cloud sync might be the culprit but there’s no evidence after some testing that it’s copying anything down from the cloud to the Windows 10 desktop.

What’s very strange is this setup worked well for over a year and suddenly started this behaviour about a month ago.

8

So my money is on your cloud copy thing.

Pause that device to get it out of the loop, run like that until you observe the issue happen again.

9

Wise advice. Thanks. Unfortunately, cloud sync stopped and deleted files still being recreated. How might I go about troubleshooting this?

10

You can enable model/scanner debugging and capture logs when it happens for us to inspect, together with names of files.

11

Sometimes the recreates appear soon, sometimes after several hours. Also have this recurring error in Console (one every minute) on one node: INFO: Failed to exchange Hello messages with KEDXRGH at [::]:22000-#############/quic-server/TLS1.3-TLS_AES_128_GCM_SHA256: Application error 0x0: closing

12

Thats not related.

13

Thank you AB Can I confirm the steps?

  1. Actions → Logs → Debugging Facilities
  2. Check scanner and model
  3. Delete the files
  4. Confirm a recreation
  5. Save sync.log
  6. Record filenames of recreated files ??
14

Yes, but log from console rather than web ui, as web ui shows only a few hundred lines.

Also, ideally capture the logs from both sides.

15

I’ve now captured the logs with some evidence that there were re-creations. Which log files located where? Also, how should they be shared? Thanks!

16

You can upload them wherever best suits you.

17

Thanks for staying with me on this. I’m not particularly pleased with myself at the moment since I thought I understood this! I’m going to need more help. Sorry.

  1. I go to SyncTrayzor and choose Syncthing → '“Open in remote browser” to open the Syncthing GUI
  2. Under “Debugging Facilities” I check ‘model’ and ‘scanner’ on both sync nodes
  3. On Windows 10, where are the resulting log files? Users → … AppData/Local/Syncthing? The log files there all record current date only.
  4. With the names of affected files, I upload the log file where?

I’m missing something here.

18

You’re using SyncTrayzor, so there shouldn’t be any need to go through such hoops :slight_smile:.

  1. In SyncTrayzor, go to “File → Settings”.
  2. Move to the “Syncthing” tab, then unfold “Advanced”.
  3. Under “Syncthing Environment Variables:”, enter STTRACE=model,scanner.
  4. Click “Save” to save the settings.
  5. Restart Syncthing by going to “Syncthing → Restart”.
  6. Reproduce the problem at hand.
  7. Go to “File → Settings → Logging”.
  8. Click “Show Syncthing Log File”.
19

My difficulty is that I can’t predict when the recreations will take place. Sometimes it’s within the hour but other times it’s a week or more. Very curious. I’ll do my best to get some log data to show the occurence.

20

I enabled logging [STTRACE=model,scanner] at about 4:15 p.m. on 17 March, then deleted folders and files. Those same folders and files reappeared on March 18, stamped 7:31 a.m.

In the “logs archive” folder there are three files (all 50 MG +/-)

What I don’t know is:

1) why timestamps in the log entries don't seem to refer to the timestamps of recreated files. All seem to record current time.
2) what I should be looking for
3) how to make the log files accessible for someone who might be able to take a look to see why deleted files reappear. (I'm assuming there's information there that shouldn't be open on the web?)