I sync some files from server A（10.206.56.52）to server B (10.206.122.21). Files in server A updated frequently, so I encountered many problems as above.
I just wonder how can I avoid this problem, could you all tell me how to fix it. Thank you all.
You can ignore those, but generally, the problem happens because the files in question either get deleted, moved or renamed on the remote device before they finish downloading. If that occurs while still downloading, the temporarily downloaded partial files will automatically be deleted later.
You may want to play with https://docs.syncthing.net/users/config.html#config-option-folder.fswatcherdelays and set it to a larger value, e.g. 60 (seconds) instead of the default 10. This will delay the sync once Syncthing detects the files, so if they disappear within that time period, it won’t even try to push them to other devices.
Just to be clear, you need to change the value on the device, which creates the files.
Thank you for reply. I have tried your suggestion, but the problem is still existed. I guess the sync rule is that when sync task is start the source file was changed, this error will appear. Plus, I am not sure if it will be sync to destination for ever when the source file updated frequently.
It depends how frequently the file is updated. If it is written into 24/7 with no breaks, then yes, it may not ever be possible to fully sync it to the other side.
Thank you very much, I have got it.
It’s going to be a very rare situation that a file just got a couple of complete blocks added. More likely, if appends are happening, “block 4” is going to be different from step one to step two (like, 100 KiB in step one and a full 128 KiB block in step two).
If the situation is precisely as you describe, which it probably isn’t, then it will indeed sync since the blocks it knows about will match.
You are right. I got it. Thank you.
I’d like to know, if the file is written into 24/7 with no breaks, could you optimize sync mode to that just base source file’s status that when syncthing try to sync the file. Thus, if we turn off [Watch for changes] item of folder, the file will be sync correctly.
We will not encounter error as below:
Actually I think something else is going on here. Appends are not a problem in any case, because the receiving side asks for a byte range and only that byte range is checksummed.
I just tested with a loop that appends a date to a text file once per second while true ; do date >> s1/file ; sleep 1 ; done and this syncs perfectly fine to the other side, periodically.
If the data inside the file changes continuously, not just getting appended, I don’t think there’s any real way to get it synced reliably.
You are right, we need to test it further and find out what was happened. Thank you for your answer.
As you see, this is the destination folder, a file is updating continuous in another server (source server), update time of TMP file and real file is big different about 10 hours.
So, I just think, could we figure out a scheme to deal with scene that updating continuously.
Such as, copy source file per 5 minute and then sync this copy to destination file or transform TMP file to real file per 5 minutes.
If this sync mode can be realized, syncthing will be fit our business requirements perfectly.
Could you please figure out better solutions? Thank you for your attention.
You can indeed make a copy or snapshot of the file and let that sync. This does not need support in Syncthing.
I agree with you. We can develop a tool and copy the files updated recently to a new folder and then sync this new folder with destination folder by syncthing.