Does syncing only happen with complete files?

I’m curious about something I am seeing. It looks like syncing will only occur from one node to another, say A to B, if A has a full copy of the file. And if it has a partial incomplete copy it will not starting syncing its pieces until it does have a full copy.

So lets take three machines A, B, & C; all syncing to each other. A has high speed bandwidth B has medium speed bandwidth C has slow speed bandwidth

A file is dropped onto A. Syncing from A>B, and A>C begins immediately. However no syncing between B<>C begins neither has a full copy of that file. Some time later B does have a full copy, but still not C, and at that point in time both A & B will sync to C.

That seems to be what I am observing. Is that expected behavior, and just curious why that is. I was sure the Bittorent protocol (which BEP is not), would start uploading its pieces of a file even though it did not have a completed copy of that file. But maybe I am wrong; its been a long time since I have used Bittorent, but I was sure I saw that behavior with BTSync/Resilio.I read the BEP doc, but couldn’t find where it talks about this particular item.

I’m not complaining at all, just asking, just curious. Syncthing is awesome; its been near flawless for me.

They should exchange blocks from partial files too, unless disableTempIndexes is set to true (false by default): https://docs.syncthing.net/users/config.html However it might be that your file is too small? It starts requesting up to 64MB immediately, so it might have already requested everything from A by the time B announces anything.

I will have to do some more observations but yesterday I did a few 4-5GB files and it looked like there was zero syncing between B<>C for the first hour or so before I went to bed. When I woke up early this morning I noticed B was “up to date” and C was indeed downloading from both A & B.

I have not modified disableTempIndexes or any other system defaults. In fact the only default I have modified is the folder scan time; change from 1H (hour) to 6H for my large volumes, and one folder is set to 1 minute. The folder in question here is set to 6H. And I have modified the sync protocol listening port from 22000 to something else.

I’ve got a busy weekend, and it will be next week before I have some larger files to use to test. If indeed I am seeing this behavior what should I look for as a cause? Any extra debug I can turn on to get insight? Thanks everyone.

Model debug logging over the lifetime of the transfer would answer it on the receiving device

Sorry for the late response; can I turn this on mid-way through a transfer or does it need to be enabled before I start one? I turned it on shortly after I started a transfer, looks like I am capturing info. Will need to read. What specific log text should I be looking for?

Also regarding this previous comment: “They should exchange blocks from partial files too…”. Do the receiving nodes all try to start at the first block and work their way towards the last block? Or do they all randomly select a block (which would be anywhere in the file) and download that first, so that over time, each receiving node builds up its own random list of completed blocks that the other receiving (and still un-synced nodes) can use to pull from instead of the node that has the file 100%?

It won’t have an effect on already started files. As there’s no known problem with this, there’s no specific message to look for. Everything at the time of the transfer might be helpful.

The description for blockPullOrder explains how it works: https://docs.syncthing.net/users/config.html
It’s essentially random, with some improvements to do more contiguous IO than entirely random.

"blockPullOrder

standard (default): The blocks of a file are split into N equal continuous sequences, where N is the number of connected devices. Each device starts downloading it’s own sequence, after which it picks other devices sequences at random. Provides acceptable data distribution and minimal spinning disk strain."

I found the mistake in my observation. The files I transfer are often 10GB or larger, and I did not watch the syncing later in the process. If I understand the above correctly, since I have three syncing nodes, then N=2. Right? Node 0 has the file, and thus node 1 and node 2 are downloading. So node 1 downloads one sequence (1/2 of the file), and node 2 downloads the other sequence (the other 1/2 of the file). Right? It looked like after node 1 had completed the first half of the transfer it began to upload to node 2. A little time later, noticing node 2 was over 50% done, it too was uploading to node 1. So it seems I was simply looking too early in the process. So if I had 10 nodes then each node would have to only download 10% of a file before it would start uploading. And if I have 100 nodes then each node would only have to download 1% before it would start uploading. Am I more or less correct with my understanding of this?

Regarding this mode, which seems to be more like how bittorrent protocol works:

random: The blocks of a file are downloaded in a random order. Provides great data distribution, but very taxing on spinning disk drives”.

Is there a way to quantify and explain this? Just curious to know if the benefits outweigh the downside. In the grand scheme of things probably not I’m guessing.

But I did notice something else I need to look into. It seems node 1 connections (to both node 0 and 2) frequently reset when actively syncing. If there is no traffic the connections do not reset, but they do reset fairly frequently when there is active syncing traffic. Node 0 and 2 have NAT port opened up, but node 1 does not yet. Could that have something to do with it? After the connections come back up to node 1, it seems to take a long time before there is syncing between nodes 1 and 2, and sometimes it appears the connection between 1 and 2 gets torn down before data can start flowing. I’ll have to dive into that when I get some time, and try to open up a NAT port for node 1 as well, which I had intended to do anyway.

Thanks again!

Various things, such as tweaking folder settings causes syncthing to reconnect to other nodes to re-do the handshake, which is probably what you are seeing.

Regarding the data distribution, I can’t really answer the question as you can’t quantify it in absolute terms as it all depends on the bandwidth available, effectively seeking around on a spinning disk might become slower than network bandwidth available and slowdown overall transfer, plus cause general bad ux using the computer while its thrashing the disks.

Nope, I am not tweaking folder settings. I changed default scan time for most of them way back, and have not touched them since. I will try to determine what is root cause, and if need be, will search for existing discussion, and then post a new topic if required.

Thanks again for your great and abundant support.

The side that decides to terminate the connection usually logs why it does that.

After reading the discussion it is still unclear to me the behavior with partial files.

I understood that in a 3 node scenario the partial file sync happens between the nodes which don’t have a full file:

  • A has the full file
  • B partially has the file
  • C can start download from B even if it has a partial file

However, let’s picture this two node scenario:

  • Node A is a cloud server with an FTP server where people upload data
  • Node B is a local server

  • An ftp client connects to A and starts uploading a 8GB file (via FTP), does A annouce the new file immediately or only after the upload completes?
  • Does B begin the partial file synchronization from A?

It will probably announce the partial, but it will go through a few error cycles until it actually syncs, because the file changes as it’s being synced/scanned.

Can I change this setting on an Android device? I don’t see that I can.

You can access the web UI, which is the same as everywhere: Actions > Advanced

1 Like