# Noob question - incremental sync?
**URL:** https://forum.syncthing.net/t/noob-question-incremental-sync/1030
**Category:** Feature
**Created:** [September 30, 2014, 8:11am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030 "2014-09-30T08:11:53Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@ionMike](https://forum.syncthing.net/u/ionMike)
#### Post date: [September 30, 2014, 8:11am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/1 "2014-09-30T08:11:53Z")
Couldn’t find any info so I’ll ask it straight - is synchronization incremental? I.e. if my 2 GB Outlook file receives a new e-mail, does the whole 2 GB + 1 kb get synced? Thanks, iM
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [September 30, 2014, 8:22am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/2 "2014-09-30T08:22:14Z")
Depends on how the outlook file format works. Hopefully not.
---
### Author:  [@ionMike](https://forum.syncthing.net/u/ionMike)
#### Post date: [September 30, 2014, 8:24am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/3 "2014-09-30T08:24:43Z")
Thanks for the quick reply. My question doesn’t concern Outlook as such, it’s more general. Does Syncthing work along the same lines as Rsync, i.e. it only transfers the changes to any given file? iM
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [September 30, 2014, 8:26am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/4 "2014-09-30T08:26:34Z")
Yes, but it depends on how those changes are done. If data is appended to the file or changed in the middle of the file, it’s fully incremental. If stuff is added to the beginning of the file and everything is shuffled backwards, it’s not. But then that requires a full rewrite of the file to do the change as well, so it’s a pretty stupid file format if that’s the case.
---
### Author:  [@ionMike](https://forum.syncthing.net/u/ionMike)
#### Post date: [September 30, 2014, 8:31am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/5 "2014-09-30T08:31:02Z")
OK, good to know. Thanks! iM
---
### Author:  [@hobarrera](https://forum.syncthing.net/u/hobarrera)
#### Post date: [October 2, 2014, 1:25pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/6 "2014-10-02T13:25:28Z")
> [@calmh](#):
>
> Yes, but it depends on how those changes are done. If data is appended to the file or changed in the middle of the file, it’s fully incremental. If stuff is added to the beginning of the file and everything is shuffled backwards, it’s not. But then that requires a full rewrite of the file to do the change as well, so it’s a pretty stupid file format if that’s the case.
Have you given any thought to using something rsync-like for syncronizing large files in future?
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [October 3, 2014, 1:43pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/7 "2014-10-03T13:43:41Z")
No, not really. This was obvious and simple and I haven’t heard a compelling use case or seen patches for anything else. 😉
---
### Author:  [@GeeGee](https://forum.syncthing.net/u/GeeGee)
#### Post date: [May 20, 2015, 9:08am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/8 "2015-05-20T09:08:55Z")
Is there any update on this topic?
I’m desperately looking for incremental sync.
---
### Author:  [@canton7](https://forum.syncthing.net/u/canton7)
#### Post date: [May 20, 2015, 9:24am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/9 "2015-05-20T09:24:19Z")
The answer is as it was: take a look at your file, and split it into blocks of 128kb. Now change the file. Only those blocks which have been altered will be synced.
This means that appends and in-file modifications will be synced in an incremental way. Changes which move data around across the whole file will not be synced in an incremental way.
---
### Author:  [@GeeGee](https://forum.syncthing.net/u/GeeGee)
#### Post date: [May 20, 2015, 9:39am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/10 "2015-05-20T09:39:46Z")
That’s quiet disappointing.
Rsync and Dropbox can detect where in a file the change occurs and can insert it at the right position when syncing. That’s much more effective. Think on a TrueCrypt-Container that needs to be synced: the current algorith will have to sync the whole file in almost any case ☹
---
### Author:  [@canton7](https://forum.syncthing.net/u/canton7)
#### Post date: [May 20, 2015, 9:41am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/11 "2015-05-20T09:41:20Z")
IIRC TrueCrypt contains completely change on _any_ save? That’s kind of the point of a secure container. So not even Dropbox/Rsync would be able to cope…
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [May 20, 2015, 10:18am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/12 "2015-05-20T10:18:19Z")
That’s not how Truecrypt behaves. It’s a disk image like any other, except encrypted. If you write a 1 MB file to it, you’ll get 1 MB of changed data (plus some blocks of filesystem metadata etc) which we’ll transfer as efficiently as anything else.
I still see this as a theoretical thing, with only Photoshop files so far being mentioned as an example of something real world that would benefit from rolling checksums, and then only in corner cases.
And rolling checksums are by their nature something peer-to-peer, not really easily implemented for a cluster such as syncthing.
@GeeGee What’s the actual use case that you are desperately looking to solve?
---
### Author:  [@GeeGee](https://forum.syncthing.net/u/GeeGee)
#### Post date: [May 29, 2015, 9:06am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/13 "2015-05-29T09:06:53Z")
As I moved from truecrypt to encfs due to the problem mentioned here, it now only applies to PST-files from Outlook. The rsync-algorithm is available to the public. Why isn’t syncthing adopting something that is working very well?
---
### Author:  [@AudriusButkevicius](https://forum.syncthing.net/u/AudriusButkevicius)
#### Post date: [May 29, 2015, 9:22am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/14 "2015-05-29T09:22:37Z")
Because rsync uses rolling hashes, which are not cryptographically secure, which is one of the properties which syncthing requires in order to prevent spoofing/DoS. We’d need a cryptographically secure rolling hash, which most likely means inventing our own crypto, which is never a good thing.
Plus, rsync relies on variable block size, which would make it very hard for synchting to maintain an index. There is already a large discussion about this in another thread, I am sure if you search for `rolling hash` you’ll be able to find it.
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [May 29, 2015, 10:24am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/15 "2015-05-29T10:24:45Z")
According to what I can see, PST files use a block based database structure that should be very well suited to our sync algorithm. Again, do you see an _actual problem_ here, _in practice_?
---
### Author:  [@stevenroose](https://forum.syncthing.net/u/stevenroose)
#### Post date: [May 30, 2015, 10:24pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/16 "2015-05-30T22:24:22Z")
What do you mean with
> rsync uses rolling hashes, which are not cryptographically secure
When is a (rolling) hash function cryptographically secure?
---
### Author:  [@AudriusButkevicius](https://forum.syncthing.net/u/AudriusButkevicius)
#### Post date: [May 30, 2015, 10:39pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/17 "2015-05-30T22:39:58Z")
When it’s designed to be.
> **[Cryptographic hash function](https://en.wikipedia.org/wiki/Cryptographic_hash_function)**
>
> A cryptographic hash function is a special class of hash function that has certain properties which make it suitable for use in cryptography. It is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert. The only way to recreate the input data from an ideal cryptographic hash function's output is to attempt a brute-force search of possible inputs to see if they p...
I am not aware of a cryptographically secure rolling hash. Though you could implement rsync like comparing, it would just be a largeamount of work.
---
### Author:  [@stevenroose](https://forum.syncthing.net/u/stevenroose)
#### Post date: [May 30, 2015, 10:50pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/18 "2015-05-30T22:50:58Z")
Well, ok. What I don’t get though, is why that is important for spoofing/DoS. Aren’t two connected syncthing devices supposed to trust each other?
---
### Author:  [@AudriusButkevicius](https://forum.syncthing.net/u/AudriusButkevicius)
#### Post date: [May 30, 2015, 10:57pm UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/19 "2015-05-30T22:57:02Z")
It’s (I guess) more todo with the likeness of collisions, given we trust a 64 byte hash for each 128kb worth of data, for potentially terabytes of it in total.
---
### Author:  [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [May 31, 2015, 5:08am UTC](https://forum.syncthing.net/t/noob-question-incremental-sync/1030/20 "2015-05-31T05:08:51Z")
The point is more that it prevents working from a known index and distributing requests among peers. Syncing a file using a rolling checksum is an operation performed by two participants who both read through the file at the same time and reporting their findings. Also **it’s not necessary** as the current setup **works perfectly fine** for all so far mentioned use cases.