Hi there!
How well does Syncthing scale, not in terms of nodes but in terms of files / sizes?
I have a data network share and would like to sync it with local copies both on my desktop and my notebook.
I’m talking about these dimensions:
103,314,954,729 bytes = 96.2 GiB
178,100 files
17,021 folders
567 kiB average file size
Since Syncthing is hashing every file, that would be a tremendous database! At least 790 kiloblocks times 1 kiB/hash is already more than 700 MiB just for the database.
Do you have any suggestions / alternatives?
Thanks you!
Lucas.
calmh
(Jakob Borg)
April 6, 2018, 2:10pm
2
It’s more like 48 bytes or so per block. It’s going to be a few tens of megabytes of index database for sure, but not as much as you expect I think.
1 Like
Thanks! SHA256 is 32 bytes, so the remaining 16 bytes are a file number / ID?
Do you recommend to try it for this amount of data?
data.syncthing.net shows stats on something you might care about.
1 Like
Thank you very much, this is really convincing!!!
Well i am experiencing real life problems with a folder of this size:
About 1 million files
About 30k folders
About 80GB of data
(in this combination)
The problems are mostly OutOfMemory crashes (System has only 1GB Ram + 2GB Swap).
Details here: Syncthing killed repeatedly - no errors
But i guess these limits don’t necessarily apply for stronger systems (esp. with more RAM).
Arneko
(Arne Ko)
April 12, 2018, 10:50pm
7
My machine is running fine with
122088 Files
4535 Folders
~853 GiB of Data
And it is just an Allwiner A20 with 2GB of RAM (Cubietruck 3)
Speeds are not that impressive though, but enough for me - most important thing is reliability, and this machine has been running almost 24/7 for the past 3-4 years without major issues.
1 Like