When I add folder on A it starts uploading, C is the fastest and gets it first. B gets it from C, cause it’s waaay faster.
The question is: can I use C as “proxy” for getting files from another server?
I don’t have many storage space on my VPS, 30Gig I think, and I wanna get some TB’s. I can upload by 20 gigs each time, but may be there better solution?
I would be cool, if I can use bandwith of another servers to get files faster.
You can use it, but you’d have to manage it manually, as you say, upload 20gb, remove, upload 20gb, remove.
Also note that files are only available from C to B once C has the whole file, so given you have a single 20gb file, the speedup will kick in only after C has got the whole file.
Well, obviously I can do it manually.
I was trying with files of several gigs now, and I get it on C pretty fast. I will take a look on speed, guess B will receive file with 8mpbs until C gets 100%. Then B goes full speed getting it from both sources.
It would be cool, if I can set up a directory and use it for caching like that. Set a limit for each server. Or be able to use some servers as “proxy”.
I recently played around with GO and Syncthing (haven’t gone into the technical part yet), it’s great! However on the slower arm machine, it took awhile just to complete the scanning (I understood this part as per the FAQ) let alone to distribute the files across the nodes.
Related to this topic, which I’m not sure this is doable without protocol changes, is a suggestion to introduce concurrent multi-part transfers could help a lot in the initial file distributions.
Taken from the quote “files are only available from C to B once C has the whole file”, instead of C doing only the receiving part from A for this 20GB file, it will also send out the partially downloaded part to B on request. Best still if A could send part of the file to B on request. I believe each node has the final 20GB file info to decide which part to send/receive through the meta exchange? The only thing is each node needs to announce how much data it already has. Segment maybe the correct term but I put as an example by part 20GB/10:
A announce "hey I have 10 parts available", B request for part1++, C request for part1++ and reached part3, then B knowing C has part2 completed will also request part2 from C
This is just my wild suggestion (I think from how BT works I’m still new in P2P ) but if Syncthing could implement this then it would be a major breakthrough.
And you can add feature to cache some data in servers. Say if server B requests file from A, it starts downloading part1 from directly from A, and C gets part2. Then A gets part2 from C, while getting part3 from A and so on.
For example with 10 caching servers the time A uploads the file will be 10 times less. Not to mention speed increase in some cases.
Imho you could also use an SSH Tunnel or IPtables to create an Proxy Server for Syncthing
For example on your VPS:
iptables -t nat -A PREROUTING -p tcp --dport 22000 -j DNAT --to-destination (IP_ADDR_OF_B):22000
And then on A you add an new Device with (IP_ADDR_OF_YOUR_VPS):22000
(Of course you need to specify IP/Port manually, because the Global Discovery doesnt work.)
It looks like you are connecting to your VPS, but it will redirect your Data to B.
) but if Syncthing could implement this then it would be a major breakthrough.