You’re going to have to be more specific. 
Wrapped my head around the query and it’s fine as is 
SELECT f.name FROM files f
WHERE NOT EXISTS (
SELECT 1 FROM files g
WHERE g.name = f.name AND g.local_flags & ? != 0
)
GROUP BY name
My point about flag magic still stands, though. All these local_flags & ? predicates are not able to use an index.
I started testing beta5 and updated to beta6. Should beta6 automatically convert my single file index-v2.db into the newer one database per folder format?
Nope. beta5 → beta6 is breaking. Nuke your DB to proceed.
Indeed. However the median user has 3 devices, 95% of devices have 9 peers or less. We have an index on the name, so I’m thinking these queries should be fairly straightforward even in the absence of an index, since they all result in a small handful of entries, and maintaining the index isn’t free in disk space or CPU?
Maybe we can give the sqlite query planner a little nudge if we know that:
https://sqlite.org/lang_corefunc.html#unlikely
SELECT f.name FROM files f
WHERE NOT EXISTS (
SELECT 1 FROM files g
WHERE unlikely(g.name = f.name) AND g.local_flags & ? != 0
)
GROUP BY name
This would tell the query planner that name has high selectivity and should be preferred.
Edit: this should be checked via EXPLAIN if it really improves execution time
1 Like
A partial index on a flag with low selectivity might also work.
Nuked database, had to manually create index-v2 folder. It works now.
Advice needed, i’m still running through the first pass of beta 6, it started shortly after 6 was released!!. Do you want me to let it carry on, or would you like me to clear the index 2 folder and start again with beta 8?
Edit: Renamed beta6 / index 2 and starting again on 8
I’m excited for the 2.0 changes, especially weak hashing removal! I manage a bunch of low-end devices (10+ years old and not the best when they were new either; ~50GB synced across ~200k files) that I expect will see very noticeable performance improvements from this.
I have a question about configurable delete retention: How do devices behave when they have differing intervals? For example, suppose interconnected Devices A, B, and C have the default 6-month interval and (also connected) Device D is set to 12 months. Device C is taken offline; then Device A deletes a file; then 9 months later, Device C reconnects. Based on my understanding of Syncthing internals, the file’s most recent change is its deletion, so I would expect Device D to “win” and the file to eventually be deleted from all devices - even if Device C manages to sync it to A and B first. Is this correct?
(My preferred setup would be default 6-month intervals on my low-end devices and a much longer or infinite interval configured on my high-end devices, but only if Device D is the winner in the above scenario. If not, I’ll be setting all of my devices to 0/infinity. I do rarely have devices come back online after being out for an unpredictable number of years.)
99% the delete would take, as it’s accepted and processed when received and only forgotten on a slow periodic task. But if it fails for permission reasons or whatever for 8 hours, or you lose the timing lottery, or we change things in the intervening year, … better set it to be what you want a guarantee for.
1 Like
Thank you for the reply, but I don’t think I quite follow. I’ll modify my scenario just slightly:
- Devices
A,B, andCforget deleted files after 180 days. DeviceDforgets after 360 days.
- Jan 1:
Cdisconnects from network - Jan 2:
Adeletes file. Deletion is synced toBandD - Jul ~1:
AandBforget file - Sep 30:
Ddisconnects from network - Oct 1:
Creconnects to network. File is synced toAandBsince it is “new” to them - Oct 2:
Dreconnects to network - ???
If I understand correctly, there is no reason for any device to forget the file around Oct 1. So this would be equivalent to losing the timing lottery, right?
(Now that I think about it, it makes sense to me that it would behave the same as v1.x does when you add an entirely new device that has a file which was recently deleted by the existing devices. Which I’m also not sure what would happen)
Garbage collection runs every 8 hours (by default). It’s possible, but not very likely, for C to get information about an old deleted file and not have time to process/sync that deletion before garbage collection kicks in, if that happens very soon after receiving the file entry.
Actually I think something interesting will happen when the delete is forgotten, as those devices will then consider themselves out of sync as they are missing the file that the long gone device was now the only one to announce. This would serve as a reminder to bring that device back from the dead.
So yeah if this is a situation you expect, set the flag accordingly.
1 Like
I’m using the Docker container packaged by LinuxServer.io, but they don’t seem to be following up with any Beta releases. I can’t just switch to the official Syncthing version because of some possible conflicts between different image configurations. Maybe try contacting them.
Hi, I am running Syncthing on my Synology DS1821+ NAS which was installed using the SynoCummunity Installer.
I saw the shiny red button to upgrade to v2 at the top so pressed it and now Syncthing shows as running a process on the NAS but does not let me connect…
Is there a command a can possibly run on the NAS to reset or is my only option to uninstall and go back to v1 for now?
Also side question… Would all of my devices need to be upgraded to v2 to sync with each other or can v2 sync with v1?
Upgrading to V2 is a one-way, unstoppable process. You can try to wait for a while. If the problem still exists, it is recommended that you package the problematic configuration files, databases, etc. and send them to the developer for troubleshooting.
I am not sure whether the configuration files of V1 and V2 are compatible. If you cannot roll back to V1 directly, you can try to manually install V1 and then manually merge the configuration items in the V2 configuration file into the V1 configuration file. Obviously, this can reduce a lot of configuration work, but rescanning all files may take a long time.
Both versions can connect and sync with one another, i.e. there is no need to upgrade all devices to v2 in order to continue synchronising them.
They config files between the two versions use the same XML format and are compatible with each other.
1 Like