Feature requests: symlinks, include list, more filter options

I’d like to make three suggestions for inprovements:

1. Symlinks:

I know that following symlinks can be a security risk, but on the other hand, it would be a great relief in many cases. E.g., if there’s a directory containing 100 sub-dirs, of which you need to sync just 10 directories. You can, of course, create 10 pairs of dirs to be synched. This will work, of course, but it any of those 10 dirs needs to be swapped by another, you will have to delete one synched pair and create another one.

On the other hand: if you you could create one special dir, inside which you create symlinks to those dirs, that need to be synched, that work would be a lot easier and much more clearly to keep track of in case of changes to be made:

dir
⌞ subdir1
⌞ subdir2
⌞ subdir3
⌞ subdir4
⌞ syncdir
  ⌞ subdir2 -> ../subdir2
  ⌞ subdir4 -> ../subdir4

Instead of creating two pairs of synching dirs, you just create one pair (syncdir), and if one day you need subdir3 to be synched instead of subdir4, you just change the symlink in syncdir and you’re done!

2. Include lists:

It would be great to use a file which contains names of files and folders that are to be synched within a directory, very much like the opposite of an ignore-file. Sometimes you don’t need to synch the hole content of a directory, just an few files and sub-dirs, and very often in this cases you can not just copy or move those files/dirs to another directory.

3. More filter options:

Filter options for sizes, ages and names of files would be very helpful.

1 Like

2 is already possible with existing system, check the docs and the ! syntax.

Oh, yes, sorry! My mistake!

1 Like

Symlinks are a can of worms; we’ve tried, quite hard, but the problems outweighed the benefits. Similarly filtering on times and sizes and such. The name based stuff is however already there as mentioned.

1 Like

Regarding point 2, the documentation on the include patterns is not described conclusively from this point of view. If, for example, only the files foo and 123 are to be synced in a directory, the Ignore list have the content

*
!foo
!123

* is set to exclude all files, only entries with ! are then included. Then only foo and 123 are synced.

Filtering according to times and sizes would be an unsafe feature, especially if the handling of metadata on the different platforms is considered. Windows, for example, has a other approach to relocating data than Linux. Or e.g. in Windows there is the meta data type “Last saved” for documents, which, as far as I know, does not exist in Linux. It is unreliable and should be watched.

Thanks for the explaining!

I understand the problems concerning filtering by date. And I also realise that in some cases you won’t even get a valid file date, e.g. if a FAT formatted SDCARD in your phone is involved. So let’s forget the idea of time based filter options!

But what about the size of files? Of course you would not be able to select the size with a precision down to a single byte, but I can not think of a case where setting a filter like “sync only those files that are smaller then 1mb” could be considered an unsafe feature.

Could you provide an example please, when and why such a filter might be a dangerous or unsafe thing to do?

It becomes confusing when files change. Say you configure it to only sync files < 1 MiB. A file is edited somewhere else and grows to 2 MiB. You ignore this change because the file no longer matches the criteria, but you have the 1 MiB variant left locally. Maybe you change that version – there’s now a sync conflict, but with an ignored file.

Similarly, if you have a file but change it so that it grows that change won’t be synced to other people, but you’ll still have it locally, and they’ll still have the old version, not knowing they’re out of sync.

And so on and so forth, it’s full of weird corner cases. Is a delete of a size-ignored file synced? What if the criteria specified a minimum size, is the deleted file zero bytes?

1 Like

I can see that filtering by size can cause problems. But, on the other hand, there are many cases in which size does matter…

For example: a navigation app keeps its files in one directory, including maps and recorded tracks. The maps don’t need to be synced, but the tracks should be copied to a pc. Maps are big files (>100MB), while recorded tracks are small (<5MB). Using name filter is not an option there, because there are other files that the app provides, using the same file pattern as the recorded tracks, while being ten times as big. So, the only way to tell those two types of file apart, is their size.

Another example: an app creates special log files for each occurrence of a certain procedure, and once these special log files have been written, they won’t grow any more. The directory also contains normal log files that can grow really big.

I can think of and I have met other situations in which a size based filter does very much make sense. And if someone decide to use that feature, they should know what it does and why they use it. And finally: no one is forced to use an optional feature!

Yeah I’m not saying there aren’t specific situations where it would be useful. However I am saying it would be something of a footgun, with the complete effects of it being difficult to reason about and causing confusing situations. Confusing situations invariably become something we need to debug and explain here, so I’m wary of that. :wink:

2 Likes

That example sounds like a deficiency of said app: It could store these different kinds of data in a way that it’s discernable - there’s plenty of options (path, name, extension, …).

That’s unfortonately not the point: In reality any feature is going to be used in any possible way, also bad ones. And users will not accept weird behaviour being due to their weird instructions, they will blame Syncthing and we get to support them.

1 Like

As always though, just because we’re skeptical doesn’t mean it can’t happen – just that we probably won’t do it. Someone could write up a small spec detailing how the thing should behave in the corner cases we can think of, and implement it cleanly, and get it merged.

1 Like

No offence intended, but that is a problem that exists with every function programmers include in their products. Even the simplest dialog like “open file” can be and will be misunderstood by users and cause a support ticket to pop up.

To get around this, would mean not to write any more programs. :wink:

No offence taken (nor do I intend it) :wink:

That’s partly true, and the reason why we don’t want to do this feature, but also obviously overly simplified. It clearly isn’t a binary problem, it’s on a scale. And Jakob explained why/in which cases this can produce confusion. What’s an acceptable amount of potential confusion isn’t a simple, objective decision to be taken. And as mentioned, this is basically two maintainers saying “too hairy for too little benefit”, not “fundamentally terrible, don’t even think about it”. So the entire discussion is a bit moot - either someone feels like dealing with the hairiness and gets to work, or this topic is moot.

4 Likes