Selective Sync Feature(most important)

The danger with AI is that they can be confidently wrong with their statements. This is often mixed with thruth which makes errors hard to spot.

Most of what ChatGPT said looks correct, though some things are confusing or just plain wrong:

Wrong. Ignore patterns are per-folder within the syncthing UI, not global.

This is confusing, because /temp/* only ignores files in a top-level folder temp, but the description makes it sound like any folder named is matched. temp/* would make more sense with the given description. The * is also unnecessary here.

Syncthing uses one ignore pattern per line. Comma separation is incorrect. The first sentence also sounds like it refers to a logical “and” of patterns, which syncthing does not do.

This makes absouletly no sense. Unless it’s talking about negated ignore patterns (! syntax), this is just wrong. It also doesn’t mention the negation syntax at all.

This may have been a markdown formatting issue, but IIRC this literally ignores a file named .. .* is what I would consider correct here.

This is okay, though

*.mp3
*.wav
*.flac

is simpler and does the same thing.

Using AI is fine, though please always fact check what it spits out and refer to official documentation for reliable information.

4 Likes

Thank you very much for your feedback and corrections. I have updated my notes. I try and document everything I learn about Syncthing in another program called Evernote.

I know that the aI chat bots are far from perfect. It gives me ideas and places to go and finalize the solution that I’m looking for so I use it as an aid not as a rule.

I find chat GPT very helpful for laying out Linux commands for me. They are always accurate. I once asked it how to make a program auto start at boot time and it gave me line by line instructions that worked perfectly so sometimes in the right situation it’s a good thing,

I usually compare the output from chat GPT to BingAI and Google Bard. between the three of them I usually get the answer I need.

@Nummer378 has already pointed out a few problems in ChatGPT’s answer. Here’s another one:

Ignore patterns do not use regular expressions. At least not what is usually considered “regular expressions”. Ignore patterns work more like globbing otherwise known from shell programming. So the AI is mixing things up.


I would suggest you checkout the official documentation which is much more reliable than some possibly (in this case definitely) mixed-up output from an AI.

If you use AI for programming (and making up an ignore pattern kind of is) you have to assume the answer is wrong.

3 Likes

My two cents: A proper selective sync feature would not be based on an ignore list. That would be a selective ignore feature.

A folder configured as selective sync would have all files and folders ignored by default. Ignored files would be replaced by a file with an extension that is unique to and opens to sync thing and retrieves the file.

Files retrieved are added to the synced files list. Right click to remove from this or all devices. There should be really no other UI needed as a good implementation of this feature would simply use the existing OS.

I agree that this would be an extremely helpful feature for Syncthing to get. As a Resilio user for the past 5 years or more, this is really the only obvious thing missing in Syncthing, but it’s an important thing. I like Sam G’s point about how this would work, but I’m not sufficiently knowledgeable to offer a pertinent opinion about the best means of implementation.

Agreed:

It’s simply my opinion that the ignore patterns being like shell programming or a descendant of shell programming makes it difficult for people that are not programmers.

Syncthing can easily, I think implement something a bit easier for the average user. The docs can be intimidating sometimes.

I have a similar program that’s Windows only that lets you right click on your selection of folders and add an exclusion or an inclusion filter just with the click and then it’s easy to see the syntax so you can learn more about how to do it yourself.

What about that ? lib/ignore Have a nice treeview for handling ignores - Next Gen Ignores by ibizaman · Pull Request #5132 · syncthing/syncthing · GitHub Sadly the author abandoned it(?)

I think the main problem is that there is really NO interface to the operating system GUI. The only interface is the Web browser. So if Syncthing creates empty/dummy files that “open in syncthing” for them to be on-demand downloaded, there is really no program for them to open in.

For me this is still a really important feature. I think the ignore list capability means that a good bit of the underlying code may be ready for this kind of feature. But the fundamental problem of a lack of integration with the GUI of the OS means it really can only be managed from the web interface. I would propose the following: (Of course, I have no idea how to do it…)

  • A folder can be identified as a selective sync folder. It should be assured at least when identifying a folder as selective sync that there is at least one device in the cluster that does not identify it as selective. That device would always have all the files in the folder.
  • Empty/Dummy files are NOT created in the selective sync folder. There’s no application to open them in. So there’s no point in them being there.
  • In the web browser on the device with the selective sync folder, the user would be able to view the folder’s directory tree and “enable” sync for files or subfolders.
  • When files or subfolders are selected for syncing, they are transferred from the pool and placed appropriately in the directory tree in the filesystem.
  • Files can be edited and would be synced back to the pool.
  • When someone is “done” with the local copy, has made any changes they desire, and ready to clean up, they would simply go back into the web UI, and “deselect” the subdirectory or files. Syncthing would then verify those files are in fact stored elsewhere in the pool, preferably on at least one device that is not selectively syncing, and then Syncthing would delete the local copy, and NOT leave the copy for the user to delete manually. This is REALLY important, because it would be too easy for a user to unsync files, and then delete other files they’re still syncing by mistake… And then, of course, they’re gone from the whole pool.
  • There would have to be a substantial amount of testing particularly if individual file syncing was permitted… I.e. if new files are created, those are also presumably selectively synced (not ignored). An alternative is to say selective sync only works on subfolders. And it’s all or nothing (the whole subfolder or not).

Additionally I would advocate this selective sync, sync list be SEPARATE from the existing ignore list. Even if it’s possible the ignore list capability is the underlying capability used to implement, the selective sync list should be completely managed from the WebUI, and the ignore list sort-of combined with the selective sync list which is perhaps hidden (Don’t need to show the user how the sausage is made.)

The benefit of this kind of solution is it doesn’t require any change to the architecture. There is no need to create “helper” applications for each OS. It’s all managed through the WebUI.

Anyway, just my thoughts after using Resilio for years and switching to Syncthing for some of the reasons mentioned here.

Let me know if you guys think this makes sense…

1 Like

I think it’s fairly clear what needs to be done here, and I think all of the maintainers agree on that.

The problem is that nobody is interested in spending time on this, and all previous efforts by external contributors have stalled.

If I knew how to code, I’d offer to assist with the additional feature because I like the idea.

One day we’ll get there.

Why not have #include file in the folder then edit it from the computer?

Because if you’re not at home, and you want to retrieve something on your phone, you can’t edit the file on the computer.

1 Like

When I really can’t do it directly on the local device I do it remotely (ssh or Remote desktop over ssh). Whatever I admit tweaking an external text file is boring, even it’s local, and that’s what I do when I need symmetric ignore or remote ignore management (via #include files). For local only management (e.g. huge remote Pictures Folder from which I just want to locally work on files from a subtree) I do with ignore patterns directly in the local GUI, what I admit is just a little less boring than with external file… ok, sorry for the noise.

Not noise. All part of the discussion.

I have a similar use case. A 30TB NAS with a ton of stuff and I’d like to selectively sync stuff on a laptop and a phone.

Currently only a small subset is under syncthing management because I can’t selectively sync. (Who has 30TB of storage on a laptop?)

Anyway im not at all familiar with the syncthing codebase but considering taking a look at what it would take to be able to implement something here.

1 Like

Hello, I switched to Resilio some time ago due to this feature missing, and could be interested in contributing some code to bring this to Syncthing. But as I’m not very involved in the community, could you (or another maintainer) point me at previous (stalled) attempts at designing or implementing things in this space? So I don’t duplicate any work, and have something to use for inspiration. Thanks!

3 Likes

From a quick GitHub search, there might be more:

Could also find some PR’s linked here:

1 Like

I’m thinking that a tree view is not the only piece of the puzzle, something needs to be done about like renaming or creating folders at the root. If you do this with a ‘*’ ignore at the end, it’s silently not synced… The correct (to me) behavior seems to not really be achievable with the current ignore feature. If a new folder is created on another node, don’t sync it to me. If I create a new folder, start syncing it.

I think ignores are smart enough to exclude things along the way of the item being excluded.

I’m contemplating of adding such a feature to Syncthing Tray. So far I added a UI for the browse-API. It would be nice if one could select files/directories from there and ignore/unignore them easily. An “ignore and remove locally” feature might also be useful. However, I’ll have too see how far I can come with ignore patterns within the time I can spend on it.

Of course it would also be more beneficial for everyone if such a feature was implemented in Syncthing’s official GUI but that’s probably harder to do.

(I’m also contemplating of porting Syncthing Tray to Android where this feature would be most beneficial to have because one the phone editing ignore patterns is most inconvenient. However, the Android/App port is probably a lot of work - although probably also a quite interesting piece of work.)

1 Like

I’m kind of curious the percentage of Syncthing users that use the tray UIs…