cosas
July 10, 2023, 5:19pm
61
When I really can’t do it directly on the local device I do it remotely (ssh or Remote desktop over ssh). Whatever I admit tweaking an external text file is boring, even it’s local, and that’s what I do when I need symmetric ignore or remote ignore management (via #include files). For local only management (e.g. huge remote Pictures Folder from which I just want to locally work on files from a subtree) I do with ignore patterns directly in the local GUI, what I admit is just a little less boring than with external file… ok, sorry for the noise.
mraneri
July 10, 2023, 5:30pm
62
Not noise. All part of the discussion.
I have a similar use case. A 30TB NAS with a ton of stuff and I’d like to selectively sync stuff on a laptop and a phone.
Currently only a small subset is under syncthing management because I can’t selectively sync. (Who has 30TB of storage on a laptop?)
Anyway im not at all familiar with the syncthing codebase but considering taking a look at what it would take to be able to implement something here.
1 Like
tbodt
April 5, 2024, 6:52am
63
AudriusButkevicius:
I think it’s fairly clear what needs to be done here, and I think all of the maintainers agree on that.
The problem is that nobody is interested in spending time on this, and all previous efforts by external contributors have stalled.
Hello, I switched to Resilio some time ago due to this feature missing, and could be interested in contributing some code to bring this to Syncthing. But as I’m not very involved in the community, could you (or another maintainer) point me at previous (stalled) attempts at designing or implementing things in this space? So I don’t duplicate any work, and have something to use for inspiration. Thanks!
3 Likes
acolomb
April 5, 2024, 10:25pm
64
From a quick GitHub search, there might be more:
syncthing:main ← p0l0us:tree-selector-for-ignores
opened 02:08AM - 26 Sep 23 UTC
### Purpose
As a user, I wish to have an easy way to select which files from … other devices will be downloaded and managed in my syncthing folder.
The main problem with current ignores is, that a user doesn't know what files exists in the global tree without downloading all files to the device or checking other device(s). So I want to expose the global item tree in the folder ignore editor tab in a tree component, allowing the user to select files which should be downloaded (not ignored). The feature target is not complex ignores management - just simply choose what to download.
Tree selection overrides the ignore patterns by something like:
```
// Generated by tree selector
!/FolderToDownload
!/AnotherFolder/FileToDownload.bar
!/AnotherFolder/FileToDownload_2.bar
**
```
The tree component is not supposed to parse any other ignore patterns than those generated by itself. It is each user's responsibility to not screw format up when using the tree selector. If the first line "// Generated by tree selector" is missing or incorrect, all patterns will be replaced by "**" and the user can un-ignore items one by one.
```
// Generated by tree selector
**
```
A broader view of the feature will be clarified by the following PR draft:
https://github.com/syncthing/syncthing/pull/9130
### Testing
Describe what testing has been done, and how the reviewer can test the change
if new tests are not included.
### Screenshots
Not receive-only folders:
Receive-only folders:
Tree expanded:
### Documentation
No API changes.
syncthing:master ← ibizaman:poc_stignore_tree
opened 02:41AM - 18 Aug 18 UTC
### Purpose
This is PR attempts to address the requirements of issue #2491 by… following the rules outlined in [this comment](https://github.com/syncthing/syncthing/issues/2491#issuecomment-359889677).
This PR is in progress. I posted it [as requested](https://github.com/syncthing/syncthing/issues/2491#issuecomment-413694849). What's notably missing is advanced testing of part of it, see [this comment](https://github.com/syncthing/syncthing/issues/2491#issuecomment-413598000).
Please see commit messages for info on the implementation.
### Testing
Unfortunately testing needs to be manual.
What would be greatly needed to gain confidence in this PR is a way to unit test the new js functions. The main scary one is the function that parses the .stignore to populate the basic view. It doesn't work right now correctly. It is a pain to test manually though.
I'm less woried about the UI part - the update of the UI when a user clicks - as there is not that much going on on that front. I'm not saying it's flawless, just that it is possible to exhaustively test the behavior with manual testing.
### Screenshots
Clicking on the Ignore Patterns shows 2 tabs, the Basic one is selected by default:
<img width="888" alt="image" src="https://user-images.githubusercontent.com/1044950/44294960-c1f7df80-a254-11e8-95fa-cc203f3e66c7.png">
This shows the advanced tab:
<img width="888" alt="image" src="https://user-images.githubusercontent.com/1044950/44294963-d3d98280-a254-11e8-8adb-fd454a04c703.png">
Opening the folder asynchronously loads the contents by getting the global state from syncthing
<img width="888" alt="image" src="https://user-images.githubusercontent.com/1044950/44294972-02eff400-a255-11e8-9baf-39bfdaee39a5.png">
Selectively syncing a few files:
<img width="888" alt="image" src="https://user-images.githubusercontent.com/1044950/44294976-156a2d80-a255-11e8-9679-721d3f8737e8.png">
Corresponding advanced rules:
<img width="271" alt="image" src="https://user-images.githubusercontent.com/1044950/44294978-2155ef80-a255-11e8-812f-78cac52e95fa.png">
Another configuration:
<img width="193" alt="image" src="https://user-images.githubusercontent.com/1044950/44294981-2e72de80-a255-11e8-8a89-8829781b949e.png">
And corresponding advanced rules:
<img width="176" alt="image" src="https://user-images.githubusercontent.com/1044950/44294982-3763b000-a255-11e8-8efb-56ce5b6460e3.png">
Extreme case 1, sync all:
<img width="170" alt="image" src="https://user-images.githubusercontent.com/1044950/44294986-42b6db80-a255-11e8-9bc8-2df9a52f8652.png">
Corresponding pattern is empty:
<img width="232" alt="image" src="https://user-images.githubusercontent.com/1044950/44294993-49dde980-a255-11e8-8207-cebe67d65985.png">
Extreme case 2, ignore all:
<img width="251" alt="image" src="https://user-images.githubusercontent.com/1044950/44294997-57936f00-a255-11e8-87c4-e6ee97f731db.png">
Corresponding patterns:
<img width="225" alt="image" src="https://user-images.githubusercontent.com/1044950/44295004-64b05e00-a255-11e8-9672-6d7193c51418.png">
### Documentation
I guess we should discuss documentation later when the implementation is more solid? Although I'm totally fine to tackle this now if requested.
Disclaimer: first PR here, I don't want to waste anyone's time so feel free to just skim over it and tell me what could help you review this more easily.
syncthing:master ← ibizaman:ignore-patterns-viewer
opened 06:16PM - 24 Feb 20 UTC
### Prelude
This PR is a second stab at #2491, after the failure of #5132. Th… is PR is not ready to be merged yet but it should be close to providing a useful feature that can be merged even though it doesn't fulfill all requirements in the issue. Think of it as a way to slowly chip away at #2491. The requirements I tried to fulfill here are:
* Must be understandable by someone who understands the interface in use by DropBox, OneDrive, etc.
* Must be a simple tree view with checkboxes
* Root of tree (=folder root) checked by default
* Power user access to flexible, underlying patterns
Feel free to give your opinion on this, even if it means changing a bunch of stuff or simply refusing to merge. I don't mind at all as that would mean we're progressing on this issue by knowing what we _don't_ want.
### Purpose
This PR introduces a tree view in the edit folder modal in the ignores tab. The tree view shows the whole global directory, not only what's local to the device/folder. Each node of the tree has a checkbox that is checked unless that file or directory is ignored. The sync is one way: updating the patterns changing what checkbox is checked but changing a checkbox has no effect. Doing this I was able to keep the PR concise.
The corresponding folder state shows that indeed, some files are ignored:
When updating the ignored patterns and clicking Save, the modal is not closed but instead the nodes in the tree view are refreshed. The goal is to see relatively quickly the impact of changing the ignore rules. It's kind of like having a REPL loop. I modified the behavior of the Save button to _not_ close the modal, otherwise refreshing the tree doesn't even make sense. I don't really like modifying the Save button behavior but I don't know what would be better.
Here is for example what happens if I modify the pattern match `documents` above and click on save:
When clicking on a node, the bottom text is updated with information pertaining to that node. It's very crude and probably useless for now but I wanted to show this is possible.
`documents` is selected:
`pictures` is selected:
Changing the checkbox state on a node has no effect at all. This is most certainly bad UX and I'm happy to change that to something that makes more sense.
The tree is lazy loaded using fancytree's lazyLoad feature. I added a new api endpoint that merges the `db/browse` and `db/file` while only keeping a few fields in the returned json. I felt it made more sense to do that in go than in javascript. Note that #5930 talks about removing the `db/browse` api endpoint and this PR goes a bit against the claim that Syncthing is not a file browser.
### Testing
Manual. I set up two Syncthing instances and shared a folder between them:
```
STNOUPGRADE=true STGUIASSETS=~/src/syncthing/gui ./bin/syncthing -no-browser
STNOUPGRADE=true STGUIASSETS=~/src/syncthing/gui ./bin/syncthing -no-browser -home ~/.config/syncthing2 -gui-address "http://127.0.0.1:8385"
```
I then made some updates to the ignored patterns.
### Documentation
I was waiting a consensus on this first.
Could also find some PR’s linked here:
opened 01:59PM - 19 Nov 15 UTC
documentation
This issue replaces the ones that relate to fundamental issues with how our igno… res work. The following is the up to date list of requirements:
### Certain Requirements
- Must be understandable by someone who understands the interface in use by DropBox, OneDrive, etc.
- Must be a simple tree view with checkboxes
- Root of tree (=folder root) checked by default
- Can uncheck root and then check just the branches that we want synced
- Must be able to black list extensions
- Must live in config (not .stignore files)
- Must "live update". That is, if I ignore a file that was going to be pulled, we should not pull it. If I unignore a file that we are missing, it should be pulled shortly thereafter.
- Must be able to specify if ignored files should be preserved or removed on directory delete.
- Per what though? File extension? Directory?
- Power user access to flexible, underlying patterns
- Be able to to remove files for folders that are not picked, as it remove things that are now ignored.
- Have inclusion and exclusion mode, ignore everything, include X, versus include everything, ignore X.
### Maybe?
- Per directory file extension black lists ("never sync `*.tmp`")
- Per directory file extension white lists ("only sync `*.doc`")
- Should sync between devices
- Ability to ignore a directory by the presence of a special file in it (`cachedir.tag`).
Comments and discussion below. A project maintainer will keep the list above in sync as the requirements change or are clarified. Note that this is not a list of problems with the current system, but requirements for a _new_ system.
1 Like
tbodt
April 5, 2024, 10:48pm
65
I’m thinking that a tree view is not the only piece of the puzzle, something needs to be done about like renaming or creating folders at the root. If you do this with a ‘*’ ignore at the end, it’s silently not synced… The correct (to me) behavior seems to not really be achievable with the current ignore feature. If a new folder is created on another node, don’t sync it to me. If I create a new folder, start syncing it.
I think ignores are smart enough to exclude things along the way of the item being excluded.
I’m contemplating of adding such a feature to Syncthing Tray. So far I added a UI for the browse-API. It would be nice if one could select files/directories from there and ignore/unignore them easily. An “ignore and remove locally” feature might also be useful. However, I’ll have too see how far I can come with ignore patterns within the time I can spend on it.
Of course it would also be more beneficial for everyone if such a feature was implemented in Syncthing’s official GUI but that’s probably harder to do.
(I’m also contemplating of porting Syncthing Tray to Android where this feature would be most beneficial to have because one the phone editing ignore patterns is most inconvenient. However, the Android/App port is probably a lot of work - although probably also a quite interesting piece of work.)
1 Like
mraneri
April 10, 2024, 4:05pm
68
I’m kind of curious the percentage of Syncthing users that use the tray UIs…
I would like to have a documents / music files on a NAS and be able to sync (make available offline) some files when I go on vacation / travel.