Such a feature is known in different synchronization applications under quite different terms: Selective Sync / On demand sync / Smart Sync / Placeholder file sync.
The discussion around this feature came up often over the last years and maybe would benefit from bundling this information together a bit. The use cases can vary a bit in the details, but the general structure is almost identical.
Here I give one case from real-life example:
Given is a directory structure such as
/projects/project_a, [7GB] /projects/project_b [11GB] /projects/project_c [45GB] /projects/project_d [3GB] […]
With each containing a sub-hierarchy such as /projects/project_a/data/raw/component1/image_0002.jpg to …/component7/image_0768.jpg /projects/project_a/data/processed /projects/project_a/summary/metadata.xlsx
This directory structure is shared among multiple colleagues both on-premise and on the road. At some point this grows beyond the available disk space. Here currently syncthing allows with the ignore files to exclude certain folders to reclaim the disk space, other sync applications allow using such a “selective sync” feature to free the disk space as well. In the meantime my colleagues might change the directory structure slightly, e.g. moving images from one component folder to another or similar.
Now I receive a request from a colleague to “Please check and update the metadata of project_c by looking into the last images”.
Here using “on-demand sync” I go into the folder, open the metadata file and start my way up from the last image opening them as necessary, this requires only a few hundred MB of disk space and transfer quota.
As far as I can see this is currently in syncthing very difficult to achieve using manual manipulation of ignore files without downloading essentially complete projects to access (read and write) a few files. I don’t know the specific file or folder names but I can navigate there quickly. It also improves collaboration as I can see if someone added files/moved/deleted files without having to sync the full contents.
Such a use case clearly requires a sync application and is not covered by a web server as it is essential to have a two-way sync between the colleagues on the road and the office. On the road both disk space and available bandwidth can be much more restricted so other syncthing features such as a differential sync are very handy.
To implement such a feature a relatively deep integration into the specific platform is required. A few years ago this would have been very challenging. Nowadays, under names such as “cloud files API”, “Finder Sync extension” there is platform support (at least on Windows, Cloud Sync Engines - Win32 apps | Microsoft Docs, and MacOSX, macOS File Provider API for Selective Sync? - #3 by calmh). Under Linux the FUSE-approach is a potential approach as neither GNOME, KDE or other desktop environments contain a similar API.
Given that there seems to be quite a lot of interest (approx. 50 issues on the bug tracker, >50 forum threads) and even a teaser by Jakob from a few years ago (Dropbox Infinite - #9 by calmh) it would be fantastic to get an update on the thoughts from a developer perspective, especially with a focus on what the humble users can do to remove potential roadblocks.