date/age ignore pattern

Hi Syncthing gurus! Thanks for making such a great product!

As a power user who frequently uses an /Archive* subfolder in my ignore patterns to store older files/folder that I wish not to sync to the clients any longer.

Would it be possible to have a date-based ignore pattern? This could be defined similar to a -mtime or -atime or -ctime in a unix find command. It could be a tag on a specific sub-folder/path.

A global filter for all members in the folder:

* -mtime +30d

For a specific sub-folder:

Videos/* -ctime +1h

Forgive me if has already been posted/shared.

Cheers,

-mt

What is the purpose of the ignore pattern? Assuming we implemented what you suggest, and things became ignored after a certain amount of time, nothing would change. The files have already been synced, ignoring them at this point has no effect. Any change to them, which could potentially be ignored, would update the mtime and thus unignore them again.

1 Like

The purpose generally would be operate like a cache where “files you are actively using, recently created or frequently changing” would be sync’d to the client. And, thus, older files you don’t really use/touch much could be left out of the synchronization. This would help manage space on the client side - where you only need recent files.

My current workaround: set an “Archive/*” ignore pattern…and then I manually copy old files/folders into that sub-folder when I don’t want to sync them to the clients.

Other ideas I can imagine a few things:

  • on a new shared folder, only files that have been “recently updated” per the time specification. Like, only sync files/folders modified in the last N days.

  • on an existing shared folder, files that are beyond the time specification would be removed from the remote client and not synchronized in the future

No other file sync tool does that, so even tho it sounds cool, I feel this is opening a can of worms where we will soon have rules that sync files based on the moon phase.

Thing is, what files should be removed on the remote client looks more like a remote client decision from where I’m looking. The typical “sync my pictures from my phone to the server, let me remove pictures on my phone” flow, which is similar, is something like

  • Set up the source and destination devices as usual
  • Set the “ignore deletes” option on the destination

New files from the source get synced to the destination. Files deleted on the source remain on the destination.

In your case you may want to set the “ignore deletes” option on the other side, and have a periodic script to remove old files on the destination, or something like that.

If you ignore it based on its age it wouldn’t be removed from the remote client. It would stay on the remote clients and the original node would ignore changes made to the file by any other clients.

Also, IMHO, the number of forum requests from someone who can’t get a file to sync because they moved it from somewhere and the mtime is within the ignore window would be crazy.

1 Like

Thanks for your great tools !, i re-launch this post, i think that the feature can have add-value in several use cases. Today i use it to synchronize some big storage between several locations, and i need several times to add or remove some servers with syncthing. When i add a new one, i wish to ignore old files because they are too heavy to be transfered, and i don’t need it anymore on the new server because i keep the files only for 30 days, after they are trashed, i search to have only the new files available on the new server. do you have idea how to do that ? the Date/age filter should be interesting

You cannot do that, and I am not aware of any other file sync solution supporting this.

For example rclone supports an option --max-age. e.g. Don’t transfer any file older than 24h: rclone copy --max-age 24h /path/to/src remote:

I find this feature useful when backing up directory with some of the files constantly changing using e.g. --min-age 5min.

In case of Syncthing I would use it to sync only latest of my photos from my phone, Because otherwise full the size of the photos is too large…

Sure. But rclone claims to be backup software. Syncthing claims to be sync software, i.e, make both sides look “the same”. I don’t think this type of feature is part of OneDrive or Google Drive etc, which is more of a target audience for syncthing.