How to sync folder structure but not files

Support
1

I think I must be missing something here. I have a directory containing lots of sub-directories with all sorts of files scattered thru it. I want to sync any markdown documents plus the whole directory structure (irrespective of whether a folder contains a markdown document or not). I feel like the following ignore patterns should accomplish this, but I can’t quite get it to work.

(?i)!*.md // include all markdown docs - works correctly
!/**/ // include all folders but not files
* // exclude all other files

I’ve played with the order and tried adding /**/*, but either I manage to sync all files, or only markdown files and the folders that contain them. But what I want is all folders but only markdown files.

Just to make it clear, the following is the structure on computer A:

/
  foo/
    bar.md
    baz.xlsx
  foo1/
    bar1.docx
    baz1.html

and what I want synced to computer B is:

/
  foo/
    bar.md
  foo1/

Am I just missing something really obvious, or is this just not possible?

2
  • ** says “match any file or directory separator”.
  • A pattern ending in a forward-slash (/) says “the contents of a directory”.

So !/**/ translates to, “Starting from the root, include any directory’s contents”. Thus if a directory has no matching files, it’s ignored.

Another way to think of it is that a directory is only created in the process of syncing a matching file when the ignore pattern feature is enabled.

There’s no token that matches an empty directory so the only method that comes to mind at the moment is to exclude all files you don’t want, leaving just the Markdown files and directory names like this…

(?i)*.docx
(?i)*.xlsx
(?i)*.html

… or more compactly:

(?i)*.{docx,xlsx,html}
3

Thanks for that.

It makes it a bit difficult if the file extensions aren’t known in advance.

From the small amount of exploration I have done so far of syncthing, it looks like it is an amazing piece of software, that is somewhat let down by the ignore parameters.

4

Assuming all your files do have an extension (and the folders don’t use . in their names), the following should do it.

!(?i)*.md
*.*
5

Unfortunately my folder names often have nn.nn in them where n is [0-9]. I’ve mucked around with various permutations on the following:

(?i)!*.md
**.*
!**{0,1,2,3,4,5,6,7,8,9}.{0,1,2,3,4,5,6,7,8,9}*
**{0,1,2,3,4,5,6,7,8,9}.{0,1,2,3,4,5,6,7,8,9}**.*
**

But again, I can only get it to either sync all folders and files, or only files and folders that contain a .md file.

6

Other sync software take the opposite approach with pattern matching but it opens a whole different can of worms.

Like Syncthing, Rsync and Duplicacy (which uses Rsync’s pattern matching syntax) include everything by default (including empty directories).

But for the filter, Rsync and Duplicacy require explicitly defining what to include and exclude rather than simply ask just what to exclude like Syncthing does.

For example, here’s my Duplicacy filter for backing up only my Firefox profile without the Firefox binary plus other files/directories bundled in the Snap package:

+snap/firefox/common/.mozilla/*
+snap/firefox/common/
+snap/firefox/
+snap/
-snap/*

The ordering of the rules is important. Like Syncthing, the first match causes all following patterns to be ignored for the file/directory being inspected.

So the filter above breaks down to:

  • Line 1: First, include the contents of snap/firefox/common/.mozilla/ otherwise it will be excluded by line 5.
  • Line 2: Include the contents of snap/firefox/common/ because otherwise the .mozilla/ directory will be excluded by line 5.
  • Line 3: Likewise, include the contents of snap/firefox/ because otherwise the common/ directory will be excluded by line 5.
  • Line 4: Ditto, the snap/ directory is required otherwise the firefox/ directory will be excluded by line 5.
  • Line 5: Finally, exclude everything under the snap/ directory that hasn’t already been explicitly included.

My Duplicacy filter currently contains 59 patterns. I decided to let some small files be included rather than maintain a more complicated filter, but I’ve seen users posting that their filters are much more elaborate.

7

Please keep in mind that ignore patterns are processed in the order they are written. In other words, you can’t use exclude after ignoring everything.

The patterns above though look overly complicated and can possibly be simplified to the following.

(?i)!*.md
!*[0-9].[0-9]*
*.*
8

Thanks @gadget and @tomasz86 for your help.

For anyone who stumbles across this thread. The following is now working for me:

(?i)!**.{md,qmd,rmd,txt}
(?i)*.[a-z]*
(?i)*.{12d}*

This maintains the directory structure (except for hidden folders starting with .) while only syncing markdown files.

  • This doesn’t work for files without extensions
  • Files with an extension that doesn’t start with a-z need to be listed separately (these are relatively rare)
1 Like
9

Great that you’ve managed to do it! :slightly_smiling_face:

Just a minor nitpick, you don’t need to use ** in the first pattern, as it’s exactly the same as just * in this case. ** itself is useful when matching something inside all subfolders of a particular path, e.g. folder/**/*.md, etc.

1 Like
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.