Directories not ignored despite .stignore pattern "*"

Support Android
1

Android 7.1.2 LineageOS Syncthing 0.10.8

I created a folder for /storage/emulated/0 (internal storage root) along with these ignore pattens:

!/Pictures/**
*

The goal being to sync the /Pictures directory and everything inside it (and possibly other dirs in the future), but nothing else.

Problem: all other directories in the root are synced as well, albeit without their content. I expect my other devices to which I sync this folder, to end up having this for content:

Pictures/
  foo.jpg

But instead they have

Alarms/
Dowloads/
...
Pictures/
  foo.jpg
Stash/
...

Is * not supposed to ignore directories? I tried adding */ instead but that didn’t help.

I took a look at the log but there was nothing helpful in there. Do you want me to enable any of the debugging facilities and post it after trying again? I’m not sure which one(s) to enable.

2

It feels like you added the ignore patterns after you added the folder, and it managed to sync the directories prior the ignore patterns were there.

3

Hmm, could be. So you’re saying the patterns do seem correct though and it should work if I do it again but more right?

4

I just did the following (folder = syncthing folder, dir = filesystem directory):

  1. removed the folder everywhere
  2. created my .stignore file with the patterns above
  3. created a new folder in the app, where the patterns from the .stignore file already showed up under Ignore Patterns before I confirmed/created the folder
  4. let it scan

Again it included 24 directories (shown in the web gui in the app) that are supposed to be ignored. Their contents are correctly ignored, but the dirs themselves aren’t. So I suspect there’s still a pattern issue (but I don’t see what if the documentation is correct) or bug.

5

I just did it again, but with 2 differences:

  • I entered my ignore rules through the app instead of making the .stignore file with an editor (shouldn’t make a difference because I checked the rules in the app)
  • I added the rule !Pictures above !/Pictures/**

Now it works right and ignores the other folders. Strange situation.

Edit: I removed the added rule again, and it still works right. Seems creating the .stignore file externally is an issue. Gonna test that.

1 Like
6

Alright, so this is what I’ve figured out I was doing wrong:

  1. my patterns weren’t entirely right. I should have also included the folder itself with !/Pictures, not just its contents.
  2. order matters. It’s best to set ignore patterns when making a folder, not afterward, to avoid… difficult synchronization situations.

Point #3 is part my bad but also part a Syncthing bug:

  1. use external editor to create .stignore file
  2. this editor does something odd, I don’t know what, I suspect it uses a quirky charset like utf-16 or something
  3. use this file

The app’s Ignore Pattern editor and syncthing’s web gui correctly display the text I wrote with my external editor, but syncthing’s internals don’t parse it correctly and end up not ignoring the dirs specified in the file. The file is parsed incorrectly by the sync engine but correctly by the gui.

In addition, after using this bad .stignore file in syncthing, it causes a glitch in the app where the web gui says “config has changed, syncthing must restart”, after which the web gui can never be opened again until after a device reboot.

Bad .stignore file to download from my personal server (won’t be available forever):

https://leaumar.net/dump/stignorebad

TLDR: writing a .stignore file externally can create a file that seems to work alright in the app and web gui, but is misinterpreted by syncthing’s internals and kills the web gui.

I’d expect the app and web gui to misinterpet the file as well so it would at least be obvious to the user the .stignore file is bad.

Do I put this on github?

7

Link to correct, syncthing-made .stignore file for comparison with the “corrupt” file:

https://leaumar.net/dump/stignoregood

Immediately obvious is that the “corrupt” file is twice the size with less content, hence I suspect a charset issue.

9

I can’t check the file as I am on a phone, yet the text editor shows utf-8 for both files.

I am not sure this is ticket worthy, as if you create a dodgy file, bad things happen. You can already edit ignores prior folder creation atleast in the web UI (yet I think you are using android, so perhaps an enhancment for android UI).

Setting ignores etc should not need a restart, but I suspect it’s the android app misreporting and not the web UI actually asking for a restart.

If syncthing doesn’t start up, the logs would be interesting as there might be something there bug worthy.

10

Yeah, the file being bad isn’t bug worthy, but the UI and internal engine interpreting the same (bad) file differently is a bug IMO. If the internals interpret the file one way, the GUI should do it the same way. I won’t be the last person using a text editor to write into exposed config files. Editing the plainly visible .stignore files isn’t any different from editing any other conf file like linux people are used to doing. It’s not like I edited a file in a typical no-go location like syncthing’s internal files. The .stignore files should be rigorously checked by both the app and syncthing since they’re user-exposed.

The thing is also, from my POV, I didn’t create a dodgy file. I created a seemingly valid .stignore file with a text editor, and for some mysterious reason syncthing can’t read it right. It’s not syncthing’s fault at all, but syncthing should guard against such things.

I’ll see if I can get a log. Are there any debug facilities that definitely will or won’t be able to tell the problem or should I just enable them all?

11

You can already edit ignores prior folder creation atleast in the web UI

in the android app as well

I suspect it’s the android app misreporting and not the web UI

No, it is the web ui asking it, after adding the folder.

12

Yeah if ot crashed and was not able to load ignores, I’d expect stuff in the logs without any facilities. You can try enabling model, config and ignores if nothing comes up.

13

Hi, I’ve struggle a while before getting nearly what I wanted. ST 'use of exclude file mixes files and folder names and that’s difficult to handle. Here is my file for inspiration/understanding (hope it helps; and don’t use an external editor, it doesn’t work anymore unless restarting/reloading I don’t remember):

// Allow specific root file BookList20180102.TxT 
(?i)!BookList2018????.txt

(?i)!/Tech/**/*.pdf
(?i)/Tech/**/*
(?i)!/Tech/**
// Attention : all root files available
(?i)!/Tech/*.pdf
(?i)/Tech/*
(?i)!/Tech

//(?i)!/English/user/folder1/*.pdf
//(?i)/English/user/folder1/*
//(?i)!/English/user/folder1
//(?i)!/English/user/folder0
//(?i)/English/user/*
//(?i)!/English/user
(?i)!/English/*.pdf
(?i)/English/*
(?i)!/English
(?i)!/French/*.pdf
(?i)/French/*
(?i)!/French
(?i)!/Dutch/*.pdf
(?i)/Dutch/*
(?i)!/Dutch
(?i)!/Deutsch/*.pdf
(?i)/Deutsch/*
(?i)!/Deutsch
(?i)!/Other/*.pdf
(?i)/Other/*
(?i)!/Other
(?i)!/Serial/*.pdf
(?i)/Serial/*
(?i)!/Serial
(?i)!*.pdf
*
14

I’m not sure how this is relevant to the thread?

15

So I suspect there’s still a pattern issue (but I don’t see what if the documentation is correct) or bug.

I’m sorry then. I was refering to your post in this thread and your title

Directories not ignored despite .stignore pattern “*”

. It shows how to exclude everything but some kind of file (.pdf) and the directories concerned. For each directory, before excluding everything else (last *), you have to refuse from exclude the directory and type of file, then allow the directory itself with it’s content and finally exclude the directory itself. A little tricky because excluding a string acts at the same time on the file names and folder names.

16

Yeah, I figured that out by the end, but that’s not really the issue anymore… The problem now is the UI and internals interpreting the same file differently.

17

Yes, sorry for that. But if it’s solved, let us know if your stignore works. Because I think there is still a mistake in your stignoregood file :

!/Pictures
!/Pictures/**
*

order matters, and you exclude a directory just to exclude right after it’s content; I think ST won’t be able to read or exclude something in a previous already excluded directory, even if it has to be reverted by the last *.

18

Hmm, could be. I need to test that.

19

You should still provide logs, if the logs point out something funny, we should open an issue on github

20

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.