Major syslog flooding since v0.14.0

I have had a rather unfortunate experience with v0.14.0 on Debian 8.

I was getting around 1000 messages/second like

Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller: shortcut: chmod /home/david/web/dwikihhg/working/configure/backup/GetAWebPlugin-backup-20160614-132658.tgz: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller (folder "Dwikihhg", file "pub/System/MimeIconPlugin/oxygen/32x32/audio-x-generic.png"): shortcut: chmod: chmod /home/david/web/dwikihhg/pub/System/MimeIconPlugin/oxygen/32x32/audio-x-generic.png: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller: shortcut: chmod /home/david/web/dwikihhg/pub/System/MimeIconPlugin/oxygen/32x32/audio-x-generic.png: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller (folder "Dwikihhg", file "pub/System/FamFamFamFlagIcons/cy.png"): shortcut: chmod: chmod /home/david/web/dwikihhg/pub/System/FamFamFamFlagIcons/cy.png: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller: shortcut: chmod /home/david/web/dwikihhg/pub/System/FamFamFamFlagIcons/cy.png: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller (folder "Dwikihhg", file "pub/System/DocumentGraphics/tablesortup.gif"): shortcut: chmod: chmod /home/david/web/dwikihhg/pub/System/DocumentGraphics/tablesortup.gif: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller: shortcut: chmod /home/david/web/dwikihhg/pub/System/DocumentGraphics/tablesortup.gif: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller (folder "Dwikihhg", file "pub/System/OriginalDocumentGraphics/award.gif"): shortcut: chmod: chmod /home/david/web/dwikihhg/pub/System/OriginalDocumentGraphics/award.gif: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller: shortcut: chmod /home/david/web/dwikihhg/pub/System/OriginalDocumentGraphics/award.gif: operation not permitted
Jul 20 13:21:03 monocerus nice[22984]: [S4TIK] INFO: Puller (folder "Dwikihhg", file "pub/System/PatternSkinTheme2009/style.css.gz"): shortcut: chmod: chmod /home/david/web/dwikihhg/pub/System/PatternSkinTheme2009/style.css.gz: operation not permitted

While System D has some control over flooding the journal, syslog grew like Topsy and I had to increase the size of a partition and perform several logrotates.

After setting ignore permissions for the folder, all settled down.

This folder wasn’t being shared with Windows, so ignore permissions wasn’t set.

Perhaps I am running syncthing in a strange way - I had understood that it was a way that was catered for.

I run syncthing under its own user, st. User david owns the files under group std with members st and david. I’ve used chmod g=u where necessary.

So, has a new requirement that the st user must now own the files, or is there something I need to do with the group permissions?

I’m surprised nobody else seems to have hit this problem.

edit correct version

I have a similar Problem here.
I’ve upgraded about 5 Machines today and got a lot of Sync Problems related to “operation not permitted” :frowning:

This was always a requirement when you want to synchronize permissions, because that’s how chmod works. Only the owner can change file permissions. Maybe you can work around this with capabilites, though.

       CAP_FOWNER
              * Bypass permission checks on operations that normally require the filesystem UID of the process to match the UID of the file (e.g., chmod(2), utime(2)), exclud‐
                ing those operations covered by CAP_DAC_OVERRIDE and CAP_DAC_READ_SEARCH;
              * set extended file attributes (see chattr(1)) on arbitrary files;
              * set Access Control Lists (ACLs) on arbitrary files;
              * ignore directory sticky bit on file deletion;
              * specify O_NOATIME for arbitrary files in open(2) and fcntl(2).

Everything else was probably already discussed in

including possible solutions

I’m guessing that in the original setup the files “originated” on the more locked down side, so there was never any reason for Syncthing to attempt to change the permissions there.

In the upgrade, since the index is recreated, Syncthing reconciles any differences detected, and probably does a chmod with the existing permissions even when they haven’t changed - not sure, would need to double check the code. That then triggers the error.

So yes, it should always have had “ignore permissions” set, but I can see that it might have sort of worked before.

1 Like

@calmh, do you think syncthing needs a mechanism to reduce log flooding?

1000/sec seems excessive.

Not particularly, but I’m open to suggestions.

OK, I see that I can reduce the impact of flooding by running logrotate more than once per day and using maxsize.

This is clearly more general and st shouldn’t need to do this.

This is fine for current users of syncthing. However, I don’t think it is acceptable for st 1.0.0 to produce so many log messages for what is, essentially one error.

All I can think to do, as a backstop, is to count messages, and if the count in a period of time (1 minute?) exceeds a threshold (6000?) more than n(3?) times in another period (1 day?) to die with a suitably unhelpful message like Stuff is wrong, see the log.

However, I don’t have any suggestions how to fix the underlying problem - that one configuration error can cause a flood of error messages, possibly affecting the stability of the system.

That would require forethought, design of multiple detection mechanisms without affecting performance too much etc. This is far too difficult for me… And it’s peripheral to the main function of st.

Perhaps all that can be done, apart from a backstop, is to compile a prominent list of gotchas in the documentation.

Thoughts?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.