V0.14 previously deleted files re-appear → ignore permission bits

Hi…

…as soon as I updated my 3 machines to syncthing 0.14, previously deleted files get back in my sync’d folders.

Well…

…after restarting all instances of syncthing it seems to get back to normal.

Previously deleted files finally appear in only one folder. And only files, previously deleted folders appeared but were empty.

Were the device completely synced before the upgrades? This should only happen if the “deleted” files still exist on one of the devices. The upgrade triggered a rebuild of the index database on all devices and since newer versions cannot speak to old versions, the sync history is lost. All files that exist on a device are therefore considered as “new” files.

As far as I understood, syncthing 0.14 will be able to speak with newer syncthing versions (0.15, …). As long as you do not upgrade all your devices at the same time, the sync history should be preserved in future.

I think I understand your explanation xduugu. And yes the 3 machines were not at the same level of synching before update to 0.14. One machine is booted less often than the 2 others which are almost always online.

So… I still have problems.

Machine 1 - online 24/7 says

    [X4DNQ] 00:33:51 INFO: Puller (folder "PALACE-Bureau", file "Art & Essai et animations/A&E 2012-2013/dates~"): shortcut: chmod: chmod /home/bureau/Art & Essai et animations/A&E 2012-2013/dates~: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller: shortcut: chmod /home/bureau/Art & Essai et animations/A&E 2012-2013/dates~: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller (folder "PALACE-Bureau", file "Art & Essai et animations/convention Palace-P Ecran2013.pdf"): shortcut: chmod: chmod /home/bureau/Art & Essai et animations/convention Palace-P Ecran2013.pdf: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller: shortcut: chmod /home/bureau/Art & Essai et animations/convention Palace-P Ecran2013.pdf: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller (folder "PALACE-Bureau", file "Art & Essai et animations/convention palace-p ecran.tif"): shortcut: chmod: chmod /home/bureau/Art & Essai et animations/convention palace-p ecran.tif: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller: shortcut: chmod /home/bureau/Art & Essai et animations/convention palace-p ecran.tif: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller (folder "PALACE-Bureau", file "Art & Essai et animations/A&E 2012-2013/convention-1.sxw"): shortcut: chmod: chmod /home/bureau/Art & Essai et animations/A&E 2012-2013/convention-1.sxw: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller: shortcut: chmod /home/bureau/Art & Essai et animations/A&E 2012-2013/convention-1.sxw: operation not permitted
    [X4DNQ] 00:33:51 INFO: Puller (folder "PALACE-Bureau", file "Banque/Remises de chèques/20131026.xls"): shortcut: chmod: chmod /home/bureau/Banque/Remises de chèques/20131026.xls: operation not permitted

Machine 2 - online 12/7 - says

[2OSOB] 01:30:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:30:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: reading length: read tcp 192.168.1.103:46711->37.187.105.88:22067: use of closed network connection
    [2OSOB] 01:30:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:30:02 INFO: Established secure connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH at 192.168.1.103:44492-31.38.172.103:56563 (TCP (Client))
    [2OSOB] 01:30:02 INFO: Device BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH client is "syncthing v0.14.0" named "Asgard"
    [2OSOB] 01:30:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: reading length: unexpected EOF
    [2OSOB] 01:30:49 INFO: Established secure connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH at 192.168.1.103:46815-37.187.105.88:22067 (Relay (Server))
    [2OSOB] 01:30:49 INFO: Device BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH client is "syncthing v0.14.0" named "Asgard"
    [2OSOB] 01:31:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:31:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: reading length: read tcp 192.168.1.103:46815->37.187.105.88:22067: use of closed network connection
    [2OSOB] 01:31:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:31:02 INFO: Established secure connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH at 192.168.1.103:44552-31.38.172.103:56563 (TCP (Client))
    [2OSOB] 01:31:02 INFO: Device BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH client is "syncthing v0.14.0" named "Asgard"
    [2OSOB] 01:31:02 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: reading length: EOF
    [2OSOB] 01:32:13 INFO: Established secure connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH at 192.168.1.103:45189-163.172.132.71:990 (Relay (Client))
    [2OSOB] 01:32:13 INFO: Device BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH client is "syncthing v0.14.0" named "Asgard"
    [2OSOB] 01:32:36 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:32:36 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: reading length: read tcp 192.168.1.103:45189->163.172.132.71:990: use of closed network connection
    [2OSOB] 01:32:36 INFO: Connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH closed: switching connections
    [2OSOB] 01:32:36 INFO: Established secure connection to BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH at 192.168.1.103:44644-31.38.172.103:56563 (TCP (Client))
    [2OSOB] 01:32:36 INFO: Device BGZRH5B-2X742Z7-FJNP2Y3-OMY66SS-AEKNIJA-US6H4H3-KUYRLUT-IOQ2VAH client is "syncthing v0.14.0" named "Asgard"

Machine 3 - online some days/7 - says

[BGZRH] 01:30:02 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: reading length: EOF
    [BGZRH] 01:30:02 INFO: Established secure connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR at 192.168.1.100:22000-80.11.36.218:44492 (TCP (Server))
    [BGZRH] 01:30:02 INFO: Device 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR client is "syncthing v0.14.0" named "ANTEC"
    [BGZRH] 01:30:02 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: writing message: write tcp 192.168.1.100:22000->80.11.36.218:44492: use of closed network connection
    [BGZRH] 01:30:49 INFO: Established secure connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR at 192.168.1.100:47440-37.187.105.88:22067 (Relay (Client))
    [BGZRH] 01:30:49 INFO: Device 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR client is "syncthing v0.14.0" named "ANTEC"
    [BGZRH] 01:31:02 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: switching connections
    [BGZRH] 01:31:02 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: reading length: EOF
    [BGZRH] 01:31:02 INFO: Established secure connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR at 192.168.1.100:22000-80.11.36.218:44552 (TCP (Server))
    [BGZRH] 01:31:02 INFO: Device 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR client is "syncthing v0.14.0" named "ANTEC"
    [BGZRH] 01:31:02 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: reading length: read tcp 192.168.1.100:22000->80.11.36.218:44552: use of closed network connection
    [BGZRH] 01:32:13 INFO: Established secure connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR at 192.168.1.100:58572-163.172.132.71:990 (Relay (Server))
    [BGZRH] 01:32:13 INFO: Device 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR client is "syncthing v0.14.0" named "ANTEC"
    [BGZRH] 01:32:37 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: reading length: EOF
    [BGZRH] 01:32:37 INFO: Connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR closed: switching connections
    [BGZRH] 01:32:37 INFO: Established secure connection to 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR at 192.168.1.100:22000-80.11.36.218:44644 (TCP (Server))
    [BGZRH] 01:32:37 INFO: Device 2OSOBTO-O4CNFU7-XLOFXDD-Y5SEGDV-MH37V7O-53VTZE7-PA6F6H2-M54OLAR client is "syncthing v0.14.0" named "ANTEC"

I don’t think I am losing files - but I’m afraid sync’ing is blocked. How do I fix that ? Should I “empty” sync’d folders on machine 2 & 3 hoping files from machine 1 will be propagated after ?


Edit: formatted code. //calmh

Your logs are quite hard to read. Could you format them as code blocks?

For the first machine, it looks like the user that runs syncthing is not the same that owns the files.

Well… I did not change anything about owners and groups in files. That worked fine until update. Indeed there is many users. One runs Syncthing, and is part of the group of users where others are, with r+w rights for users of that group.

( and yes there is samba running near )

Sorry about formatting, I’ve yet used the </> button.

Other thing : I also use syncthing-gtk on the 3 machines (ubuntu 14.04)

The permissions of the files were changed on one of the non-samba devices, and cannot be written back because the files are not owned by the user that runs syncthing. I don’t think this is related to the upgrade.

To get rid of this info message, you have to make sure that all files and folders in the synced directory are owned by the syncthing user (by either regularly chown’ing in a cron job or using samba’s force user configuration option) or disable permission synchronization (in the folder’s advanced settings Ignore Permissions).

For machines 2 and 3: Maybe you can find out why their connections are not stable by starting syncthing with STTRACE=connections.

I already make use of force user in samba directories.

The user that runs syncthing has write-right on all directories, at system level AND in samba conf.

I can’t see trace of changing permissions on files, and can’t remind why I would have changed them. It’s been almost a year it works smoothly…

With time, connection is stable between machines 2 and 3, only machine 1 appears not connected. Too bad for me, it’s the always online « server » :wink:

Maybe syncthing-gtk related ?

Ok, so I quit syncthing-gtk on the 3 machines. And run syncthing (only) from terminal and…

…after some minutes everything looks fine, all usual folders correctly updated and sync’ed.

Sorry for the noise…

…some hours later :-/ on machine 1 only I have thousands of

[X4DNQ] 19:07:59 INFO: Puller (folder "PALACE-Bureau", file "Hebdo-G/Aff A3 horaires/aff_A3_2012/Aff_A3_horaires_20120411.ods"): shortcut: chmod: chmod /home/bureau/Hebdo-G/Aff A3 horaires/aff_A3_2012/Aff_A3_horaires_20120411.ods: operation not permitted
[X4DNQ] 19:07:59 INFO: Puller: shortcut: chmod /home/bureau/Hebdo-G/Aff A3 horaires/aff_A3_2012/Aff_A3_horaires_20120411.ods: operation not permitted
[X4DNQ] 19:07:59 INFO: Puller (folder "PALACE-Bureau", file "Hebdo-G/autres grilles/Prog_A5_AVEC_synopsis/Prog_A5_synopsis_20120411.ods"): shortcut: chmod: chmod /home/bureau/Hebdo-G/autres grilles/Prog_A5_AVEC_synopsis/Prog_A5_synopsis_20120411.ods: operation not permitted
[X4DNQ] 19:07:59 INFO: Puller: shortcut: chmod /home/bureau/Hebdo-G/autres grilles/Prog_A5_AVEC_synopsis/Prog_A5_synopsis_20120411.ods: operation not permitted
[X4DNQ] 19:07:59 INFO: Exiting

I would be surprised if the issues are related to syncthing-gtk. For the chmod messages, I see no other options than fixing your files’ ownership or ignoring permissions as described in my previous reply.

2 Likes

And you are totally right !

After checking ownerships and permissions at system level in many ways, and samba level, all seems right ( to my understanding at least ).

So I’ve enabled “ignore permission bits” for each sync’d folder and this almost immediately fixes the situation ! ST and ST-gtk work both as expected, no more “chmod operation not permitted”.

Yet it seems to mean that something has changed in the way ST handles permissions as I’m really sure and careful about the settings of these 3 machines ( 2 at office and 1 at home so I almost always keep record of any changes ).

Nothing has changed how permissions are handled, and permission denied errors are come from the OS not syncthing, so there is definately something wrong with permissioms (ACLs being enabled most likely…)

Ok… so something might have changed on o.s. side at the same moment I updated syncthing and syncthing-gtk.

I mean I of course believe you I am really searching what I have done ( or which system update ) that could explain those problems with permissions.

I don’t use ACL…

…well thanx for explanations anyway ! Hope I’ll figure out the how and why…

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.