Untrusted create too long file names

[malvis_m]

I have a problem with untrusted Syncthing node with too long filenames, e.g.,

E:\OneDrive\st\u_z\3.Syncthing-enc\56\90LRVK5F3HU65M79923QCQRLF9JQO7643JTG9089CQCSJSB8HOJ8JV7GEQT8GPLGGC63D1QMAAS9DVQOCG1BFMTNV6ULSAD57V88T8A88KFSSFCLIQK6JDFLAPQ04U1ELSPN1L60MUB2RLU8FG61H8HHF4U4KQCUS5FAU47T495H04V8NTQ8592GE2A5TDFM0S4RVA78\DC3PH11M0EFSIDSRKB0BSVRBU2CRBTN68RTP1ILR8MCNQUH8A5S6R993GIQ5T3F9MIPRGH4GTJ2LV1MIR2OBK7ER9Q15AGH99Q05E0HAGIVNI99HBU2785NAJGTQ4KQOO750GCEUGEFUNV4MLLNHKIVDOHGSQJ9N0FD3248B1KRVFF6BM5RMNJ7Q0C

E:\OneDrive\st\u_z\N.Syncthing-enc\8U\0L4J5VQPS5E04CLMKLLSBAJD0SCFF8BOAMHGKMJSDM6P21RE3Q1BORKFKBMB88HKCH3AHB3TCST3C05CVBDCTP6VGC2G2QEMP9C0QAT4NB3C6PB7LFI79M8306U3FHIK6BEUQK6T23CQAA6GNAMIK2NCCM3DH6MNHKOGMJCV3SCVDIVL89D4A0A3EKHQ8J8FKRHL69J5\DNPAFF29TQLTUVDU8430L3AOVHIOELQEVE4Q8ID8MKKLUFV3Q5NFLLVPDDBQK9AE0BJ8USQEMOLR9K6MGUU8BGPRP3SIE5UARBT05S08IISTF7LPIKSGK851ACT30GG303S6U934IBQSEUBVQJPN08VOSKS8LQ059TCPEE4B4HCMTLTM52TIJ2KO2HSG0AD8AFOUGKOP

E:\OneDrive\st\u_z\N.Syncthing-enc\I6\C709F9JT57RT0SN72C911BST15RPURHGU88RNQK5HHCS9INCSR93R0OFOQBRQ96G2CISSN6I5BATUP5U3EA1JD2MTOLE7I6IT8VSKGR42LDR4JD7FDF0B2VI9H06O2AHLVJJCRV4IN735RGQ7CSBQMQDOJDH9M61VR4KQO7CJ60UH4EO6A7VDC0F880S5U3VTLOOA9UC\8MHSJJTQQ4LA5AIRLAS4OTURDBDSFVFB4T9IQVFSDD1E5FBU2T05M0MTTM73B2QTELQM2UTEDFN3DFATUFI166VQHET7FG8TV2RPDT57QCEMNG9DTSBRU73RE5D98L0G14O11G6VAEFGEN6R969CHC61PNH7ODOI27L7QPBMQP58Q2MMOLCDJ30DJF9U4E9B157GIFCK

I can’t figure out how exactly the filenames are being created, but I presume it’s file path?

I found the longest file path ~246 chars, renamed them shorter, but that didn’t change these files.

Found one ~230 chars with cyrilic symbols, renamed that one to latin, that solved the problem and got rid of one long file, although I can’t find which other files could be so long? At least the files are so long, I can’t even copy them to other directory to try to decrypt them as they are too long (don’t know how Syncthing can create them avoiding Windows system limits).

Made a test with these files:

9 автомобилей, которые не стоит покупать. Часть вторая._files S.Syncthing-enc\7T\RTR81CAOEJRJPL249IM633FEJSA8CJSB23BMO20J2NJNVVEUJM82IBF8AS2CH0P7RO9JV2IREQJ4EQKPF5T3BC0PPBOR5SBNNJJIF19JLPSR0RQ5516KVIDBCULAOPSGHISDAUA0EVEQKNHJE0JALLP2D3UK24MJHNQ71DV2THANQAKR23B5DH9JAUCQNG1NML15I

9 avtomobilei, kotorie ne stoit pokupatj. Castj vtoraj._files 9.Syncthing-enc\FO\92IC2SBPACDBP4ROSVG4G2K1TOLC9DFF8CML79AU7Q4D3J838CI5N73PVC5N1OBBSP747SQ07QMBBDS18NGGGALAEO7PLCRP0H72FN2OF01D30NOR9M7VHG7UP0D1IC

The encrypted file lenght difference is quite huge.

Any ideas how to resolve this?

[tomasz86]

Can you post screenshots from the Web GUI that show exactly the error messages that you encounter?

I’ve probably had a similar issue last year in https://forum.syncthing.net/t/failed-items-and-access-is-denied-in-a-receive-encrypted-folder/17479. Unforunately, I’ve stopped experimenting with Receive Encrypted folders after that, because not being able to sync multiple files with no clear culprit that would be fixable was a deal breaker for myself.

I did try to experiment with using symlinks in drive root paths pointing at the proper folder, but they didn’t help. I did not try moving the whole folder to a very short path, e.g. C:\u_z, so I’d suggest you try doing just that and see whether there’s any difference.

[malvis_m]

To be more precise, I’m trying to sync untrusted Syncthing which then saves files in OneDrive. OneDrive can’t sync them and hangs as due to file lenght limit. I can’t also make a copy of files etc, as they are too long, so I was thinking to decrypt them and the try to find which files are generating too long file names, but full path is over 400 chars, so it’s can’t be copied by windows explorer either…

[tomasz86]

Does this mean that there are no error in Syncthing itself? If yes, then I’d say this is really more of a OneDrive problem at this point. For the record, there is no way to make the encrypted filenames shorter in Syncthing.

If Windows Explorer’s got issues with the long filenames, you may want to try using command line tools like the built-in robocopy (see https://docs.microsoft.com/windows-server/administration/windows-commands/robocopy) which from my experience should be able to deal with them without problems.

[malvis_m]

Yes, the Syncthing is working fine. The problem is with OneDrive and Windows dealing with those file names being too long…

[imsodin (Simon)]

There’s no simple fix. Those file-name lengths are arbitrary and we just went with a high number below the typical limit supported by filesystems/platforms (256 chars), as having “synthetic” intermediate directories also carries some cost. However it turned out there’s a plethora of niche filesystems/tools/platforms which do not allow for such long names (cryptfs, synology, onedrive). While that is a deficiency of those tools, in hindsight it would have likely been the lesser disadvantage to have shorter path components and more syntethic directories. Changing that now as it’s already being used is very difficult (if at all possible without a hard reset) though.

[malvis_m]

As it states on https://docs.syncthing.net/users/untrusted.html: “Warning. This feature should still be considered beta / testing only.”, so it would still be possible to implement some kind of changes… Maybe addition checkbox “short names”, that would need to make a full resync of files or “Receive Encrypted (short filepath)”.

It seems I managed to get over the problem through going on using the commands for Linux/Win Powershell:

find . | awk '{print length,$0}' | sort -rn | less
cmd /c dir /s /b |? {$_.length -gt 210}

cmd /c dir /s /b |? {$_.length -gt 399}

and renaming the folder/files path with longest names, it looks like encryptions make the path around double the size, especially if the file name contain cyrilic or in my case also Latvian symbols (ēūīāšģķļžčņ). Renaming files to latin gave a significantly shorter file path.

[tomasz86]

Yeah, this is the general problem with computing being Latin (or rather English) centred, and the overall assumption that 1 letter/character equals 1 byte. However, even a single Chinese character takes between 3 or 4 bytes, so the so called “256 chars” limit very quickly shrinks to only 80-something characters depending on the language.

Syncthing, of course, doesn’t use non-Latin characters in the encoded filenames, but it is very likely that a non-English speaking or multilingual user will have them in the parent path.