Cross platform: character sets and filename limitations

(usernamegoeshere) #21

If there is inflight and reserved filenaming scheme already, it could be simply extended for these encoded filenames due to limitations, and every incompatible participant (e.g. the windows users in contrast to the mac users in my example) would see these encoded filenames and could rename the file (as it is only then present for them in contrast to the current situation right now) fitting their local needs and hopefully the problem would then not bounce back into the mac users realm and cause an encoded filename there if the windows user would not select a filename that would be incompatible again with the mac users local limitations.

(maybe like punycode for internantionalised domain names for webbrowsers, ascii7bit representation of punycode would work for all the browsers, and modern browsers would take the puny stuff and represent it as utf8 or whatever details that scheme and standards have and use that stuff to display at least, as far as I understand punycode is that on the dns level everything is still ascii7bit.)

I absolutely prefer to have a file, instead of not having a file. And these encoded or pending filenames would really make it possible to remedy the situation on the filesystem level that the average joe user sees, as not all my syncthing users can handle the syncthing gui (webpage) at all and can only barely cope with a commonly spread out and shared folder and can work through their local file explorer or file finder and whatnot interfaces. Its sad that apparently nobody else sees the need to have a file distributed even with problems in naming a file instead of not having a file at all. Maybe I need to fork over some money or paypal or donation or whatever to get this done, but simply the whole discussion in this thread that apparently the master leaders and coders and developers rather discuss these issues and apparently not seeing the value of it, seems pretty disappointing to me.

where do we donate or finance this project? does it even help to throw money at it or is it a mindset and developers attitude that I am experiencing here in this thread? please tell me honestly if I am causing too much trouble or am discussing in odd or nonpleasant ways to you guys, maybe my standards and experiences differ strongly from you guys, and also your priorities are of course way different from mine and so on. maybe i am even barking up the wrong tree.

thanks and regards.

(Antony Male) #22


(Audrius Butkevicius) #23

So I do see the value of it, but it’s only valuable and meaningful to the very few that experience the issue.

Most of my syncthing work is based on I think this is cool and useful to the masses hence I will spend some of my personal time to make this happen basis.

Out of the few tickets which have a bounty attached to them, only one was closed and resolved (that I am aware off), and I doubt that the guy had financial motives to spend potentially weeks writing code just to get 20 or so dollars.

You can try and get a Go freelancer on opendesk or other freelancer site to implement this, we’ll provide full support.

I think the quick win here would be to implement a remapping layer in the protocol package mutating anything that is unhandled by the local filesystem for all messages that exist.

For example foo:bar.txt becomes ~syncthing-incompatible~.foo0x58bar.txt and pray that nobody creates a a file under that name or FOO:BAR.txt, as you’ll get dataloss.

(Jakob Borg) #24

@usernamegoeshere I listed the approaches I could think of and why I don’t think they are good solutions in my first reply up the thread. If someone can think of a better approach I’m all ears and we can certainly implement something.

(Audrius Butkevicius) #25

So namespacing whatever scheme we choose to go for works, we just have to choose an escaping scheme that works too, and know that everything under that scheme is escaped.

Case differences is probably more problematic.

(Jakob Borg) #26

What’s the namespacing thing? You mean like using syncthing~BF2E33343C41.txt instead of foo:bar>baz.txt with the “syncthing~” prefix being the namespace? I can see a couple of problems with that, not the least being that I would see that file, think “that’s some crap temp file it left behind” and delete it, which should logically then delete the original named text file on the other side…

(Audrius Butkevicius) #27

What other issues do you see?

(Arne Ko) #28

How about renaming incompatible characters to their (english) names, with an escape chracter of some sort, e.g renaming foo:bar.txt to foo~st~colon~st~bar.txt

I am prepared to be shot down, but this way there wouldn’t be information loss and the information could even be “restored” when transferring to another compatible node.

Edit: Just noticed somebody else posting essentially the exact thing while I was reading and answering this

(Jakob Borg) #29

It’s really ugly? :smiley: And then the conflict stuff. What happens when I create foo:bar.txt on my Mac, it gets synced to my Windows box as syncthing~Zm9vOmJhcgo=.txt, and I manually run a copy or some other syncing thing so I get that file back to my Mac in the same directory as the original? We could stop syncing it I guess, or escape that file once more…

Apart from “see above” and generating possibly long filenames, that could conceivably work.

(Audrius Butkevicius) #30

So my idea was that you would double escape it next time you see it.

Also I am not talking about base32, I am talking about something ala urlencode where only unrepresentable chars are escaped which still makes it somewhat readable.

(Jakob Borg) #31

Could work I guess, propose something in a ticket? :slight_smile: