Towards Syncthing 2.0

Undecided, but probably not since it’s a shitload of work. If it’s possible to do it cleanly without removing the old API we’d probably do that, but — it’s then almost guaranteed that nobody will care until we actually remove it in v3 or whatever. :slight_smile:

I like gRPC. That’s probably where we’re going. But there will be a JSON & REST variant of it as well, one-to-one, automatically described by OpenAPI etc specifications. The web GUI will likely use the (new, mapped) REST API instead. We’re not going to be forcing everyone to switch to binary gRPC. It will still be possible to curl stuff. From any real programming language it will probably be nicer to use the auto generated clients based on either the gRPC spec though.

1 Like

I would care :slight_smile:

That sounds good. Being able to do simple curl calls is definitely useful. For Syncthing Tray I’ll probably switch to using gRPC, though. Qt even provides a module for it so this shouldn’t be a big deal especially when there’s the possibility to auto-generate a client.

1 Like

I remember someone starting on a rather ambitious remake of the Web GUI a while back. How far did that go?

1 Like

Scrapped in the v2 branch: https://github.com/syncthing/syncthing/commit/1a25ae32caeab45946674dce0b42a06b899eb3f5

There have been other discussions and pull requests. But all stopped dead in their tracks unfortunately…

Regarding the overhaul of our REST API (which is currently unlikely to be part of v2.0), see also this issue ticket, where concrete ideas may be collected:

https://github.com/syncthing/syncthing/issues/8796

Thanks all for the hard work. Once the dust has settled the official macOS Syncthing wrapper I made/maintain will also create a v2 baseline branch and a v1 maintaince branch. So those versions can live in parallel. I have no clue yet how to handle Sparkle update appcast updater between major change yet. As under mac updates are more intuitive with Sparkle update framework. But we will see what the future brings, maybe its a good time to (partial) rewrite Syncthing macOS to Swift only instead of Swift+Objective C (bridged). Maybe with the help of some AI coding tools.

2 Likes

There’s RoboCopy source destination /MIR in windows, after that manually delete the old folders in Syncthing folder. And only keep the news ones in Syncthing folder while the old ones in the destination backed up folder.

For Android, I haven’t found an OS alternative. There’s close sourced X-Plore with manual copy paste folder in free tier.

There are rsync in linux and rsync port in android. IDK about the RoboCopy equivalent for linux, IoS, and the auto free one in android, I also don’t have MacOs.

There’s cryptomator but it’s still problematic with Syncthing.

I only know Robocopy in Windows for copy paste or back up use case.

I’d like to put in my vote to keep XML. Its easier to edit without a specialized editor if manual changes need to be made. I get the attractiveness of JSON, but its much more difficult to make manual changes without getting all out of whack if all you have is nano/pico or vi(m).

I understand that there is a lot more to it than just the user, so I can only speak from a user who has had to work with configs in many formats. XML is (usually!) decently self-documenting, which is extremely helpful and time-saving.

1 Like

Yeah I didn’t say JSON, it’s not going to be JSON. If I’d pick anything today it’d probably be YAML

Thanks, that’s a relief! YAML is much better than JSON, though of the two I still prefer XML. I get that XML is chatty and verbose and probably takes more overhead. But its friendly!

TOML?

3 Likes

Oh, is TOML the correct name? I guess that’s slightly different than “INI” format. I like that.

I could be happy with either YAML or TOML if that simplifies the programmer’s maintenance tasks, and keeps the config clean and understandable.

I have no idea how much work it will take on the code side to switch config formats, but I can see it will need a lot of work to update the documentation for a new config format. I would like to contribute to that if I have the bandwidth. Once there’s a spec and the effort is started, I will see what I can do to help.

Actually there’s Robocopy /ZB option to make a backup. But since syncthing is a sync program, I’m not sure it has enough use cases and contributors to make something like /ZB but in Syncthing for backup folder.

Robocopy got both /MIR for syncing and /ZB for backup; and that’s probably because Ms got a bigger fund; or it’s just to streamlined a lot of programs eg xcopy into just 1 program.

As something that could be introduced in Syncthing v2, I would like to suggest improving ignore patterns handling on Windows by changing how \ is treated. This is a breaking change, but basically, \ would be used as an escape character (as is the case in other operating systems), and if one wanted to use it as a path separator, they would need to escape it as \\. Using double \\ isn’t new in the Windows world, as path separators already need to be escaped like that, e.g. in the Registry.

This change would allow Windows users to escape other characters in ignore patterns, like { or [, which isn’t possible right now, as \ is treated literally.

3 Likes

I 100% agree. It just makes things simpler for everybody. I would be happy to provide these PRs (code/docs), if you’d like.

Are you proposing to use a new filename for the new format, or what would the migration look like? TBH it doesn’t sound like an improvement to me, making normal path names more cumbersome to cater for a fairly rare corner case, but I don’t use Windows, so :person_shrugging:

1 Like

Well, as primarily a Windows user, I can only say that it’s been super annoying not to be able to use [], which are fairly common in filenames, to match in ignore patterns… On the other hand, {} are much less common, so it’s not that a of a big deal in their case.

With this change, the user would need to use directory\\file instead of directory\file to ignore it (or just use directory/file instead). When it comes to a possible migration from v1 to v2, maybe just automatically change all single \ to \\ in ignore patterns when run under Windows? I believe this should be safe, as currently \ can only be used as a path separator.

A possible alternative to this breaking change could be allowing to use one of the other, illegal Windows characters as an escape character (e.g. |). This would, of course, be Syncthing-specific, but then the current ignore patterns could continue to function with no modification necessary.

2 Likes

Yes but how do you know whether you did it or not; folders may be not present (USB sticks), stopped, have included files, get included ignore patterns synced from (or to!) v1 devices, etc.

You can’t for sure tell whether foo\[ab\]bar is a v1 or v2 pattern just by looking at it. (Though, granted, an unusual v1 style filepath :slight_smile: )

Another option would be to scan the ignore file on windows, and if it has backslashes, we try to add a #version:1 to the top end of the file. If no backslashes are found, we add a #version:2. If we can’t, we log the issue, but move on.

On future scans, if we see #version:1 (or it’s missing 'cause we couldn’t add it), we treat backslashes as path separators. Otherwise, backslashes are escape chars.

That way, we can someday switch to requiring users to add #version:1, if they want the old functionality.

Perhaps, we also add a short comment below the tag:

#version:1
// #version:1 treats backslashes as path separators
// #version:2 allows backslashes to escape *?[]{}

Or is that overkill?