syncthing 0.14.45 no longer syncs to Android (7.1.2)

The Problem

I upgraded syncthing (on linux laptops and Androids) to 0.14.45 on the 10th March, and as of yesterday, syncing to Android no longer works. syncing laptop to laptop is ok. The androids are a Google 6P and Px running 7.1.2.

Symptoms

  1. creating a new file on the Android is not copied to the laptop.

  2. creating a new file (say abc.txt) on the laptop leads to, on the android:

  • a. file .syncthing.abc.txt.tmp is created but just stays there and is never renamed.
  • b. the folder shows “Out of Sync”
  1. Updating a file on the laptop (say xyz.txt) leads to, on the android:
  • a. file .syncthing.xyz.txt.tmp is created and seems to have the new content and is the correct size. (though if I open it it appears to be empty) but just stays there and is never renamed.
  • b. the original xyz.txt just remains there.
  • c. the folder shows “Out of Sync”

4 I can solve it by:

  • a. deleting the original xyz.txt on the Android.
  • b. renaming the new .syncthing.xyz.txt.tmp to xyz.txt
  • c. and then the folder will show as synced.

The Log

the log shows the following entries:

  • tcp-client closed: protocol error: index update: “/.thumbnails”: filename is invalid
  • tcp-server closed: protocol error: index update: “/1wiki-info”: filename is invalid

the first entry seems to be spurious - there is no .thumbnails here.

the 2nd entry is the top containing folder of the files in error.

This is a bug on 0.14.44 used on the Android side. For the moment, your option is to roll back to an earlier syncthing-android release and wipe the database.

Thanks Jacob very much.

Deleting the database files - Ouch.
I guess because I have some large files (maps) and many files the way I set up a new machine is to add 1 directory at a time. so it is very manual and takes me a long time (2 days? on very fast laptops). Otherwise it just seems to fail.

Provided I manually fix the errors (renaming the .syncthing.xyz.txt.tmp to xyz.txt) the database seems ok.

And , the latest vsn on F-Droid is:

Version 0.10.4 (4137) - Added on 2018-02-01

and I have no idea how to build from the github source.

So rather than

  • stab myself in the knees 287 times,
  • then pull all my hairs out 2 by 2,
  • and then stick my head in the jaws of 27 crocodiles with bad breath,

it may be better to wait for it to be fixed in G Play store.

Just for clarification, even if you want to wait for the fixed version: You (most likely) would only need to delete the database on Android. And syncthing-android version 0.10.4 is the second newest release and bundles Syncthing v0.14.43, which is not affected by the bug.

1 Like

I think since https://github.com/syncthing/syncthing/pull/4800 is merged, it should be probably released as a hotfix all the way into android stable, outside of normal release cycle.

Otherwise it’s a broken sync for (likely) all android devices for about a month from now…

(or release an “update” to the playstore with downgrade to 0.14.43 + auto DB nuke)

2 Likes

I admit to never updating syncthing on Android once I had a verified working setup… it works fine, so the only thing I could achieve by updating would be risking running into some problem. So my version stays at 0.14.39

There’ll be a .46-rc.1 with this in it tomorrow but that’s not great to push on all Android users. If you (as in you syncthing-android maintainers) want to cherry pick it and release a .45+fix I guess that’s one option.

The 10.4 version on F-Droid would fix this, but as it’s signed differently to the Google Play Store version, we would need to uninstall that before installing the F-Droid version. I assume this would wipe the config?

@imsodin thanks, that clarified quite a bit.

Finally found the correspondence Android syncthing to main syncthing here:

for anyone else who might be looking.

But @imsodin / @calmh one question:

when reverting to an earlier version, why would I have to delete the database on android? Is there a different database structure between 0.14.43 and 0.14.45?

(I noticed that Jacob made the same point in the bug thread

“You need to remove the database also when rolling back.”

but I didn’t understand it there either.

The problem is that bad entries get added to the database. Reverting prevents adding further bad entries, but you need to get rid of the ones that are already there as well. In 0.14.46 we will clean them out on upgrade. But older versions don’t know to do that.

1 Like

Thanks Jacob.

Once I manually

  • delete the original xyz.txt on the Android.
  • rename the new .syncthing.xyz.txt.tmp to xyz.txt

the folder then shows as synced.

So, provided the folder shows as synced, this should mean no bad database entries, right?

(I’ve stopped syncing for the moment, so not creating any new bad entries)

As far as I understand the situation, what you do with files (temporary or not) should have no effect on the problem. The bad entries are in the index, which is sent in any case and produces an error on the receiving side. Did you actually connect two devices after doing what you described and it worked?

Do you keep your files in the app-specific folder (Android/data/com.nutomic.syncthingandroid) on the internal or external SD card? If not you could just export the config from syncthing android, safe it somewhere, remove syncthing, install an older playstore version e.g. from here and import the config again. Your files won’t be touched unless they are in the app specific folder.

I did so. Backed up /data/data/com.nutomic.syncthingandroid/files and /data/data/com.nutomic.syncthingandroid/shared_prefs. Switched over to 0.10.4 app and deleted the folder /data/data/com.nutomic.syncthingandroid/files/index-*.db before first run. WARNING: This could lead to data loss if devices were not fully in sync before doing this.

Yup, thanks for adding more detailed instructions. However to do it this way one needs a rooted device, my simplified explanation also works without it.

All my shared folders are elsewhere on my internal storage. I hadn’t noticed that I could export and import the config - that makes life much easier. Is that apkpure site trustworthy?

What’s this about possible data loss if not fully in sync? Given the problem with 0.10.5, I can’t seem to be fully in sync. Is it risky to downgrade?

All my shared folders are elsewhere on my internal storage. I hadn’t noticed that I could export and import the config - that makes life much easier. Is that apkpure site trustworthy?

It doesn’t really matter if the site is trustworthy as the google play releases are cryptographically signed, which means you can’t install a modified/malicious version over a genuine one installed from the play store. BUT THIS ONLY WORKS AS LONG AS THE GENUINE VERSION IS STILL INSTALLED. So to make sure you can try to install the manually downloaded apk after exporting the config. If it works, it’s not been tampered with. If there is an error like the one you get when you troy to install the F-Droid version, then you should really search for another source (btw: the f-droid version is perfectly fine, it just causes the same error message as it’s slightly different from the play store version).

What’s this about possible data loss if not fully in sync? Given the problem with 0.10.5, I can’t seem to be fully in sync. Is it risky to downgrade?

If you didn’t add any files on either device after upgrading to 0.10.5 you should be fine. Otherwise you could just sort for the ones with the latest changes/creation date and manually copy them or make a backup in case they get deleted on resync, so you can restore it in case it gets deleted. Then the worst case is that a file you deleted reappears and you have to delete it again.

@imsodin

Did you actually connect two devices after doing what you described and it worked?

yes, exactly.

I don’t normally leave syncthing running on Android - I run it when I want to sync, then exit.

So:

  1. After changing a file (say) xyx.txt on a laptop

  2. I run syncthing on the Android, but get an error on the android that the directory is “Out of Sync” and the directory contains both the old xyz.txt and .syncthing.xyz.txt.tmp

  3. I exit syncthing on the Android

  4. I delete the original xyz.txt on the Android.

  5. I rename the new .syncthing.xyz.txt.tmp to xyz.txt on the Android

  6. I run again on the Android, and the folder now shows as ‘synced’. Being a careful chap, I now exit Syncthing on the Android.

So my assumption is that the database is now ok. What do you think?

Well, if it is working, then good - it still sounds like either another problem or wodoo to me :slight_smile:

Anyway 0.15.16 is out that contains a fix for the absolute path problem that shows in your log.

woodoo: I guarantee I only use woodoo to clean my socks, never on my devices

another problem: yes, I wondered this. But honestly I just can’t understand the bug description, or how it relates to my log, well enough to comment.

In any case, my Androids are still on 0.10.15. When G play updates they will update. When it does I will try to sync, and we will see if the problem is still there.