panic / major crashing on 1.3.2-rc1

st log.txt (118.5 KB)

I use synctrazor and it’s just had a crash. Seems to be shortly after the new version. The log suggests a missing folder, but it’s there, I can see it.

Just trying Syncthing outside of synctrazor.

That’s known, persistent problem, for which the cause isn’t known unfortunately. The error usually doesn’t persist though, i.e. the panic is caught and Syncthing restarted and then it’s fine - is that not the case for you?

It’s taken several restarts of both Syncthing and SyncTrazor but it seems to have calmed down now. Only mentioned it due to it being so soon after the release.

Sure, thanks for that, I am not complaining about you mentioning it :wink:

The rate of occurrence of this actually correlates with releases, probably due the full index being transferred on upgrade: https://sentry.syncthing.net/share/issue/b230d5a6f9ca4102a23b5e3167a996b5/

@calmh @AudriusButkevicius I had a look into this and the reason is pretty simple: The striked part is wrong, we don’t actually send index updates when the remote folder is paused. The race is still real. We do send index updates regardless of whether the folder on the remote is paused. And on incoming indexes we check the config wrapper to see if our folder is paused, and ignore the index if so. That leaves a small window between a config change and that change being reflected in the model for a race.

To be honest I see no reason to panic here, I’d just bail instead (we even just info log if we don’t have the folder at all in config). Regardless a more general question is:

  • Should we accept index updates regardless of whether the folder is paused (locally)?

If yes, we need to leave filesets in the model if a folder gets paused, if no we shouldn’t send indexes if the CC indicates that the folder is paused on the remote.

It seems we should just not send them if remote cluster config does not have the folder anymore. On the other side we should just drop it on the floor, and not crash, IMHO.

I guess if we start sending CC msgs while connected, we should not drop the index messages, but the pausing becomes a lie, as indexes arriving can cause a lot of io.

I was talking BS, we already don’t send indexes when the remote folder is paused - sorry for the noise. I’ll just change the index handling to drop index updates instead of panicking.

syncthing.log (1.3 MB)

A send only PC has now got the same issue, however I have not been able to get St / Synctrazor to start. All the folders are visible on the network and have full access. It had been fine up until this morning. No folders are paused on the sending end.

As a workaround you can pause all devices in config before starting or start syncthing with -paused command line option, then unpause folders, wait until everything is idle and then unpause devices.

I am working on a solution (or at least mitigation) for the next release cycle.