Syncthing 0.14.51 : unable to reach generated api key, api key in config file doesn't change/update

[fulstop]

Not sure if it’s a bug or worth an issue but

Syncthing installed on a new ubuntu server, with apt-get from

deb http://apt.syncthing.net/ syncthing release

I wanted to make this server reachable with the REST api (that I already use a lot), so I went to the GUI and generated a new key from the configuration panel. GUI is password protected and https is activated. Saved and tried to reach my endpoint : “Not authorized”. Tried to restart syncthing : curl standard commands still “not authorized”

To be sure I tested the command on 2 other endpoints with their respective api keys, they work as expected :

curl -X GET -k -H "X-API-Key: REDACTED" https://127.0.0.1:8384/rest/system/status

Came back to the ubuntu server, restarted syncthing anew and at some point when opening the configuration panel, the api key was gone from the text field. I generated another one, again, retried, relaunched, updated syncthing, still nothing.

Then I logged into my command line, verified that syncthing was running as syncthing user, went into ~/.config/syncthing/ and did a

cat config.xml | grep apikey

and to my surprise the apikey saved in the config file was none of the 3-4 I generated from the start.

I tested this API key, and to my surprise, it worked… Double checked the config file, it’s the one from the running instance (matching folders, devices, some custom settings,…)

Tried again to generate a new key from configuration panel, saved the changes, restared to be sure and the new key shown in config panel still doesn’t work… and there is always the same key written in my config.xml … and this key still works.

If I stop syncthing from servicectl and try to reach it with the key from the config file… it doesn’t work anymore…

So to my conclusion : it seems that in some way, the key written in the config file is hardcoded, which worries me a lot. Am I missing something ? I must be missing something.

Any idea ?

[AudriusButkevicius]

Do other settings get saved in the config correctly if you change them?

[fulstop]

Yes, and it worries me :

How I tested (all these commands where entered one after one in a row on the same shell :

Initially, syncthing is configured in the GUI to leave 1% of disk free, the config file reflects that setting :

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep minHomeDiskFree
    <minHomeDiskFree unit="%">1</minHomeDiskFree>
    <minHomeDiskFreePct>0</minHomeDiskFreePct>

Then in the GUI i change this rule to 10 Gb, and hit save, the config file changes accordingly :

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep minHomeDiskFree
    <minHomeDiskFree unit="GB">10</minHomeDiskFree>
    <minHomeDiskFreePct>0</minHomeDiskFreePct>

I then hit the “generate api key” button, I see a new API key 4T3H(shortened) appear, I copy it and hit ‘save’. The config file stays locked to the same API key (the same since the beginning of this support request).

The first four letters I provide are the real ones.

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep apikey
    <apikey>3MDF(shortened)</apikey>

Tried again, generated another one Eaf7(shortened), hit ‘Save’, restart syncthing, the config file is still stuck to the same api key :

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep apikey
    <apikey>3MDF(shortened)</apikey>

I then reset the changes I made to minHomeDiskFree var in the GUI (from 1% to 10Gb)

Before (as expected) :

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep minHomeDiskFree
    <minHomeDiskFree unit="GB">10</minHomeDiskFree>
    <minHomeDiskFreePct>0</minHomeDiskFreePct>

After :

syncthing@irbg:~/.config/syncthing$ cat config.xml | grep minHomeDiskFree
    <minHomeDiskFree unit="%">1</minHomeDiskFree>
    <minHomeDiskFreePct>0</minHomeDiskFreePct>

Weird. I don’t want to become paranoid, but it seems that it only happens to the api key… which makes my syncthing cluster vulnerable if someone has this key.

[imsodin (Simon)]

Any errors in the JS console?

I just tried to reproduce and changing the API key works fine here, with our without https and/or auth.

[fulstop]

No errors, normal regular logging (refreshSystem, refreshCache, refreshErrors (empty) ) The only different line after generating a new key and hitting ‘save’ :

settings equals - options: true gui: true ignDev: true ignFol: true

(2 times) Maybe the api key regeneration is not detected as a change on saving ?

[imsodin (Simon)]

Yes that’s it then, in this line gui should be false after generating a new key (and it is false for me).

Just to be sure: When you generate a new key, it is actually displayed correctly. However when you click save and reopen the settings window, the old key is displayed?

You could investigate by monitoring the value of $scope.tmpGUI.apiKey and setting breakpoints on $scope.saveSettings, $scope.setAPIKey and $scope.settingsModified.

[fulstop]

When generating a new key it is actually displayed correctly. When I click save and reopen the settings window there is no old key displayed, it’s the new (non-working) generated one that is (correctly) displayed (but not subsequently written in config.xml file).

I will try to investigate your suggestions.

[fulstop]

Did my best with mozilla debugger, it seems that the values never change between

$scope.config.gui.apiKey = 3MDF(old key)
$scope.tmpGUI.apiKey = 3MDF(old key)

So it seems like some GUI actions are broken and then I tested on Safari instead of Firefox and regenerating API key worked… like a charm. It went straight to the config file.

What I doesn’t understand is why it doesn’t work on Firefox. It generates a new key, stores it in some browser memory, but never retrieves the correct actual key from the config file, and never stores the new generated key.

My Firefox is version 63.0, only extension that could complicate things is uBlock Origin, but it doesnt trigger any filtering with syncthing GUI instances.

Pretty good news in a way, but I can’t still understand what blocks on Firefox.

[AudriusButkevicius]

Can you open a bug on github?

[imsodin (Simon)]

Even though the problem is totally funky? I mean it does not work for him on Firefox 63, works on Safari and for me on Firefox 63 and chromium.

[AudriusButkevicius]

Are you on the same browser on the same os tho?

[imsodin (Simon)]

Ah no, not OS probably, as safari is involved…

[fulstop]

(post-mortem)

I did a lot of cross tests, varying OS, browsers, and syncthing instances.

In the end, and after a lot of hair-pulling it’s my Lastpass Password Manager extension that creates this mess with syncthing instances in Firefox.

I have Firefox Sync activated, and so this extension is replicated on all my sessions, it’s when I tried from a Firefox of someone else (and regenerating a new key did work) that it clicked to me.

I think that Lastpass Password Manager understands that the API key is a secret, and the extension wants to “smartly” handle it like it does when you set up a new account on some website (with popup “save this password” toolbars and so on). That’s why this problem only occured with API Key settings and I had no problem with modifying regular settings.

But it’s not the first time that this extension and its css and javascript overloading create a complete mess on the visited service/website.

This time thou, it was completely invisible, no visible clue of Lastpass messing with syncthing settings panel.

I never imagined that the error could be on GUI/browser side nor only on my specific Firefox config.

I could have been better at faster testing of another browser suite, I really thought that everything was working fine on the GUI and that the problem shall be on backend side. I tried deactivating a lot of extensions while searching for this problem, but never thought of my password manager.

Thanks for your help