Beta test of Syncthing build with alternate database backend ("boltdb")

Announce Pre-release
1

@kahun, @Tom, any others seeing these weird corruption crashes - mind testing this snapshot?

http://build.syncthing.net/job/syncthing-bolt/

It uses a different database backend, so if there’s a bug in that it ought to be fixed (per definition). If the root cause is still flaky hardware or so, you’ll probably experience some new, different funkiness…

(Different database means full rescan on launch… Also means the normal database is untouched, so just reverting back to a vanilla 0.11.8 (or any other version) is a full undo of the experiment.)

2

The boltdb version is indexing…be 48-72h before we’re running normally. This version doesn’t create an index folder but stores everything in a file?

3

Yep.

4

ok I am testing it, I will report it later.

5

indexing completed overnight, big surprise! It takes previous versions >48h for the same work (14k files representing 4600GB).

index-v0.12.0.boltdb is 7.340.000KB (7GB), which is about 1GB larger than the index files have been in previous versions.

6

Yeah, bolt doesn’t use compression while leveldb does, so the index will be a bit larger on disk due to that if nothing else.

7

not worried about size on disk. Storage is cheap. I was just relaying the info.

Question : do both sides of a share need to be on this boltdb build ?

8

Only the ones which you seem to experience the crash with.

9

Yep. No difference in the protocol, talks perfectly fine with “normal” builds.

10

panic http://6f7fcc1928bb0b30.paste.se/

11

That’s a new bug for sure. :slight_smile: Either in Bolt or my code around it; I’ll assume the latter to start with.

12

syncthing restarted right after, and has started to sync with the remote host again.

13

Yeah, that’s my bug. It’ll deadlock (and then restart) under some circumstances.

Bolt build won’t work very well for the moment, requires some re-architecting that I won’t have time to do this evening. Leave it for a future experiment.

14

Well, mine never finished or I did not wait long enough to finish. I waited about 4 hours I think, normally it does not take that long. Shall I retry it?

ST was using very little cpu and my exfat process was using the process more compared to the standard ST builds.

15

still no issues… question : if i let the boltdb version update to 11.9, will it revert to goleveldb ?

16

Yes.

17

then i wont :stuck_out_tongue:

18

So, do I interpret that as the bolt version actually working reasonably well, deadlocks notwithstanding?

19

that’s a big yes. Haven’t had a single db-related panic.

20

@Tom @kahun I’ve made a new build which is v0.11.10+bolt, with (I think) a fix for the deadlock. It works for me. :smile:

http://build.syncthing.net/job/syncthing-bolt/lastSuccessfulBuild/artifact/