panic: non-increasing sequence

Ahh OK re diagnostic only. Thank you.

Erm, LDB files are all 131MB …

The total value is relevant, but that’s still only ~1.4GB < 8GB. I wouldn’t have expected unmarshalled stuff to be that much larger. I just checked for my 500MB database the maximum memory usage during stindex -mode idxck was 2.5GB, so a factor 5 and thus ~7GB for you. The 16GB machine should be able to handle that :slight_smile:

OK, I’ll believe you :slight_smile: I’ll try it on that machine now!

For the record, I’m checking my DB too - and I’ve run out of RAM…

My DB folder is 21GB, and I’m running it on a Mac with 24GB RAM (and nothing else running) - so I’d expect a bit of contention to be going on. Looking at Activity Monitor, I can see:

  • 23.4GB used;
  • 20.2GB compressed RAM;
  • stindex ‘using’ 29.7GB…

I’m just transferring onto a machine with 64GB RAM now - so it’s not a problem for me* - but I thought you should know!

*Unless stindex really does need 5x the DB size to run…

Still on 1.3.4, right? Then it’s probably not 5x db size, as 1.4.0 reduces the db size significantly. And as Jakob wrote before, some db inconsistencies are known to exist on 1.3.4 and are fixed in 1.4.0.

…And on the Mac with 64GB RAM, stindex usage peaked around 35ishGB - and the process completed successfully within a couple of minutes. :slight_smile:

And there we go - took about 2 mins total!

Sequence mismatch for "SBD Software Updates/Installed/sg501601004.exe", folder "Sage", 8634 (key) != 8863 (FileInfo)
Sequence mismatch for "Backups/redacted 2013-04-12 16-47-31.001", folder "Sage", 8551 (key) != 8782 (FileInfo)
Sequence mismatch for "Backups/redacted 2013-06-18 15-03-22.001", folder "Sage", 8560 (key) != 8791 (FileInfo)
Sequence mismatch for "Backups/redacted B4 Processing SLR-190811-0954.001", folder "Sage", 8614 (key) != 8845 (FileInfo)
Sequence mismatch for "Backups/redacted 2012-09-25 17-19-38.001", folder "Sage", 8519 (key) != 8750 (FileInfo)
Duplicate sequence entry for "SBD Software Updates/Installed/sg501601004.exe", folder "Sage", seq 8863 (prev 8634)

There were about 100 of these lines. Last line was:

1 block list entries out of 768331 needs GC

1 Like

Thanks a lot for the info. So for now just keep running without model debug logging and either keep an eye on https://github.com/syncthing/syncthing/pull/6309 for a fix or just wait until I post here once there is something.

Will do, thank you ever so much. Will that allow syncthing to keep running? Or will I have to wait for a fix before it’ll work again?

And also, I’m so sorry - I’ve got Syncthing and SyncTrayzor. Could you tell me how to run without model debug logging? (so sorry)

I am not quite sure what you are apologizing for, thank you for going through the debugging!

Regardless of whether you use Syncthing through Synctrayzor or not, you can control the debug facilities through the actions>logs menu. If I remember correctly in the Synctrayzor settings there’s an option to setup environment variables: If you setup STTRACE=model there, remove that.

(ha, I was apologising as it was such basic requests!)

I cannot get into Syncthing - it doesn’t get as far as showing me the UI/webpage as the panic kicks in too quickly (I can see on the console log).

However, I see this with SyncTrayzor

Syncthing settings - https://i.imgur.com/OJ7m2uM.png Logging settings - https://i.imgur.com/1qHOccL.png

I can’t see the STTRACE=model setting anywhere. Is it in some config xml file for me to check somewhere? (rather than relying on UI)

What I meant was the “Syncthing Environment Variables” field, and that’s empty - curious. Does the Synctrayzor log say something about how Syncthing is started or env vars or anything?

Another complete guess: Try disabling “Start Syncthing when…”, then check the settings again before starting. Maybe the panic “messes something up in Synctrayzor”.

Hi there

I’ve disabled starting syncthing at all. And can see there’s no settings different to what I saw/shared before.

This is the log:

2020-02-27 20:19:55.7390 #8324 [Info] SyncTrayzor.Bootstrapper: SyncTrazor version 1.1.23.0 (Amd64) started at C:\Program Files\SyncTrayzor\SyncTrayzor.exe (.NET version: Unknown (528040)) 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: LogFilePath: C:\Users\mark\AppData\Roaming\SyncTrayzor\logs 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: SyncthingBackupPath: C:\Program Files\SyncTrayzor\syncthing.exe 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: ConfigurationFilePath: C:\Users\mark\AppData\Roaming\SyncTrayzor\config.xml 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: ConfigurationFileBackupPath: C:\Users\mark\AppData\Roaming\SyncTrayzor\config-backups 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: CefCachePath: C:\Users\mark\AppData\Local\SyncTrayzor\cef\cache 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: DefaultSyncthingPath: C:\Users\mark\AppData\Roaming\SyncTrayzor\syncthing.exe 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Config.ApplicationPathsProvider: DefaultSyncthingHomePath:  
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Ipc.IpcCommsClientFactory: Checking for other SyncTrayzor processes 
2020-02-27 20:19:55.7700 #8324 [Debug] SyncTrayzor.Services.Ipc.IpcCommsClientFactory: Did not find any other processes, or they all responded with an error 
2020-02-27 20:19:56.8931 #8324 [Info] SyncTrayzor.Services.Config.ConfigurationProvider: Loaded default configuration: <Configuration ShowTrayIconOnlyOnClose=False MinimizeToTray=False CloseToTray=True ShowDeviceConnectivityBalloons=True ShowDeviceOrFolderRejectedBalloons=True SyncthingAddress=localhost:8384 StartSyncthingAutomatically=True SyncthingCommandLineFlags=[] SyncthingEnvironmentalVariables=[] SyncthingDenyUpgrade=False SyncthingPriorityLevel=Normal Folders=[] NotifyOfNewVersions=True LatestNotifiedVersion= ObfuscateDeviceIDs=True UseComputerCulture=True SyncthingConsoleHeight=100 WindowPlacement= SyncthingWebBrowserZoomLevel=0 LastSeenInstallCount=0 SyncthingCustomPath= SyncthingCustomHomePath= ShowSynchronizedBalloonEvenIfNothingDownloaded=False DisableHardwareRendering=False EnableFailedTransferAlerts=True EnableConflictFileMonitoring=True ConflictResolverDeletesToRecycleBin=True PauseDevicesOnMeteredNetworks=True HaveDonated=False IconAnimationMode=DataTransferring OpenFolderCommand=explorer.exe "{0}" ShowFileInFolderCommand=explorer.exe /select, "{0}"LogLevel=Info> 
2020-02-27 20:19:57.0085 #8324 [Debug] SyncTrayzor.Services.Config.ConfigurationProvider: Found existing configuration at C:\Users\mark\AppData\Roaming\SyncTrayzor\config.xml 
2020-02-27 20:19:58.1537 #8324 [Debug] SyncTrayzor.Services.AutostartProvider: Have read/write access to the registry 
2020-02-27 20:20:03.1524 #8324 [Debug] SyncTrayzor.Services.Conflicts.ConflictFileWatcher: Refreshing conflicted files. Found 0 from 0 options 
2020-02-27 20:20:15.0478 #8324 [Info] SyncTrayzor.Services.UpdateManagement.UpdateChecker: No updates available

Had a thought …

Am I better to delete the DB completely, remove the mapping, and recreate it?

(I’ve doubled the RAM in the “server” by the way - so 16GB now)

I need to get this working again pronto (I’m many days out of sync and apart from Crashplan, I also use Syncthing to backup from work to home constantly) … so decided to delete the indexdb folder completely and recreate the files. Maybe a bad move, I’m not sure :slight_smile:

Weird that I’ve set the debug to “info (default)” yet I’m seeing thousands of model.go: DEBUG: lines in the console. Is that expected? And DEBUG: Index update (in): and other DEBUG lines. Sounds to me like the default isn’t in fact “show me only errors” but the default is to show DEBUG info?

Resetting the db should indeed fix the sequence out problem.

The debug stuff though is definitely not the default. Given the info you provided (using synctrayzor and the settings in the screenshots) I have no clue why debug logging is enabled (anything else to check @canton7?). Though now when it’s not crashing you should be able to use actions/logging to disable it.

Hiya

Deleting the DB worked well. In fact, when starting up i was impressed I didn’t need to do anything and it’s just started again. Really good.

I’ve already checked re debugging and all log entries are blank everywhere. Anything else you’d like me to check or send through? Most odd hey!

Thanks for the continued replies :slight_smile:

Nope, I don’t do anything with logging these days.

@marky_uk Just check under File -> Settings -> Syncthing -> Advanced -> Syncthing Environmental Variables, see if there’s anything related to STTRACE.

Hi @canton7 - sure. I just did, and it’s all blank -

Sometimes I’m in love with Syncthing as it works so flippin well, and then others, I want to cry

So my deleting of the DB and the folder details worked for a moment but within a hour or so, since March 2, it’s done NOTHING MORE :frowning: Same syncing percentage (87%) and same number of remaining files on both sides. Baaah.

What do you suggest? And if the answer is “pay me and I’ll sort”, let me know.