STGCBLOCKSEVERY hasn’t made any difference, it’s not made it lag or queue go up. I will kill, remove the setting and try again and see if I can record the resource monitor somehow to see what happens
2.avi (11.5 MB)
Here’s an edited review of last nights sync. Key events (im using the time on the original video, not the time on the file).
9:04:38 Syncthings uptime jumps from 6h 8m to 14h 35m
I scroll the video towards the end but watch the disk que
9:21:27 The disk que comes from high to ‘normal’
9:22:16 Indexes are being read / written, disk que goes up
9:23:52 Indexes activity increases, Disk IO ramps up to 101MB/sec (top of resource monitor) que increasing
9:24:13 Synctrazor gui no longer responding
9:25:00 Its just indexing
At this point im trying to run a cpu profile, but the system is starting to go slow so I save the video.
I have reapplied the STGCBLOCKSEVERY variable and will see if that helps. It seems that when it indexes in chunks is where it struggles, but only after a period of time ( memory related, high mem - index issue?). If the variable is updating the index more often, then in theory I should not get locked up.
Please don’t!
As I wrote before, that was only an experiment to forcefully trigger GC to check if that immediately locks freezes Syncthing. Running GC continuously is not helping/mitigating/doing anything good.
Please share the logs from that time and ideally include the folder side into screenshots/-casts (starting around 22.16 the download rate becomes really high, so I’d expect folders are syncing the two files added to local state).
Stopped and removed.
Can you advise what logs you need
I should use “the log” not “logs”. Unless otherwise stated, the required log is always syncthings own log. If they get archived after some time (they are on Windows i think) then the file with entries from the time in question. If unclear which, just take all.
They’ve been overwritten as I have restarted a few times. I will pull them next time it happens.
Task manager shows seek times/duration which is more interesting metric.
Hard drive issues usually manifest not in dropped throughput but in longer seek times.
The USB drive is rarely turned off. It has performed impeccably for a year and no issues on earlier versions. It will sit quite happily with an average que of 16, but after a period of time it will be unusually busy, but then calms down after a while. If the drive were to be exhibiting issues would it not do it all the time? As I type, St has been running for 5h 45m, and it’s flat lined for almost all that time (I have 3 screens, the resource monitors and synctrazor is always on top.)
I’m only reporting what I see, and yes, at the moment I do feel that I have a unique issue since no one else has come forward and said anything, but perhaps that’s because they don’t have 45 sync jobs spread over 3 USB drives.
For the weekend i’m just letting it catch up and then will see what happens when everything is up to date.
Sorry, so we are talking about spinning drives or usb drives now? Is the database also on a usb drive?
the synced files are on usb spinning drives. the indexes are on SSD (c drive)
And when you say disk queue goes up, which disk are you referring to
The usb drive which contains the data. It’s locked up again this morning
thats interesting, in the time I was making these snips this happened
ahh, rc6 has been released literally as I was typing this…
So I will see how it goes.
Are your usb enclosures proper branded enclosures or some cheap nameless Chinese produce for a few bucks off ebay?
I’ve had cheap stuff before and it seems the usb controller panics and stops working when under load.
1 Like
Western Digital My Book 8TB
1 Like
I had the same issues with cheap enclosures and gender adapters, and recently I have also encountered disconnecting problems with WD external drives when using the default Microsoft Windows 10 drivers for USB 3.0. They stopped only after I installed AMD USB drivers instead.
@terry Can you check what kind of USB chip you are using and the drivers which are currently installed for it? You can do it through the Device Manager.
Intel 8 series / C220 With MS drivers
But as mentioned before, i’ve not had an issue prior to 1.4 and I have sync data spread over 3 USB drives, but the WD does have 22 jobs and around 6Tb of data, which I do expect it to take around 2 days to index from a start/restart.
It might be tempting fate, but after 13h running, everything is ‘normal’, with the disk queue now at 12. Mem useage is 1.1Gb and the issue usually appears when I get above 2.5Gb ram.
So fingers crossed that whats changed in RC6 / 7 may have improved things. So long as there is no RC8 released today I will know by this evening how things are going.
Nothing relevant changed between any of the RCs. I honestly suspect you’re just seeing increased database activity as a result of the migration and full index exchanges after upgrade.
But the io balloons on the data volumes… Which makes no sense at all, as nothing could have changed there between 1.3.4 and 1.4.0
could the io be a red herring? the issue is that the web UI freezes/looses connection. even a completely broken/overloaded disk shouldn’t be able to make the api unresponsive (with the db on a separate disk).