when opening or reloading the UI all folders are displayed as “Unknown”, then one by one they switch to “Up to date” or whatever state they are really in.
This effect takes longer, the bigger (or more files?) the folder is/has and/or if the computer or HDD is slower.
Once it is done loading: (38 seconds later)
I would assume that the Syncthing process knows the status of each folder at (almost) any time.
And it should be able to show it instantly.
Can you fix this?
Maybe if the data is not yet readily available a cache for this data might be a good idea.
Being constantly updated when things change, it could deliver this data to the UI instantly.
There is a database query behind each, which takes longer on slower machines and larger folders.
But 38 Seconds?
Maybe the database needs an index or some other way to speed this up?
In modern big-data systems data is often stored multiple times in different representations to speed up the different use-cases.
table1 = 2 million entries - query time: 38 seconds
table2 = generated from: select count(files), otherStuff from table1 group by folderId
this table is updated regularly (or if possible incrementally in order to not have to run the 38 seconds query over and over again)
UI builds from table2 only → fast
Is there a way to enable showing the queries (with run times)?
No. Syncthing isn’t a modern big data system in this case, the queries the gui needs to start up take a while with a lot of files.
It’s not a SQL database, it’s a key value store and we have to walk the whole keyspace to work out sizes.
If we were to maintain a secondary thing for fast UI summaries that means we’d have an extra operation for every file we handle, which would slow down the process of syncing.
Wouldn’t it make sense to store the last status the GUI calculated for the folders to be stored in localStorage and used by the GUI until the data is fetched from the API?
This data should be marked in some way to make it clear that it is cached data from the last visit.
Stale data in my books is as good as no data.
The Syncthing process itself caches some of this data in RAM, not sure off the top of my head what it is that we’re missing and if it could be added without significant pain.
Thanks! This sounds promising.
You are right, this data could be kept in RAM.
This way we don’t need another DB write operation every time a file is synced.
It’s just a fast update in the RAM.
And what if you store this data in a file on shutdown? And load it again on startup?
This might even solve this issue:
I noticed this now several times:
When i restart Syncthing the UI is not available for several minutes.
I checked the log and found that it is scanning folders before it makes the UI available.
Could you change this so the UI is made available first?
This becomes more relevant the larger and/or slower the folders/scanning get.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.