Deleting a folder from the GUI does not delete all index files

Hello,

I would have expected that when a folder is deleted in the GUI, all index files related to the contents of the folder would be deleted as well. This seems not to happen at times. Is the assumption correct? Are there known situations in which there are “leftover” files to be found?

Example:

  • all folders in GUI deleted
  • giving Syncthing time to do its thing (watching …/index-v0.14.0.db/LOG for activities)
  • all sync partners deleted (just for good measure to be sure there is no sync in any way)
  • checking files in folder …/index-v0.14.1.db:
    DE71364-SV-U5:#/.config/syncthing/index-v0.14.0.db # ll
    total 724K
    -rw-r--r-- 1 tst tstgrp 399K Jan 22 11:59 000279.ldb
    -rw-r--r-- 1 tst tstgrp 234K Jan 22 11:59 000280.ldb
    -rw-r--r-- 1 tst tstgrp    0 Jan 22 12:06 000289.log
    -rw-r--r-- 1 tst tstgrp   16 Jan 22 12:06 CURRENT
    -rw-r--r-- 1 tst tstgrp    0 Aug  3  2016 LOCK
    -rw-r--r-- 1 tst tstgrp  75K Jan 22 12:06 LOG
    -rw-r--r-- 1 tst tstgrp  185 Jan 22 12:06 MANIFEST-000290
    

two ldb-files there…

  • checking the contents of the *.ldb files shows that those are files that were synced before and belonged in/to the deleted folder
  • tail -f …/index-v0.14.1.db/LOG while Syncthing is restarted multiple times shows:
    =============== Jan 22, 2018 (CET) ===============
    12:04:49.288830 log@legend F·NumFile S·FileSize N·Entry C·BadEntry B·BadBlock Ke·KeyError D·DroppedEntry L·Level Q·SeqNum T·TimeElapsed
    12:04:49.298769 version@stat F·[1 1] S·632KiB[233KiB 398KiB] Sc·[0.25 0.00]
    12:04:49.300121 db@open opening
    12:04:49.302642 journal@recovery F·1
    12:04:49.320406 journal@recovery recovering @285
    12:04:49.347332 version@stat F·[1 1] S·632KiB[233KiB 398KiB] Sc·[0.25 0.00]
    12:04:49.376067 db@janitor F·4 G·0
    12:04:49.391570 db@open done T·76.439034ms
    =============== Jan 22, 2018 (CET) ===============
    12:06:46.260026 log@legend F·NumFile S·FileSize N·Entry C·BadEntry B·BadBlock Ke·KeyError D·DroppedEntry L·Level Q·SeqNum T·TimeElapsed
    12:06:46.266109 version@stat F·[1 1] S·632KiB[233KiB 398KiB] Sc·[0.25 0.00]
    12:06:46.273597 db@open opening
    12:06:46.275987 journal@recovery F·1
    12:06:46.337315 journal@recovery recovering @287
    12:06:46.359354 version@stat F·[1 1] S·632KiB[233KiB 398KiB] Sc·[0.25 0.00]
    12:06:46.454475 db@janitor F·4 G·0
    12:06:46.457041 db@open done T·181.934964ms
    

looks good. Nothing else happens in the log once syncthing has started, so I assume that Syncthing has finished whatever is has been doing while deleting the folder. Still, there are two files left.

Have you actually confirmed that data relating to the folders which were deleted remains in the database? Or are you just observing that the database continues to exist after all folders were deleted?

If it’s the latter, I don’t think that’s an issue.

I’ve confirmed that:

checking the contents of the *.ldb files shows that those are files that were synced before and belonged in/to the deleted folder

Yeah, but you gave no details on that. How did you check them? Did you ensure that a compaction / garbage collection operation was performed first?

I just 'cat’enated them and saw entries that I recognize.For the test I just synced a folder full of bash scripts and then deleted it again. When catenating one of the files, I find things like “# this is a comment that was in one of the files” which makes me believe that this has something to do with the files that were once synced and should have been deleted.

Did you ensure that a compaction / garbage collection operation was performed first?

No. I am a user. I would a) expect Syncthing to do that itself, if necessary and b) wouldn’t know about how to do that. Can you give me a hint?

You claim to be a user, but you’re caring about the precise contents of the database used by Syncthing, which makes you a bit more than a normal user :wink:

Syncthing probably will do compaction if necessary, but it’s idea of what’s necessary probably doesn’t align with yours.

I’m surprised to see file contents in the database – my understanding is that should never be the case.

The database is much like other databases - deleting a record marks the space as available in some bitmask. At some later point it becomes overwritten or cleared out as part of compaction. The database is shared between all folders (and other stuff). There are no user servicable parts here.

The database does not contain file contents.

I’m sorry: With “contents” I meant filenames.

You claim to be a user, but you’re caring about the precise contents of the database used by Syncthing, which makes you a bit more than a normal user :wink:

Ack, but lightyears from any developer level. :wink: Just interested in Synching and trying to understand how Syncthing works. Just learning.

I am almost afraid to ask, since a simple question became an argument of who I am and why I want to know things (isn’t curiosity enough?), but… I still have no definitive answer to my original question “I would have expected that when a folder is deleted in the GUI, all index files related to the contents of the folder would be deleted as well.”.
May I assume that some *.ldb files are always “lying around” (no don’t get all upset about this wording, I just can’t think of more neutral right now), even if there is no sync-folder configured in Syncthing?

Yes. There’s a -reset-database option for destroying it completely, IIRC.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.