Hi, I have just setup Syncthing on a new computer and added a new folder. However, the local state (and global state, but no syncing has been done so far) show almost 400GiB as the size of this folder while the real folder is about 20GiB.
I have checked the folder size multiple times, there are (as far as I can tell) no weird symlink things going on) and as a funny bonus I know the shown folder size cannot be correct because my hard drive is not big enough for it.
What is going on ? And importantly how can I fix this ?
Cheers !
Update: It seems looking at the file size through a graphical file manager (nautilus) also gives a wrong file size, utilities like âduâ show the correct size.
So Iâm guessing this is a bug, but not native to Syncthing and rather the different methods used to calculate total folder/file size ?
I have narrowed down the sub-folders which seem to be problematic, and they contain netkit VMs which show the same folder size problem when viewing in the exact same way (terminal vs Syncthing/nautilus file size) So Iâm not quite sure what type of file it really is, the VM filesystem folders seem to be the most I can narrow this down.
Actually, now that Iâm thinking about this, Iâve got the exact same thing happening on my system. I sync VirtualBox VMs with their size displayed as â25 GBâ in Syncthing, while in reality they take ~15 GB on the disk.
Well thatâs it then, glad to know my system is not just crazy ! I unfortunately donât know how Syncthing (or nautilus as in my example) calculate file sizes, but it doesnât seem like the sort of issue that would be too hard to resolves ? Considering âduâ apparently gets the file sizes more accurate (although you canât just rely on using a GNU-Core tool for a multiplatform program like Syncthing)
EDIT: I canât be sure but I suspect this also make the Syncthing of these files slower ? Or am I just being paranoid now haha
âSize of a file [on disk]â is highly relative and depends on how you look at it.
For example, do you include the overhead of a file or not? When on an extX filesystem, do you count bytes that belong to content blocks but are actually unused? Do you count the bytes needed for the inodes? The bytes for the inode bitmaps, directory entries or group descriptor tables?
IIRC, du -h tries to estimate the size of the file as it exists on disk. It also has to make a bunch of assumptions and will never satisfy every perspective. Syncthing on the other hand uses the filesize as reported by the OS (I think).
The filesize as reported by the OS usually is just the number âhow many bytes are in this fileâ. This can differ hugely for the actual âphysical size on diskâ, if features like sparse blocks are used - which I believe is probably whatâs happening for the virtual disk images. The files are really as huge as reported by Syncthing - the OS has allocated this number of bytes for the file, and this is the number of bytes that can be read from the file.
The actual physical size on disk is lower, probably because a bunch of bytes part of the file are not actually stored on disk - theyâre probably compressed, or left out entirely (sparse blocks).
Correct, but even here we can still argue that itâs âwrongâ, because Windows (assuming an NTFS filesystem) does not include MFT-overhead including most non-data attributes, which is part of the size on disk - just not of the content. It does however consider the bytes required for partially filled clusters (blocks).
What Iâm trying to say is: filesize is always relative and thereâs not really a universal truth that covers everyoneâs perspective.
[In addition to that Microsoft has always had trouble with SI vs IEC units, which is highly annoying when working with filesizes as displayed in the Explorer]