Update on 2023 "directory requires a lot of resources and might slow down your system" warning

General
1

Relates to TrueNAS Core: TrueNAS-13.0-U6.1 Syncthing Version: 1.27.2_1 Install type: plugin

Hi all,

First of all, well done to all you mods, helping the experienced and inexperienced alike :+1:

I thought I would give an update to the following thread, that I last responded to a year ago.

Just wish to say that I still receive this warning upon a restart of ST on the server, however there’s been no excessive CPU usage or reliability issues and it’s only applicable to the following folders:

Folder 1 - 9694 files / 1462 folders Folder 2 - 26174 files / 8066 folders

The warning does not occur on the following folders:

Folder 3 64 files / no folders Folder 4 2 files / no folders Folder 5 5 files / 1 folder Folder 6 5547 files / 946 folders.

I hope that helps someone. Of course as time goes by, this could be a moot issue as TrueNAS Core users are gently nudged to migrate to Scale.

Screenshot 2024-04-14 114012
Screenshot 2024-04-14 1140121290×317 9.63 KB

2

IIRC the warning triggers at 10k files+dirs so this is correct as intended.

1 Like
3

FWIW, I have Syncthing running on XigmaNAS (FreeBSD) with close to 300k files/folders. But its got plenty of RAM and CPU, so I’ve never noticed it choking although I receive the warnings. XigmaNAS seems to be well tuned out of the box for this.

1 Like
4

Yes indeed, there must have been some improvements as last year it also triggered a warning, though sometimes with only a dozen files in a folder.

5

That’s very reassuring to hear, thank you very much for letting me know. I have only a modest amount of RAM (32GB), but it seems all has always been well and good.

6

There was a bug that triggered the warning too early; it’s been fixed. But as many such things it’s just a threshold for a warning. We trigger it at 10k open file descriptors, but your system may be fine with 100k or a million, or might struggle at a thousand. Specifics differ…

7

That’s more than sufficient RAM, I have 8GB. I have a couple of remote VPS FreeBSD servers handling ~130k files with only 512MB RAM.

You may have to increase file handles in FreeBSD if you receive the message Too many files:

# sysctl -a |grep files
kern.maxfiles: 113359
kern.maxfilesperproc: 102015

# sysctl kern.maxfiles=2000000
kern.maxfiles: 113359 -> 2000000

# sysctl kern.maxfilesperproc=250000
kern.maxfilesperproc: 102015 -> 250000

# service sycnthing restart

To make the changes apply on the next reboot:

# echo kern.maxfiles=2000000 >> /etc/sysctl.conf
# echo kern.maxfilesperproc=250000 >> /etc/sysctl.conf

In XigmaNAS (and similar possibly for TrueNAS Core), these settings will instead need to be added (or edited) in System > Advanced > sysctl.conf.

1 Like