Scanning slow on MacOS

I tried to enable the Set Low Priority option but it had no effect on the performance. During the time I reported the behaviour I had running messages, mail, terminal (mc), Chrome and Signal so nothing CPU/IO intensive (90%+ CPU idle). Any energy saving is turned off on the system.

You have to disable it, as its enabled by default.

Sorry for confusion, I disabled it, but no impact on performance.

Just to exclude one more possile aspect: Did you restart Syncthing after toggline said setting? It does not take effect immediately.

Yes. I rather restart after each change as I do not know when the settings are applied.

1 Like

How do you start/install Syncthing on the Mac? Any chance that some wrapper/task-manager/… is setting any funky flags (I have no idea about macos, thus the very non-specific questions :slight_smile: ).

This behavior seems to be purely I/O related and is tied to what kind of storage is used(SSD vs HDD) from what a quick Google search revealed.

I installed the package from the official distribution https://github.com/syncthing/syncthing-macos/releases/tag/v1.19.0-1

Well, the Syncthing application is for whatever reason sleeping the majority of the time. The I/O is not the issue here, even if I run Syncthing as a single application (no other applications running, nothing in the tray) the performance is in KiB/s while the storage provides xxxMiB/s. Disabling throttling is resolving the issue so the I/O is definitely not an issue. Why the macOS thinks that Syncthing should be throttled I have no clue.

1 Like

Searching for that flag, this seems to be a common problem e.g. also for time-machine: https://github.com/tnk3r/osx_hacks/blob/master/lowpri_throttle, Speed Up Time machines First back up - Apple Community

I don’t think we can do anything on Syncthing’s side (the flag change requires root) - maybe worth adding a FAQ about this.

To be clear: The OS seems to throttle I/O operations only for processes with low priority and not in general.

So your storage might be able to provide a lot more performance but Syncthing or other low priority processes are artificially limited.

Maybe we should rethink the low process priority default for Mac in order to avoid this?

1 Like

That’s not the issue here. @pporozin confirmed that the issue persists when disabling “Set Low Priority”. This is macos setting a bunch of apps as a special kind of low-prio, under unknown circumstances - in this case including Syncthing. If there’s a way of telling macos to not do that, then that would be great. However workarounds floating around on the interwebs (and my personal bias) suggests there’s no such possibility.

1 Like

The time machine backups are considered low priority to allow the user focus on his work while the macOS somehow backups the data. On a heavy loaded system the time machine backups might not complete even in days as Apple consider it as a feature. With Syncthing it might be similar, from macOS point of view it is probably a low priority process demanding a lot of resources. The option to disable the Low priority mode might be one of the issues (maybe something more is required?). The other issue to me seems that the macOS thinks for whatever reason that it is a good idea to throttle Syncthing even on a idle system. I do not have a deep understanding here so what I can do is to spend some time and test different scenarios:

  1. loaded system, low priority disabled for Syncthing
  2. idle system, low priority disabled for synchting
  3. loaded system, low priority enabled for Syncthing
  4. idle system, low priority enabled for synchting
  5. loaded system, low priority disabled for Syncthing, Syncthing process manually re-niced to a normal priority
  6. idle system, low priority disabled for synchting, Syncthing process manually re-niced to a normal priority
  7. loaded system, low priority enabled for Syncthing, Syncthing process manually re-niced to a normal priority
  8. idle system, low priority enabled for synchting, Syncthing process manually re-niced to a normal priority
  9. loaded system, low priority disabled for Syncthing, Syncthing process manually re-niced to a higher priority
  10. idle system, low priority disabled for synchting, Syncthing process manually re-niced to a higher priority
  11. loaded system, low priority enabled for Syncthing, Syncthing process manually re-niced to a higher priority
  12. idle system, low priority enabled for synchting, Syncthing process manually re-niced to a higher priority

Does this make sense or there is something different I can test? One more thing - all the above may behave differently when the macOS is running on battery.

The concerned reader can study the throttling mechanism in the source code…

I’m not that curious and it’s fairly dense, but my guess from a quick scan is that the throttling kicks in for low-priority I/O to non-SSD devices when there is also high priority I/O to those devices. I’m also thinking that “high priority” and “low priority” is less controlled by traditional unix niceness and more by whether a program is graphical, has a window in front of the user, and is considered interactive. Syncthing obviously doesn’t do any such things and will probably be considered low prio regardless of niceness.

2 Likes

Thank you Jokob! I would say that there is no real solution other than disabling the throttling in case of non SSD storage.

1 Like

The not-that concerned, casually un-informed macos grumbler (read I) is positivly impressed and surprised :smiley:
(by the source code availability, not so sure about the mechanism)

1 Like

I’m guessing this is relevant - but I wouldn’t have any idea how to apply it to the Syncthing executable…

Bumped into this while googling for this exact same issue. Sync’ing a local (NVME) raid to a NAS connected using dual 10G-T (LAG). Just scanning ~10TB took over 20 hours. Transfer throughput is < 10MBs (still going after a couple of days). When I rsync the entire raid to the NAS it takes only a few minutes. The local raid sustains >5GB/s random access. Access to the NAS saturates the network link so it’s not a NAS disk speed issue.

I too am curious to see if there is something that can be done to speed this up as the intent is to (auto) sync a few workstations to the (same) NAS.

Mac Pro with a crap-ton of memory. Qnap NAS, ZFS in “raid 6” mode.

I’ve been experimenting with different (bi-directional) sync applications and Syncthing seemed to have the most ratings.

All the relevant info to get rid of the problem discussed in this topic is in a post above:

If that doesn’t help you, there’s a different problem and please open a new topic for that.

1 Like

Disabling throttle alone didn’t make much of a difference. Yes, it did get better but still far from optimum. I can’t find documentation to all the settings under “Advanced/Folder”. Messing with Hashers (explained above) and especially “Copiers” did make a significant difference but it’s still around 100MBs. But then again, I won’t complain about a 10x increase and I can live with this as is.