# Memory usage slowly rising after migration to 1.4.0

**URL:** https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689
**Category:** Support
**Created:** [March 24, 2020, 5:58pm UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689 "2020-03-24T17:58:36Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 24, 2020, 5:58pm UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/1 "2020-03-24T17:58:36Z")

</div>

Hi everyone,

I’m running Syncthing on a small NAS with 2GB of RAM. The NAS basically has a couple of large send-only folders (about 1TB and 300GB respectively), and a few very small (I mean, megabytes) two-way folders. With this kind of setup, I know precisely when it’s time for Syncthing to wake up.

Today I heard the drives were spinning for no reason, logged in and found Syncthing was using 2.7 Gib (!) of RAM; the entire machine was close to unresponsive, so I restarted Syncthing from the GUI; immediately after the restart, memory was about 150Mib, and after say, 10 minutes, was 250Mib. Syncthing didn’t do anything in the meanwhile.

All devices are idle, cpu usage is \< 0.1% The only odd thing I noticed is that there were two syncthing processes running, one using ~ 0 memory, but that may be normal. I’ll take a heap profile next time.

If you have a clue in the meanwhile, I’d appreciate.

The db does not look huge to me. Not sure if this is of any help:

```
[start] 18:22:04 INFO: syncthing v1.4.0 "Fermium Flea" (go1.13.8 linux-arm64) teamcity@build.syncthing.net 2020-03-06 19:52:22 UTC

[start] 18:22:04 INFO: Using large-database tuning

134320728 Mar 22 21:27 076608.ldb
 51699228 Mar 22 21:27 076609.ldb
134369757 Mar 23 10:28 076614.ldb
134332590 Mar 23 10:28 076615.ldb
134343390 Mar 23 10:29 076616.ldb
134322887 Mar 23 10:29 076617.ldb
134355440 Mar 23 10:29 076618.ldb
134300150 Mar 23 10:29 076620.ldb
134242502 Mar 23 10:30 076621.ldb
     8775 Mar 24 18:22 076632.ldb
     4054 Mar 24 18:31 076633.log
134358341 Mar 24 18:22 076635.ldb
       16 Mar 24 18:22 CURRENT
       16 Mar 24 18:22 CURRENT.bak
        0 Aug 26 2019 LOCK
   207825 Mar 24 18:22 LOG
  1048718 Jan 16 19:24 LOG.old
     2463 Mar 24 18:22 MANIFEST-076634
134343390 Mar 23 10:29 ./.config/syncthing/index-v0.14.0.db/076616.ldb
        0 Aug 26 2019 ./.config/syncthing/index-v0.14.0.db/LOCK
 51699228 Mar 22 21:27 ./.config/syncthing/index-v0.14.0.db/076609.ldb
134300150 Mar 23 10:29 ./.config/syncthing/index-v0.14.0.db/076620.ldb
134355440 Mar 23 10:29 ./.config/syncthing/index-v0.14.0.db/076618.ldb
       16 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/CURRENT.bak
134369757 Mar 23 10:28 ./.config/syncthing/index-v0.14.0.db/076614.ldb
     4054 Mar 24 18:31 ./.config/syncthing/index-v0.14.0.db/076633.log
134320728 Mar 22 21:27 ./.config/syncthing/index-v0.14.0.db/076608.ldb
       16 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/CURRENT
134242502 Mar 23 10:30 ./.config/syncthing/index-v0.14.0.db/076621.ldb
134358341 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/076635.ldb
  1048718 Jan 16 19:24 ./.config/syncthing/index-v0.14.0.db/LOG.old
     2463 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/MANIFEST-076634
134322887 Mar 23 10:29 ./.config/syncthing/index-v0.14.0.db/076617.ldb
     8775 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/076632.ldb
   207825 Mar 24 18:22 ./.config/syncthing/index-v0.14.0.db/LOG
134332590 Mar 23 10:28 ./.config/syncthing/index-v0.14.0.db/076615.ldb
```

---

<div class="post-metadata">

### Author: ![imsodin](https://forum.syncthing.net/user_avatar/forum.syncthing.net/imsodin/32/3317_2.png) [@imsodin](https://forum.syncthing.net/u/imsodin)
#### Post date: [March 24, 2020, 6:58pm UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/2 "2020-03-24T18:58:30Z")

</div>

Please get a heap profile to see what is using the memory: [https://docs.syncthing.net/users/profiling.html](https://docs.syncthing.net/users/profiling.html). Alternatively run syncthing with the `STHEAPPROFILE` environment var set, which will continuously create heap-profiles.

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 25, 2020, 5:29am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/3 "2020-03-25T05:29:01Z")

</div>

thanks, I will.

one more question: this instance was installed originally with Syncthing 1.21 and auto-upgraded since then. Is there any special maintenance to do? e.g. manually clean or compact the db? or any configuration setting to add?

thanks for the support

MH

---

<div class="post-metadata">

### Author: ![Andy](https://forum.syncthing.net/user_avatar/forum.syncthing.net/andy/32/7128_2.png) [@Andy](https://forum.syncthing.net/u/Andy)
#### Post date: [March 25, 2020, 5:52am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/4 "2020-03-25T05:52:59Z")

</div>

Which NAS you are using?

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 25, 2020, 6:27am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/5 "2020-03-25T06:27:19Z")

</div>

an Asustor 4004. It’s the standard arm64 binary, not the pre-pacakged app.

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 25, 2020, 6:28am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/6 "2020-03-25T06:28:55Z")

</div>

this morning memory is fluctuating between 350mb and 650mb.

[syncthing-heap-linux-arm64-v1.4.0-072530.pprof](https://forum.syncthing.net/uploads/short-url/yZPKrpfwBFaMhMH1HKeuV8VNzyy.pprof) (74.2 KB)

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 25, 2020, 7:31am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/7 "2020-03-25T07:31:22Z")

</div>

One more thing I didn’t mention: the NAS is _dedicated_ to Syncthing, it’s not doing anything else (except sharing folders over the LAN, obviously).

---

<div class="post-metadata">

### Author: ![Andy](https://forum.syncthing.net/user_avatar/forum.syncthing.net/andy/32/7128_2.png) [@Andy](https://forum.syncthing.net/u/Andy)
#### Post date: [March 25, 2020, 8:56am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/8 "2020-03-25T08:56:00Z")

</div>

It depends, this memory allocation does not have to be unusual. Because every file, every directory, i.e. every element must also be managed. But I guess that each element occupies +/- 1kB in RAM. Simon can say more or better about that.

Hence my question, how many peers (folders) have you created and how many files and directories (top right under “This Device”) are managed?

---

<div class="post-metadata">

### Author: ![calmh](https://forum.syncthing.net/user_avatar/forum.syncthing.net/calmh/32/15311_2.png) [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [March 25, 2020, 8:59am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/9 "2020-03-25T08:59:06Z")

</div>

What Andy says, to some extent. There will be some overhead in handling a large setup \> 1 TB. Additionally, in detecting this Syncthing makes a trade off for performance over memory usage:

```
[start] 18:22:04 INFO: Using large-database tuning

```

You can set the database tuning to `small` in the advanced config. This will reduce RAM usage somewhat. Your profile is busy scanning and running a database GC. You might want to reduce the GC scanning interval to make that happen less frequently, as you have a large database and slow computer:

```
export STGCINDIRECTEVERY=720h

```

(default is every 13h)

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 26, 2020, 10:10am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/10 "2020-03-26T10:10:17Z")

</div>

thanks for the hints.

Let me point out that the reason I raised the issue is not because the allocation is unusually large, it’s just because memory usage seems to increase over time for no evident reason (for example, now it’s fluctuating between 600 and 800Mib, according to ST web interface, and ST has always been idle - no new file was added/touched anywhere).

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 26, 2020, 10:19am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/11 "2020-03-26T10:19:14Z")

</div>

there are 6 folders and 4 peers.

peers A and B are always on. that remaining 2 peers are off 99% of the time.

folder A and B are shared send-only with peers A and B respectively. the other 4 folders are basically empty.

folder A is 115000 files, roughly 1TB.

folder B is 25000, roughly 300GB.

both trees are well balanced (i.e. no single folder in the tree contains, say, more than 100 files).

as I pointed out, I’m not saying that memory usage is large or small, I’m just saying that it seems to grow over time (something that never happened before the upgrade to 1.40).

---

<div class="post-metadata">

### Author: ![calmh](https://forum.syncthing.net/user_avatar/forum.syncthing.net/calmh/32/15311_2.png) [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [March 26, 2020, 10:24am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/12 "2020-03-26T10:24:17Z")

</div>

I’m hearing more “fluctuate” (grows and then decreases) than “grows over time” (continuous increase until, presumable, crash). I suspect this is the (new in 1.4) database GC which traverses the whole database periodically and (in large database tuning, at least) will likely result in fair amount of in-memory caching.

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 26, 2020, 10:33am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/13 "2020-03-26T10:33:06Z")

</div>

“fluctuate” does not exclude “grow over time”. it’s not a flat sinusoid: the trendline seems to grow steadily.

---

<div class="post-metadata">

### Author: ![calmh](https://forum.syncthing.net/user_avatar/forum.syncthing.net/calmh/32/15311_2.png) [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [March 26, 2020, 10:38am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/15 "2020-03-26T10:38:28Z")

</div>

The one thing in your profile that _might_ generate continuous growth is the QUIC usage. We’ve seen some reports of leaks on that previously. You could try and disable QUIC (set listen address to `tcp://:22000, dynamic+https://relays.syncthing.net/endpoint` instead of `default`, on both sides) and see what effect that has.

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 26, 2020, 10:44am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/16 "2020-03-26T10:44:49Z")

</div>

thanks for the hint. I’ll definitely try it here. As the entire country is in “soft lockdown”, now I cannot physically reach the other two peer machines.

Is this a change in 1.40? I’m asking, because I never had any issue since I started using SyncThing 1.21.

anyway, if I see another unusually high memory usage, I’ll take another dump.

---

<div class="post-metadata">

### Author: ![calmh](https://forum.syncthing.net/user_avatar/forum.syncthing.net/calmh/32/15311_2.png) [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [March 26, 2020, 11:34am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/17 "2020-03-26T11:34:35Z")

</div>

No, only the db GC thing is new in 1.4. But, you might not have noticed the slight upward trend until the larger fluctuations started happening, causing you to investigate?

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 27, 2020, 8:49am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/18 "2020-03-27T08:49:30Z")

</div>

I don’t think so.

Memory usage climbs up pretty quickly, today it’s 1.01Gib. So roughly every ~10 days it will need a reboot. I would have noticed.

[syncthing-heap-linux-arm64-v1.4.0-094858.pprof](https://forum.syncthing.net/uploads/short-url/1xIvMSSQDFcrU9ijDKHdYiDLmKf.pprof) (108.6 KB)

---

<div class="post-metadata">

### Author: ![m.holmes](https://forum.syncthing.net/letter_avatar_proxy/v4/letter/m/87869e/32.png) [@m.holmes](https://forum.syncthing.net/u/m.holmes)
#### Post date: [March 27, 2020, 8:53am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/19 "2020-03-27T08:53:42Z")

</div>

Additionally, it looks like logging in the web interface does trigger something. After a couple of seconds from login, memory usage spikes up, cpu usage briefly goes up too, but eventually _some_ memory is released (some is leaked, so the final result is higher than the initial).

[syncthing-heap-linux-arm64-v1.4.0-095019.pprof](https://forum.syncthing.net/uploads/short-url/VekGPHjl4PJjJNP8PJyYe3axCt.pprof) (110.9 KB)

here’s memory usage, as reported from the NAS itself:

 ![Screenshot 2020-03-27 at 09.57.36](https://forum-uploads.syncthingcdn.net/original/2X/e/e9715d89d7f52b208fae2a7eb8f6f6e0bd0769b8.png)

---

<div class="post-metadata">

### Author: ![calmh](https://forum.syncthing.net/user_avatar/forum.syncthing.net/calmh/32/15311_2.png) [@calmh](https://forum.syncthing.net/u/calmh)
#### Post date: [March 27, 2020, 9:34am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/20 "2020-03-27T09:34:11Z")

</div>

The GUI does a bunch of db operations which drives up memory usage.

Both your profiles are 55% QUIC memory. Something is broken with QUIC, but I doubt it’s new in 1.4 because nothing QUIC related changed there.

---

<div class="post-metadata">

### Author: ![imsodin](https://forum.syncthing.net/user_avatar/forum.syncthing.net/imsodin/32/3317_2.png) [@imsodin](https://forum.syncthing.net/u/imsodin)
#### Post date: [March 27, 2020, 10:04am UTC](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689/21 "2020-03-27T10:04:16Z")

</div>

> [@m.holmes](#):
>
> Is this a change in 1.40? I’m asking, because I never had any issue since I started using SyncThing 1.21.

Did you upgrade regularly or directly from 1.2.1 to 1.4.0? In 1.3.0 and 1.3.1 we updated the quic library versions, and the leak might have been introduced there.

[Next page](https://forum.syncthing.net/t/memory-usage-slowly-rising-after-migration-to-1-4-0/14689.md?page=2)
