Windows 64-bit builds using 35GB virtual RAM

On Windows using 64-bit builds of syncthing (current release and snapshot gdab0aec) I notice that syncthing.exe uses up about 35GB virtual RAM. The physical RAM use is sensible at just under 100MB.

I’ve tried using a brand new config with no repos and observe the same symptoms. If I use the x86 version it does not claim such a large amount of VRAM. This behaviour is the same on both my Windows 7 SP1 64-bit and Windows Vista SP2 64-bit.

I can’t say I notice any actual problems for syncthing or the operating system in general but it strikes me as unusual behaviour.

Running Syncthing 0.9.5 (64-bit) on OS X 10.9.4 here.

With just a couple of very small repos in use, top shows Syncthing using 25MB physical RAM and allocating a whopping 139GB in virtual memory. In comparison, all other system processes tend to allocate a fairly consistent 1GB / 2GB / 2.5GB of virtual memory at most.

Is it just a typical behaviour of Go?

Why does my Go process use so much virtual memory?

The Go memory allocator reserves a large region of virtual memory as an arena for allocations. This virtual memory is local to the specific Go process; the reservation does not deprive other processes of memory.

To find the amount of actual memory allocated to a Go process, use the Unix top command and consult the RES (Linux) or RSIZE (Mac OS X) columns.

1 Like

The first four paragraphs on this page throw some light on why Go applications allocate large amounts of virtual RAM in comparison to other processes (it’s all about Go’s garbage collector). It does suggest up to 16GB being typical so I’m not sure if that fully explains the 139GB I see here.

That seems like a sensible explanation. With any luck, anyone with the same question should be able to find this thread.

FWIW I see the same on Mac OS X;

Note though that virtual memory really is virtual, it’s a convenient fiction for the program to use and has no impact on the system. The relevant numbers are the “real memory” or just “memory” columns here (I’m not sure what’s going on with compressed memory etc, seems new in 10.10).

Well, compared to 500M on Linux AMD64 I think this may be a bug in go’s OSX implementation. Maybe someone could raise this with the go project: https://code.google.com/p/go/issues/list (Virtual Memory is limited, too. And I think 300 GB is a bit much even for that.)

It is for all intents and purposes unlimited. This is not a problem.

https://developer.apple.com/library/mac/documentation/performance/Conceptual/ManagingMemory/Articles/AboutMemory.html

It does seem to be potentially problematic mostly on 32 bit: http://support.microsoft.com/kb/294418 and in detail http://blogs.technet.com/b/markrussinovich/archive/2008/11/17/3155406.aspx

As I think processes share the virtual memory space of 2-4 GB (32 bit) and 8 TB (64 bit) in Windows. But even then on 64 bit there would only be a problem with several large users of virtual memory, e.g. 27 instances of syncthing :wink:

And on 32 bit I guess go deals more conservatively with virtual memory?

To be devil’s advocate, I have 102 processes running right now. If they all used up that much virtual memory, I would certainly be in trouble, if you are correct that 27 instances of syncthing could be a problem.

No, you wouldn’t, because virtual memory is per process.

Oh, I didn’t know that. And even then you’d need over 200 processes :wink: