Optimize JVM Memory Arguments to Prevent PermGen and Java.lang.outOfMemory errors

Oops, wrong resolution

Increased default heap size to 512 megs

There is already a ticket in for that, we've just never done it.

Few comments.

I don't think removing the -Xms setting will solve your out of memory issues. This is just the startup memory for the heap. Also, I made it start the same as the max as it allows for the OS to snapshot that memory beforehand without throttling to get more and more when it needs it. It is a best practice to set them the same on startup for those issues. In either case, it should not affect the memory allocation of your program if you do not set the Xms the same as the max, but try it, maybe something does affect it. You can change this in the ServerSerice for now to test.

As for permgen, we do not have a setting for it, as well we would have to have a setting for java 7 and 8. So not sure how we would tackle this, but maybe just allowing ala-carte JVM arguments. Thoughts

Brad,

Yourself and Luis are probably JVM experts more so than I, but I'll try to outline my reasoning below:

Setting a min size lower than your max size isn't really a cushion. It just means that the JVM doesn't bother asking the OS for all the memory up front in case it doesn't need it. Setting the min the same as the max shouldn't make the JVM any more or less likely to run out of memory, it just means that the JVM won't need to go back to the OS to ask for more since it gets it all at once.

Since I'm running in to the errors in running large test suites, I can assume that the problem is Garbage Collection related. I've run in to the issue in production of OOM errors before when the Min/Max were set the same. Something to do with the OOM being triggered before the GC routine is forced to run. In this situation, it may be caused by the PermGen issues with class compilation in tandem with the memory limits, but it's been a standard practice of mine for a long time to set Xmx to a higher value than Xsx, for that reason.

setting the PermGen min/max to the minHeap value

I've never had a production server that needed a perm gen size anywhere near my heap size. For instance, I may use 2, 4, or even 8 Gig heaps by perm gen usage rarely ever goes above 100 to 200 megs. What correlation are you seeing between the heap size and perm gen? I also suspect that most people are already in Java 8 by now (or will be soon) so perm gen should become a non-issue since it will be part of the heap as metaspace.

I agree. PermGen doesn't need to be near that size so that could be handled separately. My solution was a "set it and forget it" kind of deal. My understanding is that setting PermGen and Max PermGen to the same value, eliminates the full GC from running when the limit is reached and forces parallel Sweeping. I do know that I'm seeing PermGen errors regularly on big test suites, even when I'm setting heapSize to 1024 or even 2048 - without using the explicit 1:1 setting. I haven't tested enough to tell you what the minimum thresholds are for that to stop. Metaspace, as I understand it, is implemented in a similar fashion to PermGen so, while it may be improved, this will still be a factor.

which will consistently cause java.lang.OutOfMemory errors
I've actually never gotten an OOM error with CommandBox yet. Can you share more about the ones you're seeing? What is the exact error? Is it a heap OOM or a permgen OOM?

Both Commandbox testbox runner and the HTML runner are causing issues. I ran full suites yesterday on three different apps, all started at 512 or above heapSize and saw them, either in full or as errors on individual tests, on all three apps. One of the Apps is a CBORM app, one is a CB3.8/Contentbox app, and the other is a non-ORM mongo app. All experienced the same results.

var maxHeapSize = minHeapSize*1.5;
I might see setting the min lower, but I wouldn't think we'd want to assign more than the user asked for. If I told CommandBox to start with a 1Gb heap I would not expect it to actually use up to 1.5 Gigs. Since JVM heap has always been a hard limit, I add in my own cushion when choosing a heap size.

Then I would suggest removing -Xms and clarifying that it's the maximum heap size. My development machine has 16GB but if I set the heapSize setting to 2048, I still don't want that memory reserved by the system unless I need it.

Alternately, I would suggest removing the -Xms argument altogether
I actually agree with this. Perhaps Luis Majano can comment (when he's back from vacation) on why he set the minimum as well. My reasoning for this would simply be because CommandBox is a development tool and people don't have unlimited RAM on their PCs so I don't want my dev servers taking up tons of RAM unless they actually need it. I only set the min equal to the max on production servers where RAM is dedicated and I don't want the JVM to waste time asking the OS for more as it warms up.

Agreed.

Also, for what it's worth, 256MB seemed to be the out-of-the-box default for Java processes, so that's why we chose it as the default here. In theory, the CommandBox server should be behaving the same before and after the "heapSize" argument addition (with the exception of the full amount being allocated at startup due to setting the min value). We've also never set a permgen size, but we probably should since CF engines use a lot of perm space (pre 1.8, of course).

256 is pretty low these days, IMHO, but having the heapSize argument allows me to work around that (though I would like to see a box.json/server.json config option). When I tried to start up a fairly straightforward CBORM app with a lot of relationships, I couldn't even get the app to load to the default login page.

If you feel this is worth pursuing, I'll downgrade my server settings and post some stack traces.