How to know about OutOfMemory or StackOverflow errors ahead of time

How to know about OutOfMemory or StackOverflow errors ahead of time

飘然心甜 发布于 2021-11-26 字数 764 浏览 816 回复 11 原文

In Java, is there a way to know that a StackOverflow error or OutOfMemory exception may happen soon?

The OutOfMemory exception might be an easier one to catch, if one is capable of getting memory usage statistics programmatically, and if one knows ahead of time how much memory needs to be used before the OutOfMemory exception is thrown. But are those values knowable?

For the StackOverflow error, is there a way to get recursion depth, and how does one know what value for recursion depth would cause the error to occur?

By knowing ahead of time whether these errors will happen, I feel I can recover the application more gracefully, instead of watching it crash.

如果你对这篇文章有疑问,欢迎到本站 社区 发帖提问或使用手Q扫描下方二维码加群参与讨论,获取更多帮助。

扫码加入群聊

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(11

柠檬 2022-06-07 11 楼

Implementation example using MemoryPoolMXBean

import static java.lang.management.ManagementFactory.getMemoryMXBean;
import static java.lang.management.ManagementFactory.getMemoryPoolMXBeans;
import static java.lang.management.MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED;
import static java.lang.management.MemoryType.HEAP;
import java.lang.management.MemoryPoolMXBean;
import java.lang.management.MemoryUsage;
import java.util.Collection;
import java.util.concurrent.CopyOnWriteArrayList;
import javax.management.Notification;
import javax.management.NotificationEmitter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MemoryWatcher {

    public interface Listener {
        void memoryUsageLow(long usedMemory, long maxMemory);
    }

    private static final MemoryPoolMXBean tenuredGenPool = findTenuredGenPool();
    private static final Logger logger = LoggerFactory.getLogger(MemoryWatcher.class);

    private static MemoryPoolMXBean findTenuredGenPool() {
        for (MemoryPoolMXBean pool : getMemoryPoolMXBeans())
            if (pool.getType() == HEAP && pool.isUsageThresholdSupported())
                return pool;
        return null;
    }

    public static MemoryPoolMXBean getTenuredGenPool() {
        return tenuredGenPool;
    }

    private final Collection<Listener> listeners = new CopyOnWriteArrayList<>();

    public MemoryWatcher(double usageThresholdPercent) {
        if (tenuredGenPool == null) {
            logger.warn("Tenured pool is not used");
            return;
        }
        if (tenuredGenPool.getUsageThreshold() != 0)
            logger.warn("Overriding tenured usage threshold {} with {}", tenuredGenPool.getUsage().getMax() / (double) tenuredGenPool.getUsageThreshold(), usageThresholdPercent);
        tenuredGenPool.setUsageThreshold((long) (tenuredGenPool.getUsage().getMax() * usageThresholdPercent));

        NotificationEmitter emitter = (NotificationEmitter) getMemoryMXBean();
        emitter.addNotificationListener((Notification n, Object hb) -> {
            MemoryUsage usage = tenuredGenPool.getUsage();
            if (n.getType().equals(MEMORY_THRESHOLD_EXCEEDED) && usage.getMax() == usage.getCommitted())
                listeners.forEach(listener -> listener.memoryUsageLow(usage.getUsed(), usage.getMax()));
        }, null, null);
    }

    public boolean addListener(Listener listener) {
        return listeners.add(listener);
    }

    public boolean removeListener(Listener listener) {
        return listeners.remove(listener);
    }
}
天邊彩虹 2022-06-07 10 楼

For StackOverflowError:

To know the current depth, usually it's either:

  1. using a stateful function (storing the depth in outside the function)
  2. using an accumulator (passing the depth as an argument to the function)

Knowing the depth it will occur is difficult. There are several factors:

  1. The stack space allocated to the JVM (you can change this with the -Xss option)
  2. The amount of stack space already used
  3. The amount used by the current function.

Why not try it out using something like this?

public static void main(String[] args) {
    try {
        recurs();
    } catch (Throwable t) {
        // not a good idea in production code....
    }
}
static int depth = 0;
static void recurs() {
    System.out.println(depth++);
    recurs();
}

Run it several times. Also try adding dummy variables. It can be seen that even the same code may halt at different depths and adding more variables cause it to end earlier. So yeah, pretty much it's unpredictable.

I suppose that besides rewriting the algorithm the only option would be to increase the stack space with the -Xss option.

For OutOfMemoryError, there's the -Xmx option

夏九 2022-06-07 9 楼

You could discover a lot about recursion depth, by creating a Throwable object, and querying its getStackTrace() method. But this is expensive to do.

If you really have a method with a small potential of throwing a StackOverflowError or an OutOfMemoryError why not just insert try-catch blocks and catch those errors? They can be caught and handled just like checked exceptions.

夜光 2022-06-07 8 楼

I don't know anything about working this out at run time, or what you might be able to do to avoid it once you predict it is going to happen. Better to try and avoid it occurring in the first place.

1) You could use Findbugs which may indicate some StackOverFlow errors occurring from inadvertently calling the same method from itself.

2) You could store data likely to cause you to run out of memory with a SoftReference and have a null check on accessing it so it can be reloaded if it's been garbage collected.

If either of these things are actually issues for you then the solution probably isn't in detecting it happening but architecting your solution differently to avoid them occuring if at all possible.

鱼忆七猫命九 2022-06-07 7 楼

The MemoryMXBean can emit notifications if your memory reaches a certain threshold.

沦落红尘 2022-06-07 6 楼

One useful thing you can do is use SoftReferences for caches. That will give you a gradual performance slide as you run out of memory. Just don't use WeakReference even in a WeakHashMap because it will annoy me when your application dies nastily on me.

吃颗糖壮壮胆 2022-06-07 5 楼

Usually memory useage is hard to predict.

Stack Overflows from infinite regression generally show up in the write/debug cycle.

Harder to catch are memory issues on things like large held collections, caches, etc.etc. As has been pointed out you can check runtime free and max memory, but be careful, as their meanings aren't obvious - "free" here, means "free now", e.g. meaning if you ran a full gc you might get more. Unfortunately there's no way to get the "total possible free, including garbage-collectible" without running System.gc(), which is not a nice thing to do on a production application (where you're liable to have large enough data sets to cause the problem in the first place) because the entire JVM will come to a screeching halt for a few seconds (or more, in a large app). Note that even System.gc() is not guaranteed to run "now", but my experience has been that it has whenever I've played with it.

You can print gc activity from the running jvm by starting java with -verbose:gc, -XX:+PrintGCTimeStamps, and -XX:+PrintGCDetails (more detail here), and in general if the collector starts to run more frequently, it's probably a sign that you're running out of memory.

仅此而已 2022-06-07 4 楼

You should never see a StackOverflow exception if your application is designed and implemented correctly!

Generally, if you get a StackOverflow exception then it's a sign that there's a bug in your recursion code.

挽你眉间 2022-06-07 3 楼

Most StackOverflow errors come out of bad recursion. Unfortunately, the problem of determining if a recursion will stop is generally not decidable (this is a central concept in CS). There are cases, however, you could get warnings, for example some IDEs will let you know if you're invoking a function recursively with no parameters.

流绪微梦 2022-06-07 2 楼

You can anticipate out-of-memory conditions with Runtime.freeMemory() and Runtime.maxMemory(). Most times it'll be hard recovering gracefully, but I leave that to you.

沉睡月亮 2022-06-07 1 楼

Anticipating Out of Memory Errors

I'm surprised I didn't see this mentioned in the other posts, but you can use ManagementFactory in Java 5/6 to get at a lot of the memory usage information.

Look at the platform mbean server page for more information on detecting low memory conditions in Java. I believe you can setup notifiers to call code when memory usage reaches a certain threshold.