DeleteOnExitHook

严格来说，DeleteOnExitHook 导致的内存占用不应该归结为内存泄漏，开发者应该恰当地使用 java.io.File#deleteOnExit 方法，如果持续对不同的文件调用该方法，那么长时间运行的 JVM 实例的内存最终将被填满。为了规避这个问题，建议应用程序自行进行临时文件管理，而不是依赖于 JVM 退出来进行相应的文件清理操作。

如果通过 Google 搜索 java.io.DeleteOnExitHook memory leak，可知有不少开源项目存在该问题，主要受影响的为长时间运行的服务，java.io.DeleteOnExitHook#files 占用的内存往往在数百兆以上而且随着服务的持续运行会持续增长。

如 HIVE-11768 中提到的 .pipeout 文件问题，其解决方案为自行编写了 ShutdownHookManager 类，该类相比 JDK 提供的 DeleteOnExitHook 新增了 cancelDeleteOnExit 方法，且应用自行管理这部分临时文件的生命周期，不再使用后删除文件并调用 cancelDeleteOnExit 方法移除这部分文件名，以解决内存占用问题。本次 commit 可参考：HIVE-11768 : java.io.DeleteOnExitHook leaks memory on long running Hi… · apache/hive@1a81c26 · GitHub。

附上 JDK 8 中 DeleteOnExitHook 的源码：

/**
 * This class holds a set of filenames to be deleted on VM exit through a shutdown hook.
 * A set is used both to prevent double-insertion of the same file as well as offer
 * quick removal.
 */
class DeleteOnExitHook {
    private static LinkedHashSet<String> files = new LinkedHashSet<>();
    static {
        // DeleteOnExitHook must be the last shutdown hook to be invoked.
        // Application shutdown hooks may add the first file to the
        // delete on exit list and cause the DeleteOnExitHook to be
        // registered during shutdown in progress. So set the
        // registerShutdownInProgress parameter to true.
        sun.misc.SharedSecrets.getJavaLangAccess()
            .registerShutdownHook(2 /* Shutdown hook invocation order */,
                true /* register even if shutdown in progress */,
                new Runnable() {
                    public void run() {
                       runHooks();
                    }
                }
        );
    }

    private DeleteOnExitHook() {}

    static synchronized void add(String file) {
        if(files == null) {
            // DeleteOnExitHook is running. Too late to add a file
            throw new IllegalStateException("Shutdown in progress");
        }

        files.add(file);
    }

    static void runHooks() {
        LinkedHashSet<String> theFiles;

        synchronized (DeleteOnExitHook.class) {
            theFiles = files;
            files = null;
        }

        ArrayList<String> toBeDeleted = new ArrayList<>(theFiles);

        // reverse the list to maintain previous jdk deletion order.
        // Last in first deleted.
        Collections.reverse(toBeDeleted);
        for (String filename : toBeDeleted) {
            (new File(filename)).delete();
        }
    }
}

Reference

jdk/DeleteOnExitHook.java at jdk8-b120 · openjdk/jdk · GitHub
Memory Leak on DeleteOnExitHook - Stack Overflow
JDK-4513817 File.deleteOnExit consumes memory - Java Bug System
SPARK-14261 Memory leak in Spark Thrift Server - ASF JIRA