Http2Stream.waitForIo()

同事反馈有一组应用的定时任务不执行了，简单看了下，该应用未定义过基于 TaskScheduler 或 ScheduledExecutorService 实现的 Bean，此时 Spring 会在 ScheduledTaskRegistrar.scheduleTasks() 中创建一个单线程的任务调度器：

/**
 * Schedule all registered tasks against the underlying {@linkplain
 * #setTaskScheduler(TaskScheduler) task scheduler}.
 */
protected void scheduleTasks() {
  if (this.taskScheduler == null) {
    this.localExecutor = Executors.newSingleThreadScheduledExecutor();
    this.taskScheduler = new ConcurrentTaskScheduler(this.localExecutor);
  }
  if (this.triggerTasks != null) {
    for (TriggerTask task : this.triggerTasks) {
      addScheduledTask(scheduleTriggerTask(task));
    }
  }
  if (this.cronTasks != null) {
    for (CronTask task : this.cronTasks) {
      addScheduledTask(scheduleCronTask(task));
    }
  }
  if (this.fixedRateTasks != null) {
    for (IntervalTask task : this.fixedRateTasks) {
      addScheduledTask(scheduleFixedRateTask(task));
    }
  }
  if (this.fixedDelayTasks != null) {
    for (IntervalTask task : this.fixedDelayTasks) {
      addScheduledTask(scheduleFixedDelayTask(task));
    }
  }
}

正如 EnableScheduling (Spring Framework 5.3.14 API) 中提到的这一段逻辑：

By default, Spring will search for an associated scheduler definition: either a unique TaskScheduler bean in the context, or a TaskScheduler bean named “taskScheduler” otherwise; the same lookup will also be performed for a ScheduledExecutorService bean. If neither of the two is resolvable, a local single-threaded default scheduler will be created and used within the registrar.

在出问题的应用中，即采用的默认创建的基于单个线程的定时任务调度器，而根据定时任务线程池的原理，只有一个线程时，当前线程要处理完当前任务后才会去队列中检查下一个任务。我对该应用拉取了一次栈帧，发现负责定时任务的线程的栈帧如下：

java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at okhttp3.internal.http2.Http2Stream.waitForIo(Http2Stream.java:645)
    at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:151)
    - locked <0x00000006f12faee8> (a okhttp3.internal.http2.Http2Stream)
    at okhttp3.internal.http2.Http2ExchangeCodec.readResponseHeaders(Http2ExchangeCodec.java:136)
    at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:115)
    at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
    at okhttp3.RealCall.execute(RealCall.java:81)

可以看出在 Http2Stream.waitForIo() 方法中调用 Object.wait() 后一直处于等待状态了，关于该问题，至少有两三个 issue 反馈过，由于并不好复现，所以一直没有得到解决，且读取 header 部分不受读取超时时间的控制。对于该问题，我们对调用增加了全局超时时间来避免一直卡在此处。同时配置了多线程的定时任务调度器以避免单线程的定时任务调度器执行任务卡住时导致其他定时任务未被触发执行。

关于 Http2Stream.waitForIo() 卡住的问题，作者的最新回复如下：

No clear answer, but it’s likely to be one request being starved on an otherwise healthy shared connection. So either the request is not getting a response from the server, or the request is “head of line” blocked by another request that is saturating the socket.

Reference

OkHttp3 block on Http2Stream.waitForIo() - Stack Overflow
HTTP/2 connection coalescing | daniel.haxx.se
HTTP: Optimizing Application Delivery - High Performance BrowserNetworking (O’Reilly)