Poison


  • 首页

  • 归档

  • 标签

  • 搜索
close
Poison

RateLimiter

发表于 2021-11-03

我之前曾使用过阿里云离线版的 IP 地理位置库,在该 SDK 中,使用了 RateLimiter 去对用户的调用速率进行限制,记得上限为 15w,早期的版本采用了 tryAcquire 方法去尝试获取许可,即使用的非阻塞版本,该问题导致我们集成至 Spark 集群后,在进行离线计算时因为超过 QPS 上限使任务失败,后面向他们反馈该问题后,他们将限速实现调整为了基于 acquire 方法的阻塞版本,在限速的基础上支持了离线计算环境下的正常运行,本文简要记录 RateLimiter 的限流实现机制。

首先根据 RateLimiter 的官方文档我们知道,RateLimiter 支持以可配置的速率分发许可,它支持并发调用,且将限制来自所有线程的总调用率,但是不保证公平性。

阅读全文 »
Poison

Decorator

发表于 2021-10-31

之前在编写基于 ScheduledExecutorService (Java Platform SE 8 ) 的定时任务处理逻辑时,发现若任务出现异常,将不会被再调度执行,其文档中也有如下说明:

If any execution of the task encounters an exception, subsequent executions are suppressed.

比如如下代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
package me.tianshuang;

import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class ScheduledExecutorServiceTest {

public static void main(String[] args) {
Executors.newSingleThreadScheduledExecutor()
.scheduleAtFixedRate(() -> System.out.println("I'm running..."), 1, 1, TimeUnit.SECONDS);
}

}
阅读全文 »
Poison

Isolated Agent Classloader

发表于 2021-10-30

关于 Java Agent 为何需要做类加载隔离,我在实际开发 Java Agent 之前是不清楚的,直到业务需要将 Java Agent 用于应用监控,在开发过程中,对整个类加载器层次及类隔离有了更深入的理解,本文简要记录。

在早期我们用于监控的 Java Agent 的实现中,是没有做类加载隔离的,因为起初的 Java Agent 实现非常简单,仅仅是监控是否有堆转储文件产生,然后触发告警,此时 Java Agent 没有任何依赖。随着业务发展,越来越多的依赖加入至 Java Agent 后,我们发现集成至 JVM 应用后,会触发各种关于类加载的异常,如:X cannot be cast to X exceptions。

阅读全文 »
Poison

Executor of Tomcat

发表于 2021-10-19

首先我们看看 JDK 中 java.util.concurrent.ThreadPoolExecutor 提交任务的实现:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
/**
* Executes the given task sometime in the future. The task
* may execute in a new thread or in an existing pooled thread.
*
* If the task cannot be submitted for execution, either because this
* executor has been shutdown or because its capacity has been reached,
* the task is handled by the current {@code RejectedExecutionHandler}.
*
* @param command the task to execute
* @throws RejectedExecutionException at discretion of
* {@code RejectedExecutionHandler}, if the task
* cannot be accepted for execution
* @throws NullPointerException if {@code command} is null
*/
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reject(command);
}

根据以上代码及注释,我们知道,在 JDK 的线程池实现中,当线程池中的 worker 数量小于 corePoolSize 时,会尝试创建 worker 并执行任务,而如果线程池中的 worker 数量大于等于 corePoolSize 时,会尝试将任务放入队列,仅当放入队列失败时才会尝试创建 worker 并执行任务。那么可以理解为 JDK 中的实现偏向于尽量少创建线程,优先放入队列,更加适合于 CPU 密集型的任务。

阅读全文 »
Poison

Thread.getId()

发表于 2021-10-18

记录该方法是因为我在开发基于 Java Agent 的监控时,根据物理机的线程 id 来匹配 JVM 线程转储中的线程时,发现 JVM 中的线程 id 并不等于物理机上的线程 id,本文简要记录。

首先与 Thread.getId() 相关的源码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
/*
* Thread ID
*/
private long tid;

/**
* Returns the identifier of this Thread. The thread ID is a positive
* <tt>long</tt> number generated when this thread was created.
* The thread ID is unique and remains unchanged during its lifetime.
* When a thread is terminated, this thread ID may be reused.
*
* @return this thread's ID.
* @since 1.5
*/
public long getId() {
return tid;
}

/* For generating thread ID */
private static long threadSeqNumber;

private static synchronized long nextThreadID() {
return ++threadSeqNumber;
}

/**
* Initializes a Thread.
*
* @param g the Thread group
* @param target the object whose run() method gets called
* @param name the name of the new Thread
* @param stackSize the desired stack size for the new thread, or
* zero to indicate that this parameter is to be ignored.
* @param acc the AccessControlContext to inherit, or
* AccessController.getContext() if null
* @param inheritThreadLocals if {@code true}, inherit initial values for
* inheritable thread-locals from the constructing thread
*/
private void init(ThreadGroup g, Runnable target, String name,
long stackSize, AccessControlContext acc,
boolean inheritThreadLocals) {
if (name == null) {
throw new NullPointerException("name cannot be null");
}

this.name = name;

Thread parent = currentThread();
SecurityManager security = System.getSecurityManager();
if (g == null) {
/* Determine if it's an applet or not */

/* If there is a security manager, ask the security manager
what to do. */
if (security != null) {
g = security.getThreadGroup();
}

/* If the security doesn't have a strong opinion of the matter
use the parent thread group. */
if (g == null) {
g = parent.getThreadGroup();
}
}

/* checkAccess regardless of whether or not threadgroup is
explicitly passed in. */
g.checkAccess();

/*
* Do we have the required permissions?
*/
if (security != null) {
if (isCCLOverridden(getClass())) {
security.checkPermission(SUBCLASS_IMPLEMENTATION_PERMISSION);
}
}

g.addUnstarted();

this.group = g;
this.daemon = parent.isDaemon();
this.priority = parent.getPriority();
if (security == null || isCCLOverridden(parent.getClass()))
this.contextClassLoader = parent.getContextClassLoader();
else
this.contextClassLoader = parent.contextClassLoader;
this.inheritedAccessControlContext =
acc != null ? acc : AccessController.getContext();
this.target = target;
setPriority(priority);
if (inheritThreadLocals && parent.inheritableThreadLocals != null)
this.inheritableThreadLocals =
ThreadLocal.createInheritedMap(parent.inheritableThreadLocals);
/* Stash the specified stack size in case the VM cares */
this.stackSize = stackSize;

/* Set thread ID */
tid = nextThreadID();
}

根据以上源码,可以看出,Java 内部的线程 id 与底层操作系统的线程 id 无关,Java 中在每次创建线程时使用同步方法 nextThreadID 对静态变量 threadSeqNumber 加一的值作为新生成的线程的 id,对于底层平台的线程 id,我们在线程转储中可以看到,其中 nid=0x7dc8 就是使用十六进制表示的底层平台的线程 id。参见:find out the correspondence between the tid/nid of Java threads as shown from jstack/JMX, on HotSpot/Linux · GitHub。

我们利用以上映射关系,实现了基于 top 命令的高 CPU 使用率 JVM 线程定位。

1…121314…26

130 日志
119 标签
GitHub LeetCode
© 2025 Poison 蜀ICP备16000644号
由 Hexo 强力驱动
主题 - NexT.Mist