Poison

发表于

今天帮同事查了个问题，在此简单记录。起因是同事在本地开发一段时间后会触发这样的异常：

1
2
3
4
5
2022-09-05 18:10:28.172 ERROR - [nio-8079-exec-2] o.a.c.c.C.[.[.[.[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.ClassCastException: me.tianshuang.dto.BizDTO cannot be cast to me.tianshuang.dto.BizDTO] with root cause

java.lang.ClassCastException: me.tianshuang.dto.BizDTO cannot be cast to me.tianshuang.dto.BizDTO
    at net.sf.cglib.empty.Object$$BeanCopierByCGLIB$$ac6a32c6.copy(<generated>) ~[na:na]
    at me.tianshuang.bean.BeanCopy.copy(BeanCopy.java:51) ~[commons-tool-1.0-SNAPSHOT.jar:na]
阅读全文 »
Poison

发表于

今天排查了个内存申请率较高的问题，单个节点上 Allocation Rate 为 700MB/s，Promoted Rate 仅 300KB/s，而通过查看该组应用的历史监控，发现 Allocation Rate 之前从未超过 200MB/s。自最近一个版本发布后，Allocation Rate 翻了几倍，通过 async-profiler 对内存申请进行分析发现为业务中一段类型转换的代码导致，即 String.valueOf(long l) 方法，最后通过优化业务代码实现解决该问题。

2022-09-20

类似地，今天查了个由业务中滥用 RequestMapping 中的正则匹配导致内存申请率高的问题，此时不能直接通过请求的 URL 直接查询出关联的 HandlerMethod，而需要遍历所有的 mapping 去尝试正则匹配，导致内存申请率高，相关源码位于 AbstractHandlerMethodMapping.java at v3.2.18:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
/**
 * Look up the best-matching handler method for the current request.
 * If multiple matches are found, the best match is selected.
 * @param lookupPath mapping lookup path within the current servlet mapping
 * @param request the current request
 * @return the best-matching handler method, or {@code null} if no match
 * @see #handleMatch(Object, String, HttpServletRequest)
 * @see #handleNoMatch(Set, String, HttpServletRequest)
 */
protected HandlerMethod lookupHandlerMethod(String lookupPath, HttpServletRequest request) throws Exception {
  List<Match> matches = new ArrayList<Match>();
  List<T> directPathMatches = this.urlMap.get(lookupPath);
  if (directPathMatches != null) {
    addMatchingMappings(directPathMatches, matches, request);
  }
  if (matches.isEmpty()) {
    // No choice but to go through all mappings...
    addMatchingMappings(this.handlerMethods.keySet(), matches, request);
  }

  if (!matches.isEmpty()) {
    // ...
    return bestMatch.handlerMethod;
  }
  else {
    return handleNoMatch(handlerMethods.keySet(), lookupPath, request);
  }
}

private void addMatchingMappings(Collection<T> mappings, List<Match> matches, HttpServletRequest request) {
  for (T mapping : mappings) {
    T match = getMatchingMapping(mapping, request);
    if (match != null) {
      matches.add(new Match(match, this.handlerMethods.get(mapping)));
    }
  }
}
阅读全文 »
Poison

发表于

今天看了个因字符串拼接导致的 CPU 高的问题，首先是监控 Agent 持续对一个实例进行告警，并抓取到了 CPU 高的相关 Java 线程：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
lwpId: 17, CPU usage: 52.9
"G1 Concurrent Refinement Thread#0" os_prio=0 tid=0x00007fa73002e000 nid=0x11 runnable 

lwpId: 198, CPU usage: 47.1
"http-nio-80-exec-42" #188 daemon prio=5 os_prio=0 tid=0x00007fa71409e800 nid=0xc6 runnable [0x00007fa6c942b000]
   java.lang.Thread.State: RUNNABLE
  at java.util.Arrays.copyOf(Arrays.java:3332)
  at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
  at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
  at java.lang.StringBuilder.append(StringBuilder.java:136)
  at me.tianshuang.tool.StringUtil.listToString(StringUtil.java:433)
  at sun.reflect.GeneratedMethodAccessor1008.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
阅读全文 »
Poison

发表于

今天帮同事查了个参数丢失的问题，确认是由参数大小超过 Tomcat 默认限制引起，源码位于 tomcat/Request.java:

1
2
3
4
5
6
7
8
9
10
if ((maxPostSize >= 0) && (len > maxPostSize)) {
    Context context = getContext();
    if (context != null && context.getLogger().isDebugEnabled()) {
        context.getLogger().debug(
                sm.getString("coyoteRequest.postTooLarge"));
    }
    checkSwallowInput();
    parameters.setParseFailedReason(FailReason.POST_TOO_LARGE);
    return;
}
阅读全文 »
Poison

发表于

OpenSSH 8.8 存在不兼容的改动，导致 macOS 升级至 Ventura 后因 macOS 上的 OpenSSH 版本升级至了 9.0 后当连接的服务端 OpenSSH 版本为低版本（如：6.6.1）时 SSH 连接时无可互用的签名算法导致提示输入密码（服务端允许密码登录时）或者直接提示 Permission denied, please try again.（服务端不允许密码登录时）。如果使用 verbose 模式查看 ssh 时的 debug 信息，可以发现如下输出：

1
2
3
4
5
6
7
8
debug1: send_pubkey_test: no mutual signature algorithm
debug1: Trying private key: /Users/Poison/.ssh/id_ecdsa
debug1: Trying private key: /Users/Poison/.ssh/id_ecdsa_sk
debug1: Trying private key: /Users/Poison/.ssh/id_ed25519
debug1: Trying private key: /Users/Poison/.ssh/id_ed25519_sk
debug1: Trying private key: /Users/Poison/.ssh/id_xmss
debug1: Trying private key: /Users/Poison/.ssh/id_dsa
debug1: Next authentication method: password

建议的解决方案为升级服务端的 OpenSSH 版本，如果实在无法升级，则推荐修改客户端 ssh 配置临时启用使用 SHA-1 哈希算法的 RSA 签名。OpenSSH 8.8 Release 文档中关于此部分的描述如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Potentially-incompatible changes
================================

This release disables RSA signatures using the SHA-1 hash algorithm
by default. This change has been made as the SHA-1 hash algorithm is
cryptographically broken, and it is possible to create chosen-prefix
hash collisions for <USD$50K [1]

For most users, this change should be invisible and there is
no need to replace ssh-rsa keys. OpenSSH has supported RFC8332
RSA/SHA-256/512 signatures since release 7.2 and existing ssh-rsa keys
will automatically use the stronger algorithm where possible.

Incompatibility is more likely when connecting to older SSH
implementations that have not been upgraded or have not closely tracked
improvements in the SSH protocol. For these cases, it may be necessary
to selectively re-enable RSA/SHA1 to allow connection and/or user
authentication via the HostkeyAlgorithms and PubkeyAcceptedAlgorithms
options. For example, the following stanza in ~/.ssh/config will enable
RSA/SHA1 for host and user authentication for a single destination host:

    Host old-host
        HostkeyAlgorithms +ssh-rsa
        PubkeyAcceptedAlgorithms +ssh-rsa

We recommend enabling RSA/SHA1 only as a stopgap measure until legacy
implementations can be upgraded or reconfigured with another key type
(such as ECDSA or Ed25519).

[1] "SHA-1 is a Shambles: First Chosen-Prefix Collision on SHA-1 and
    Application to the PGP Web of Trust" Leurent, G and Peyrin, T
    (2020) https://eprint.iacr.org/2020/014.pdf
Reference

OpenSSH - release 8.8
ssh(1) - Linux manual page