From b7d6497ca8f971d2cbe1cadca17f6c39b4be9f30 Mon Sep 17 00:00:00 2001 From: Layue13 Date: Fri, 23 Oct 2026 16:40:48 +0800 Subject: [PATCH] =?UTF-8?q?perf:=20WebSocket=20=E7=BB=8F=20CDN=EF=BC=9Awss?= =?UTF-8?q?=20=E6=94=B9=E7=94=A8=20BoringSSL=E3=80=81=E4=BF=9D=E6=B4=BB=20?= =?UTF-8?q?Ping=E3=80=81=E6=95=B4=E5=90=88=E5=8C=85=E5=BD=A2=E6=80=81?= =?UTF-8?q?=E5=9F=BA=E5=87=86?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - ProxySessionBenchmark 新增 --profile modded（仿大型整合包的双向包组与 2 MiB 登录突发，逐字节校验）、 --websocket-tls 1（keytool 一次性证书的 wss）和 --client-coalesce 1。 - wss 监听改用随包附带的 BoringSSL（netty-tcnative-boringssl-static 2.0.84）：TLS 占 wss 代理 I/O CPU 的 55%， 换用后每帧 CPU 在每一对配对测量中都更低（窗口 1 -22%，窗口 8 -36%），帧率 +31%/+34%；其他平台回落 JDK， 启动日志标明所用实现。jar 由 10.4 MB 增至 16.5 MB。 - 新增 websocket.pingIntervalSeconds（默认 30，0 关闭）：升级后代理静默不超过该时长，期间无写出就发空 Ping， 避免 CDN/负载均衡在安静的登录、选服阶段按空闲超时断开。IdleStateHandler 版本在窗口 8 多耗 7.6% CPU， 未达规则；改为写时置位标志、每半个间隔检查的实现，JFR 中几乎不可见。 - 文档：WebSocket 接入新增“经 CDN 部署”，更正消息大小描述；记录见 testing/benchmarks/results/2026-10-23-websocket-cdn.md。 Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 2 +- docs/operations.md | 2 +- docs/testing.md | 4 + docs/websocket.md | 19 +- gradle/libs.versions.toml | 2 + proxy/core/build.gradle.kts | 4 + .../moonbridge/app/ProxyConfiguration.java | 7 +- .../app/ProxyConfigurationLoader.java | 3 +- .../java/dev/moonbridge/app/ProxyMain.java | 6 +- .../core/websocket/WebSocketFrontend.java | 21 +- .../core/websocket/WebSocketKeepalive.java | 59 ++ .../app/ProxyConfigurationTest.java | 8 + .../core/session/WebSocketListenerTest.java | 49 +- .../websocket/WebSocketKeepaliveTest.java | 84 +++ testing/benchmarks/ProxySessionBenchmark.java | 387 ++++++++++- .../results/2026-10-23-websocket-cdn-raw.txt | 642 ++++++++++++++++++ .../results/2026-10-23-websocket-cdn.md | 139 ++++ 17 files changed, 1383 insertions(+), 55 deletions(-) create mode 100644 proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketKeepalive.java create mode 100644 proxy/core/src/test/java/dev/moonbridge/core/websocket/WebSocketKeepaliveTest.java create mode 100644 testing/benchmarks/results/2026-10-23-websocket-cdn-raw.txt create mode 100644 testing/benchmarks/results/2026-10-23-websocket-cdn.md diff --git a/CLAUDE.md b/CLAUDE.md index 68c84df..d2b74e1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -49,7 +49,7 @@ testing/smoke/ smoke scripts and dated records; testing/benchmarks/ likewis **Session threading.** `ProxySessionListener` owns the listener, event loops and the UUID→session index. Each `Session` (one per player connection) confines its mutable state to the frontend event loop, and its backend channels are created on that same loop (`SessionChannels.backendBootstrap` enforces it). Login verification, backend DNS and plugin callbacks run off that loop and re-enter it by posting a task; results are checked against the current connection identity so a stale async result cannot act on a reconnected player. Plugin work goes through bounded pools that reject when full. -**Session layout.** `core.session` holds `Session` (the Netty handler and lifecycle owner) plus the pieces it delegates to: `FrontendLogin` (handshake, status, encryption, Mojang verification), `InitialRouter` (placement decision and first backend dial), `BackendHandshakes`. `core.session.transfer` is `TransferCoordinator`, which reaches the session only through the `TransferHost` port (Session supplies it as an adapter) and owns all transfer state. `core.session.play` has the PLAY data path (`RawRelay`, the per-frame handlers, relay start-up buffering); `core.session.channel` has `NetworkTransport`, backend dialing, pipeline lookups and `clientAddress` (the player IP: a trusted forwarded address or the socket peer; use it instead of `remoteAddress()`). `core.websocket` is the optional WebSocket frontend; after the upgrade the pipeline behind `WebSocketFrameAdapter` is the same as for TCP. Dependencies point one way: `session` -> `transfer` -> `play`/`channel`. +**Session layout.** `core.session` holds `Session` (the Netty handler and lifecycle owner) plus the pieces it delegates to: `FrontendLogin` (handshake, status, encryption, Mojang verification), `InitialRouter` (placement decision and first backend dial), `BackendHandshakes`. `core.session.transfer` is `TransferCoordinator`, which reaches the session only through the `TransferHost` port (Session supplies it as an adapter) and owns all transfer state. `core.session.play` has the PLAY data path (`RawRelay`, the per-frame handlers, relay start-up buffering); `core.session.channel` has `NetworkTransport`, backend dialing, pipeline lookups and `clientAddress` (the player IP: a trusted forwarded address or the socket peer; use it instead of `remoteAddress()`). `core.websocket` is the optional WebSocket frontend; after the upgrade the pipeline behind `WebSocketFrameAdapter` is the same as for TCP, and `WebSocketKeepalive` pings a connection the proxy has not written to for `pingIntervalSeconds` so CDN idle timers do not cut quiet logins. Dependencies point one way: `session` -> `transfer` -> `play`/`channel`. **Data path.** After login, `RawRelay` forwards already-framed PLAY frames between the two channels without copying. Per-frame concerns are separate inbound handlers (`KeepAliveBridge`, `PlayerCommandInterceptor`, `TabCompletionBridge`, `TransferFrameHandler`) in front of the relay. Backpressure works by pausing reads on the source when the peer is not writable; the relay flushes once per read batch (`channelReadComplete`). diff --git a/docs/operations.md b/docs/operations.md index 832d3d3..d20583f 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -62,7 +62,7 @@ status: - 对外服务使用 `ONLINE_BUNGEE`，并将后端限制为代理可达，禁止客户端绕过代理直连；后端需启用兼容的旧版 Bungee 身份转发。会话验证与加密路径有自动化测试，当前正版账号的真实 Mojang 会话验证、UUID 和皮肤签名转发也已完成端到端实测。 - 可选的 `websocket` 段在 TCP `listen` 之外再开一个 WebSocket 玩家监听，供经 CDN 接入的客户端使用；两种入口同时可用，见 [WebSocket 接入](websocket.md)。OFFLINE 模式对 `websocket.listen` 同样要求字面量回环 IP，除非设置 `allowOfflinePublicAccess: true`。 - `maxConnections` 默认 4096，范围为 1–1,000,000；计数包括登录中的客户端，以及尚未完成 WebSocket 升级的连接。达到上限时新连接直接关闭。按部署资源和连接规模配置，不将其解释为后端容量。 -- `transport` 选择玩家连接、后端连接和 DNS 使用的网络传输层，默认 `AUTO`：Linux 上用 epoll，macOS 上用 kqueue，其他平台（如 Windows）或原生库无法加载时（如 ARM/x86 以外的架构、musl 发行版）回退到 NIO，启动日志会输出实际选用的传输层。发行包附带 Linux x86_64/aarch64 的 epoll 与 macOS x86_64/aarch64 的 kqueue 原生库，启动脚本已加 `--enable-native-access=ALL-UNNAMED`。显式写 `NIO`、`EPOLL` 或 `KQUEUE` 会强制使用对应实现；指定的原生传输层不可用时启动失败，不会静默回退。后端控制通道使用独立的阻塞套接字，不受此项影响。 +- `transport` 选择玩家连接、后端连接和 DNS 使用的网络传输层，默认 `AUTO`：Linux 上用 epoll，macOS 上用 kqueue，其他平台（如 Windows）或原生库无法加载时（如 ARM/x86 以外的架构、musl 发行版）回退到 NIO，启动日志会输出实际选用的传输层。发行包附带 Linux x86_64/aarch64 的 epoll 与 macOS x86_64/aarch64 的 kqueue 原生库，以及供 `wss://` 监听使用的 BoringSSL（上述平台加 Windows x86_64，见 [WebSocket 接入](websocket.md)），启动脚本已加 `--enable-native-access=ALL-UNNAMED`。显式写 `NIO`、`EPOLL` 或 `KQUEUE` 会强制使用对应实现；指定的原生传输层不可用时启动失败，不会静默回退。后端控制通道使用独立的阻塞套接字，不受此项影响。 - 发行包默认关闭 Netty 的内存泄漏检测（`-Dio.netty.leakDetection.level=disabled`）：它在每个手写转发/编码路径上都有逐帧开销，合成基准中 burst 场景每帧 CPU 中位数约低 26%（范围有重叠，见[记录](../testing/benchmarks/results/2026-10-05-event-loop-queue-and-netty.md)）。代替它的是测试期的 `LeakGate`：`proxy-core` 的每个测试类都以 PARANOID 级别运行，发现未释放的缓冲区即让该类失败，且有自测证明它能抓到故意的泄漏。引用计数的手写位置已集中到 `ByteBufs`、`FrameTransformHandler` 与 Netty 自带的编解码基类里；已评估的手写 release 位置都有测试守护，范围与方法见[验证说明](testing.md)。这个默认值由 `ProxyMain` 在加载 Netty 前设置；需要在某个部署上重新启用时，启动时传入 `java -Dio.netty.leakDetection.level=simple -jar moonbridge.jar`，显式给出的值优先。 - 日志同时写到控制台和 `logs/latest.log`（按天滚动，保留 14 天）；`logging.playerAddresses: false` 让玩家事件不带 IP。各级别写什么、怎样只打开某一块的 DEBUG，见[日志策略](logging.md)。 - `plugins.eventTimeoutSeconds` 为统一事件链期限，默认 5 秒，可设为 1–30 秒。玩家准入事件按订阅注册顺序共享一个期限；无订阅者时跳过派发。准入队列容量为 128，最多 1024 个未决请求。队列饱和、超时、异常或空结果都会拒绝当前访问：连接准入阶段关闭 TCP 连接，玩家准入阶段返回通用错误文本；插件可通过显式拒绝结果提供登录断开原因。生命周期通知有独立的单线程 FIFO 队列，容量 128，最多一个事件正在派发；单事件链受同一事件期限约束。过载时尽力丢弃并聚合记录，不影响连接处理。 diff --git a/docs/testing.md b/docs/testing.md index 75ca9a3..773dff3 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -88,6 +88,10 @@ sh testing/smoke/container-channel.sh `--websocket 1`（直接运行 Java 时）让代理阶段的客户端通过 `ws://` 连接代理的 WebSocket 监听器：完成升级后，客户端每次 flush 发送一条带掩码的二进制消息，读取时按 RFC 6455 拆出服务器消息的负载。直接阶段仍用 TCP 连接同一个模拟后端，所以代理/直连之比混入了 WebSocket 客户端自身的开销；比较 WebSocket 路径时以代理 I/O 线程每帧 CPU 为准。[2026-10-22 WebSocket 合并记录](../testing/benchmarks/results/2026-10-22-websocket-coalescing.md)用它比较了逐帧消息与按 flush 合并。 +`--websocket-tls 1`（需同时给 `--websocket 1`）让代理的 WebSocket 监听器提供 `wss://`：启动时用 JDK 的 `keytool` 生成一次性 ECDSA P-256 自签名证书，代理端按运行时可用的提供者（BoringSSL 可加载时为 OpenSSL，否则为 JDK）终结 TLS，客户端用 JDK `SSLSocket`。加 `-Dio.netty.handler.ssl.noOpenSsl=true` 可强制代理使用 JDK 提供者。 + +`--profile modded` 把等长回显换成仿大型整合包的固定种子流量：每个往返是客户端一个 tick（模组自定义负载、位置包、着地包共 3 个包），后端回以该 tick 的下行包组（平均约 10 个包、6 KB：大量 20–600 字节的模组负载与方块实体更新，约每 3 个 tick 一个 2–8 KiB 负载，约每 20 个 tick 一个 30–60 KiB 区块）。每次 Position and Look 之后后端还发送约 2 MiB 的 30,000 字节负载，模拟 Forge 注册表同步；其耗时单独报告为 `login_burst_ms`，不计入计时。每个下行包都带 tick 序号并逐字节校验；帧数与每帧 CPU 计入双向。默认客户端每个包单独写出并 flush（与已发布的客户端 mod 一样每包一条消息），`--client-coalesce 1` 改为每个 tick 一次写出。[2026-10-23 CDN 场景记录](../testing/benchmarks/results/2026-10-23-websocket-cdn.md)用这两个模式测量了 ws/wss 基线、BoringSSL 与 JDK TLS 的对比和保活 Ping 的开销。 + 原 stop-and-wait 小样本见 `testing/benchmarks/results/2026-09-25-proxy-session-benchmark-smoke.md`；Window 参数的小样本和 Window=1/16 重复测量见 `testing/benchmarks/results/2026-09-25-proxy-session-window.md`。这些是环回网络上的合成帧对照，不能代表 Forge 整合包、真实后端或跨主机部署的性能。 ## 后端宿主真实服务端验收 diff --git a/docs/websocket.md b/docs/websocket.md index 01306fd..d6fcc0e 100644 --- a/docs/websocket.md +++ b/docs/websocket.md @@ -10,8 +10,9 @@ CDN 能否降低延迟或丢包，取决于玩家到 CDN 边缘、以及边缘 - 标准 RFC 6455 WebSocket。客户端向配置的 `path` 发送 HTTP/1.1 `GET` 升级请求，不需要子协议，也不协商扩展（不压缩）。 - 升级完成后，每条消息都是**二进制**帧，载荷是原始 Minecraft TCP 字节流的一段。这些字节与直连 TCP 时完全相同：带 VarInt 长度前缀，在线模式下还经过 Minecraft 自己的 AES/CFB8 加密。 -- 消息边界没有含义。一条消息可以只含半个 Minecraft 帧，也可以含多个帧；接收方必须按字节流拼接。代理对每次写出发送一条消息，单条消息载荷上限为一个 1.7.10 帧的最大值（2 MiB + 3 字节）。 +- 消息边界没有含义。一条消息可以只含半个 Minecraft 帧，也可以含多个帧；接收方必须按字节流拼接。代理把每次 flush 之前写出的数据合并为一条消息（单条上限 64 KiB，超过则拆成多条）；代理接收的单条消息载荷上限为一个 1.7.10 帧的最大值（2 MiB + 3 字节）。 - 收到文本帧视为协议错误，代理直接关闭连接。Ping 由代理回复 Pong；关闭帧关闭连接。代理关闭会话时先发送关闭帧。 +- 升级完成后，代理保证朝客户端方向的静默不超过 `pingIntervalSeconds`：期间没有数据时发送一个空 Ping，客户端须回复 Pong（RFC 6455 本就要求）。这让 CDN 和负载均衡在登录、选服等安静阶段也能看到流量，不会按空闲超时切断连接。 - 请求其他路径、非升级请求返回 `404` 并关闭。连接建立后，若在 `handshakeTimeoutSeconds` 内没有完成升级，代理关闭连接；尚未升级的连接同样占用 `maxConnections` 名额。 ## 配置 @@ -26,16 +27,18 @@ websocket: trustedProxies: ["127.0.0.1"] # 只有这些来源可以提供真实 IP 头 realIpHeader: "X-Forwarded-For" # 默认值；也可用 CF-Connecting-IP、X-Real-IP handshakeTimeoutSeconds: 10 # 1–60，默认 10 + pingIntervalSeconds: 30 # 0–300，默认 30；0 不发送 Ping ``` | 键 | 说明 | | --- | --- | | `listen` | 必填，`host:port`，端口须大于 0。`OFFLINE` 模式下与 TCP `listen` 相同，必须是字面量回环 IP，除非设置 `allowOfflinePublicAccess: true`。 | | `path` | 升级请求的路径，须以 `/` 开头，不含查询串和空白。 | -| `tls` | 可选。客户端 mod 只用 TLS 1.2（Minecraft 自带的 Netty 无法完成 TLS 1.3 握手），因此任何 TLS 终端都不能只开放 TLS 1.3。配置后监听端直接提供 `wss://`；PEM 证书链与**未加密的 PKCS#8** 私钥，路径相对于配置文件。PKCS#1 私钥（`BEGIN RSA PRIVATE KEY`）可用 `openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.pk8.pem` 转换。不配置时为明文 `ws://`，由 CDN 或反向代理终结 TLS。 | +| `tls` | 可选。客户端 mod 只用 TLS 1.2（Minecraft 自带的 Netty 无法完成 TLS 1.3 握手），因此任何 TLS 终端都不能只开放 TLS 1.3。配置后监听端直接提供 `wss://`；PEM 证书链与**未加密的 PKCS#8** 私钥，路径相对于配置文件。PKCS#1 私钥（`BEGIN RSA PRIVATE KEY`）可用 `openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.pk8.pem` 转换。不配置时为明文 `ws://`，由 CDN 或反向代理终结 TLS。发行包附带 BoringSSL（Linux x86_64/aarch64、macOS、Windows x86_64），监听端用它处理 TLS，CPU 开销明显低于 JDK 实现；其他平台回落到 JDK，启动日志的 `TLS via OPENSSL/JDK` 标明实际所用。 | | `trustedProxies` | IP 或 CIDR 列表。仅当 TCP 对端属于该列表时才读取真实 IP 头；为空时始终使用 TCP 对端地址。 | | `realIpHeader` | 真实 IP 头名称，大小写不敏感。`X-Forwarded-For` 从右往左取第一个不属于 `trustedProxies` 的地址（全部可信时取最左项）；其他头必须恰好出现一次且是 IP 字面量。可信来源发送了格式错误的头时拒绝连接，不会回退。 | | `handshakeTimeoutSeconds` | 从建连到升级完成的期限。 | +| `pingIntervalSeconds` | 代理朝客户端方向的静默不超过这个时长：每半个间隔检查一次，上半个间隔内没有写出就发送一个空 WebSocket Ping。须小于玩家与代理之间所有 CDN、负载均衡和反向代理的空闲超时；默认 30 秒低于常见的 60–100 秒。连接不可写时不发送（数据已在排队）。 | `--validate-config` 同样会加载并校验 TLS 证书和 `trustedProxies`。启动日志会打印 WebSocket 监听的地址与路径。 @@ -45,6 +48,17 @@ websocket: 经 WebSocket 接入的连接在升级完成后才触发 `ConnectionAdmissionEvent`，这时已经确定了真实地址。 +## 经 CDN 部署 + +以下适用于任何能转发 WebSocket 的 CDN 或负载均衡，Cloudflare 的具体设置见下一节示例。 + +- **空闲超时。** CDN 和负载均衡会切断一段时间没有流量的连接，常见值为 60–100 秒，也有更短的。PLAY 阶段后端的 Keep Alive 足以保持连接；大型整合包登录、等待选服或后端卡顿时则可能长时间没有数据，`pingIntervalSeconds` 用来覆盖这些阶段。若链路上某一跳的空闲超时短于 30 秒，把它调到更小的值。nginx 等自建反向代理的 `proxy_read_timeout` 也须大于该值。 +- **真实 IP。** 在 `trustedProxies` 中列出 CDN 公布的回源 IP 段，`realIpHeader` 设为 CDN 写入客户端地址的头（`X-Forwarded-For`、`CF-Connecting-IP`、`True-Client-IP` 等，以 CDN 文档为准）。 +- **回源走 ws 还是 wss。** CDN 到源站走 TLS（`tls` 配置或前置 nginx 终结）时，这一段的 TLS 开销落在源站；回源走明文 ws 时，须用防火墙确保只有 CDN 能连到该端口。客户端到 CDN 边缘始终建议用 `wss://`。 +- **TLS 版本。** 客户端 mod 只用 TLS 1.2，CDN 边缘不能设为仅 TLS 1.3。 +- **不压缩。** 代理和客户端都不协商 permessage-deflate：在线模式下数据在分帧前已经 AES 加密，无法压缩；1.7.10 的区块数据本身已是 zlib 压缩；客户端使用的 Netty 4.0.10 也没有该扩展。 +- **延迟。** 每个连接要多走客户端到边缘和边缘到源站两段，加上 TLS 与 HTTP 升级各一到两次往返；登录与 Forge 握手的往返次数由协议决定。CDN 是否改善体验取决于实际线路，应在玩家所在地实测。 + ## 部署示例 ### Cloudflare + 本机 nginx @@ -103,6 +117,7 @@ Cloudflare 侧： - 错误路径返回 404、文本帧关闭连接； - 未升级连接占用并归还 `maxConnections` 名额，以及升级期限； - 由配置加载 PEM 证书后提供 `wss://`。 +- 升级后空闲的连接收到代理的 Ping；有写出时推迟 Ping，不可写时不发送（`WebSocketKeepaliveTest`）。 另用客户端 mod 自己的管线（Netty 4.0.10，Java 8 与 21）对已安装的 MoonBridge 发行包实测过 `ws://` 与 `wss://` 的状态查询和登录阶段断开消息，以及错误路径的 404。 diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index d876635..85ba557 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,6 +1,7 @@ [versions] junit = "5.14.1" netty = "4.2.18.Final" +netty-tcnative = "2.0.84.Final" jackson = "2.20.0" slf4j = "2.0.18" logback = "1.6.3" @@ -20,6 +21,7 @@ netty-transport-classes-epoll = { module = "io.netty:netty-transport-classes-epo netty-transport-native-epoll = { module = "io.netty:netty-transport-native-epoll", version.ref = "netty" } netty-transport-classes-kqueue = { module = "io.netty:netty-transport-classes-kqueue", version.ref = "netty" } netty-transport-native-kqueue = { module = "io.netty:netty-transport-native-kqueue", version.ref = "netty" } +netty-tcnative-boringssl-static = { module = "io.netty:netty-tcnative-boringssl-static", version.ref = "netty-tcnative" } jackson-databind = { module = "com.fasterxml.jackson.core:jackson-databind", version.ref = "jackson" } jackson-yaml = { module = "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml", version.ref = "jackson" } slf4j-api = { module = "org.slf4j:slf4j-api", version.ref = "slf4j" } diff --git a/proxy/core/build.gradle.kts b/proxy/core/build.gradle.kts index bcb59db..2fc43eb 100644 --- a/proxy/core/build.gradle.kts +++ b/proxy/core/build.gradle.kts @@ -30,6 +30,10 @@ dependencies { for (classifier in listOf("osx-x86_64", "osx-aarch_64")) { runtimeOnly(variantOf(libs.findLibrary("netty-transport-native-kqueue").get()) { classifier(classifier) }) } + // BoringSSL for wss:// listeners; Netty uses it when it loads and falls back to the JDK provider otherwise. + for (classifier in listOf("linux-x86_64", "linux-aarch_64", "osx-x86_64", "osx-aarch_64", "windows-x86_64")) { + runtimeOnly(variantOf(libs.findLibrary("netty-tcnative-boringssl-static").get()) { classifier(classifier) }) + } implementation(libs.findLibrary("jackson-databind").get()) implementation(libs.findLibrary("jackson-yaml").get()) implementation(libs.findLibrary("adventure-gson").get()) diff --git a/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfiguration.java b/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfiguration.java index db82ed5..334b60a 100644 --- a/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfiguration.java +++ b/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfiguration.java @@ -131,7 +131,7 @@ public record ProxyConfiguration(String listen, Authentication authentication, L * The TCP {@code listen} keeps working; both share {@code maxConnections}. */ public record WebSocket(String listen, String path, Tls tls, List trustedProxies, String realIpHeader, - Integer handshakeTimeoutSeconds) { + Integer handshakeTimeoutSeconds, Integer pingIntervalSeconds) { public WebSocket { if (listen == null || listen.isBlank()) throw new IllegalArgumentException("websocket.listen is required"); if (parseAddress(listen).getPort() == 0) { @@ -147,6 +147,11 @@ public record ProxyConfiguration(String listen, Authentication authentication, L if (handshakeTimeoutSeconds < 1 || handshakeTimeoutSeconds > 60) { throw new IllegalArgumentException("websocket.handshakeTimeoutSeconds must be between 1 and 60"); } + // Below the common CDN and load-balancer idle limits (60-100 s); 0 turns the ping off. + pingIntervalSeconds = pingIntervalSeconds == null ? 30 : pingIntervalSeconds; + if (pingIntervalSeconds < 0 || pingIntervalSeconds > 300) { + throw new IllegalArgumentException("websocket.pingIntervalSeconds must be between 0 and 300"); + } } public InetSocketAddress listenAddress() { return resolve(parseAddress(listen)); } diff --git a/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfigurationLoader.java b/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfigurationLoader.java index 16d11d6..3f20e28 100644 --- a/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfigurationLoader.java +++ b/proxy/core/src/main/java/dev/moonbridge/app/ProxyConfigurationLoader.java @@ -57,7 +57,8 @@ public final class ProxyConfigurationLoader { } return new WebSocketFrontend.Settings(websocket.path(), tls, new ClientAddressPolicy(websocket.trustedProxies(), websocket.realIpHeader()), - Duration.ofSeconds(websocket.handshakeTimeoutSeconds())); + Duration.ofSeconds(websocket.handshakeTimeoutSeconds()), + Duration.ofSeconds(websocket.pingIntervalSeconds())); } private static String loadFavicon(Path configPath, String configuredPath) throws IOException { diff --git a/proxy/core/src/main/java/dev/moonbridge/app/ProxyMain.java b/proxy/core/src/main/java/dev/moonbridge/app/ProxyMain.java index 62a2ea7..c04095d 100644 --- a/proxy/core/src/main/java/dev/moonbridge/app/ProxyMain.java +++ b/proxy/core/src/main/java/dev/moonbridge/app/ProxyMain.java @@ -140,7 +140,11 @@ public final class ProxyMain { var listeners = new java.util.ArrayList(); listeners.add("TCP " + serverChannel.localAddress()); if (webSocket != null) { - LOGGER.info("Listening for WebSocket{} on {}{}", webSocket.tls() == null ? "" : " (TLS)", + // OPENSSL is the bundled BoringSSL; JDK means it could not load on this platform. + LOGGER.info("Listening for WebSocket{} on {}{}", webSocket.tls() == null ? "" + : " (TLS via " + (webSocket.tls() + instanceof io.netty.handler.ssl.ReferenceCountedOpenSslContext + ? "OPENSSL" : "JDK") + ")", listener.webSocketChannel().localAddress(), webSocket.path()); listeners.add((webSocket.tls() == null ? "ws " : "wss ") + listener.webSocketChannel().localAddress() + webSocket.path()); diff --git a/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketFrontend.java b/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketFrontend.java index 6945054..711e9bd 100644 --- a/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketFrontend.java +++ b/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketFrontend.java @@ -36,13 +36,20 @@ import java.util.function.BiConsumer; */ public final class WebSocketFrontend { public static final String FRAME_ADAPTER = "websocket-frames"; + static final String KEEPALIVE = "websocket-keepalive"; private static final Logger LOGGER = LoggerFactory.getLogger(WebSocketFrontend.class); private static final int MAX_UPGRADE_REQUEST_BYTES = 8192; // One Minecraft 1.7.10 frame (three-byte length plus at most 0x1FFFFF bytes) fits in one message. private static final int MAX_FRAME_PAYLOAD = 0x1FFFFF + 3; - /** Immutable listener settings; a null TLS context means plain ws:// (TLS ends at a CDN or proxy). */ - public record Settings(String path, SslContext tls, ClientAddressPolicy addresses, Duration handshakeTimeout) { + /** + * Immutable listener settings; a null TLS context means plain ws:// (TLS ends at a CDN or proxy). A zero + * {@code pingInterval} sends no keepalive pings. + */ + public record Settings(String path, SslContext tls, ClientAddressPolicy addresses, Duration handshakeTimeout, + Duration pingInterval) { + public static final Duration DEFAULT_PING_INTERVAL = Duration.ofSeconds(30); + public Settings { Objects.requireNonNull(path, "path"); if (!path.startsWith("/")) throw new IllegalArgumentException("websocket path must start with /"); @@ -51,6 +58,12 @@ public final class WebSocketFrontend { if (handshakeTimeout.isNegative() || handshakeTimeout.isZero()) { throw new IllegalArgumentException("handshake timeout must be positive"); } + Objects.requireNonNull(pingInterval, "pingInterval"); + if (pingInterval.isNegative()) throw new IllegalArgumentException("ping interval must not be negative"); + } + + public Settings(String path, SslContext tls, ClientAddressPolicy addresses, Duration handshakeTimeout) { + this(path, tls, addresses, handshakeTimeout, DEFAULT_PING_INTERVAL); } } @@ -128,6 +141,10 @@ public final class WebSocketFrontend { return; } ctx.pipeline().replace(this, FRAME_ADAPTER, new WebSocketFrameAdapter()); + if (!settings.pingInterval().isZero()) { + // Behind the encoder, so pings are framed and the adapter's writes count as activity. + ctx.pipeline().addBefore(FRAME_ADAPTER, KEEPALIVE, new WebSocketKeepalive(settings.pingInterval())); + } ready.accept(ctx.channel(), address); } diff --git a/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketKeepalive.java b/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketKeepalive.java new file mode 100644 index 0000000..c27286b --- /dev/null +++ b/proxy/core/src/main/java/dev/moonbridge/core/websocket/WebSocketKeepalive.java @@ -0,0 +1,59 @@ +package dev.moonbridge.core.websocket; + +import io.netty.channel.ChannelHandlerContext; +import io.netty.channel.ChannelOutboundHandlerAdapter; +import io.netty.channel.ChannelPromise; +import io.netty.handler.codec.http.websocketx.PingWebSocketFrame; + +import java.time.Duration; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeUnit; + +/** + * Sends an empty WebSocket ping when the proxy has written nothing for a while, so a CDN or load balancer between + * the player and the proxy sees traffic during quiet stretches (a long modded login, a placement wait, a stalled + * backend) and does not cut the connection at its idle limit. The client's pong is dropped by Netty's protocol + * handler. + * + *

A write only sets a flag; a check every half interval pings if the last half interval had no write. So the + * proxy is never silent for longer than the interval, never pings after less than half of it, and the data path + * pays nothing beyond the flag (an {@code IdleStateHandler}, which adds a listener to every write, measured + * slower; see testing/benchmarks/results/2026-10-23-websocket-cdn.md). An unwritable connection is skipped: bytes + * are already queued there. + */ +final class WebSocketKeepalive extends ChannelOutboundHandlerAdapter { + private final long checkNanos; + private boolean wrote; + private ScheduledFuture check; + + WebSocketKeepalive(Duration interval) { + checkNanos = Math.max(1, interval.toNanos() / 2); + } + + @Override public void handlerAdded(ChannelHandlerContext ctx) { + check = ctx.executor().scheduleAtFixedRate(() -> check(ctx), checkNanos, checkNanos, TimeUnit.NANOSECONDS); + } + + @Override public void handlerRemoved(ChannelHandlerContext ctx) { + check.cancel(false); + } + + @Override public void write(ChannelHandlerContext ctx, Object message, ChannelPromise promise) { + wrote = true; + ctx.write(message, promise); + } + + private void check(ChannelHandlerContext ctx) { + if (!ctx.channel().isActive()) { + check.cancel(false); + return; + } + boolean idle = !wrote; + wrote = false; + if (idle && ctx.channel().isWritable()) { + // Written from this context, so it bypasses write(); count it as this period's write by hand. + ctx.writeAndFlush(new PingWebSocketFrame()); // A failed ping means a closing channel; nothing to do. + wrote = true; + } + } +} diff --git a/proxy/core/src/test/java/dev/moonbridge/app/ProxyConfigurationTest.java b/proxy/core/src/test/java/dev/moonbridge/app/ProxyConfigurationTest.java index ae64458..f6e4f41 100644 --- a/proxy/core/src/test/java/dev/moonbridge/app/ProxyConfigurationTest.java +++ b/proxy/core/src/test/java/dev/moonbridge/app/ProxyConfigurationTest.java @@ -350,10 +350,18 @@ final class ProxyConfigurationTest { var websocket = new ProxyConfigurationLoader().load(config).websocket(); assertEquals("/mc", websocket.path()); assertEquals(10, websocket.handshakeTimeoutSeconds()); + assertEquals(30, websocket.pingIntervalSeconds()); assertEquals(8080, websocket.listenAddress().getPort()); var settings = new ProxyConfigurationLoader().loadWebSocket(new ProxyConfigurationLoader().load(config), config); assertEquals("/mc", settings.path()); assertNull(settings.tls()); + assertEquals(java.time.Duration.ofSeconds(30), settings.pingInterval()); + + Files.writeString(config, base + "websocket:\n listen: \"127.0.0.1:8080\"\n pingIntervalSeconds: 0\n"); + assertEquals(java.time.Duration.ZERO, new ProxyConfigurationLoader() + .loadWebSocket(new ProxyConfigurationLoader().load(config), config).pingInterval()); + Files.writeString(config, base + "websocket:\n listen: \"127.0.0.1:8080\"\n pingIntervalSeconds: 301\n"); + assertThrows(com.fasterxml.jackson.databind.JsonMappingException.class, () -> new ProxyConfigurationLoader().load(config)); Files.writeString(config, base + "websocket:\n listen: \"127.0.0.1:8080\"\n"); assertEquals("/", new ProxyConfigurationLoader().load(config).websocket().path()); diff --git a/proxy/core/src/test/java/dev/moonbridge/core/session/WebSocketListenerTest.java b/proxy/core/src/test/java/dev/moonbridge/core/session/WebSocketListenerTest.java index 5612300..9ab8258 100644 --- a/proxy/core/src/test/java/dev/moonbridge/core/session/WebSocketListenerTest.java +++ b/proxy/core/src/test/java/dev/moonbridge/core/session/WebSocketListenerTest.java @@ -359,6 +359,27 @@ final class WebSocketListenerTest { } } + @Test + void anIdleUpgradedConnectionIsPinged() throws Exception { + var listener = new ProxySessionListener(LOOPBACK_ANY_PORT, new InMemoryBackendCatalog()); + listener.setPlacement(player -> CompletableFuture.completedFuture(Optional.empty())); + listener.setWebSocket(LOOPBACK_ANY_PORT, new WebSocketFrontend.Settings("/", null, ClientAddressPolicy.direct(), + Duration.ofSeconds(5), Duration.ofMillis(200))); + try { + listener.start().toCompletableFuture().get(5, TimeUnit.SECONDS); + try (Socket socket = new Socket(InetAddress.getLoopbackAddress(), port(listener.webSocketChannel()))) { + socket.setSoTimeout(5000); + assertTrue(TestWebSocketClient.upgrade(socket, "/", Map.of()).startsWith("HTTP/1.1 101")); + // Nothing is sent after the upgrade; the first server frame is an empty, unmasked ping. + InputStream input = socket.getInputStream(); + assertEquals(0x80 | TestWebSocketClient.PING, input.read()); + assertEquals(0, input.read()); + } + } finally { + listener.close().toCompletableFuture().get(5, TimeUnit.SECONDS); + } + } + @Test void idleConnectionIsClosedAtTheHandshakeDeadline() throws Exception { var listener = new ProxySessionListener(LOOPBACK_ANY_PORT, new InMemoryBackendCatalog()); @@ -387,7 +408,7 @@ final class WebSocketListenerTest { var configuration = new ProxyConfiguration("127.0.0.1:0", ProxyConfiguration.Authentication.OFFLINE, List.of(), null, null, false, null, null, null, null, new ProxyConfiguration.WebSocket("127.0.0.1:1", "/mc", - new ProxyConfiguration.Tls("certs/chain.pem", "certs/key.pem"), null, null, null)); + new ProxyConfiguration.Tls("certs/chain.pem", "certs/key.pem"), null, null, null, null)); var settings = new ProxyConfigurationLoader().loadWebSocket(configuration, directory.resolve("moonbridge.yml")); var listener = new ProxySessionListener(LOOPBACK_ANY_PORT, new InMemoryBackendCatalog()); @@ -395,17 +416,25 @@ final class WebSocketListenerTest { listener.setWebSocket(LOOPBACK_ANY_PORT, settings); try { listener.start().toCompletableFuture().get(5, TimeUnit.SECONDS); + // The distribution bundles BoringSSL for this platform; the client mod offers TLS 1.2 only. + assertTrue(settings.tls() instanceof io.netty.handler.ssl.ReferenceCountedOpenSslContext, + settings.tls().getClass().getName()); SSLContext context = SSLContext.getInstance("TLS"); context.init(null, certificate.toTrustManagerFactory().getTrustManagers(), null); - var socket = (SSLSocket) context.getSocketFactory().createSocket(InetAddress.getLoopbackAddress(), - port(listener.webSocketChannel())); - var parameters = socket.getSSLParameters(); - parameters.setServerNames(List.of(new SNIHostName("localhost"))); - parameters.setEndpointIdentificationAlgorithm("HTTPS"); - socket.setSSLParameters(parameters); - socket.setSoTimeout(5000); - try (var ws = TestWebSocketClient.connect(socket, "/mc", Map.of())) { - assertEquals("MoonBridge", requestStatus(ws.input(), ws.output()).path("description").path("text").asText()); + for (String[] protocols : List.of(new String[] {"TLSv1.2"}, new String[] {"TLSv1.3", "TLSv1.2"})) { + var socket = (SSLSocket) context.getSocketFactory().createSocket(InetAddress.getLoopbackAddress(), + port(listener.webSocketChannel())); + var parameters = socket.getSSLParameters(); + parameters.setServerNames(List.of(new SNIHostName("localhost"))); + parameters.setEndpointIdentificationAlgorithm("HTTPS"); + parameters.setProtocols(protocols); + socket.setSSLParameters(parameters); + socket.setSoTimeout(5000); + try (var ws = TestWebSocketClient.connect(socket, "/mc", Map.of())) { + assertEquals(protocols[0], socket.getSession().getProtocol()); + assertEquals("MoonBridge", + requestStatus(ws.input(), ws.output()).path("description").path("text").asText()); + } } } finally { listener.close().toCompletableFuture().get(5, TimeUnit.SECONDS); diff --git a/proxy/core/src/test/java/dev/moonbridge/core/websocket/WebSocketKeepaliveTest.java b/proxy/core/src/test/java/dev/moonbridge/core/websocket/WebSocketKeepaliveTest.java new file mode 100644 index 0000000..255ea10 --- /dev/null +++ b/proxy/core/src/test/java/dev/moonbridge/core/websocket/WebSocketKeepaliveTest.java @@ -0,0 +1,84 @@ +package dev.moonbridge.core.websocket; + +import io.netty.buffer.Unpooled; +import io.netty.channel.WriteBufferWaterMark; +import io.netty.channel.embedded.EmbeddedChannel; +import io.netty.handler.codec.http.websocketx.BinaryWebSocketFrame; +import io.netty.handler.codec.http.websocketx.PingWebSocketFrame; +import io.netty.handler.codec.http.websocketx.WebSocketFrame; +import org.junit.jupiter.api.Test; + +import java.time.Duration; +import java.util.concurrent.TimeUnit; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNull; + +/** A 30-second interval: checks run every 15 seconds and ping after a check period without writes. */ +final class WebSocketKeepaliveTest { + private static EmbeddedChannel channel() { + var channel = new EmbeddedChannel(); + channel.freezeTime(); + channel.pipeline().addLast(new WebSocketKeepalive(Duration.ofSeconds(30))); + return channel; + } + + private static void advance(EmbeddedChannel channel, long seconds) { + channel.advanceTimeBy(seconds, TimeUnit.SECONDS); + channel.runScheduledPendingTasks(); + } + + @Test + void anIdleConnectionGetsAnEmptyPingEveryHalfInterval() { + var channel = channel(); + advance(channel, 14); + assertNull(channel.readOutbound()); + advance(channel, 1); + WebSocketFrame ping = assertInstanceOf(PingWebSocketFrame.class, channel.readOutbound()); + assertEquals(0, ping.content().readableBytes()); + ping.release(); + // The ping itself is a write, so the next check skips and the one after pings again. + advance(channel, 15); + assertNull(channel.readOutbound()); + advance(channel, 15); + assertInstanceOf(PingWebSocketFrame.class, channel.readOutbound()).release(); + assertFalse(channel.finishAndReleaseAll()); + } + + @Test + void writesPostponeThePingSoSilenceNeverExceedsTheInterval() { + var channel = channel(); + advance(channel, 10); + channel.writeOutbound(new BinaryWebSocketFrame(Unpooled.buffer().writeByte(1))); + channel.readOutbound().release(); + advance(channel, 20); // Checks at 15 (saw the write) and 30 (nothing since 15) -> ping 20 s after the write. + assertInstanceOf(PingWebSocketFrame.class, channel.readOutbound()).release(); + assertFalse(channel.finishAndReleaseAll()); + } + + @Test + void anUnwritableConnectionGetsNoPing() { + var channel = channel(); + advance(channel, 15); + channel.readOutbound().release(); + channel.config().setWriteBufferWaterMark(new WriteBufferWaterMark(1, 2)); + channel.write(new BinaryWebSocketFrame(Unpooled.buffer().writeBytes(new byte[64]))); + assertFalse(channel.isWritable()); + advance(channel, 30); + channel.flushOutbound(); + assertInstanceOf(BinaryWebSocketFrame.class, channel.readOutbound()).release(); + assertNull(channel.readOutbound()); + assertFalse(channel.finishAndReleaseAll()); + } + + @Test + void closingStopsTheChecks() { + var channel = channel(); + channel.close(); + advance(channel, 60); + assertNull(channel.readOutbound()); + assertFalse(channel.finishAndReleaseAll()); + } +} diff --git a/testing/benchmarks/ProxySessionBenchmark.java b/testing/benchmarks/ProxySessionBenchmark.java index 2635353..a00099a 100644 --- a/testing/benchmarks/ProxySessionBenchmark.java +++ b/testing/benchmarks/ProxySessionBenchmark.java @@ -50,6 +50,9 @@ public final class ProxySessionBenchmark { int commandInterceptor = intArg(args, "command-interceptor", 1); int burst = intArg(args, "burst", 1); int webSocket = intArg(args, "websocket", 0); + int webSocketTls = intArg(args, "websocket-tls", 0); + int clientCoalesce = intArg(args, "client-coalesce", 0); + String profile = stringArg(args, "profile", "echo"); String mode = modeArg(args); var transport = dev.moonbridge.core.session.channel.NetworkTransport.select( dev.moonbridge.core.session.channel.NetworkTransport.Preference.valueOf( @@ -59,18 +62,30 @@ public final class ProxySessionBenchmark { || warmup < 0 || warmup > 10_000 || payload < 5 || payload > 1_048_576 || repeats < 1 || repeats > 10 || window < 1 || window > 1024 || (commandInterceptor != 0 && commandInterceptor != 1) || burst < 1 || burst > 256 - || (webSocket != 0 && webSocket != 1)) { + || (webSocket != 0 && webSocket != 1) || (webSocketTls != 0 && webSocketTls != 1) + || (clientCoalesce != 0 && clientCoalesce != 1)) { throw new IllegalArgumentException("bounds: connections 1..32, messages 1..100000, warmup 0..10000, " + "payload 5..1048576, repeats 1..10, window 1..1024, burst 1..256"); } + if (!profile.equals("echo") && !profile.equals("modded")) + throw new IllegalArgumentException("profile must be echo or modded"); + if (webSocketTls == 1 && webSocket == 0) throw new IllegalArgumentException("--websocket-tls needs --websocket 1"); + ModdedTraffic modded = profile.equals("modded") ? new ModdedTraffic() : null; + if (modded != null && burst != 1) throw new IllegalArgumentException("--profile modded sets its own frame mix; " + + "--burst does not apply"); + if (modded == null && clientCoalesce == 1) + throw new IllegalArgumentException("--client-coalesce applies to --profile modded"); byte[] playPayload = new byte[payload]; playPayload[0] = 0x03; // PLAY packet ID; bytes 1..4 carry the per-request sequence. for (int i = 1; i < payload; i++) playPayload[i] = (byte) (i * 31 + 7); byte[] playFrame = frame(playPayload); + BenchmarkTls tls = webSocketTls == 1 ? BenchmarkTls.create() : null; + ClientOptions options = new ClientOptions(webSocket == 1, tls == null ? null : tls.client(), modded, + clientCoalesce == 1); - try (FakeBackend backend = new FakeBackend(connections, 42, burst); - FakeBackend replacement = afterTransfer ? new FakeBackend(connections, 99, burst) : null) { + try (FakeBackend backend = new FakeBackend(connections, 42, burst, modded); + FakeBackend replacement = afterTransfer ? new FakeBackend(connections, 99, burst, modded) : null) { backend.start(); if (replacement != null) replacement.start(); InMemoryBackendCatalog catalog = new InMemoryBackendCatalog(); @@ -88,7 +103,8 @@ public final class ProxySessionBenchmark { if (commandInterceptor == 1) proxy.setCommandDispatcher((player, message, reply, admission) -> false); if (webSocket == 1) { proxy.setWebSocket(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0), - new dev.moonbridge.core.websocket.WebSocketFrontend.Settings(WEBSOCKET_PATH, null, + new dev.moonbridge.core.websocket.WebSocketFrontend.Settings(WEBSOCKET_PATH, + tls == null ? null : tls.server(), dev.moonbridge.core.websocket.ClientAddressPolicy.direct(), Duration.ofSeconds(10))); } try { @@ -98,16 +114,19 @@ public final class ProxySessionBenchmark { // because the fake backend is a Minecraft TCP server. int proxyPort = webSocket == 1 ? ((InetSocketAddress) proxy.webSocketChannel().localAddress()).getPort() : tcpPort; - System.out.printf("Offline synthetic Minecraft 1.7.10 PLAY echo; mode=%s payload=%d bytes frame=%d bytes " - + "connections=%d measured_roundtrips_per_connection=%d warmup_per_connection=%d " - + "window_per_connection=%d repeats=%d command_interceptor=%d burst=%d transport=%s " - + "proxy_frontend=%s%n", - mode, playPayload.length, playFrame.length, connections, messages, warmup, window, repeats, - commandInterceptor, burst, transport.name(), webSocket == 1 ? "websocket" : "tcp"); - if (webSocket == 1) System.out.println("Proxy clients connect over plain ws:// and send one masked binary " + System.out.printf("Offline synthetic Minecraft 1.7.10 PLAY echo; mode=%s profile=%s payload=%d bytes " + + "frame=%d bytes connections=%d measured_roundtrips_per_connection=%d " + + "warmup_per_connection=%d window_per_connection=%d repeats=%d command_interceptor=%d " + + "burst=%d transport=%s proxy_frontend=%s client_coalesce=%d%n", + mode, profile, playPayload.length, playFrame.length, connections, messages, warmup, window, + repeats, commandInterceptor, burst, transport.name(), + webSocket == 0 ? "tcp" : tls == null ? "websocket" : "websocket-tls", clientCoalesce); + if (webSocket == 1) System.out.println("Proxy clients connect over " + (tls == null ? "plain ws://" + : "wss:// (JDK TLS client, " + tls.description() + ")") + " and send one masked binary " + "message per client flush; direct clients use TCP to the same fake backend."); if (burst > 1) System.out.println("Each roundtrip writes " + burst + " frames in one client write; " + "the fake backend echoes them with one flush. Window counts bursts."); + if (modded != null) System.out.println(modded.description(clientCoalesce == 1)); System.out.println("Direct and proxy phases use the same JVM, fake backend, client code, framed payload, " + "connection concurrency, and per-client in-flight window. Login/setup and warmup are excluded from timing."); if (afterTransfer) System.out.println("Proxy clients switch from bench to replacement before warmup; " @@ -119,14 +138,14 @@ public final class ProxySessionBenchmark { for (int round = 0; round < repeats; round++) { if (proxyFirst) { runPhase("proxy-" + (round + 1), proxyPort, connections, messages, warmup, window, - playFrame, playPayload, proxy, afterTransfer, burst, webSocket == 1); + playFrame, playPayload, proxy, afterTransfer, burst, options); runPhase("direct-" + (round + 1), directPort, connections, messages, warmup, window, - playFrame, playPayload, proxy, afterTransfer, burst, webSocket == 1); + playFrame, playPayload, proxy, afterTransfer, burst, options); } else { runPhase("direct-" + (round + 1), directPort, connections, messages, warmup, window, - playFrame, playPayload, proxy, afterTransfer, burst, webSocket == 1); + playFrame, playPayload, proxy, afterTransfer, burst, options); runPhase("proxy-" + (round + 1), proxyPort, connections, messages, warmup, window, - playFrame, playPayload, proxy, afterTransfer, burst, webSocket == 1); + playFrame, playPayload, proxy, afterTransfer, burst, options); } proxyFirst = !proxyFirst; } @@ -138,7 +157,7 @@ public final class ProxySessionBenchmark { private static void runPhase(String name, int port, int connections, int messages, int warmup, int window, byte[] playFrame, byte[] playPayload, ProxySessionListener proxy, - boolean afterTransfer, int burst, boolean webSocket) throws Exception { + boolean afterTransfer, int burst, ClientOptions options) throws Exception { List clients = new ArrayList<>(); CountDownLatch prepared = new CountDownLatch(connections); CountDownLatch begin = new CountDownLatch(1); @@ -146,7 +165,7 @@ public final class ProxySessionBenchmark { for (int i = 0; i < connections; i++) { ClientWorker client = new ClientWorker(name, i, port, messages, warmup, window, playFrame, playPayload, prepared, begin, finished, clients, proxy, afterTransfer, burst, - webSocket && name.startsWith("proxy")); + options.forPhase(name.startsWith("proxy"))); clients.add(client); } for (ClientWorker client : clients) { @@ -182,8 +201,12 @@ public final class ProxySessionBenchmark { Arrays.sort(samples); long measured = (long) connections * messages; double seconds = elapsed / 1_000_000_000.0; - long frames = measured * burst; - double oneWayMib = frames * (double) playPayload.length / (1024.0 * 1024.0) / seconds; + boolean modded = options.modded() != null; + // The modded profile counts frames and bytes in both directions; the echo profile counts one direction. + long frames = modded ? clients.stream().mapToLong(client -> client.frames).sum() : measured * burst; + double oneWayMib = modded + ? clients.stream().mapToLong(client -> client.serverBytes).sum() / (1024.0 * 1024.0) / seconds + : frames * (double) playPayload.length / (1024.0 * 1024.0) / seconds; long echoes = clients.stream().mapToLong(client -> client.echoes).sum(); long correct = clients.stream().mapToLong(client -> client.correct).sum(); long errors = clients.stream().mapToLong(client -> client.errors).sum(); @@ -197,6 +220,11 @@ public final class ProxySessionBenchmark { percentile(samples, .50) / 1e6, percentile(samples, .95) / 1e6, percentile(samples, .99) / 1e6, elapsed / 1e6, frames / seconds, ioCpu / (double) frames, segments < 0 ? "n/a" : String.format("%.3f", segments / (double) frames)); + if (modded) { + long[] bursts = clients.stream().mapToLong(client -> client.loginBurstNanos).sorted().toArray(); + System.out.printf("%s: modded server_to_client_MiB/s=%.2f login_burst_ms_p50=%.3f max=%.3f%n", name, + oneWayMib, percentile(bursts, .50) / 1e6, bursts[bursts.length - 1] / 1e6); + } } private static void closeClients(List clients) { @@ -213,43 +241,58 @@ public final class ProxySessionBenchmark { final ProxySessionListener proxy; final boolean afterTransfer; final int burst; - final boolean webSocket; + final ClientOptions options; + final ModdedTraffic modded; volatile Exception failure; volatile Socket socket; long[] latencies; long echoes, correct, errors; + long frames, serverBytes, loginBurstNanos; ClientWorker(String phase, int index, int port, int messages, int warmup, int window, byte[] playFrame, byte[] playPayload, CountDownLatch prepared, CountDownLatch begin, CountDownLatch finished, List peers, ProxySessionListener proxy, boolean afterTransfer, int burst, - boolean webSocket) { + ClientOptions options) { this.phase = phase; this.index = index; this.port = port; this.messages = messages; this.warmup = warmup; this.window = window; this.playFrame = playFrame.clone(); this.playPayload = playPayload; this.prepared = prepared; this.begin = begin; this.finished = finished; this.peers = peers; - this.proxy = proxy; this.afterTransfer = afterTransfer; this.burst = burst; this.webSocket = webSocket; + this.proxy = proxy; this.afterTransfer = afterTransfer; this.burst = burst; this.options = options; + this.modded = options.modded(); } @Override public void run() { - try (Socket socket = new Socket()) { - this.socket = socket; - socket.setTcpNoDelay(true); - socket.setSoTimeout(IO_TIMEOUT_MILLIS); - socket.connect(new InetSocketAddress("127.0.0.1", port), 5000); + try (Socket plain = new Socket()) { + this.socket = plain; + plain.setTcpNoDelay(true); + plain.setSoTimeout(IO_TIMEOUT_MILLIS); + plain.connect(new InetSocketAddress("127.0.0.1", port), 5000); + Socket socket = plain; + if (options.tls() != null) { + var secure = (javax.net.ssl.SSLSocket) options.tls().getSocketFactory() + .createSocket(plain, "localhost", port, true); + secure.startHandshake(); + this.socket = socket = secure; + } DataInputStream input; DataOutputStream output; - if (webSocket) { + if (options.webSocket()) { var streams = new WebSocketStreams(socket, port); input = new DataInputStream(streams.input); output = new DataOutputStream(streams.output); } else { - input = new DataInputStream(socket.getInputStream()); - output = new DataOutputStream(socket.getOutputStream()); + // The modded profile reads and writes packet by packet; buffering gives TCP the same read cost + // as the WebSocket client's buffered input and the same write boundaries as its messages. + input = new DataInputStream(modded == null ? socket.getInputStream() + : new java.io.BufferedInputStream(socket.getInputStream(), 64 * 1024)); + output = new DataOutputStream(modded == null ? socket.getOutputStream() + : new java.io.BufferedOutputStream(socket.getOutputStream(), 64 * 1024)); } String username = "B" + (phase.startsWith("proxy") ? "P" : "D") + index + "X"; sendHandshakeAndLogin(output, username); readLoginSuccess(input, username); if (packetId(readFrame(input)) != 1) throw new IOException("expected Join Game before PLAY"); if (packetId(readFrame(input)) != 8) throw new IOException("expected Position and Look before PLAY"); + if (modded != null) readLoginBurst(input); if (afterTransfer && phase.startsWith("proxy")) { PlayerView player = awaitPlayer(proxy, username); var transfer = proxy.transfer(player.identity(), "replacement").toCompletableFuture(); @@ -257,6 +300,7 @@ public final class ProxySessionBenchmark { throw new IOException("expected two Respawns from replacement backend"); if (packetId(readFrame(input)) != 8) throw new IOException("expected Position and Look from replacement backend"); + if (modded != null) readLoginBurst(input); var result = transfer.get(SETUP_TIMEOUT_SECONDS, TimeUnit.SECONDS); if (result.status() != TransferStatus.NETWORK_READY) throw new IOException("replacement transfer failed: " + result.status() @@ -264,12 +308,15 @@ public final class ProxySessionBenchmark { if (!proxy.find(player.identity()).flatMap(PlayerView::currentServer).orElse("") .equals("replacement")) throw new IOException("replacement player view missing"); } - exchange(output, input, warmup, null, 0); - echoes = 0; correct = 0; errors = 0; // Warmup is validated, then omitted from reported phase counts. + if (modded != null) exchangeModded(output, input, warmup, null, 0); + else exchange(output, input, warmup, null, 0); + // Warmup is validated, then omitted from reported phase counts. + echoes = 0; correct = 0; errors = 0; frames = 0; serverBytes = 0; latencies = new long[messages]; prepared.countDown(); begin.await(); - exchange(output, input, messages, latencies, warmup); + if (modded != null) exchangeModded(output, input, messages, latencies, warmup); + else exchange(output, input, messages, latencies, warmup); } catch (Exception problem) { failure = problem; if (latencies == null) prepared.countDown(); @@ -317,6 +364,57 @@ public final class ProxySessionBenchmark { } } + /** The backend's registry-sync stand-in after each Position and Look; its wall time is reported, not timed. */ + private void readLoginBurst(DataInputStream input) throws IOException { + long started = System.nanoTime(); + ModdedTraffic.Packet[] login = modded.login; + for (int i = 0; i < login.length; i++) modded.verify(login[i], readFrame(input), i); + loginBurstNanos = System.nanoTime() - started; + } + + /** + * One roundtrip is one client tick: three small serverbound packets, answered by the backend with that + * tick's clientbound mix. Without client coalescing each packet is its own write and flush, as the released + * client mod sends one WebSocket message per vanilla packet. + */ + private void exchangeModded(DataOutputStream output, DataInputStream input, int count, long[] samples, + int firstSequence) throws Exception { + long[] outstandingSince = new long[Math.min(window, Math.max(1, count))]; + int sent = 0; + int received = 0; + while (received < count) { + while (sent < count && sent - received < outstandingSince.length) { + int sequence = firstSequence + sent; + outstandingSince[sent % outstandingSince.length] = System.nanoTime(); + for (ModdedTraffic.Packet packet : modded.client(sequence)) { + output.write(modded.frame(packet, sequence)); + if (!options.coalesce()) output.flush(); + frames++; + } + if (options.coalesce()) output.flush(); + sent++; + } + int sequence = firstSequence + received; + for (ModdedTraffic.Packet packet : modded.server(sequence)) { + byte[] body = readFrame(input); + try { + modded.verify(packet, body, sequence); + } catch (IOException mismatch) { + errors++; + throw new IOException("tick " + received + ": " + mismatch.getMessage(), mismatch); + } + frames++; + serverBytes += body.length; + } + if (samples != null) { + samples[received] = System.nanoTime() - outstandingSince[received % outstandingSince.length]; + } + echoes++; + correct++; + received++; + } + } + void closeSocket() { Socket current = socket; if (current != null) try { current.close(); } catch (IOException ignored) { } @@ -424,6 +522,205 @@ public final class ProxySessionBenchmark { } } + /** How a phase's clients connect and what they send; direct phases always use plain TCP to the fake backend. */ + private record ClientOptions(boolean webSocket, javax.net.ssl.SSLContext tls, ModdedTraffic modded, + boolean coalesce) { + ClientOptions forPhase(boolean proxyPhase) { + return proxyPhase ? this : new ClientOptions(false, null, modded, coalesce); + } + } + + /** + * Fixed-seed traffic shaped like a large 1.7.10 modpack. Each client tick sends a mod custom payload, a position + * packet and an on-ground packet. Each server tick answers with 6-14 small packets (mod custom payloads and tile + * entity updates of 20-600 bytes), a 2-8 KiB custom payload on about one tick in three and a 30-60 KiB chunk on + * about one in twenty. After each Position and Look the backend also sends a ~2 MiB burst of 30,000-byte custom + * payloads, standing in for Forge registry sync. Every clientbound packet carries the tick sequence at a known + * offset, so the client checks every byte. 256 templates repeat by sequence. + */ + private static final class ModdedTraffic { + private static final int TEMPLATES = 256; + private static final int LOGIN_PACKETS = 70; + private static final String[] CHANNELS = {"gregtech", "appliedenergistics2", "Forestry", "IC2", + "Thaumcraft", "BuildCraft|TE", "OpenComputers", "journeymap"}; + + /** A packet body; {@code stamp} is where the 4-byte sequence goes, or -1 for none. */ + record Packet(byte[] body, int stamp) { } + + final Packet[] login = new Packet[LOGIN_PACKETS]; + private final Packet[][] client = new Packet[TEMPLATES][]; + private final Packet[][] server = new Packet[TEMPLATES][]; + + ModdedTraffic() { + try { + for (int t = 0; t < TEMPLATES; t++) { + var random = new java.util.SplittableRandom(0x5EED0000L + t); + client[t] = new Packet[] { + customPayload(0x17, channel(random), 10 + random.nextInt(71), random), + position(random), new Packet(new byte[] {0x03, 1}, -1)}; + List packets = new ArrayList<>(); + int small = 6 + random.nextInt(9); + for (int i = 0; i < small; i++) { + packets.add(random.nextInt(10) < 7 + ? customPayload(0x3F, channel(random), 20 + random.nextInt(281), random) + : tileEntity(100 + random.nextInt(501), random)); + } + if (random.nextInt(3) == 0) { + packets.add(customPayload(0x3F, channel(random), 2048 + random.nextInt(6145), random)); + } + if (random.nextInt(20) == 0) packets.add(chunk(30 * 1024 + random.nextInt(30 * 1024 + 1), random)); + server[t] = packets.toArray(Packet[]::new); + } + var random = new java.util.SplittableRandom(0x10C1L); + for (int i = 0; i < LOGIN_PACKETS; i++) login[i] = customPayload(0x3F, "gregtech", 30_000, random); + } catch (IOException impossible) { + throw new java.io.UncheckedIOException(impossible); + } + } + + Packet[] client(int sequence) { return client[Math.floorMod(sequence, TEMPLATES)]; } + Packet[] server(int sequence) { return server[Math.floorMod(sequence, TEMPLATES)]; } + + String description(boolean coalesce) { + long frames = 0, bytes = 0; + for (Packet[] tick : server) for (Packet packet : tick) { frames++; bytes += packet.body().length; } + long loginBytes = 0; + for (Packet packet : login) loginBytes += packet.body().length; + return String.format("Modded profile: per roundtrip the client sends 3 packets (%s) and the backend " + + "answers with %.1f packets / %.0f bytes on average over %d templates; frame counts " + + "include both directions. Login burst: %d packets / %d bytes after each Position and Look.", + coalesce ? "one write and flush per tick" : "one write and flush per packet", + frames / (double) TEMPLATES, bytes / (double) TEMPLATES, TEMPLATES, LOGIN_PACKETS, loginBytes); + } + + byte[] frame(Packet packet, int sequence) throws IOException { + byte[] body = packet.body().clone(); + if (packet.stamp() >= 0) stamp(body, packet.stamp(), sequence); + return ProxySessionBenchmark.frame(body); + } + + void verify(Packet expected, byte[] body, int sequence) throws IOException { + byte[] want = expected.body(); + int at = expected.stamp(); + if (body.length != want.length) throw new IOException("length " + body.length + " expected " + want.length); + if (Arrays.mismatch(body, 0, at, want, 0, at) >= 0 + || Arrays.mismatch(body, at + 4, body.length, want, at + 4, want.length) >= 0) { + throw new IOException("packet bytes differ from the template"); + } + int actual = ((body[at] & 0xff) << 24) | ((body[at + 1] & 0xff) << 16) + | ((body[at + 2] & 0xff) << 8) | (body[at + 3] & 0xff); + if (actual != sequence) throw new IOException("sequence " + actual + " expected " + sequence); + } + + /** The backend reads the tick sequence from the client's custom payload, which each tick sends first. */ + static int sequenceOf(byte[] body) throws IOException { + ByteArrayInputStream input = new ByteArrayInputStream(body); + if (readVarInt(input) != 0x17) throw new IOException("expected the tick's custom payload first"); + readString(input, 20); + DataInputStream data = new DataInputStream(input); + data.readUnsignedShort(); + return data.readInt(); + } + + private static String channel(java.util.SplittableRandom random) { + return CHANNELS[random.nextInt(CHANNELS.length)]; + } + + /** C17/S3F: channel, a short length (Forge's VarShort is the same two bytes below 32 KiB), then data. */ + private static Packet customPayload(int id, String channel, int length, java.util.SplittableRandom random) + throws IOException { + ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + writeVarInt(bytes, id); + writeString(bytes, channel); + new DataOutputStream(bytes).writeShort(length); + int stamp = bytes.size(); + bytes.write(randomBytes(length, random)); + return new Packet(bytes.toByteArray(), stamp); + } + + /** S35 Update Tile Entity: the sequence sits in x, then y, z, action and the NBT bytes. */ + private static Packet tileEntity(int nbtLength, java.util.SplittableRandom random) throws IOException { + ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + DataOutputStream out = new DataOutputStream(bytes); + writeVarInt(bytes, 0x35); + out.writeInt(0); out.writeShort(64); out.writeInt(random.nextInt(1 << 16)); out.writeByte(1); + out.writeShort(nbtLength); + bytes.write(randomBytes(nbtLength, random)); + return new Packet(bytes.toByteArray(), 1); + } + + /** S21 Chunk Data: the sequence sits in the chunk x, then z, flags and the (already deflated) data. */ + private static Packet chunk(int dataLength, java.util.SplittableRandom random) throws IOException { + ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + DataOutputStream out = new DataOutputStream(bytes); + writeVarInt(bytes, 0x21); + out.writeInt(0); out.writeInt(random.nextInt(64)); out.writeBoolean(true); + out.writeShort(0xFFFF); out.writeShort(0); out.writeInt(dataLength); + bytes.write(randomBytes(dataLength, random)); + return new Packet(bytes.toByteArray(), 1); + } + + /** C04 Player Position: x, feet y, stance, z, on ground. */ + private static Packet position(java.util.SplittableRandom random) throws IOException { + ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + DataOutputStream out = new DataOutputStream(bytes); + writeVarInt(bytes, 0x04); + double y = 64 + random.nextDouble(); + out.writeDouble(random.nextDouble() * 100); out.writeDouble(y); out.writeDouble(y + 1.62); + out.writeDouble(random.nextDouble() * 100); out.writeBoolean(true); + return new Packet(bytes.toByteArray(), -1); + } + + private static byte[] randomBytes(int length, java.util.SplittableRandom random) { + byte[] bytes = new byte[length]; + random.nextBytes(bytes); + return bytes; + } + + private static void stamp(byte[] body, int at, int sequence) { + body[at] = (byte) (sequence >>> 24); + body[at + 1] = (byte) (sequence >>> 16); + body[at + 2] = (byte) (sequence >>> 8); + body[at + 3] = (byte) sequence; + } + } + + /** A throwaway ECDSA P-256 key and self-signed certificate made with the JDK's keytool, for --websocket-tls. */ + private record BenchmarkTls(io.netty.handler.ssl.SslContext server, javax.net.ssl.SSLContext client, + String description) { + static BenchmarkTls create() throws Exception { + java.nio.file.Path directory = java.nio.file.Files.createTempDirectory("moonbridge-bench-tls"); + java.nio.file.Path store = directory.resolve("bench.p12"); + String password = "benchmark"; + var keys = java.security.KeyStore.getInstance("PKCS12"); + try { + Process keytool = new ProcessBuilder( + java.nio.file.Path.of(System.getProperty("java.home"), "bin", "keytool").toString(), + "-genkeypair", "-alias", "bench", "-keyalg", "EC", "-groupname", "secp256r1", + "-dname", "CN=localhost", "-ext", "SAN=dns:localhost", "-validity", "2", + "-storetype", "PKCS12", "-keystore", store.toString(), + "-storepass", password, "-keypass", password).redirectErrorStream(true).start(); + String output = new String(keytool.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + if (keytool.waitFor() != 0) throw new IOException("keytool failed: " + output); + try (var input = java.nio.file.Files.newInputStream(store)) { keys.load(input, password.toCharArray()); } + } finally { + java.nio.file.Files.deleteIfExists(store); + java.nio.file.Files.deleteIfExists(directory); + } + var keyManagers = javax.net.ssl.KeyManagerFactory.getInstance( + javax.net.ssl.KeyManagerFactory.getDefaultAlgorithm()); + keyManagers.init(keys, password.toCharArray()); + var trust = javax.net.ssl.TrustManagerFactory.getInstance( + javax.net.ssl.TrustManagerFactory.getDefaultAlgorithm()); + trust.init(keys); + var client = javax.net.ssl.SSLContext.getInstance("TLS"); + client.init(null, trust.getTrustManagers(), null); + var server = io.netty.handler.ssl.SslContextBuilder.forServer(keyManagers).build(); + return new BenchmarkTls(server, client, "proxy SslProvider " + + io.netty.handler.ssl.SslContext.defaultServerProvider() + ", ECDSA P-256 certificate"); + } + } + private static PlayerView awaitPlayer(ProxySessionListener proxy, String username) throws Exception { long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(SETUP_TIMEOUT_SECONDS); while (System.nanoTime() < deadline) { @@ -465,9 +762,12 @@ public final class ProxySessionBenchmark { private volatile boolean closed; private Thread acceptThread; - FakeBackend(int concurrency, int entityId, int burst) throws IOException { + private final ModdedTraffic modded; + + FakeBackend(int concurrency, int entityId, int burst, ModdedTraffic modded) throws IOException { this.entityId = entityId; this.burst = burst; + this.modded = modded; clients = Executors.newFixedThreadPool(concurrency); server.bind(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0), concurrency * 2); } @@ -505,7 +805,21 @@ public final class ProxySessionBenchmark { writeFrame(output, success.toByteArray()); writeFrame(output, joinGame(entityId)); writeFrame(output, positionAndLook()); - if (burst == 1) { + if (modded != null) { + ByteArrayOutputStream batch = new ByteArrayOutputStream(); + for (int i = 0; i < modded.login.length; i++) batch.write(modded.frame(modded.login[i], i)); + batch.writeTo(output); + output.flush(); + while (!closed && !socket.isClosed()) { + int sequence = ModdedTraffic.sequenceOf(readFrame(input)); + for (int k = 1; k < modded.client(sequence).length; k++) readFrame(input); + batch.reset(); + for (ModdedTraffic.Packet packet : modded.server(sequence)) + batch.write(modded.frame(packet, sequence)); + batch.writeTo(output); + output.flush(); + } + } else if (burst == 1) { while (!closed && !socket.isClosed()) { byte[] body = readFrame(input); writeFrame(output, body); @@ -663,7 +977,8 @@ public final class ProxySessionBenchmark { int found = fallback; boolean seen = false; List known = List.of("connections", "messages", "warmup", "payload", "repeats", "window", - "mode", "command-interceptor", "burst", "transport", "websocket"); + "mode", "command-interceptor", "burst", "transport", "websocket", "websocket-tls", "profile", + "client-coalesce"); for (int i = 0; i < args.length; i++) { if (args[i].startsWith("--") && !known.contains(args[i].substring(2))) throw new IllegalArgumentException("unknown option " + args[i]); diff --git a/testing/benchmarks/results/2026-10-23-websocket-cdn-raw.txt b/testing/benchmarks/results/2026-10-23-websocket-cdn-raw.txt new file mode 100644 index 0000000..a20602f --- /dev/null +++ b/testing/benchmarks/results/2026-10-23-websocket-cdn-raw.txt @@ -0,0 +1,642 @@ +# Raw output for 2026-10-23-websocket-cdn.md. Lines are: workload variant round|pair phase: metrics. + +## 1. Baseline matrix, run 1 (main 6224d3d; quiet host apart from appstoreagent; TCP rows have the unbuffered-client artifact) +w1 tcp round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=61230.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=355.08 latency_ms_p50=0.058 p95=0.115 p99=0.144 elapsed_ms=326.635 frames/s=811156.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round1 direct-1: modded server_to_client_MiB/s=355.08 login_burst_ms_p50=4.391 max=4.616 +w1 tcp round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35160.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=203.90 latency_ms_p50=0.108 p95=0.179 p99=0.217 elapsed_ms=568.820 frames/s=465792.6 proxy_io_cpu_ns/frame=3114 tcp_out_segments/frame=n/a +w1 tcp round1 proxy-1: modded server_to_client_MiB/s=203.90 login_burst_ms_p50=10.105 max=11.608 +w1 tcp round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43509.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=252.32 latency_ms_p50=0.085 p95=0.145 p99=0.178 elapsed_ms=459.675 frames/s=576389.9 proxy_io_cpu_ns/frame=2388 tcp_out_segments/frame=n/a +w1 tcp round1 proxy-2: modded server_to_client_MiB/s=252.32 login_burst_ms_p50=3.313 max=3.839 +w1 tcp round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=70596.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=409.40 latency_ms_p50=0.051 p95=0.093 p99=0.117 elapsed_ms=283.302 frames/s=935227.4 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round1 direct-2: modded server_to_client_MiB/s=409.40 login_burst_ms_p50=2.325 max=2.605 +w1 ws round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=60403.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=350.29 latency_ms_p50=0.059 p95=0.117 p99=0.146 elapsed_ms=331.108 frames/s=800197.4 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round1 direct-1: modded server_to_client_MiB/s=350.29 login_burst_ms_p50=4.079 max=5.158 +w1 ws round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=40404.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=234.31 latency_ms_p50=0.095 p95=0.158 p99=0.204 elapsed_ms=494.993 frames/s=535263.9 proxy_io_cpu_ns/frame=3567 tcp_out_segments/frame=n/a +w1 ws round1 proxy-1: modded server_to_client_MiB/s=234.31 login_burst_ms_p50=11.532 max=14.292 +w1 ws round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=50216.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=291.21 latency_ms_p50=0.074 p95=0.128 p99=0.157 elapsed_ms=398.279 frames/s=665241.8 proxy_io_cpu_ns/frame=2668 tcp_out_segments/frame=n/a +w1 ws round1 proxy-2: modded server_to_client_MiB/s=291.21 login_burst_ms_p50=2.524 max=3.023 +w1 ws round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=69377.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=402.33 latency_ms_p50=0.053 p95=0.094 p99=0.118 elapsed_ms=288.278 frames/s=919086.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round1 direct-2: modded server_to_client_MiB/s=402.33 login_burst_ms_p50=1.687 max=2.106 +w1 wss round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=60955.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=353.49 latency_ms_p50=0.058 p95=0.116 p99=0.145 elapsed_ms=328.109 frames/s=807513.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round1 direct-1: modded server_to_client_MiB/s=353.49 login_burst_ms_p50=4.461 max=4.634 +w1 wss round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=23606.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=136.90 latency_ms_p50=0.154 p95=0.313 p99=0.402 elapsed_ms=847.232 frames/s=312726.5 proxy_io_cpu_ns/frame=7201 tcp_out_segments/frame=n/a +w1 wss round1 proxy-1: modded server_to_client_MiB/s=136.90 login_burst_ms_p50=66.954 max=73.345 +w1 wss round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32989.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=191.31 latency_ms_p50=0.110 p95=0.216 p99=0.305 elapsed_ms=606.250 frames/s=437034.0 proxy_io_cpu_ns/frame=4366 tcp_out_segments/frame=n/a +w1 wss round1 proxy-2: modded server_to_client_MiB/s=191.31 login_burst_ms_p50=5.720 max=8.659 +w1 wss round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=69806.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=404.82 latency_ms_p50=0.052 p95=0.093 p99=0.118 elapsed_ms=286.506 frames/s=924769.3 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w1 wss round1 direct-2: modded server_to_client_MiB/s=404.82 login_burst_ms_p50=1.939 max=2.191 +w1 ws-coalesce round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=63304.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=367.11 latency_ms_p50=0.056 p95=0.114 p99=0.143 elapsed_ms=315.933 frames/s=838632.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 direct-1: modded server_to_client_MiB/s=367.11 login_burst_ms_p50=4.210 max=4.619 +w1 ws-coalesce round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=40158.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=232.89 latency_ms_p50=0.097 p95=0.165 p99=0.214 elapsed_ms=498.021 frames/s=532009.3 proxy_io_cpu_ns/frame=3478 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 proxy-1: modded server_to_client_MiB/s=232.89 login_burst_ms_p50=10.221 max=11.944 +w1 ws-coalesce round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=50994.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=295.73 latency_ms_p50=0.072 p95=0.127 p99=0.160 elapsed_ms=392.199 frames/s=675555.9 proxy_io_cpu_ns/frame=2497 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 proxy-2: modded server_to_client_MiB/s=295.73 login_burst_ms_p50=4.078 max=4.732 +w1 ws-coalesce round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=76788.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=445.31 latency_ms_p50=0.048 p95=0.082 p99=0.103 elapsed_ms=260.457 frames/s=1017256.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 direct-2: modded server_to_client_MiB/s=445.31 login_burst_ms_p50=2.020 max=2.131 +w1 ws-coalesce round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=65405.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=379.30 latency_ms_p50=0.054 p95=0.109 p99=0.137 elapsed_ms=305.785 frames/s=866466.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 direct-1: modded server_to_client_MiB/s=379.30 login_burst_ms_p50=4.330 max=4.421 +w1 ws-coalesce round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41823.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=242.54 latency_ms_p50=0.089 p95=0.164 p99=0.215 elapsed_ms=478.198 frames/s=554063.4 proxy_io_cpu_ns/frame=3323 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 proxy-1: modded server_to_client_MiB/s=242.54 login_burst_ms_p50=10.284 max=15.108 +w1 ws-coalesce round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=54871.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=318.21 latency_ms_p50=0.065 p95=0.121 p99=0.150 elapsed_ms=364.491 frames/s=726908.7 proxy_io_cpu_ns/frame=2266 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 proxy-2: modded server_to_client_MiB/s=318.21 login_burst_ms_p50=3.414 max=4.909 +w1 ws-coalesce round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=76930.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=446.13 latency_ms_p50=0.047 p95=0.082 p99=0.105 elapsed_ms=259.976 frames/s=1019141.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 direct-2: modded server_to_client_MiB/s=446.13 login_burst_ms_p50=1.982 max=2.210 +w1 wss round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59042.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=342.40 latency_ms_p50=0.060 p95=0.119 p99=0.145 elapsed_ms=338.739 frames/s=782170.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round2 direct-1: modded server_to_client_MiB/s=342.40 login_burst_ms_p50=4.937 max=5.430 +w1 wss round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=22680.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=131.53 latency_ms_p50=0.159 p95=0.318 p99=0.422 elapsed_ms=881.817 frames/s=300461.4 proxy_io_cpu_ns/frame=7529 tcp_out_segments/frame=n/a +w1 wss round2 proxy-1: modded server_to_client_MiB/s=131.53 login_burst_ms_p50=52.112 max=54.008 +w1 wss round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35144.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=203.81 latency_ms_p50=0.103 p95=0.197 p99=0.280 elapsed_ms=569.078 frames/s=465581.2 proxy_io_cpu_ns/frame=4217 tcp_out_segments/frame=n/a +w1 wss round2 proxy-2: modded server_to_client_MiB/s=203.81 login_burst_ms_p50=5.233 max=7.204 +w1 wss round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=67848.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=393.46 latency_ms_p50=0.053 p95=0.095 p99=0.118 elapsed_ms=294.774 frames/s=898832.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round2 direct-2: modded server_to_client_MiB/s=393.46 login_burst_ms_p50=1.989 max=2.151 +w1 ws round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59406.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=344.50 latency_ms_p50=0.059 p95=0.118 p99=0.152 elapsed_ms=336.666 frames/s=786987.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round2 direct-1: modded server_to_client_MiB/s=344.50 login_burst_ms_p50=3.864 max=4.074 +w1 ws round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34914.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=202.47 latency_ms_p50=0.109 p95=0.187 p99=0.253 elapsed_ms=572.837 frames/s=462526.3 proxy_io_cpu_ns/frame=4208 tcp_out_segments/frame=n/a +w1 ws round2 proxy-1: modded server_to_client_MiB/s=202.47 login_burst_ms_p50=12.140 max=14.185 +w1 ws round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46610.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=270.30 latency_ms_p50=0.079 p95=0.138 p99=0.178 elapsed_ms=429.089 frames/s=617475.7 proxy_io_cpu_ns/frame=2885 tcp_out_segments/frame=n/a +w1 ws round2 proxy-2: modded server_to_client_MiB/s=270.30 login_burst_ms_p50=4.556 max=4.944 +w1 ws round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=62835.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=364.39 latency_ms_p50=0.056 p95=0.112 p99=0.146 elapsed_ms=318.290 frames/s=832423.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round2 direct-2: modded server_to_client_MiB/s=364.39 login_burst_ms_p50=2.300 max=2.671 +w1 tcp round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=61834.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=358.59 latency_ms_p50=0.058 p95=0.114 p99=0.143 elapsed_ms=323.443 frames/s=819160.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round2 direct-1: modded server_to_client_MiB/s=358.59 login_burst_ms_p50=4.047 max=4.555 +w1 tcp round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35336.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=204.92 latency_ms_p50=0.107 p95=0.180 p99=0.219 elapsed_ms=565.991 frames/s=468120.9 proxy_io_cpu_ns/frame=3038 tcp_out_segments/frame=n/a +w1 tcp round2 proxy-1: modded server_to_client_MiB/s=204.92 login_burst_ms_p50=6.925 max=9.701 +w1 tcp round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=39676.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=230.09 latency_ms_p50=0.091 p95=0.165 p99=0.225 elapsed_ms=504.077 frames/s=525618.1 proxy_io_cpu_ns/frame=2536 tcp_out_segments/frame=n/a +w1 tcp round2 proxy-2: modded server_to_client_MiB/s=230.09 login_burst_ms_p50=3.703 max=4.018 +w1 tcp round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=70069.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=406.34 latency_ms_p50=0.052 p95=0.093 p99=0.117 elapsed_ms=285.433 frames/s=928246.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round2 direct-2: modded server_to_client_MiB/s=406.34 login_burst_ms_p50=1.795 max=2.305 +w1 wss round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59400.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=344.47 latency_ms_p50=0.060 p95=0.119 p99=0.148 elapsed_ms=336.696 frames/s=786917.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round3 direct-1: modded server_to_client_MiB/s=344.47 login_burst_ms_p50=4.286 max=4.646 +w1 wss round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=21959.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=127.34 latency_ms_p50=0.157 p95=0.342 p99=0.503 elapsed_ms=910.782 frames/s=290906.1 proxy_io_cpu_ns/frame=7484 tcp_out_segments/frame=n/a +w1 wss round3 proxy-1: modded server_to_client_MiB/s=127.34 login_burst_ms_p50=55.163 max=60.688 +w1 wss round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30681.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=177.93 latency_ms_p50=0.120 p95=0.234 p99=0.332 elapsed_ms=651.858 frames/s=406456.5 proxy_io_cpu_ns/frame=4815 tcp_out_segments/frame=n/a +w1 wss round3 proxy-2: modded server_to_client_MiB/s=177.93 login_burst_ms_p50=8.359 max=9.651 +w1 wss round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=66962.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=388.32 latency_ms_p50=0.054 p95=0.096 p99=0.123 elapsed_ms=298.676 frames/s=887089.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round3 direct-2: modded server_to_client_MiB/s=388.32 login_burst_ms_p50=1.803 max=2.534 +w1 tcp round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=56364.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=326.87 latency_ms_p50=0.063 p95=0.125 p99=0.158 elapsed_ms=354.831 frames/s=746698.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round3 direct-1: modded server_to_client_MiB/s=326.87 login_burst_ms_p50=4.007 max=4.627 +w1 tcp round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34592.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=200.61 latency_ms_p50=0.109 p95=0.184 p99=0.225 elapsed_ms=578.166 frames/s=458263.1 proxy_io_cpu_ns/frame=3032 tcp_out_segments/frame=n/a +w1 tcp round3 proxy-1: modded server_to_client_MiB/s=200.61 login_burst_ms_p50=8.182 max=10.737 +w1 tcp round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=39073.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=226.60 latency_ms_p50=0.094 p95=0.168 p99=0.209 elapsed_ms=511.851 frames/s=517635.4 proxy_io_cpu_ns/frame=2636 tcp_out_segments/frame=n/a +w1 tcp round3 proxy-2: modded server_to_client_MiB/s=226.60 login_burst_ms_p50=3.311 max=3.809 +w1 tcp round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=56266.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=326.30 latency_ms_p50=0.064 p95=0.123 p99=0.151 elapsed_ms=355.454 frames/s=745390.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round3 direct-2: modded server_to_client_MiB/s=326.30 login_burst_ms_p50=2.093 max=2.236 +w1 ws-coalesce round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=62548.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=362.73 latency_ms_p50=0.056 p95=0.113 p99=0.142 elapsed_ms=319.751 frames/s=828618.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 direct-1: modded server_to_client_MiB/s=362.73 login_burst_ms_p50=4.687 max=4.785 +w1 ws-coalesce round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37060.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=214.92 latency_ms_p50=0.099 p95=0.188 p99=0.279 elapsed_ms=539.662 frames/s=490959.2 proxy_io_cpu_ns/frame=3612 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 proxy-1: modded server_to_client_MiB/s=214.92 login_burst_ms_p50=11.569 max=15.776 +w1 ws-coalesce round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=49567.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=287.45 latency_ms_p50=0.073 p95=0.134 p99=0.177 elapsed_ms=403.490 frames/s=656651.4 proxy_io_cpu_ns/frame=2572 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 proxy-2: modded server_to_client_MiB/s=287.45 login_burst_ms_p50=2.482 max=3.646 +w1 ws-coalesce round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=66340.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=384.72 latency_ms_p50=0.052 p95=0.107 p99=0.151 elapsed_ms=301.477 frames/s=878846.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 direct-2: modded server_to_client_MiB/s=384.72 login_burst_ms_p50=2.178 max=2.538 +w1 ws round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=48892.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=283.53 latency_ms_p50=0.074 p95=0.139 p99=0.175 elapsed_ms=409.064 frames/s=647703.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round3 direct-1: modded server_to_client_MiB/s=283.53 login_burst_ms_p50=5.198 max=5.388 +w1 ws round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33145.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=192.22 latency_ms_p50=0.115 p95=0.193 p99=0.247 elapsed_ms=603.394 frames/s=439102.7 proxy_io_cpu_ns/frame=4709 tcp_out_segments/frame=n/a +w1 ws round3 proxy-1: modded server_to_client_MiB/s=192.22 login_burst_ms_p50=9.908 max=13.010 +w1 ws round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41820.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=242.52 latency_ms_p50=0.091 p95=0.151 p99=0.196 elapsed_ms=478.239 frames/s=554015.5 proxy_io_cpu_ns/frame=3321 tcp_out_segments/frame=n/a +w1 ws round3 proxy-2: modded server_to_client_MiB/s=242.52 login_burst_ms_p50=4.331 max=5.503 +w1 ws round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=63775.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=369.84 latency_ms_p50=0.056 p95=0.106 p99=0.134 elapsed_ms=313.599 frames/s=844874.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round3 direct-2: modded server_to_client_MiB/s=369.84 login_burst_ms_p50=2.098 max=2.148 +w8 tcp round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45432.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=263.47 latency_ms_p50=0.598 p95=1.073 p99=3.084 elapsed_ms=440.216 frames/s=601868.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round1 direct-1: modded server_to_client_MiB/s=263.47 login_burst_ms_p50=4.418 max=4.763 +w8 tcp round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37194.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=215.70 latency_ms_p50=0.690 p95=1.879 p99=4.183 elapsed_ms=537.713 frames/s=492738.4 proxy_io_cpu_ns/frame=3975 tcp_out_segments/frame=n/a +w8 tcp round1 proxy-1: modded server_to_client_MiB/s=215.70 login_burst_ms_p50=10.193 max=11.028 +w8 tcp round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35834.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=207.81 latency_ms_p50=0.757 p95=1.884 p99=3.702 elapsed_ms=558.126 frames/s=474717.4 proxy_io_cpu_ns/frame=4457 tcp_out_segments/frame=n/a +w8 tcp round1 proxy-2: modded server_to_client_MiB/s=207.81 login_burst_ms_p50=5.235 max=6.439 +w8 tcp round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46691.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=270.77 latency_ms_p50=0.637 p95=1.039 p99=1.158 elapsed_ms=428.348 frames/s=618544.1 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 tcp round1 direct-2: modded server_to_client_MiB/s=270.77 login_burst_ms_p50=2.336 max=2.777 +w8 ws round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44973.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=260.81 latency_ms_p50=0.671 p95=1.046 p99=1.286 elapsed_ms=444.707 frames/s=595789.4 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round1 direct-1: modded server_to_client_MiB/s=260.81 login_burst_ms_p50=4.482 max=5.063 +w8 ws round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=67445.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=391.13 latency_ms_p50=0.404 p95=0.913 p99=1.350 elapsed_ms=296.536 frames/s=893491.3 proxy_io_cpu_ns/frame=3208 tcp_out_segments/frame=n/a +w8 ws round1 proxy-1: modded server_to_client_MiB/s=391.13 login_burst_ms_p50=12.084 max=15.123 +w8 ws round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=82703.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=479.61 latency_ms_p50=0.322 p95=0.725 p99=1.450 elapsed_ms=241.827 frames/s=1095626.6 proxy_io_cpu_ns/frame=2725 tcp_out_segments/frame=n/a +w8 ws round1 proxy-2: modded server_to_client_MiB/s=479.61 login_burst_ms_p50=3.199 max=4.072 +w8 ws round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44619.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=258.76 latency_ms_p50=0.684 p95=1.024 p99=1.197 elapsed_ms=448.235 frames/s=591100.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round1 direct-2: modded server_to_client_MiB/s=258.76 login_burst_ms_p50=2.805 max=3.508 +w8 wss round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47401.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=274.89 latency_ms_p50=0.628 p95=1.038 p99=1.229 elapsed_ms=421.928 frames/s=627955.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round1 direct-1: modded server_to_client_MiB/s=274.89 login_burst_ms_p50=4.729 max=5.702 +w8 wss round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30335.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=175.92 latency_ms_p50=0.922 p95=1.965 p99=2.906 elapsed_ms=659.293 frames/s=401872.8 proxy_io_cpu_ns/frame=7799 tcp_out_segments/frame=n/a +w8 wss round1 proxy-1: modded server_to_client_MiB/s=175.92 login_burst_ms_p50=77.045 max=82.726 +w8 wss round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46689.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=270.76 latency_ms_p50=0.596 p95=1.145 p99=1.789 elapsed_ms=428.363 frames/s=618521.7 proxy_io_cpu_ns/frame=4815 tcp_out_segments/frame=n/a +w8 wss round1 proxy-2: modded server_to_client_MiB/s=270.76 login_burst_ms_p50=6.075 max=8.768 +w8 wss round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45942.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=266.43 latency_ms_p50=0.665 p95=0.995 p99=1.162 elapsed_ms=435.325 frames/s=608630.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round1 direct-2: modded server_to_client_MiB/s=266.43 login_burst_ms_p50=2.012 max=3.485 +w8 ws-coalesce round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47554.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=275.78 latency_ms_p50=0.634 p95=0.967 p99=1.073 elapsed_ms=420.567 frames/s=629987.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 direct-1: modded server_to_client_MiB/s=275.78 login_burst_ms_p50=4.725 max=5.361 +w8 ws-coalesce round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=86918.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=504.06 latency_ms_p50=0.302 p95=0.741 p99=1.187 elapsed_ms=230.100 frames/s=1151466.5 proxy_io_cpu_ns/frame=2712 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 proxy-1: modded server_to_client_MiB/s=504.06 login_burst_ms_p50=11.270 max=12.312 +w8 ws-coalesce round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=107676.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=624.43 latency_ms_p50=0.264 p95=0.475 p99=0.616 elapsed_ms=185.742 frames/s=1426449.2 proxy_io_cpu_ns/frame=2348 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 proxy-2: modded server_to_client_MiB/s=624.43 login_burst_ms_p50=3.907 max=4.270 +w8 ws-coalesce round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=48194.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=279.49 latency_ms_p50=0.641 p95=0.955 p99=1.070 elapsed_ms=414.983 frames/s=638464.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 direct-2: modded server_to_client_MiB/s=279.49 login_burst_ms_p50=2.214 max=2.997 +w8 ws-coalesce round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47063.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=272.93 latency_ms_p50=0.642 p95=0.965 p99=1.067 elapsed_ms=424.960 frames/s=623475.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 direct-1: modded server_to_client_MiB/s=272.93 login_burst_ms_p50=4.090 max=4.847 +w8 ws-coalesce round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=84945.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=492.61 latency_ms_p50=0.347 p95=0.618 p99=0.827 elapsed_ms=235.445 frames/s=1125324.4 proxy_io_cpu_ns/frame=2611 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 proxy-1: modded server_to_client_MiB/s=492.61 login_burst_ms_p50=10.105 max=11.316 +w8 ws-coalesce round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=104292.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=604.81 latency_ms_p50=0.278 p95=0.486 p99=0.652 elapsed_ms=191.769 frames/s=1381621.2 proxy_io_cpu_ns/frame=2099 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 proxy-2: modded server_to_client_MiB/s=604.81 login_burst_ms_p50=4.003 max=4.252 +w8 ws-coalesce round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45758.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=265.36 latency_ms_p50=0.662 p95=0.970 p99=1.078 elapsed_ms=437.082 frames/s=606184.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 direct-2: modded server_to_client_MiB/s=265.36 login_burst_ms_p50=2.132 max=2.256 +w8 wss round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47499.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=275.46 latency_ms_p50=0.628 p95=1.034 p99=1.202 elapsed_ms=421.053 frames/s=629260.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round2 direct-1: modded server_to_client_MiB/s=275.46 login_burst_ms_p50=4.270 max=5.115 +w8 wss round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33560.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=194.62 latency_ms_p50=0.764 p95=1.808 p99=2.396 elapsed_ms=595.944 frames/s=444592.2 proxy_io_cpu_ns/frame=6952 tcp_out_segments/frame=n/a +w8 wss round2 proxy-1: modded server_to_client_MiB/s=194.62 login_burst_ms_p50=70.047 max=72.153 +w8 wss round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46968.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=272.38 latency_ms_p50=0.574 p95=1.262 p99=1.916 elapsed_ms=425.814 frames/s=622225.3 proxy_io_cpu_ns/frame=4850 tcp_out_segments/frame=n/a +w8 wss round2 proxy-2: modded server_to_client_MiB/s=272.38 login_burst_ms_p50=8.910 max=14.386 +w8 wss round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46022.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=266.89 latency_ms_p50=0.643 p95=1.026 p99=1.147 elapsed_ms=434.573 frames/s=609683.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round2 direct-2: modded server_to_client_MiB/s=266.89 login_burst_ms_p50=2.206 max=2.916 +w8 ws round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46837.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=271.62 latency_ms_p50=0.634 p95=1.027 p99=1.154 elapsed_ms=427.007 frames/s=620486.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round2 direct-1: modded server_to_client_MiB/s=271.62 login_burst_ms_p50=4.135 max=5.256 +w8 ws round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=70718.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=410.11 latency_ms_p50=0.403 p95=0.756 p99=1.321 elapsed_ms=282.812 frames/s=936850.2 proxy_io_cpu_ns/frame=2776 tcp_out_segments/frame=n/a +w8 ws round2 proxy-1: modded server_to_client_MiB/s=410.11 login_burst_ms_p50=11.749 max=14.319 +w8 ws round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=83925.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=486.70 latency_ms_p50=0.307 p95=0.629 p99=0.942 elapsed_ms=238.307 frames/s=1111809.4 proxy_io_cpu_ns/frame=2726 tcp_out_segments/frame=n/a +w8 ws round2 proxy-2: modded server_to_client_MiB/s=486.70 login_burst_ms_p50=3.278 max=3.791 +w8 ws round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47907.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=277.82 latency_ms_p50=0.625 p95=1.038 p99=1.154 elapsed_ms=417.469 frames/s=634662.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round2 direct-2: modded server_to_client_MiB/s=277.82 login_burst_ms_p50=2.525 max=3.033 +w8 tcp round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44460.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=257.83 latency_ms_p50=0.670 p95=1.079 p99=1.218 elapsed_ms=449.842 frames/s=588988.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round2 direct-1: modded server_to_client_MiB/s=257.83 login_burst_ms_p50=4.341 max=5.621 +w8 tcp round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35242.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=204.38 latency_ms_p50=0.827 p95=1.493 p99=2.916 elapsed_ms=567.499 frames/s=466877.0 proxy_io_cpu_ns/frame=4006 tcp_out_segments/frame=n/a +w8 tcp round2 proxy-1: modded server_to_client_MiB/s=204.38 login_burst_ms_p50=9.919 max=11.424 +w8 tcp round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30839.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=178.85 latency_ms_p50=1.000 p95=1.433 p99=2.125 elapsed_ms=648.510 frames/s=408554.9 proxy_io_cpu_ns/frame=5282 tcp_out_segments/frame=n/a +w8 tcp round2 proxy-2: modded server_to_client_MiB/s=178.85 login_burst_ms_p50=5.144 max=5.243 +w8 tcp round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46142.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=267.59 latency_ms_p50=0.649 p95=1.071 p99=1.203 elapsed_ms=433.440 frames/s=611277.9 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 tcp round2 direct-2: modded server_to_client_MiB/s=267.59 login_burst_ms_p50=1.921 max=2.335 +w8 wss round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47197.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=273.71 latency_ms_p50=0.629 p95=1.057 p99=1.226 elapsed_ms=423.750 frames/s=625255.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round3 direct-1: modded server_to_client_MiB/s=273.71 login_burst_ms_p50=5.093 max=5.875 +w8 wss round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32518.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=188.58 latency_ms_p50=0.853 p95=1.788 p99=2.362 elapsed_ms=615.026 frames/s=430797.8 proxy_io_cpu_ns/frame=7545 tcp_out_segments/frame=n/a +w8 wss round3 proxy-1: modded server_to_client_MiB/s=188.58 login_burst_ms_p50=74.948 max=76.406 +w8 wss round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44003.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=255.18 latency_ms_p50=0.627 p95=1.233 p99=1.691 elapsed_ms=454.513 frames/s=582935.4 proxy_io_cpu_ns/frame=5332 tcp_out_segments/frame=n/a +w8 wss round3 proxy-2: modded server_to_client_MiB/s=255.18 login_burst_ms_p50=14.210 max=15.055 +w8 wss round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45401.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=263.29 latency_ms_p50=0.650 p95=1.048 p99=1.697 elapsed_ms=440.517 frames/s=601457.6 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 wss round3 direct-2: modded server_to_client_MiB/s=263.29 login_burst_ms_p50=2.086 max=3.026 +w8 tcp round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41713.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=241.90 latency_ms_p50=0.668 p95=1.103 p99=2.942 elapsed_ms=479.467 frames/s=552596.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round3 direct-1: modded server_to_client_MiB/s=241.90 login_burst_ms_p50=4.464 max=4.889 +w8 tcp round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36502.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=211.69 latency_ms_p50=0.693 p95=1.616 p99=4.033 elapsed_ms=547.904 frames/s=483573.7 proxy_io_cpu_ns/frame=3708 tcp_out_segments/frame=n/a +w8 tcp round3 proxy-1: modded server_to_client_MiB/s=211.69 login_burst_ms_p50=7.543 max=14.291 +w8 tcp round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=31567.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=183.06 latency_ms_p50=0.989 p95=1.469 p99=2.055 elapsed_ms=633.568 frames/s=418190.1 proxy_io_cpu_ns/frame=4912 tcp_out_segments/frame=n/a +w8 tcp round3 proxy-2: modded server_to_client_MiB/s=183.06 login_burst_ms_p50=3.436 max=6.006 +w8 tcp round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45371.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=263.12 latency_ms_p50=0.657 p95=1.057 p99=1.169 elapsed_ms=440.806 frames/s=601063.3 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 tcp round3 direct-2: modded server_to_client_MiB/s=263.12 login_burst_ms_p50=2.407 max=2.715 +w8 ws-coalesce round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45113.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=261.62 latency_ms_p50=0.678 p95=0.974 p99=1.089 elapsed_ms=443.325 frames/s=597647.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 direct-1: modded server_to_client_MiB/s=261.62 login_burst_ms_p50=4.092 max=5.100 +w8 ws-coalesce round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=69931.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=405.54 latency_ms_p50=0.364 p95=0.837 p99=1.430 elapsed_ms=285.996 frames/s=926419.6 proxy_io_cpu_ns/frame=2989 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 proxy-1: modded server_to_client_MiB/s=405.54 login_burst_ms_p50=10.606 max=14.774 +w8 ws-coalesce round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=104539.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=606.24 latency_ms_p50=0.267 p95=0.522 p99=0.810 elapsed_ms=191.315 frames/s=1384898.0 proxy_io_cpu_ns/frame=2329 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 proxy-2: modded server_to_client_MiB/s=606.24 login_burst_ms_p50=3.840 max=6.884 +w8 ws-coalesce round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45492.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=263.82 latency_ms_p50=0.679 p95=0.956 p99=1.096 elapsed_ms=439.633 frames/s=602666.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 direct-2: modded server_to_client_MiB/s=263.82 login_burst_ms_p50=2.856 max=3.122 +w8 ws round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47381.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=274.77 latency_ms_p50=0.647 p95=0.974 p99=1.123 elapsed_ms=422.109 frames/s=627685.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round3 direct-1: modded server_to_client_MiB/s=274.77 login_burst_ms_p50=4.681 max=4.940 +w8 ws round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=64613.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=374.70 latency_ms_p50=0.444 p95=0.836 p99=1.221 elapsed_ms=309.533 frames/s=855973.6 proxy_io_cpu_ns/frame=3061 tcp_out_segments/frame=n/a +w8 ws round3 proxy-1: modded server_to_client_MiB/s=374.70 login_burst_ms_p50=11.101 max=13.143 +w8 ws round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=62275.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=361.15 latency_ms_p50=0.347 p95=0.871 p99=2.914 elapsed_ms=321.153 frames/s=825002.5 proxy_io_cpu_ns/frame=2885 tcp_out_segments/frame=n/a +w8 ws round3 proxy-2: modded server_to_client_MiB/s=361.15 login_burst_ms_p50=3.455 max=3.811 +w8 ws round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=27703.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=160.65 latency_ms_p50=0.450 p95=1.916 p99=18.411 elapsed_ms=721.941 frames/s=366999.6 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 ws round3 direct-2: modded server_to_client_MiB/s=160.65 login_burst_ms_p50=2.636 max=3.675 + +## 2. Baseline matrix, run 2 (TCP client fixed; discarded: host under external load, see record) +w1 tcp round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=76622.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=444.35 latency_ms_p50=0.048 p95=0.088 p99=0.120 elapsed_ms=261.020 frames/s=1015062.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round1 direct-1: modded server_to_client_MiB/s=444.35 login_burst_ms_p50=3.943 max=4.751 +w1 tcp round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=29453.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=170.81 latency_ms_p50=0.122 p95=0.217 p99=0.350 elapsed_ms=679.032 frames/s=390190.5 proxy_io_cpu_ns/frame=4250 tcp_out_segments/frame=n/a +w1 tcp round1 proxy-1: modded server_to_client_MiB/s=170.81 login_burst_ms_p50=7.714 max=11.669 +w1 tcp round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35460.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=205.64 latency_ms_p50=0.108 p95=0.161 p99=0.196 elapsed_ms=564.009 frames/s=469765.4 proxy_io_cpu_ns/frame=3662 tcp_out_segments/frame=n/a +w1 tcp round1 proxy-2: modded server_to_client_MiB/s=205.64 login_burst_ms_p50=8.220 max=8.453 +w1 tcp round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=63254.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=366.82 latency_ms_p50=0.058 p95=0.097 p99=0.122 elapsed_ms=316.184 frames/s=837968.9 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w1 tcp round1 direct-2: modded server_to_client_MiB/s=366.82 login_burst_ms_p50=3.268 max=3.867 +w1 ws round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=80619.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=467.52 latency_ms_p50=0.046 p95=0.081 p99=0.104 elapsed_ms=248.079 frames/s=1068014.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round1 direct-1: modded server_to_client_MiB/s=467.52 login_burst_ms_p50=3.230 max=3.616 +w1 ws round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37102.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=215.17 latency_ms_p50=0.103 p95=0.171 p99=0.218 elapsed_ms=539.042 frames/s=491523.8 proxy_io_cpu_ns/frame=3947 tcp_out_segments/frame=n/a +w1 ws round1 proxy-1: modded server_to_client_MiB/s=215.17 login_burst_ms_p50=11.492 max=12.653 +w1 ws round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43434.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=251.88 latency_ms_p50=0.087 p95=0.142 p99=0.180 elapsed_ms=460.463 frames/s=575403.8 proxy_io_cpu_ns/frame=3129 tcp_out_segments/frame=n/a +w1 ws round1 proxy-2: modded server_to_client_MiB/s=251.88 login_burst_ms_p50=2.740 max=4.491 +w1 ws round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=84369.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=489.27 latency_ms_p50=0.043 p95=0.078 p99=0.097 elapsed_ms=237.051 frames/s=1117698.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round1 direct-2: modded server_to_client_MiB/s=489.27 login_burst_ms_p50=1.842 max=2.335 +w1 wss round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=78709.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=456.45 latency_ms_p50=0.048 p95=0.083 p99=0.109 elapsed_ms=254.098 frames/s=1042716.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round1 direct-1: modded server_to_client_MiB/s=456.45 login_burst_ms_p50=3.242 max=3.528 +w1 wss round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=21050.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=122.07 latency_ms_p50=0.167 p95=0.339 p99=0.488 elapsed_ms=950.115 frames/s=278863.1 proxy_io_cpu_ns/frame=7834 tcp_out_segments/frame=n/a +w1 wss round1 proxy-1: modded server_to_client_MiB/s=122.07 login_burst_ms_p50=68.157 max=73.875 +w1 wss round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24093.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=139.72 latency_ms_p50=0.154 p95=0.266 p99=0.367 elapsed_ms=830.113 frames/s=319176.0 proxy_io_cpu_ns/frame=6243 tcp_out_segments/frame=n/a +w1 wss round1 proxy-2: modded server_to_client_MiB/s=139.72 login_burst_ms_p50=9.530 max=11.515 +w1 wss round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=68974.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=400.00 latency_ms_p50=0.054 p95=0.088 p99=0.108 elapsed_ms=289.961 frames/s=913750.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round1 direct-2: modded server_to_client_MiB/s=400.00 login_burst_ms_p50=1.845 max=3.023 +w1 ws-coalesce round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=61427.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=356.23 latency_ms_p50=0.052 p95=0.105 p99=0.221 elapsed_ms=325.588 frames/s=813765.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 direct-1: modded server_to_client_MiB/s=356.23 login_burst_ms_p50=3.601 max=4.177 +w1 ws-coalesce round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33751.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=195.73 latency_ms_p50=0.096 p95=0.232 p99=0.348 elapsed_ms=592.567 frames/s=447125.8 proxy_io_cpu_ns/frame=4813 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 proxy-1: modded server_to_client_MiB/s=195.73 login_burst_ms_p50=16.636 max=18.772 +w1 ws-coalesce round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35508.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=205.92 latency_ms_p50=0.095 p95=0.196 p99=0.295 elapsed_ms=563.245 frames/s=470402.5 proxy_io_cpu_ns/frame=3675 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 proxy-2: modded server_to_client_MiB/s=205.92 login_burst_ms_p50=4.190 max=6.224 +w1 ws-coalesce round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=61239.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=355.14 latency_ms_p50=0.051 p95=0.101 p99=0.168 elapsed_ms=326.586 frames/s=811278.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round1 direct-2: modded server_to_client_MiB/s=355.14 login_burst_ms_p50=4.079 max=4.612 +w1 ws-coalesce round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47484.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=275.37 latency_ms_p50=0.072 p95=0.151 p99=0.250 elapsed_ms=421.186 frames/s=629061.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 direct-1: modded server_to_client_MiB/s=275.37 login_burst_ms_p50=5.879 max=8.795 +w1 ws-coalesce round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=20389.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=118.24 latency_ms_p50=0.165 p95=0.326 p99=0.600 elapsed_ms=980.923 frames/s=270104.8 proxy_io_cpu_ns/frame=6219 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 proxy-1: modded server_to_client_MiB/s=118.24 login_burst_ms_p50=23.091 max=23.617 +w1 ws-coalesce round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=20905.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=121.24 latency_ms_p50=0.151 p95=0.345 p99=0.627 elapsed_ms=956.675 frames/s=276950.9 proxy_io_cpu_ns/frame=4998 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 proxy-2: modded server_to_client_MiB/s=121.24 login_burst_ms_p50=6.373 max=9.330 +w1 ws-coalesce round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45860.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=265.95 latency_ms_p50=0.073 p95=0.151 p99=0.278 elapsed_ms=436.106 frames/s=607541.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round2 direct-2: modded server_to_client_MiB/s=265.95 login_burst_ms_p50=2.746 max=3.109 +w1 wss round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41845.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=242.67 latency_ms_p50=0.078 p95=0.160 p99=0.301 elapsed_ms=477.950 frames/s=554351.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round2 direct-1: modded server_to_client_MiB/s=242.67 login_burst_ms_p50=6.396 max=6.760 +w1 wss round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=10432.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=60.50 latency_ms_p50=0.303 p95=0.728 p99=1.486 elapsed_ms=1917.110 frames/s=138203.9 proxy_io_cpu_ns/frame=15465 tcp_out_segments/frame=n/a +w1 wss round2 proxy-1: modded server_to_client_MiB/s=60.50 login_burst_ms_p50=175.999 max=183.148 +w1 wss round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=15905.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=92.24 latency_ms_p50=0.196 p95=0.452 p99=0.749 elapsed_ms=1257.447 frames/s=210706.4 proxy_io_cpu_ns/frame=8107 tcp_out_segments/frame=n/a +w1 wss round2 proxy-2: modded server_to_client_MiB/s=92.24 login_burst_ms_p50=15.052 max=17.237 +w1 wss round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46348.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=268.78 latency_ms_p50=0.077 p95=0.133 p99=0.215 elapsed_ms=431.517 frames/s=614001.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round2 direct-2: modded server_to_client_MiB/s=268.78 login_burst_ms_p50=2.357 max=4.139 +w1 ws round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=84473.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=489.87 latency_ms_p50=0.043 p95=0.079 p99=0.101 elapsed_ms=236.762 frames/s=1119065.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round2 direct-1: modded server_to_client_MiB/s=489.87 login_burst_ms_p50=3.003 max=3.469 +w1 ws round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35033.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=203.17 latency_ms_p50=0.110 p95=0.176 p99=0.228 elapsed_ms=570.876 frames/s=464114.6 proxy_io_cpu_ns/frame=4258 tcp_out_segments/frame=n/a +w1 ws round2 proxy-1: modded server_to_client_MiB/s=203.17 login_burst_ms_p50=10.456 max=14.998 +w1 ws round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=38671.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=224.26 latency_ms_p50=0.099 p95=0.151 p99=0.187 elapsed_ms=517.182 frames/s=512298.9 proxy_io_cpu_ns/frame=3565 tcp_out_segments/frame=n/a +w1 ws round2 proxy-2: modded server_to_client_MiB/s=224.26 login_burst_ms_p50=3.625 max=3.888 +w1 ws round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=77001.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=446.54 latency_ms_p50=0.047 p95=0.082 p99=0.101 elapsed_ms=259.735 frames/s=1020085.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round2 direct-2: modded server_to_client_MiB/s=446.54 login_burst_ms_p50=2.142 max=2.171 +w1 tcp round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=84056.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=487.46 latency_ms_p50=0.044 p95=0.080 p99=0.104 elapsed_ms=237.934 frames/s=1113551.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round2 direct-1: modded server_to_client_MiB/s=487.46 login_burst_ms_p50=3.165 max=3.444 +w1 tcp round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41014.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=237.85 latency_ms_p50=0.095 p95=0.145 p99=0.188 elapsed_ms=487.631 frames/s=543345.6 proxy_io_cpu_ns/frame=3307 tcp_out_segments/frame=n/a +w1 tcp round2 proxy-1: modded server_to_client_MiB/s=237.85 login_burst_ms_p50=7.236 max=11.441 +w1 tcp round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45615.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=264.53 latency_ms_p50=0.084 p95=0.130 p99=0.164 elapsed_ms=438.446 frames/s=604297.5 proxy_io_cpu_ns/frame=2863 tcp_out_segments/frame=n/a +w1 tcp round2 proxy-2: modded server_to_client_MiB/s=264.53 login_burst_ms_p50=5.733 max=6.450 +w1 tcp round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=85655.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=496.73 latency_ms_p50=0.041 p95=0.078 p99=0.096 elapsed_ms=233.494 frames/s=1134725.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round2 direct-2: modded server_to_client_MiB/s=496.73 login_burst_ms_p50=2.036 max=2.179 +w1 wss round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=81143.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=470.56 latency_ms_p50=0.046 p95=0.082 p99=0.111 elapsed_ms=246.477 frames/s=1074956.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round3 direct-1: modded server_to_client_MiB/s=470.56 login_burst_ms_p50=3.437 max=4.265 +w1 wss round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=18998.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=110.17 latency_ms_p50=0.184 p95=0.358 p99=0.511 elapsed_ms=1052.737 frames/s=251679.2 proxy_io_cpu_ns/frame=9001 tcp_out_segments/frame=n/a +w1 wss round3 proxy-1: modded server_to_client_MiB/s=110.17 login_burst_ms_p50=70.791 max=74.910 +w1 wss round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=17888.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=103.74 latency_ms_p50=0.197 p95=0.371 p99=0.568 elapsed_ms=1118.054 frames/s=236976.0 proxy_io_cpu_ns/frame=7915 tcp_out_segments/frame=n/a +w1 wss round3 proxy-2: modded server_to_client_MiB/s=103.74 login_burst_ms_p50=8.182 max=10.504 +w1 wss round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=55057.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=319.29 latency_ms_p50=0.067 p95=0.109 p99=0.133 elapsed_ms=363.255 frames/s=729383.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 wss round3 direct-2: modded server_to_client_MiB/s=319.29 login_burst_ms_p50=2.722 max=2.909 +w1 tcp round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=83953.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=486.86 latency_ms_p50=0.043 p95=0.081 p99=0.105 elapsed_ms=238.227 frames/s=1112184.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 tcp round3 direct-1: modded server_to_client_MiB/s=486.86 login_burst_ms_p50=3.432 max=4.039 +w1 tcp round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32383.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=187.80 latency_ms_p50=0.112 p95=0.203 p99=0.306 elapsed_ms=617.592 frames/s=429008.3 proxy_io_cpu_ns/frame=3795 tcp_out_segments/frame=n/a +w1 tcp round3 proxy-1: modded server_to_client_MiB/s=187.80 login_burst_ms_p50=8.567 max=10.007 +w1 tcp round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35227.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=204.29 latency_ms_p50=0.109 p95=0.160 p99=0.196 elapsed_ms=567.745 frames/s=466674.1 proxy_io_cpu_ns/frame=3681 tcp_out_segments/frame=n/a +w1 tcp round3 proxy-2: modded server_to_client_MiB/s=204.29 login_burst_ms_p50=5.878 max=6.646 +w1 tcp round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59904.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=347.39 latency_ms_p50=0.062 p95=0.101 p99=0.127 elapsed_ms=333.867 frames/s=793585.6 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w1 tcp round3 direct-2: modded server_to_client_MiB/s=347.39 login_burst_ms_p50=2.758 max=2.854 +w1 ws-coalesce round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32060.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=185.92 latency_ms_p50=0.087 p95=0.227 p99=0.568 elapsed_ms=623.831 frames/s=424717.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 direct-1: modded server_to_client_MiB/s=185.92 login_burst_ms_p50=6.235 max=7.222 +w1 ws-coalesce round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=12242.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=71.00 latency_ms_p50=0.199 p95=0.662 p99=3.086 elapsed_ms=1633.608 frames/s=162188.2 proxy_io_cpu_ns/frame=7262 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 proxy-1: modded server_to_client_MiB/s=71.00 login_burst_ms_p50=35.900 max=40.947 +w1 ws-coalesce round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=22722.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=131.77 latency_ms_p50=0.159 p95=0.265 p99=0.431 elapsed_ms=880.196 frames/s=301014.9 proxy_io_cpu_ns/frame=5240 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 proxy-2: modded server_to_client_MiB/s=131.77 login_burst_ms_p50=4.927 max=5.945 +w1 ws-coalesce round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=50007.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=290.00 latency_ms_p50=0.074 p95=0.118 p99=0.157 elapsed_ms=399.942 frames/s=662476.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws-coalesce round3 direct-2: modded server_to_client_MiB/s=290.00 login_burst_ms_p50=3.230 max=3.953 +w1 ws round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33097.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=191.94 latency_ms_p50=0.080 p95=0.194 p99=0.468 elapsed_ms=604.277 frames/s=438460.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round3 direct-1: modded server_to_client_MiB/s=191.94 login_burst_ms_p50=5.573 max=6.395 +w1 ws round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=12933.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=75.00 latency_ms_p50=0.236 p95=0.527 p99=1.125 elapsed_ms=1546.415 frames/s=171333.1 proxy_io_cpu_ns/frame=9466 tcp_out_segments/frame=n/a +w1 ws round3 proxy-1: modded server_to_client_MiB/s=75.00 login_burst_ms_p50=26.914 max=34.107 +w1 ws round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=13824.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=80.17 latency_ms_p50=0.206 p95=0.583 p99=1.585 elapsed_ms=1446.682 frames/s=183144.6 proxy_io_cpu_ns/frame=6442 tcp_out_segments/frame=n/a +w1 ws round3 proxy-2: modded server_to_client_MiB/s=80.17 login_burst_ms_p50=8.274 max=11.573 +w1 ws round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24283.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=140.82 latency_ms_p50=0.095 p95=0.267 p99=0.783 elapsed_ms=823.603 frames/s=321698.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w1 ws round3 direct-2: modded server_to_client_MiB/s=140.82 login_burst_ms_p50=3.472 max=4.170 +w8 tcp round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47878.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=277.65 latency_ms_p50=0.278 p95=1.200 p99=6.206 elapsed_ms=417.729 frames/s=634268.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round1 direct-1: modded server_to_client_MiB/s=277.65 login_burst_ms_p50=8.195 max=19.895 +w8 tcp round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=26243.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=152.19 latency_ms_p50=0.752 p95=2.626 p99=8.459 elapsed_ms=762.095 frames/s=347662.6 proxy_io_cpu_ns/frame=4737 tcp_out_segments/frame=n/a +w8 tcp round1 proxy-1: modded server_to_client_MiB/s=152.19 login_burst_ms_p50=25.697 max=26.880 +w8 tcp round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30144.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=174.81 latency_ms_p50=0.474 p95=1.754 p99=7.627 elapsed_ms=663.466 frames/s=399345.2 proxy_io_cpu_ns/frame=3120 tcp_out_segments/frame=n/a +w8 tcp round1 proxy-2: modded server_to_client_MiB/s=174.81 login_burst_ms_p50=10.187 max=14.712 +w8 tcp round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59126.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=342.88 latency_ms_p50=0.270 p95=0.757 p99=5.912 elapsed_ms=338.260 frames/s=783278.1 proxy_io_cpu_ns/frame=1 tcp_out_segments/frame=n/a +w8 tcp round1 direct-2: modded server_to_client_MiB/s=342.88 login_burst_ms_p50=4.446 max=8.351 +w8 ws round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34162.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=198.11 latency_ms_p50=0.233 p95=1.385 p99=15.976 elapsed_ms=585.443 frames/s=452566.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round1 direct-1: modded server_to_client_MiB/s=198.11 login_burst_ms_p50=7.480 max=7.793 +w8 ws round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=12866.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=74.61 latency_ms_p50=0.872 p95=7.975 p99=39.368 elapsed_ms=1554.485 frames/s=170443.5 proxy_io_cpu_ns/frame=7356 tcp_out_segments/frame=n/a +w8 ws round1 proxy-1: modded server_to_client_MiB/s=74.61 login_burst_ms_p50=55.716 max=60.834 +w8 ws round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=23686.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=137.36 latency_ms_p50=0.561 p95=2.231 p99=22.486 elapsed_ms=844.348 frames/s=313795.0 proxy_io_cpu_ns/frame=4876 tcp_out_segments/frame=n/a +w8 ws round1 proxy-2: modded server_to_client_MiB/s=137.36 login_burst_ms_p50=10.136 max=12.484 +w8 ws round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=56942.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=330.22 latency_ms_p50=0.227 p95=0.675 p99=4.256 elapsed_ms=351.229 frames/s=754357.2 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round1 direct-2: modded server_to_client_MiB/s=330.22 login_burst_ms_p50=7.405 max=9.709 +w8 wss round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37243.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=215.98 latency_ms_p50=0.312 p95=1.117 p99=16.179 elapsed_ms=537.004 frames/s=493388.9 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round1 direct-1: modded server_to_client_MiB/s=215.98 login_burst_ms_p50=8.096 max=8.200 +w8 wss round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=10523.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=61.03 latency_ms_p50=1.682 p95=9.358 p99=27.041 elapsed_ms=1900.520 frames/s=139410.3 proxy_io_cpu_ns/frame=13455 tcp_out_segments/frame=n/a +w8 wss round1 proxy-1: modded server_to_client_MiB/s=61.03 login_burst_ms_p50=414.155 max=455.923 +w8 wss round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=16727.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=97.01 latency_ms_p50=1.112 p95=4.004 p99=18.654 elapsed_ms=1195.609 frames/s=221604.2 proxy_io_cpu_ns/frame=9078 tcp_out_segments/frame=n/a +w8 wss round1 proxy-2: modded server_to_client_MiB/s=97.01 login_burst_ms_p50=20.053 max=21.309 +w8 wss round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=70506.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=408.88 latency_ms_p50=0.433 p95=0.555 p99=1.235 elapsed_ms=283.662 frames/s=934040.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round1 direct-2: modded server_to_client_MiB/s=408.88 login_burst_ms_p50=3.523 max=4.890 +w8 ws-coalesce round1 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=81872.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=474.79 latency_ms_p50=0.230 p95=0.602 p99=3.347 elapsed_ms=244.284 frames/s=1084607.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 direct-1: modded server_to_client_MiB/s=474.79 login_burst_ms_p50=4.558 max=4.851 +w8 ws-coalesce round1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34284.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=198.82 latency_ms_p50=0.575 p95=2.014 p99=7.907 elapsed_ms=583.346 frames/s=454193.2 proxy_io_cpu_ns/frame=5377 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 proxy-1: modded server_to_client_MiB/s=198.82 login_burst_ms_p50=23.617 max=26.075 +w8 ws-coalesce round1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41258.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=239.27 latency_ms_p50=0.487 p95=1.294 p99=6.055 elapsed_ms=484.744 frames/s=546581.5 proxy_io_cpu_ns/frame=4308 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 proxy-2: modded server_to_client_MiB/s=239.27 login_burst_ms_p50=9.976 max=11.937 +w8 ws-coalesce round1 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=98761.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=572.73 latency_ms_p50=0.313 p95=0.419 p99=0.675 elapsed_ms=202.508 frames/s=1308350.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round1 direct-2: modded server_to_client_MiB/s=572.73 login_burst_ms_p50=4.286 max=6.024 +w8 ws-coalesce round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=88746.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=514.66 latency_ms_p50=0.238 p95=0.743 p99=2.532 elapsed_ms=225.360 frames/s=1175682.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 direct-1: modded server_to_client_MiB/s=514.66 login_burst_ms_p50=4.145 max=5.122 +w8 ws-coalesce round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34399.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=199.49 latency_ms_p50=0.573 p95=1.699 p99=4.487 elapsed_ms=581.411 frames/s=455705.4 proxy_io_cpu_ns/frame=5007 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 proxy-1: modded server_to_client_MiB/s=199.49 login_burst_ms_p50=21.060 max=27.079 +w8 ws-coalesce round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=47058.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=272.90 latency_ms_p50=0.447 p95=1.296 p99=4.874 elapsed_ms=425.002 frames/s=623413.3 proxy_io_cpu_ns/frame=3830 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 proxy-2: modded server_to_client_MiB/s=272.90 login_burst_ms_p50=6.718 max=7.179 +w8 ws-coalesce round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=97027.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=562.68 latency_ms_p50=0.308 p95=0.411 p99=0.715 elapsed_ms=206.126 frames/s=1285387.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round2 direct-2: modded server_to_client_MiB/s=562.68 login_burst_ms_p50=3.331 max=5.009 +w8 wss round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=63878.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=370.44 latency_ms_p50=0.467 p95=0.868 p99=1.475 elapsed_ms=313.096 frames/s=846232.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round2 direct-1: modded server_to_client_MiB/s=370.44 login_burst_ms_p50=5.238 max=5.813 +w8 wss round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=19304.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=111.95 latency_ms_p50=1.535 p95=2.883 p99=4.137 elapsed_ms=1036.007 frames/s=255743.4 proxy_io_cpu_ns/frame=12674 tcp_out_segments/frame=n/a +w8 wss round2 proxy-1: modded server_to_client_MiB/s=111.95 login_burst_ms_p50=119.653 max=130.606 +w8 wss round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=25150.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=145.85 latency_ms_p50=1.157 p95=2.192 p99=3.368 elapsed_ms=795.227 frames/s=333177.7 proxy_io_cpu_ns/frame=8342 tcp_out_segments/frame=n/a +w8 wss round2 proxy-2: modded server_to_client_MiB/s=145.85 login_burst_ms_p50=19.707 max=22.906 +w8 wss round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=66473.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=385.49 latency_ms_p50=0.483 p95=0.585 p99=0.960 elapsed_ms=300.872 frames/s=880613.4 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round2 direct-2: modded server_to_client_MiB/s=385.49 login_burst_ms_p50=2.756 max=4.467 +w8 ws round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=92330.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=535.44 latency_ms_p50=0.237 p95=0.711 p99=1.700 elapsed_ms=216.614 frames/s=1223152.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round2 direct-1: modded server_to_client_MiB/s=535.44 login_burst_ms_p50=3.764 max=3.853 +w8 ws round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=42634.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=247.24 latency_ms_p50=0.492 p95=1.414 p99=4.141 elapsed_ms=469.106 frames/s=564802.3 proxy_io_cpu_ns/frame=3757 tcp_out_segments/frame=n/a +w8 ws round2 proxy-1: modded server_to_client_MiB/s=247.24 login_burst_ms_p50=17.931 max=18.645 +w8 ws round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43595.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=252.81 latency_ms_p50=0.461 p95=1.112 p99=3.661 elapsed_ms=458.768 frames/s=577529.0 proxy_io_cpu_ns/frame=3754 tcp_out_segments/frame=n/a +w8 ws round2 proxy-2: modded server_to_client_MiB/s=252.81 login_burst_ms_p50=4.843 max=5.935 +w8 ws round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=52062.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=301.92 latency_ms_p50=0.443 p95=0.859 p99=3.960 elapsed_ms=384.154 frames/s=689702.3 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round2 direct-2: modded server_to_client_MiB/s=301.92 login_burst_ms_p50=4.109 max=4.507 +w8 tcp round2 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=87548.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=507.71 latency_ms_p50=0.225 p95=0.797 p99=2.140 elapsed_ms=228.446 frames/s=1159802.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round2 direct-1: modded server_to_client_MiB/s=507.71 login_burst_ms_p50=4.780 max=5.164 +w8 tcp round2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=55879.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=324.06 latency_ms_p50=0.439 p95=1.108 p99=2.477 elapsed_ms=357.911 frames/s=740272.6 proxy_io_cpu_ns/frame=3673 tcp_out_segments/frame=n/a +w8 tcp round2 proxy-1: modded server_to_client_MiB/s=324.06 login_burst_ms_p50=13.183 max=15.691 +w8 tcp round2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=58344.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=338.35 latency_ms_p50=0.430 p95=0.903 p99=1.981 elapsed_ms=342.791 frames/s=772925.5 proxy_io_cpu_ns/frame=3682 tcp_out_segments/frame=n/a +w8 tcp round2 proxy-2: modded server_to_client_MiB/s=338.35 login_burst_ms_p50=4.087 max=8.016 +w8 tcp round2 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=82091.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=476.06 latency_ms_p50=0.382 p95=0.466 p99=0.552 elapsed_ms=243.632 frames/s=1087508.7 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round2 direct-2: modded server_to_client_MiB/s=476.06 login_burst_ms_p50=3.011 max=3.328 +w8 wss round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=138111.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=800.93 latency_ms_p50=0.203 p95=0.392 p99=0.686 elapsed_ms=144.811 frames/s=1829642.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round3 direct-1: modded server_to_client_MiB/s=800.93 login_burst_ms_p50=4.054 max=4.160 +w8 wss round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32080.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=186.04 latency_ms_p50=0.871 p95=1.910 p99=2.592 elapsed_ms=623.434 frames/s=424988.1 proxy_io_cpu_ns/frame=8220 tcp_out_segments/frame=n/a +w8 wss round3 proxy-1: modded server_to_client_MiB/s=186.04 login_burst_ms_p50=51.910 max=55.573 +w8 wss round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43162.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=250.31 latency_ms_p50=0.654 p95=1.127 p99=1.750 elapsed_ms=463.366 frames/s=571798.5 proxy_io_cpu_ns/frame=5382 tcp_out_segments/frame=n/a +w8 wss round3 proxy-2: modded server_to_client_MiB/s=250.31 login_burst_ms_p50=13.237 max=14.929 +w8 wss round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=87015.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=504.61 latency_ms_p50=0.368 p95=0.447 p99=0.583 elapsed_ms=229.845 frames/s=1152740.0 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 wss round3 direct-2: modded server_to_client_MiB/s=504.61 login_burst_ms_p50=2.037 max=3.324 +w8 tcp round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=141861.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=822.67 latency_ms_p50=0.204 p95=0.403 p99=0.649 elapsed_ms=140.983 frames/s=1879318.8 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round3 direct-1: modded server_to_client_MiB/s=822.67 login_burst_ms_p50=3.685 max=3.791 +w8 tcp round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=84959.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=492.70 latency_ms_p50=0.303 p95=0.651 p99=1.369 elapsed_ms=235.405 frames/s=1125514.4 proxy_io_cpu_ns/frame=2419 tcp_out_segments/frame=n/a +w8 tcp round3 proxy-1: modded server_to_client_MiB/s=492.70 login_burst_ms_p50=6.460 max=9.767 +w8 tcp round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=95932.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=556.32 latency_ms_p50=0.298 p95=0.529 p99=0.825 elapsed_ms=208.481 frames/s=1270869.6 proxy_io_cpu_ns/frame=2472 tcp_out_segments/frame=n/a +w8 tcp round3 proxy-2: modded server_to_client_MiB/s=556.32 login_burst_ms_p50=3.970 max=4.554 +w8 tcp round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=129683.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=752.05 latency_ms_p50=0.241 p95=0.313 p99=0.353 elapsed_ms=154.222 frames/s=1717991.4 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 tcp round3 direct-2: modded server_to_client_MiB/s=752.05 login_burst_ms_p50=2.008 max=2.849 +w8 ws-coalesce round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=168878.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=979.35 latency_ms_p50=0.167 p95=0.347 p99=0.475 elapsed_ms=118.428 frames/s=2237234.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 direct-1: modded server_to_client_MiB/s=979.35 login_burst_ms_p50=3.550 max=3.748 +w8 ws-coalesce round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=79110.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=458.77 latency_ms_p50=0.341 p95=0.760 p99=1.077 elapsed_ms=252.811 frames/s=1048025.6 proxy_io_cpu_ns/frame=3090 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 proxy-1: modded server_to_client_MiB/s=458.77 login_burst_ms_p50=13.675 max=15.365 +w8 ws-coalesce round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=98922.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=573.66 latency_ms_p50=0.305 p95=0.490 p99=0.588 elapsed_ms=202.179 frames/s=1310480.1 proxy_io_cpu_ns/frame=2400 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 proxy-2: modded server_to_client_MiB/s=573.66 login_burst_ms_p50=2.489 max=2.888 +w8 ws-coalesce round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=165065.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=957.24 latency_ms_p50=0.177 p95=0.296 p99=0.412 elapsed_ms=121.164 frames/s=2186726.6 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws-coalesce round3 direct-2: modded server_to_client_MiB/s=957.24 login_burst_ms_p50=2.751 max=3.004 +w8 ws round3 direct-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=105998.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=614.70 latency_ms_p50=0.159 p95=0.463 p99=2.150 elapsed_ms=188.683 frames/s=1404220.1 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round3 direct-1: modded server_to_client_MiB/s=614.70 login_burst_ms_p50=3.543 max=3.809 +w8 ws round3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43381.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=251.57 latency_ms_p50=0.488 p95=1.654 p99=4.509 elapsed_ms=461.029 frames/s=574697.1 proxy_io_cpu_ns/frame=4469 tcp_out_segments/frame=n/a +w8 ws round3 proxy-1: modded server_to_client_MiB/s=251.57 login_burst_ms_p50=11.803 max=16.509 +w8 ws round3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=79236.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=459.50 latency_ms_p50=0.337 p95=0.638 p99=1.109 elapsed_ms=252.411 frames/s=1049686.7 proxy_io_cpu_ns/frame=3054 tcp_out_segments/frame=n/a +w8 ws round3 proxy-2: modded server_to_client_MiB/s=459.50 login_burst_ms_p50=4.252 max=4.570 +w8 ws round3 direct-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=91873.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=532.79 latency_ms_p50=0.244 p95=0.827 p99=1.793 elapsed_ms=217.690 frames/s=1217105.5 proxy_io_cpu_ns/frame=0 tcp_out_segments/frame=n/a +w8 ws round3 direct-2: modded server_to_client_MiB/s=532.79 login_burst_ms_p50=3.156 max=3.240 + +## 3. Paired: wss JDK vs OpenSSL (candidate jar, -Dio.netty.handler.ssl.noOpenSsl=true for JDK); ws main vs IdleStateHandler keepalive +w1 wss-jdk pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=22427.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=130.06 latency_ms_p50=0.157 p95=0.337 p99=0.467 elapsed_ms=891.745 frames/s=297116.4 proxy_io_cpu_ns/frame=7480 tcp_out_segments/frame=n/a +w1 wss-jdk pair1 proxy-1: modded server_to_client_MiB/s=130.06 login_burst_ms_p50=67.690 max=73.964 +w1 wss-jdk pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33573.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=194.70 latency_ms_p50=0.110 p95=0.202 p99=0.294 elapsed_ms=595.708 frames/s=444768.1 proxy_io_cpu_ns/frame=4389 tcp_out_segments/frame=n/a +w1 wss-jdk pair1 proxy-2: modded server_to_client_MiB/s=194.70 login_burst_ms_p50=6.203 max=6.779 +w1 wss-openssl pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24115.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=139.85 latency_ms_p50=0.150 p95=0.289 p99=0.424 elapsed_ms=829.331 frames/s=319476.7 proxy_io_cpu_ns/frame=5981 tcp_out_segments/frame=n/a +w1 wss-openssl pair1 proxy-1: modded server_to_client_MiB/s=139.85 login_burst_ms_p50=56.428 max=59.952 +w1 wss-openssl pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35736.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=207.24 latency_ms_p50=0.104 p95=0.185 p99=0.256 elapsed_ms=559.658 frames/s=473417.4 proxy_io_cpu_ns/frame=3926 tcp_out_segments/frame=n/a +w1 wss-openssl pair1 proxy-2: modded server_to_client_MiB/s=207.24 login_burst_ms_p50=6.724 max=7.080 +w1 ws-main pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35096.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=203.53 latency_ms_p50=0.109 p95=0.188 p99=0.241 elapsed_ms=569.862 frames/s=464940.5 proxy_io_cpu_ns/frame=4237 tcp_out_segments/frame=n/a +w1 ws-main pair1 proxy-1: modded server_to_client_MiB/s=203.53 login_burst_ms_p50=12.532 max=13.629 +w1 ws-main pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=46351.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=268.80 latency_ms_p50=0.080 p95=0.138 p99=0.174 elapsed_ms=431.489 frames/s=614041.1 proxy_io_cpu_ns/frame=2920 tcp_out_segments/frame=n/a +w1 ws-main pair1 proxy-2: modded server_to_client_MiB/s=268.80 login_burst_ms_p50=3.898 max=4.660 +w1 ws-keepalive pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32578.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=188.93 latency_ms_p50=0.115 p95=0.200 p99=0.260 elapsed_ms=613.895 frames/s=431592.0 proxy_io_cpu_ns/frame=4604 tcp_out_segments/frame=n/a +w1 ws-keepalive pair1 proxy-1: modded server_to_client_MiB/s=188.93 login_burst_ms_p50=11.121 max=13.377 +w1 ws-keepalive pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45398.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=263.27 latency_ms_p50=0.082 p95=0.142 p99=0.181 elapsed_ms=440.542 frames/s=601423.4 proxy_io_cpu_ns/frame=3013 tcp_out_segments/frame=n/a +w1 ws-keepalive pair1 proxy-2: modded server_to_client_MiB/s=263.27 login_burst_ms_p50=4.009 max=4.970 +w1 wss-openssl pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=25252.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=146.44 latency_ms_p50=0.148 p95=0.262 p99=0.366 elapsed_ms=792.012 frames/s=334530.1 proxy_io_cpu_ns/frame=6085 tcp_out_segments/frame=n/a +w1 wss-openssl pair2 proxy-1: modded server_to_client_MiB/s=146.44 login_burst_ms_p50=49.099 max=53.781 +w1 wss-openssl pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36647.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=212.52 latency_ms_p50=0.100 p95=0.181 p99=0.261 elapsed_ms=545.742 frames/s=485489.9 proxy_io_cpu_ns/frame=3779 tcp_out_segments/frame=n/a +w1 wss-openssl pair2 proxy-2: modded server_to_client_MiB/s=212.52 login_burst_ms_p50=6.288 max=6.584 +w1 wss-jdk pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=20294.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=117.69 latency_ms_p50=0.172 p95=0.341 p99=0.457 elapsed_ms=985.482 frames/s=268855.2 proxy_io_cpu_ns/frame=8550 tcp_out_segments/frame=n/a +w1 wss-jdk pair2 proxy-1: modded server_to_client_MiB/s=117.69 login_burst_ms_p50=59.382 max=62.601 +w1 wss-jdk pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24570.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=142.49 latency_ms_p50=0.129 p95=0.277 p99=0.599 elapsed_ms=813.997 frames/s=325494.9 proxy_io_cpu_ns/frame=4944 tcp_out_segments/frame=n/a +w1 wss-jdk pair2 proxy-2: modded server_to_client_MiB/s=142.49 login_burst_ms_p50=7.063 max=8.361 +w1 ws-keepalive pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35301.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=204.72 latency_ms_p50=0.109 p95=0.185 p99=0.236 elapsed_ms=566.550 frames/s=467658.3 proxy_io_cpu_ns/frame=4283 tcp_out_segments/frame=n/a +w1 ws-keepalive pair2 proxy-1: modded server_to_client_MiB/s=204.72 login_burst_ms_p50=14.300 max=17.489 +w1 ws-keepalive pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45836.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=265.81 latency_ms_p50=0.081 p95=0.139 p99=0.173 elapsed_ms=436.331 frames/s=607227.4 proxy_io_cpu_ns/frame=2972 tcp_out_segments/frame=n/a +w1 ws-keepalive pair2 proxy-2: modded server_to_client_MiB/s=265.81 login_burst_ms_p50=2.947 max=3.993 +w1 ws-main pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36378.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=210.97 latency_ms_p50=0.105 p95=0.180 p99=0.231 elapsed_ms=549.771 frames/s=481931.6 proxy_io_cpu_ns/frame=4066 tcp_out_segments/frame=n/a +w1 ws-main pair2 proxy-1: modded server_to_client_MiB/s=210.97 login_burst_ms_p50=9.634 max=11.638 +w1 ws-main pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45039.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=261.19 latency_ms_p50=0.082 p95=0.143 p99=0.182 elapsed_ms=444.051 frames/s=596670.8 proxy_io_cpu_ns/frame=2991 tcp_out_segments/frame=n/a +w1 ws-main pair2 proxy-2: modded server_to_client_MiB/s=261.19 login_burst_ms_p50=3.903 max=4.392 +w1 wss-jdk pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=20294.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=117.69 latency_ms_p50=0.170 p95=0.363 p99=0.520 elapsed_ms=985.511 frames/s=268847.3 proxy_io_cpu_ns/frame=8540 tcp_out_segments/frame=n/a +w1 wss-jdk pair3 proxy-1: modded server_to_client_MiB/s=117.69 login_burst_ms_p50=58.410 max=59.254 +w1 wss-jdk pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=29465.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=170.87 latency_ms_p50=0.128 p95=0.228 p99=0.316 elapsed_ms=678.768 frames/s=390342.2 proxy_io_cpu_ns/frame=5129 tcp_out_segments/frame=n/a +w1 wss-jdk pair3 proxy-2: modded server_to_client_MiB/s=170.87 login_burst_ms_p50=8.111 max=10.766 +w1 wss-openssl pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24860.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=144.17 latency_ms_p50=0.151 p95=0.265 p99=0.366 elapsed_ms=804.493 frames/s=329340.2 proxy_io_cpu_ns/frame=6201 tcp_out_segments/frame=n/a +w1 wss-openssl pair3 proxy-1: modded server_to_client_MiB/s=144.17 login_burst_ms_p50=56.702 max=60.574 +w1 wss-openssl pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37952.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=220.09 latency_ms_p50=0.094 p95=0.175 p99=0.244 elapsed_ms=526.982 frames/s=502772.4 proxy_io_cpu_ns/frame=3613 tcp_out_segments/frame=n/a +w1 wss-openssl pair3 proxy-2: modded server_to_client_MiB/s=220.09 login_burst_ms_p50=7.205 max=7.425 +w1 ws-main pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34579.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=200.53 latency_ms_p50=0.111 p95=0.187 p99=0.245 elapsed_ms=578.380 frames/s=458093.1 proxy_io_cpu_ns/frame=4353 tcp_out_segments/frame=n/a +w1 ws-main pair3 proxy-1: modded server_to_client_MiB/s=200.53 login_burst_ms_p50=13.204 max=16.755 +w1 ws-main pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45480.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=263.75 latency_ms_p50=0.082 p95=0.140 p99=0.180 elapsed_ms=439.752 frames/s=602502.9 proxy_io_cpu_ns/frame=2997 tcp_out_segments/frame=n/a +w1 ws-main pair3 proxy-2: modded server_to_client_MiB/s=263.75 login_burst_ms_p50=3.525 max=4.036 +w1 ws-keepalive pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34211.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=198.40 latency_ms_p50=0.112 p95=0.186 p99=0.239 elapsed_ms=584.601 frames/s=453218.4 proxy_io_cpu_ns/frame=4434 tcp_out_segments/frame=n/a +w1 ws-keepalive pair3 proxy-1: modded server_to_client_MiB/s=198.40 login_burst_ms_p50=13.037 max=14.119 +w1 ws-keepalive pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=42698.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=247.61 latency_ms_p50=0.089 p95=0.148 p99=0.189 elapsed_ms=468.402 frames/s=565650.7 proxy_io_cpu_ns/frame=3277 tcp_out_segments/frame=n/a +w1 ws-keepalive pair3 proxy-2: modded server_to_client_MiB/s=247.61 login_burst_ms_p50=3.260 max=4.714 +w1 wss-openssl pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=25625.5 warmup_roundtrips=2000 window=1 one_way_MiB/s=148.61 latency_ms_p50=0.147 p95=0.255 p99=0.353 elapsed_ms=780.472 frames/s=339476.8 proxy_io_cpu_ns/frame=6114 tcp_out_segments/frame=n/a +w1 wss-openssl pair4 proxy-1: modded server_to_client_MiB/s=148.61 login_burst_ms_p50=40.410 max=45.461 +w1 wss-openssl pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37935.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=219.99 latency_ms_p50=0.095 p95=0.173 p99=0.240 elapsed_ms=527.214 frames/s=502551.4 proxy_io_cpu_ns/frame=3604 tcp_out_segments/frame=n/a +w1 wss-openssl pair4 proxy-2: modded server_to_client_MiB/s=219.99 login_burst_ms_p50=5.861 max=6.696 +w1 wss-jdk pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=19149.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=111.05 latency_ms_p50=0.176 p95=0.342 p99=0.498 elapsed_ms=1044.398 frames/s=253688.7 proxy_io_cpu_ns/frame=8377 tcp_out_segments/frame=n/a +w1 wss-jdk pair4 proxy-1: modded server_to_client_MiB/s=111.05 login_burst_ms_p50=72.720 max=75.477 +w1 wss-jdk pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=29227.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=169.49 latency_ms_p50=0.131 p95=0.228 p99=0.324 elapsed_ms=684.299 frames/s=387187.6 proxy_io_cpu_ns/frame=5205 tcp_out_segments/frame=n/a +w1 wss-jdk pair4 proxy-2: modded server_to_client_MiB/s=169.49 login_burst_ms_p50=6.747 max=8.365 +w1 ws-keepalive pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35774.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=207.46 latency_ms_p50=0.107 p95=0.178 p99=0.236 elapsed_ms=559.062 frames/s=473922.7 proxy_io_cpu_ns/frame=4115 tcp_out_segments/frame=n/a +w1 ws-keepalive pair4 proxy-1: modded server_to_client_MiB/s=207.46 login_burst_ms_p50=12.596 max=14.371 +w1 ws-keepalive pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43230.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=250.70 latency_ms_p50=0.085 p95=0.148 p99=0.216 elapsed_ms=462.642 frames/s=572693.4 proxy_io_cpu_ns/frame=3130 tcp_out_segments/frame=n/a +w1 ws-keepalive pair4 proxy-2: modded server_to_client_MiB/s=250.70 login_burst_ms_p50=2.972 max=4.470 +w1 ws-main pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=33405.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=193.72 latency_ms_p50=0.114 p95=0.188 p99=0.242 elapsed_ms=598.708 frames/s=442539.5 proxy_io_cpu_ns/frame=4599 tcp_out_segments/frame=n/a +w1 ws-main pair4 proxy-1: modded server_to_client_MiB/s=193.72 login_burst_ms_p50=11.292 max=12.710 +w1 ws-main pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43381.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=251.57 latency_ms_p50=0.087 p95=0.146 p99=0.188 elapsed_ms=461.030 frames/s=574696.1 proxy_io_cpu_ns/frame=3180 tcp_out_segments/frame=n/a +w1 ws-main pair4 proxy-2: modded server_to_client_MiB/s=251.57 login_burst_ms_p50=3.602 max=4.921 +w8 wss-jdk pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=39409.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=228.54 latency_ms_p50=0.728 p95=1.455 p99=1.982 elapsed_ms=507.498 frames/s=522075.1 proxy_io_cpu_ns/frame=6259 tcp_out_segments/frame=n/a +w8 wss-jdk pair1 proxy-1: modded server_to_client_MiB/s=228.54 login_burst_ms_p50=143.162 max=146.442 +w8 wss-jdk pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=51545.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=298.92 latency_ms_p50=0.568 p95=1.036 p99=1.340 elapsed_ms=388.004 frames/s=682859.2 proxy_io_cpu_ns/frame=4611 tcp_out_segments/frame=n/a +w8 wss-jdk pair1 proxy-2: modded server_to_client_MiB/s=298.92 login_burst_ms_p50=5.697 max=9.157 +w8 wss-openssl pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44960.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=260.73 latency_ms_p50=0.627 p95=1.355 p99=2.015 elapsed_ms=444.835 frames/s=595618.2 proxy_io_cpu_ns/frame=5061 tcp_out_segments/frame=n/a +w8 wss-openssl pair1 proxy-1: modded server_to_client_MiB/s=260.73 login_burst_ms_p50=106.198 max=109.719 +w8 wss-openssl pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=57991.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=336.30 latency_ms_p50=0.508 p95=0.920 p99=1.168 elapsed_ms=344.881 frames/s=768242.8 proxy_io_cpu_ns/frame=3583 tcp_out_segments/frame=n/a +w8 wss-openssl pair1 proxy-2: modded server_to_client_MiB/s=336.30 login_burst_ms_p50=5.157 max=7.407 +w8 ws-main pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=68196.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=395.48 latency_ms_p50=0.429 p95=0.772 p99=1.036 elapsed_ms=293.268 frames/s=903445.9 proxy_io_cpu_ns/frame=2944 tcp_out_segments/frame=n/a +w8 ws-main pair1 proxy-1: modded server_to_client_MiB/s=395.48 login_burst_ms_p50=12.196 max=12.457 +w8 ws-main pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=72810.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=422.24 latency_ms_p50=0.313 p95=0.647 p99=1.747 elapsed_ms=274.684 frames/s=964569.9 proxy_io_cpu_ns/frame=2669 tcp_out_segments/frame=n/a +w8 ws-main pair1 proxy-2: modded server_to_client_MiB/s=422.24 login_burst_ms_p50=3.327 max=5.090 +w8 ws-keepalive pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=41110.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=238.41 latency_ms_p50=0.489 p95=1.465 p99=5.701 elapsed_ms=486.494 frames/s=544614.7 proxy_io_cpu_ns/frame=3530 tcp_out_segments/frame=n/a +w8 ws-keepalive pair1 proxy-1: modded server_to_client_MiB/s=238.41 login_burst_ms_p50=12.211 max=14.502 +w8 ws-keepalive pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=63787.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=369.91 latency_ms_p50=0.408 p95=1.003 p99=1.851 elapsed_ms=313.543 frames/s=845027.4 proxy_io_cpu_ns/frame=3008 tcp_out_segments/frame=n/a +w8 ws-keepalive pair1 proxy-2: modded server_to_client_MiB/s=369.91 login_burst_ms_p50=4.733 max=5.185 +w8 wss-openssl pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=25575.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=148.31 latency_ms_p50=0.699 p95=3.946 p99=9.558 elapsed_ms=782.010 frames/s=338809.1 proxy_io_cpu_ns/frame=4047 tcp_out_segments/frame=n/a +w8 wss-openssl pair2 proxy-1: modded server_to_client_MiB/s=148.31 login_burst_ms_p50=108.094 max=115.270 +w8 wss-openssl pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=56628.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=328.40 latency_ms_p50=0.437 p95=0.930 p99=1.580 elapsed_ms=353.177 frames/s=750196.9 proxy_io_cpu_ns/frame=3549 tcp_out_segments/frame=n/a +w8 wss-openssl pair2 proxy-2: modded server_to_client_MiB/s=328.40 login_burst_ms_p50=4.224 max=6.759 +w8 wss-jdk pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=29735.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=172.44 latency_ms_p50=0.875 p95=1.952 p99=2.970 elapsed_ms=672.591 frames/s=393927.4 proxy_io_cpu_ns/frame=7599 tcp_out_segments/frame=n/a +w8 wss-jdk pair2 proxy-1: modded server_to_client_MiB/s=172.44 login_burst_ms_p50=77.321 max=78.454 +w8 wss-jdk pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34320.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=199.03 latency_ms_p50=0.777 p95=1.608 p99=2.722 elapsed_ms=582.737 frames/s=454668.6 proxy_io_cpu_ns/frame=6279 tcp_out_segments/frame=n/a +w8 wss-jdk pair2 proxy-2: modded server_to_client_MiB/s=199.03 login_burst_ms_p50=9.495 max=13.896 +w8 ws-keepalive pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44335.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=257.11 latency_ms_p50=0.492 p95=1.350 p99=2.860 elapsed_ms=451.110 frames/s=587333.1 proxy_io_cpu_ns/frame=3780 tcp_out_segments/frame=n/a +w8 ws-keepalive pair2 proxy-1: modded server_to_client_MiB/s=257.11 login_burst_ms_p50=12.965 max=14.457 +w8 ws-keepalive pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=50301.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=291.70 latency_ms_p50=0.396 p95=0.825 p99=3.960 elapsed_ms=397.607 frames/s=666367.2 proxy_io_cpu_ns/frame=2813 tcp_out_segments/frame=n/a +w8 ws-keepalive pair2 proxy-2: modded server_to_client_MiB/s=291.70 login_burst_ms_p50=6.213 max=6.404 +w8 ws-main pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=59633.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=345.82 latency_ms_p50=0.406 p95=1.054 p99=1.974 elapsed_ms=335.384 frames/s=789996.3 proxy_io_cpu_ns/frame=3449 tcp_out_segments/frame=n/a +w8 ws-main pair2 proxy-1: modded server_to_client_MiB/s=345.82 login_burst_ms_p50=11.226 max=13.102 +w8 ws-main pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=80436.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=466.46 latency_ms_p50=0.352 p95=0.616 p99=0.846 elapsed_ms=248.644 frames/s=1065587.7 proxy_io_cpu_ns/frame=2667 tcp_out_segments/frame=n/a +w8 ws-main pair2 proxy-2: modded server_to_client_MiB/s=466.46 login_burst_ms_p50=2.957 max=4.702 +w8 wss-jdk pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=27206.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=157.78 latency_ms_p50=0.967 p95=2.139 p99=3.423 elapsed_ms=735.107 frames/s=360426.5 proxy_io_cpu_ns/frame=8393 tcp_out_segments/frame=n/a +w8 wss-jdk pair3 proxy-1: modded server_to_client_MiB/s=157.78 login_burst_ms_p50=64.478 max=69.160 +w8 wss-jdk pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=43318.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=251.21 latency_ms_p50=0.614 p95=1.132 p99=1.545 elapsed_ms=461.700 frames/s=573862.3 proxy_io_cpu_ns/frame=4827 tcp_out_segments/frame=n/a +w8 wss-jdk pair3 proxy-2: modded server_to_client_MiB/s=251.21 login_burst_ms_p50=7.807 max=8.741 +w8 wss-openssl pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=42310.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=245.36 latency_ms_p50=0.614 p95=1.303 p99=1.993 elapsed_ms=472.697 frames/s=560511.2 proxy_io_cpu_ns/frame=4673 tcp_out_segments/frame=n/a +w8 wss-openssl pair3 proxy-1: modded server_to_client_MiB/s=245.36 login_burst_ms_p50=47.378 max=48.552 +w8 wss-openssl pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=53384.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=309.58 latency_ms_p50=0.498 p95=0.980 p99=1.316 elapsed_ms=374.641 frames/s=707215.1 proxy_io_cpu_ns/frame=3877 tcp_out_segments/frame=n/a +w8 wss-openssl pair3 proxy-2: modded server_to_client_MiB/s=309.58 login_burst_ms_p50=6.872 max=9.627 +w8 ws-main pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=57617.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=334.13 latency_ms_p50=0.467 p95=0.982 p99=1.713 elapsed_ms=347.115 frames/s=763297.3 proxy_io_cpu_ns/frame=3210 tcp_out_segments/frame=n/a +w8 ws-main pair3 proxy-1: modded server_to_client_MiB/s=334.13 login_burst_ms_p50=12.258 max=13.366 +w8 ws-main pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=79300.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=459.88 latency_ms_p50=0.295 p95=0.667 p99=2.023 elapsed_ms=252.204 frames/s=1050546.4 proxy_io_cpu_ns/frame=2367 tcp_out_segments/frame=n/a +w8 ws-main pair3 proxy-2: modded server_to_client_MiB/s=459.88 login_burst_ms_p50=3.726 max=5.032 +w8 ws-keepalive pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=64156.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=372.05 latency_ms_p50=0.371 p95=0.880 p99=2.094 elapsed_ms=311.737 frames/s=849921.0 proxy_io_cpu_ns/frame=3137 tcp_out_segments/frame=n/a +w8 ws-keepalive pair3 proxy-1: modded server_to_client_MiB/s=372.05 login_burst_ms_p50=14.104 max=14.511 +w8 ws-keepalive pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=77199.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=447.69 latency_ms_p50=0.374 p95=0.650 p99=0.832 elapsed_ms=259.070 frames/s=1022703.8 proxy_io_cpu_ns/frame=2791 tcp_out_segments/frame=n/a +w8 ws-keepalive pair3 proxy-2: modded server_to_client_MiB/s=447.69 login_burst_ms_p50=2.651 max=4.187 +w8 wss-openssl pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=45198.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=262.11 latency_ms_p50=0.597 p95=1.304 p99=2.223 elapsed_ms=442.489 frames/s=598776.0 proxy_io_cpu_ns/frame=4906 tcp_out_segments/frame=n/a +w8 wss-openssl pair4 proxy-1: modded server_to_client_MiB/s=262.11 login_burst_ms_p50=53.288 max=57.083 +w8 wss-openssl pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=53777.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=311.87 latency_ms_p50=0.490 p95=1.068 p99=1.485 elapsed_ms=371.901 frames/s=712426.5 proxy_io_cpu_ns/frame=3970 tcp_out_segments/frame=n/a +w8 wss-openssl pair4 proxy-2: modded server_to_client_MiB/s=311.87 login_burst_ms_p50=6.731 max=8.480 +w8 wss-jdk pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=28003.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=162.40 latency_ms_p50=0.936 p95=2.101 p99=3.539 elapsed_ms=714.191 frames/s=370981.8 proxy_io_cpu_ns/frame=8411 tcp_out_segments/frame=n/a +w8 wss-jdk pair4 proxy-1: modded server_to_client_MiB/s=162.40 login_burst_ms_p50=78.766 max=81.678 +w8 wss-jdk pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=39320.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=228.02 latency_ms_p50=0.651 p95=1.471 p99=2.531 elapsed_ms=508.648 frames/s=520895.1 proxy_io_cpu_ns/frame=5613 tcp_out_segments/frame=n/a +w8 wss-jdk pair4 proxy-2: modded server_to_client_MiB/s=228.02 login_burst_ms_p50=12.631 max=13.411 +w8 ws-keepalive pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=64401.4 warmup_roundtrips=2000 window=8 one_way_MiB/s=373.47 latency_ms_p50=0.419 p95=0.902 p99=1.451 elapsed_ms=310.552 frames/s=853163.9 proxy_io_cpu_ns/frame=3542 tcp_out_segments/frame=n/a +w8 ws-keepalive pair4 proxy-1: modded server_to_client_MiB/s=373.47 login_burst_ms_p50=10.783 max=11.340 +w8 ws-keepalive pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=80595.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=467.39 latency_ms_p50=0.371 p95=0.638 p99=0.795 elapsed_ms=248.152 frames/s=1067702.6 proxy_io_cpu_ns/frame=2126 tcp_out_segments/frame=n/a +w8 ws-keepalive pair4 proxy-2: modded server_to_client_MiB/s=467.39 login_burst_ms_p50=3.014 max=4.426 +w8 ws-main pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=39356.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=228.23 latency_ms_p50=0.500 p95=1.574 p99=4.879 elapsed_ms=508.182 frames/s=521372.7 proxy_io_cpu_ns/frame=4061 tcp_out_segments/frame=n/a +w8 ws-main pair4 proxy-1: modded server_to_client_MiB/s=228.23 login_burst_ms_p50=13.381 max=16.990 +w8 ws-main pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=72521.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=420.56 latency_ms_p50=0.318 p95=0.809 p99=2.162 elapsed_ms=275.782 frames/s=960731.0 proxy_io_cpu_ns/frame=2765 tcp_out_segments/frame=n/a +w8 ws-main pair4 proxy-2: modded server_to_client_MiB/s=420.56 login_burst_ms_p50=3.787 max=5.202 + +## 4. Paired: ws main vs flag keepalive (discarded: external load, per-pair ratios 0.67-1.56) +w1 ws-main pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24594.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=142.62 latency_ms_p50=0.148 p95=0.245 p99=0.363 elapsed_ms=813.207 frames/s=325811.4 proxy_io_cpu_ns/frame=5755 tcp_out_segments/frame=n/a +w1 ws-main pair1 proxy-1: modded server_to_client_MiB/s=142.62 login_burst_ms_p50=12.784 max=15.951 +w1 ws-main pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=26131.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=151.54 latency_ms_p50=0.146 p95=0.214 p99=0.272 elapsed_ms=765.374 frames/s=346173.4 proxy_io_cpu_ns/frame=5311 tcp_out_segments/frame=n/a +w1 ws-main pair1 proxy-2: modded server_to_client_MiB/s=151.54 login_burst_ms_p50=5.265 max=5.918 +w1 ws-keepalive pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34315.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=199.00 latency_ms_p50=0.111 p95=0.194 p99=0.251 elapsed_ms=582.831 frames/s=454594.7 proxy_io_cpu_ns/frame=4456 tcp_out_segments/frame=n/a +w1 ws-keepalive pair1 proxy-1: modded server_to_client_MiB/s=199.00 login_burst_ms_p50=12.890 max=13.748 +w1 ws-keepalive pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36088.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=209.28 latency_ms_p50=0.090 p95=0.161 p99=0.271 elapsed_ms=554.190 frames/s=478089.1 proxy_io_cpu_ns/frame=3178 tcp_out_segments/frame=n/a +w1 ws-keepalive pair1 proxy-2: modded server_to_client_MiB/s=209.28 login_burst_ms_p50=2.921 max=3.616 +w1 ws-keepalive pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36316.3 warmup_roundtrips=2000 window=1 one_way_MiB/s=210.60 latency_ms_p50=0.105 p95=0.181 p99=0.230 elapsed_ms=550.717 frames/s=481104.1 proxy_io_cpu_ns/frame=4008 tcp_out_segments/frame=n/a +w1 ws-keepalive pair2 proxy-1: modded server_to_client_MiB/s=210.60 login_burst_ms_p50=11.617 max=14.471 +w1 ws-keepalive pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44853.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=260.11 latency_ms_p50=0.084 p95=0.140 p99=0.178 elapsed_ms=445.901 frames/s=594195.1 proxy_io_cpu_ns/frame=3047 tcp_out_segments/frame=n/a +w1 ws-keepalive pair2 proxy-2: modded server_to_client_MiB/s=260.11 login_burst_ms_p50=2.576 max=4.425 +w1 ws-main pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36437.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=211.30 latency_ms_p50=0.105 p95=0.180 p99=0.233 elapsed_ms=548.893 frames/s=482702.2 proxy_io_cpu_ns/frame=4061 tcp_out_segments/frame=n/a +w1 ws-main pair2 proxy-1: modded server_to_client_MiB/s=211.30 login_burst_ms_p50=15.207 max=17.833 +w1 ws-main pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=44444.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=257.74 latency_ms_p50=0.084 p95=0.143 p99=0.181 elapsed_ms=449.995 frames/s=588788.7 proxy_io_cpu_ns/frame=3073 tcp_out_segments/frame=n/a +w1 ws-main pair2 proxy-2: modded server_to_client_MiB/s=257.74 login_burst_ms_p50=2.759 max=3.912 +w1 ws-main pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=22362.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=129.68 latency_ms_p50=0.138 p95=0.345 p99=0.657 elapsed_ms=894.374 frames/s=296242.9 proxy_io_cpu_ns/frame=4123 tcp_out_segments/frame=n/a +w1 ws-main pair3 proxy-1: modded server_to_client_MiB/s=129.68 login_burst_ms_p50=14.144 max=14.782 +w1 ws-main pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24020.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=139.30 latency_ms_p50=0.137 p95=0.319 p99=0.489 elapsed_ms=832.619 frames/s=318215.1 proxy_io_cpu_ns/frame=4265 tcp_out_segments/frame=n/a +w1 ws-main pair3 proxy-2: modded server_to_client_MiB/s=139.30 login_burst_ms_p50=6.900 max=7.526 +w1 ws-keepalive pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=18345.2 warmup_roundtrips=2000 window=1 one_way_MiB/s=106.39 latency_ms_p50=0.188 p95=0.366 p99=0.591 elapsed_ms=1090.203 frames/s=243030.0 proxy_io_cpu_ns/frame=6881 tcp_out_segments/frame=n/a +w1 ws-keepalive pair3 proxy-1: modded server_to_client_MiB/s=106.39 login_burst_ms_p50=19.122 max=22.844 +w1 ws-keepalive pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=21661.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=125.62 latency_ms_p50=0.164 p95=0.286 p99=0.508 elapsed_ms=923.294 frames/s=286963.9 proxy_io_cpu_ns/frame=5784 tcp_out_segments/frame=n/a +w1 ws-keepalive pair3 proxy-2: modded server_to_client_MiB/s=125.62 login_burst_ms_p50=4.682 max=5.812 +w1 ws-keepalive pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=13973.6 warmup_roundtrips=2000 window=1 one_way_MiB/s=81.03 latency_ms_p50=0.217 p95=0.567 p99=1.217 elapsed_ms=1431.275 frames/s=185116.0 proxy_io_cpu_ns/frame=6783 tcp_out_segments/frame=n/a +w1 ws-keepalive pair4 proxy-1: modded server_to_client_MiB/s=81.03 login_burst_ms_p50=26.837 max=34.467 +w1 ws-keepalive pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=14082.9 warmup_roundtrips=2000 window=1 one_way_MiB/s=81.67 latency_ms_p50=0.200 p95=0.643 p99=1.600 elapsed_ms=1420.161 frames/s=186564.7 proxy_io_cpu_ns/frame=5858 tcp_out_segments/frame=n/a +w1 ws-keepalive pair4 proxy-2: modded server_to_client_MiB/s=81.67 login_burst_ms_p50=18.784 max=19.833 +w1 ws-main pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=13772.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=79.87 latency_ms_p50=0.224 p95=0.582 p99=1.459 elapsed_ms=1452.149 frames/s=182455.1 proxy_io_cpu_ns/frame=7013 tcp_out_segments/frame=n/a +w1 ws-main pair4 proxy-1: modded server_to_client_MiB/s=79.87 login_burst_ms_p50=21.745 max=22.895 +w1 ws-main pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=15952.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=92.51 latency_ms_p50=0.190 p95=0.520 p99=1.103 elapsed_ms=1253.752 frames/s=211327.2 proxy_io_cpu_ns/frame=5853 tcp_out_segments/frame=n/a +w1 ws-main pair4 proxy-2: modded server_to_client_MiB/s=92.51 login_burst_ms_p50=9.700 max=14.592 +w1 ws-main pair5 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=14659.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=85.01 latency_ms_p50=0.194 p95=0.526 p99=1.143 elapsed_ms=1364.342 frames/s=194197.6 proxy_io_cpu_ns/frame=6854 tcp_out_segments/frame=n/a +w1 ws-main pair5 proxy-1: modded server_to_client_MiB/s=85.01 login_burst_ms_p50=22.799 max=24.871 +w1 ws-main pair5 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=16294.7 warmup_roundtrips=2000 window=1 one_way_MiB/s=94.50 latency_ms_p50=0.180 p95=0.411 p99=1.360 elapsed_ms=1227.396 frames/s=215865.1 proxy_io_cpu_ns/frame=6251 tcp_out_segments/frame=n/a +w1 ws-main pair5 proxy-2: modded server_to_client_MiB/s=94.50 login_burst_ms_p50=6.997 max=8.195 +w1 ws-keepalive pair5 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32161.1 warmup_roundtrips=2000 window=1 one_way_MiB/s=186.51 latency_ms_p50=0.119 p95=0.185 p99=0.238 elapsed_ms=621.870 frames/s=426057.0 proxy_io_cpu_ns/frame=4688 tcp_out_segments/frame=n/a +w1 ws-keepalive pair5 proxy-1: modded server_to_client_MiB/s=186.51 login_burst_ms_p50=15.548 max=16.184 +w1 ws-keepalive pair5 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34515.0 warmup_roundtrips=2000 window=1 one_way_MiB/s=200.16 latency_ms_p50=0.112 p95=0.165 p99=0.203 elapsed_ms=579.458 frames/s=457241.2 proxy_io_cpu_ns/frame=4033 tcp_out_segments/frame=n/a +w1 ws-keepalive pair5 proxy-2: modded server_to_client_MiB/s=200.16 login_burst_ms_p50=3.776 max=5.097 +w1 ws-keepalive pair6 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30491.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=176.83 latency_ms_p50=0.123 p95=0.201 p99=0.260 elapsed_ms=655.915 frames/s=403942.8 proxy_io_cpu_ns/frame=4871 tcp_out_segments/frame=n/a +w1 ws-keepalive pair6 proxy-1: modded server_to_client_MiB/s=176.83 login_burst_ms_p50=11.868 max=14.450 +w1 ws-keepalive pair6 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32262.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=187.10 latency_ms_p50=0.119 p95=0.178 p99=0.220 elapsed_ms=619.908 frames/s=427405.2 proxy_io_cpu_ns/frame=4375 tcp_out_segments/frame=n/a +w1 ws-keepalive pair6 proxy-2: modded server_to_client_MiB/s=187.10 login_burst_ms_p50=3.266 max=4.622 +w1 ws-main pair6 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=18871.4 warmup_roundtrips=2000 window=1 one_way_MiB/s=109.44 latency_ms_p50=0.182 p95=0.353 p99=0.649 elapsed_ms=1059.807 frames/s=250000.1 proxy_io_cpu_ns/frame=7102 tcp_out_segments/frame=n/a +w1 ws-main pair6 proxy-1: modded server_to_client_MiB/s=109.44 login_burst_ms_p50=19.191 max=20.529 +w1 ws-main pair6 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24055.8 warmup_roundtrips=2000 window=1 one_way_MiB/s=139.50 latency_ms_p50=0.160 p95=0.236 p99=0.297 elapsed_ms=831.399 frames/s=318682.2 proxy_io_cpu_ns/frame=5802 tcp_out_segments/frame=n/a +w1 ws-main pair6 proxy-2: modded server_to_client_MiB/s=139.50 login_burst_ms_p50=6.211 max=7.406 +w8 ws-main pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=69198.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=401.29 latency_ms_p50=0.384 p95=0.868 p99=1.378 elapsed_ms=289.023 frames/s=916715.7 proxy_io_cpu_ns/frame=3250 tcp_out_segments/frame=n/a +w8 ws-main pair1 proxy-1: modded server_to_client_MiB/s=401.29 login_burst_ms_p50=11.886 max=15.323 +w8 ws-main pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=75934.0 warmup_roundtrips=2000 window=8 one_way_MiB/s=440.35 latency_ms_p50=0.373 p95=0.708 p99=1.103 elapsed_ms=263.387 frames/s=1005942.6 proxy_io_cpu_ns/frame=2686 tcp_out_segments/frame=n/a +w8 ws-main pair1 proxy-2: modded server_to_client_MiB/s=440.35 login_burst_ms_p50=2.875 max=4.208 +w8 ws-keepalive pair1 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=49922.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=289.51 latency_ms_p50=0.564 p95=0.998 p99=1.322 elapsed_ms=400.617 frames/s=661359.2 proxy_io_cpu_ns/frame=2626 tcp_out_segments/frame=n/a +w8 ws-keepalive pair1 proxy-1: modded server_to_client_MiB/s=289.51 login_burst_ms_p50=12.537 max=16.884 +w8 ws-keepalive pair1 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=53101.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=307.95 latency_ms_p50=0.433 p95=1.110 p99=3.595 elapsed_ms=376.635 frames/s=703471.3 proxy_io_cpu_ns/frame=3064 tcp_out_segments/frame=n/a +w8 ws-keepalive pair1 proxy-2: modded server_to_client_MiB/s=307.95 login_burst_ms_p50=4.011 max=5.162 +w8 ws-keepalive pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=24119.1 warmup_roundtrips=2000 window=8 one_way_MiB/s=139.87 latency_ms_p50=0.696 p95=2.202 p99=6.287 elapsed_ms=829.219 frames/s=319519.9 proxy_io_cpu_ns/frame=4509 tcp_out_segments/frame=n/a +w8 ws-keepalive pair2 proxy-1: modded server_to_client_MiB/s=139.87 login_burst_ms_p50=23.745 max=27.405 +w8 ws-keepalive pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=31821.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=184.54 latency_ms_p50=0.834 p95=1.965 p99=2.997 elapsed_ms=628.510 frames/s=421555.9 proxy_io_cpu_ns/frame=2739 tcp_out_segments/frame=n/a +w8 ws-keepalive pair2 proxy-2: modded server_to_client_MiB/s=184.54 login_burst_ms_p50=4.297 max=7.858 +w8 ws-main pair2 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=18438.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=106.93 latency_ms_p50=1.202 p95=5.287 p99=8.400 elapsed_ms=1084.670 frames/s=244269.6 proxy_io_cpu_ns/frame=3942 tcp_out_segments/frame=n/a +w8 ws-main pair2 proxy-1: modded server_to_client_MiB/s=106.93 login_burst_ms_p50=23.237 max=24.564 +w8 ws-main pair2 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35319.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=204.82 latency_ms_p50=0.681 p95=1.439 p99=4.600 elapsed_ms=566.255 frames/s=467902.0 proxy_io_cpu_ns/frame=2998 tcp_out_segments/frame=n/a +w8 ws-main pair2 proxy-2: modded server_to_client_MiB/s=204.82 login_burst_ms_p50=7.622 max=8.458 +w8 ws-main pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=21083.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=122.27 latency_ms_p50=0.776 p95=5.039 p99=10.594 elapsed_ms=948.619 frames/s=279303.0 proxy_io_cpu_ns/frame=4691 tcp_out_segments/frame=n/a +w8 ws-main pair3 proxy-1: modded server_to_client_MiB/s=122.27 login_burst_ms_p50=27.385 max=40.186 +w8 ws-main pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=34119.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=197.86 latency_ms_p50=0.710 p95=1.861 p99=4.469 elapsed_ms=586.174 frames/s=452002.7 proxy_io_cpu_ns/frame=3083 tcp_out_segments/frame=n/a +w8 ws-main pair3 proxy-2: modded server_to_client_MiB/s=197.86 login_burst_ms_p50=14.078 max=14.638 +w8 ws-keepalive pair3 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=14722.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=85.38 latency_ms_p50=1.068 p95=5.215 p99=24.152 elapsed_ms=1358.495 frames/s=195033.5 proxy_io_cpu_ns/frame=4925 tcp_out_segments/frame=n/a +w8 ws-keepalive pair3 proxy-1: modded server_to_client_MiB/s=85.38 login_burst_ms_p50=24.295 max=28.728 +w8 ws-keepalive pair3 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=30923.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=179.33 latency_ms_p50=0.485 p95=2.616 p99=6.525 elapsed_ms=646.761 frames/s=409660.1 proxy_io_cpu_ns/frame=3675 tcp_out_segments/frame=n/a +w8 ws-keepalive pair3 proxy-2: modded server_to_client_MiB/s=179.33 login_burst_ms_p50=5.921 max=8.029 +w8 ws-keepalive pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=17243.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=100.00 latency_ms_p50=0.903 p95=5.358 p99=21.217 elapsed_ms=1159.856 frames/s=228435.2 proxy_io_cpu_ns/frame=6442 tcp_out_segments/frame=n/a +w8 ws-keepalive pair4 proxy-1: modded server_to_client_MiB/s=100.00 login_burst_ms_p50=70.026 max=73.733 +w8 ws-keepalive pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=27984.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=162.29 latency_ms_p50=0.535 p95=2.157 p99=13.068 elapsed_ms=714.680 frames/s=370728.1 proxy_io_cpu_ns/frame=4555 tcp_out_segments/frame=n/a +w8 ws-keepalive pair4 proxy-2: modded server_to_client_MiB/s=162.29 login_burst_ms_p50=17.754 max=17.925 +w8 ws-main pair4 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=13564.2 warmup_roundtrips=2000 window=8 one_way_MiB/s=78.66 latency_ms_p50=1.336 p95=6.847 p99=11.519 elapsed_ms=1474.470 frames/s=179693.0 proxy_io_cpu_ns/frame=4374 tcp_out_segments/frame=n/a +w8 ws-main pair4 proxy-1: modded server_to_client_MiB/s=78.66 login_burst_ms_p50=24.940 max=26.714 +w8 ws-main pair4 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=35738.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=207.25 latency_ms_p50=0.685 p95=1.782 p99=5.733 elapsed_ms=559.617 frames/s=473452.1 proxy_io_cpu_ns/frame=2690 tcp_out_segments/frame=n/a +w8 ws-main pair4 proxy-2: modded server_to_client_MiB/s=207.25 login_burst_ms_p50=13.555 max=17.258 +w8 ws-main pair5 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=17382.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=100.81 latency_ms_p50=0.903 p95=5.140 p99=16.365 elapsed_ms=1150.554 frames/s=230282.0 proxy_io_cpu_ns/frame=5236 tcp_out_segments/frame=n/a +w8 ws-main pair5 proxy-1: modded server_to_client_MiB/s=100.81 login_burst_ms_p50=26.094 max=28.460 +w8 ws-main pair5 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=36547.8 warmup_roundtrips=2000 window=8 one_way_MiB/s=211.95 latency_ms_p50=0.711 p95=1.529 p99=3.834 elapsed_ms=547.229 frames/s=484170.6 proxy_io_cpu_ns/frame=3509 tcp_out_segments/frame=n/a +w8 ws-main pair5 proxy-2: modded server_to_client_MiB/s=211.95 login_burst_ms_p50=6.020 max=11.309 +w8 ws-keepalive pair5 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=16728.9 warmup_roundtrips=2000 window=8 one_way_MiB/s=97.01 latency_ms_p50=1.478 p95=4.431 p99=6.748 elapsed_ms=1195.533 frames/s=221618.3 proxy_io_cpu_ns/frame=3755 tcp_out_segments/frame=n/a +w8 ws-keepalive pair5 proxy-1: modded server_to_client_MiB/s=97.01 login_burst_ms_p50=17.932 max=25.117 +w8 ws-keepalive pair5 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=28985.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=168.09 latency_ms_p50=0.745 p95=3.061 p99=4.707 elapsed_ms=690.004 frames/s=383986.1 proxy_io_cpu_ns/frame=3058 tcp_out_segments/frame=n/a +w8 ws-keepalive pair5 proxy-2: modded server_to_client_MiB/s=168.09 login_burst_ms_p50=13.912 max=16.896 +w8 ws-keepalive pair6 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=32355.5 warmup_roundtrips=2000 window=8 one_way_MiB/s=187.63 latency_ms_p50=0.881 p95=1.725 p99=2.365 elapsed_ms=618.133 frames/s=428632.4 proxy_io_cpu_ns/frame=4135 tcp_out_segments/frame=n/a +w8 ws-keepalive pair6 proxy-1: modded server_to_client_MiB/s=187.63 login_burst_ms_p50=35.123 max=37.394 +w8 ws-keepalive pair6 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=37230.7 warmup_roundtrips=2000 window=8 one_way_MiB/s=215.91 latency_ms_p50=0.760 p95=1.442 p99=2.435 elapsed_ms=537.191 frames/s=493217.7 proxy_io_cpu_ns/frame=2822 tcp_out_segments/frame=n/a +w8 ws-keepalive pair6 proxy-2: modded server_to_client_MiB/s=215.91 login_burst_ms_p50=5.074 max=5.405 +w8 ws-main pair6 proxy-1: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=16948.6 warmup_roundtrips=2000 window=8 one_way_MiB/s=98.29 latency_ms_p50=1.341 p95=5.078 p99=7.215 elapsed_ms=1180.035 frames/s=224528.9 proxy_io_cpu_ns/frame=2942 tcp_out_segments/frame=n/a +w8 ws-main pair6 proxy-1: modded server_to_client_MiB/s=98.29 login_burst_ms_p50=28.294 max=30.341 +w8 ws-main pair6 proxy-2: roundtrips=20000 echoes=20000 correct=20000 errors=0 roundtrips/s=40665.3 warmup_roundtrips=2000 window=8 one_way_MiB/s=235.82 latency_ms_p50=0.705 p95=1.425 p99=2.318 elapsed_ms=491.820 frames/s=538717.6 proxy_io_cpu_ns/frame=2753 tcp_out_segments/frame=n/a +w8 ws-main pair6 proxy-2: modded server_to_client_MiB/s=235.82 login_burst_ms_p50=17.999 max=22.534 + +## 5. JFR exclusive attribution, moonbridge-session-io ExecutionSample, --window 8, 20000 ticks x4 +wss (JDK provider): +io samples 397 + 54.9% TLS + 14.4% WebSocket codec + 11.8% MoonBridge handlers + 7.1% Netty pipeline/buffers/other + 7.1% Minecraft-frame decode (ByteToMessageDecoder) + 4.8% socket read/write syscalls +ws: +io samples 153 + 31.4% WebSocket codec + 24.8% MoonBridge handlers + 18.3% socket read/write syscalls + 13.7% Minecraft-frame decode (ByteToMessageDecoder) + 11.8% Netty pipeline/buffers/other + +## 6. JFR keepalive attribution, ws --window 8, 30000 ticks x4 +IdleStateHandler candidate: + jdk.ExecutionSample: 20/154 samples in the keepalive (12.99%) + jdk.ObjectAllocationSample: 6/100 samples in the keepalive (6.00%), weight 15.0/517.2 MB +flag candidate: + jdk.ExecutionSample: 1/141 samples in the keepalive (0.71%) + jdk.ObjectAllocationSample: 0/100 samples in the keepalive (0.00%), weight 0.0/528.9 MB diff --git a/testing/benchmarks/results/2026-10-23-websocket-cdn.md b/testing/benchmarks/results/2026-10-23-websocket-cdn.md new file mode 100644 index 0000000..2a313e4 --- /dev/null +++ b/testing/benchmarks/results/2026-10-23-websocket-cdn.md @@ -0,0 +1,139 @@ +# WebSocket behind a CDN with modpack-shaped traffic: baseline, BoringSSL, keepalive — 2026-10-23 + +Base revision: `6224d3d` (main). This record covers the proxy half of the work: a modpack-shaped benchmark workload, the cost of `wss://` on the proxy, and a keepalive ping for CDN idle limits. The client-mod half (TCP_NODELAY, its own WebSocket codec, per-tick coalescing) has its own record. + +**Outcomes:** +- `--profile modded` and `--websocket-tls 1` added to `ProxySessionBenchmark`. +- **BoringSSL adopted** for `wss://` listeners: `netty-tcnative-boringssl-static` 2.0.84.Final is a runtime dependency, and Netty uses it whenever it loads. +- **WebSocket keepalive adopted in its flag form.** The first form, Netty's `IdleStateHandler`, failed its rule and was replaced. + +## Benchmark modes added + +`--profile modded` replaces the fixed-size echo with fixed-seed traffic shaped like a large 1.7.10 pack: +- **Client tick:** each roundtrip is one tick of 3 serverbound packets: a C17 mod custom payload, a C04 position packet and a C03 on-ground packet. +- **Server answer:** the backend replies with that tick's clientbound mix, on average 10.2 packets and 6.1 KB over 256 templates: + - 6–14 small packets: S3F custom payloads on mod channels, and S35 tile entity updates of 20–600 bytes; + - a 2–8 KiB payload on about one tick in three; + - a 30–60 KiB S21 chunk on about one in twenty. +- **Login burst:** after each Position and Look, the backend sends 70 × 30,000-byte S3F payloads (2.1 MB), standing in for Forge registry sync. Its wall time is reported as `login_burst_ms` and not counted in the timed phase. +- **Validation:** every clientbound packet carries the tick sequence and is checked byte for byte. +- **Counting:** frames and CPU per frame count both directions. + +Client sending modes: +- **Default:** the client writes and flushes each packet on its own, which over WebSocket is one message per packet, as the released client mod sends. +- **`--client-coalesce 1`:** one write per tick. + +`--websocket-tls 1` serves `wss://`: +- the certificate is a throwaway ECDSA P-256 one from the JDK's `keytool`; +- the proxy uses whichever `SslProvider` is available; +- the client is a JDK `SSLSocket`; +- `-Dio.netty.handler.ssl.noOpenSsl=true` forces the proxy onto the JDK provider. + +## Environment and method + +- **Hardware and software:** Apple M4 (10 cores), macOS, Zulu OpenJDK 25.0.4.1, Netty 4.2.18.Final, kqueue, loopback. +- **Settings:** `-Xms512m -Xmx512m`, `--connections 4 --messages 5000 --warmup 500 --repeats 2`, at `--window 1` and `--window 8`. +- **Ordering:** variant order rotated across JVM runs. +- **Errors:** every phase in every run below reports `errors=0`. + +**Host load.** +- The host was not quiet. `appstoreagent` held about one core for the whole session. +- From about 16:15, separate Uranium Gradle integration runs and another session's builds pushed the load average to 5–7 intermittently. +- Two runs were discarded for this (raw output kept, sections 2 and 4). Decisions rest on the clean run, on per-pair ratios and on JFR attribution, as noted below. + +## Baseline (clean run) + +Median and [min–max] of 6 proxy phases each. Run 1 started at 16:09, before the external load. Its plain-TCP rows are omitted. In that run the modded TCP client read its input unbuffered, one syscall per VarInt byte, so TCP was limited by the client. That was fixed for later runs; the WebSocket client was already buffered. + +| Window | Variant | Proxy frames/s | p50 ms | Proxy I/O CPU ns/frame | Login burst ms | +| --- | --- | ---: | ---: | ---: | ---: | +| 1 | ws | 544640 [439103–665242] | 0.093 [0.074–0.115] | 3444 [2668–4709] | 7.2 [2.5–12.1] | +| 1 | ws, client coalesces per tick | 605357 [490959–726909] | 0.081 [0.065–0.099] | 2948 [2266–3612] | 7.1 [2.5–11.6] | +| 1 | wss (JDK TLS) | 359592 [290906–465581] | 0.137 [0.103–0.159] | 6008 [4217–7529] | 30.2 [5.2–67.0] | +| 8 | ws | 915171 [825003–1111809] | 0.375 [0.307–0.444] | 2831 [2725–3208] | 7.3 [3.2–12.1] | +| 8 | ws, client coalesces per tick | 1266544 [926420–1426449] | 0.290 [0.264–0.364] | 2480 [2099–2989] | 7.1 [3.8–11.3] | +| 8 | wss (JDK TLS) | 513764 [401873–622225] | 0.696 [0.574–0.922] | 6142 [4815–7799] | 42.1 [6.1–77.0] | + +- **Client coalescing:** sending a tick as one message instead of three saves 12–14% of proxy CPU per frame. That is the proxy's share of the client-mod change. +- **wss with the JDK provider** costs about twice the proxy CPU of plain ws. + +A JFR `profile` recording at window 8 attributes each `moonbridge-session-io` CPU sample to the first component found walking down from the top frame (section 5 of the raw file): + +| Component | ws | wss (JDK TLS) | +| --- | ---: | ---: | +| TLS (SSLEngine, SslHandler, JDK crypto) | — | 54.9% | +| WebSocket codec | 31.4% | 14.4% | +| MoonBridge handlers | 24.8% | 11.8% | +| Minecraft frame decode | 13.7% | 7.1% | +| Socket syscalls | 18.3% | 4.8% | +| Other Netty | 11.8% | 7.1% | +| Samples | 153 | 397 | + +## BoringSSL for wss + +**Decision rule (from the plan):** +- Try OpenSSL only if TLS is the dominant per-frame cost under `modded` wss. It is: 55% of samples. +- Adopt only if wss proxy CPU per frame is at least 20% lower, with no drop in frames/s. + +**Method:** +- Same candidate jar for both variants, so only the provider differs. JDK was forced with `-Dio.netty.handler.ssl.noOpenSsl=true`, and the benchmark header confirmed `SslProvider JDK`. +- Four alternating pairs (8 phases per cell). + +| Window | Provider | Proxy frames/s | p50 ms | Proxy I/O CPU ns/frame | Login burst ms | +| --- | --- | ---: | ---: | ---: | ---: | +| 1 | JDK | 311306 [253689–444768] | 0.144 [0.110–0.176] | 6343 [4389–8550] | 33.3 [6.2–72.7] | +| 1 | OpenSSL | 406447 [319477–502772] | 0.126 [0.094–0.151] | 4954 [3604–6201] | 23.8 [5.9–56.7] | +| 8 | JDK | 487782 [360427–682859] | 0.752 [0.568–0.967] | 6269 [4611–8411] | 38.6 [5.7–143.2] | +| 8 | OpenSSL | 652996 [338809–768243] | 0.552 [0.437–0.699] | 4009 [3549–5061] | 27.1 [4.2–108.1] | + +Per-pair CPU ratios (OpenSSL / JDK): +- window 1: 0.83, 0.73, 0.72, 0.72; +- window 8: 0.80, 0.55, 0.65, 0.63. + +**Verdict:** the median is 22% lower at window 1 and 36% lower at window 8. Every pair is lower, and frames/s rose 31% and 34%. **Adopted.** + +**Cost of adopting:** +- The fat jar grows from 10.4 MB to 16.5 MB, with natives for linux-x86_64/aarch_64, osx-x86_64/aarch_64 and windows-x86_64. +- Other platforms fall back to the JDK provider. The startup line `Listening for WebSocket (TLS via OPENSSL|JDK)` says which is in use. +- `WebSocketListenerTest` now checks that the loaded context is OpenSSL, and that a TLS 1.2-only client, which is what the client mod offers, completes against it. + +## WebSocket keepalive + +The change: after the upgrade, the proxy sends an empty Ping when it has written nothing for `websocket.pingIntervalSeconds` (default 30, 0 disables). That keeps CDN and load-balancer idle timers, typically 60–100 s, from cutting quiet stretches: long modded logins, placement waits, stalled backends. + +This is a correctness change, but it sits on the write path, so it had its own rule: proxy CPU per frame on ws at most 5% above main. + +**First form: Netty's `IdleStateHandler(writerIdle)`.** It adds a listener to every outbound message. Measured in the same four pairs as the OpenSSL comparison: + +| Window | Variant | Proxy frames/s | Proxy I/O CPU ns/frame | +| --- | --- | ---: | ---: | +| 1 | main | 528314 [442540–614041] | 3623 [2920–4599] | +| 1 | IdleStateHandler | 519787 [431592–607227] | 3696 [2972–4604] | +| 8 | main | 932088 [521373–1065588] | 2855 [2367–4061] | +| 8 | IdleStateHandler | 847474 [544615–1067703] | 3073 [2126–3780] | + +- Per-pair ratios: window 1: 1.06, 1.03, 1.05, 0.93; window 8: 1.16, 1.08, 1.06, 0.83. +- At window 8 the median is +7.6%, so the rule fails. **Rejected.** + +**Second form, adopted: `WebSocketKeepalive`.** +- **Design:** `write` only sets a flag. A task every half interval pings if the last half interval had no write, and counts its own ping as that period's write. +- **Bounds:** the proxy is then never silent for longer than the interval, and never pings after less than half of it. +- **Throughput comparison discarded:** a six-pair run happened under the external load. Per-pair ratios ranged 0.67–1.56 and the medians moved in opposite directions at the two windows (section 4 of the raw file). +- **Evidence instead, from JFR** at window 8, ws, both candidates on the same host minutes apart. These are the samples whose stack reaches the keepalive handler before handing off to the next handler (section 6): + +| Candidate | CPU samples in the handler | Allocation samples in the handler | +| --- | ---: | ---: | +| IdleStateHandler | 20 / 154 (13.0%) | 6 / 100, 15.0 of 517 MB sampled weight | +| WebSocketKeepalive (flag) | 1 / 141 (0.7%) | 0 / 100 | + +These sample counts are small. They show where the first form's cost was (the per-write listener and its completion) and that the flag form has essentially none. They are not a precise overhead figure. + +## Limits + +- One desktop, loopback only, synthetic traffic from a fake backend. There was no CDN hop, no real RTT, no Nagle/delayed-ACK interplay (loopback ACKs at once), no real client mod and no real modpack. +- The workload's packet mix is an informed guess, not a capture from a pack. +- The host carried external load for much of the session. The OpenSSL verdict rests on consistent per-pair ratios; the keepalive verdict on JFR attribution. +- The JDK `SSLSocket` client is not the client mod's Netty 4.0.10 `SslHandler` (TLS 1.2). +- Single-host loopback numbers are never capacity claims. + +Raw output: [2026-10-23-websocket-cdn-raw.txt](2026-10-23-websocket-cdn-raw.txt).