mirror of
https://github.com/kunkundi/crossdesk.git
synced 2026-11-02 12:02:42 +08:00
[perf] accelerate Kitty and ANSI rendering
This commit is contained in:
dijunkun
@@ -20,6 +20,8 @@
|
||||
|
||||
凭据不能放在 connect JSON 中。专用描述符可以由 Node `spawn` 的第四条 pipe 提供。成功读取后会话不会保留用户密码供上层访问；MiniRTC 内部为现有 Join/重连流程持有必要凭据直到销毁 Peer。下一次 connect 必须有可读的凭据来源；Pi 关闭视图后直接结束进程。
|
||||
|
||||
`viewport.fps` 是本地输出目标。对端支持预览订阅时，超过 15 的目标使用已有交互模式（协议 `fps=60`、编码 `frame_rate=30`），不再折算成 15 FPS；低目标仍按能力选择 1/8/15。bridge 始终输出字符行并要求宿主 ACK，不使用交互 Kitty 的共享内存传图。
|
||||
|
||||
帧事件：
|
||||
|
||||
```json
|
||||
|
||||
@@ -45,13 +45,17 @@ crossdesk-cli 123456789 --server your-server.example --port 9099
|
||||
|
||||
ANSI 默认使用四象限字符：每个字符从 2×2 样本选择误差最小的两个颜色和象限形状，比原先上下半方块多一倍水平采样点；每个字符仍只有前景/背景两色。使用 `--ansi-detail half` 可回到上下半方块以兼容字体。缩小时采用面积加权滤波，避免最近邻采样漏掉细线。支持 TrueColor、256 色和基础 8 色，256 色选择实际颜色立方体或灰阶条中更接近的颜色。环境的 COLORTERM/TERM 决定颜色模式；非 UTF-8 locale 使用背景色空格。TERM=dumb、非 TTY 输出明确拒绝交互连接，不输出全屏转义序列。
|
||||
|
||||
四象限模式将每个 RGB 通道变化均不超过 6/255 的低对比单元合并为纯背景色，减少平坦区域的字形接缝。其余单元在独立拟合后参考上下左右邻格的量化误差调整选色，保留源图的真实渐变，并在高对比边缘减弱约束；调整设有失真上限，超过低对比阈值且能精确表示的文字细线和双色边缘保持原样。这些处理自动应用于独立 CLI 和 Pi／bridge 的四象限画面，实际效果仍受终端字体和行距影响。
|
||||
|
||||
画面按字符单元比例等比缩放，留黑边及状态栏。交互连接通过终端字节流查询字符尺寸，支持通过 SSH 返回的回复：优先使用 `CSI 16 t` 的字符像素尺寸，否则组合 `CSI 14 t` 的完整文本区像素尺寸和 `CSI 18 t` 的完整行列数计算，兼容 GNOME/VTE 的文本区报告。计算后再扣除状态栏及画面行列上限，不把预留行当作图像高度。窗口改变时重新查询，每两秒刷新一次以覆盖字体／行距变化；回复到达后会重绘静止画面和更新鼠标映射。查询不阻塞连接；没有有效回复时使用系统提供的像素尺寸，再缺失时回退为宽高比 0.5。曾收到有效回复时保留最近有效尺寸，直到下一次有效回复。字符栅格的整数舍入仍可能产生小幅比例误差。格式依据 [XTerm 窗口尺寸报告](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html) 和 [GNOME VTE 实现](https://github.com/GNOME/vte/blob/master/src/vteseq.cc)。
|
||||
|
||||
Windows 的可见传统控制台优先读取当前字体尺寸；Windows Terminal／ConPTY 使用显示终端的尺寸回复，不采用隐藏控制台宿主的字体值。不同终端版本可能不支持查询，或返回虚拟图形尺寸而非实际字体尺寸，此时需要手动校准。
|
||||
|
||||
默认 `--cell-aspect auto` 自动确定比例。如果终端屏蔽查询或需要手动校准，可设置 `--cell-aspect 0.45`，数值表示单个字符的**宽／高**，允许 0.2～2；例如字符为 9×20 像素时使用 0.45。画面被压窄时调小此值，被拉宽时调大。此参数优先于自动检测，同时用于全景、细节和鼠标坐标映射。
|
||||
|
||||
每次只输出变化行，合并连续颜色设置；缩放在完整输出边界清理旧画面。上限 240 列、100 画面行；默认顶部预留一行状态栏、底部预留两行快捷键，另预留一列避免自动换行。顶部状态栏用反色区分，左侧为控制模式和连接进度，右侧为画面参数及网络信息；底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始，鼠标坐标按视频区域映射，点击状态栏不会传给远端。按键分组使用统一列宽和左右留白，窄窗口自动减少列数、合并按键组并省略次要参数，累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作，进入本地命令模式后才显示平移和缩放按键；终端高度不足时优先收起底部提示行。
|
||||
独立 CLI 每次只输出变化行中首尾变化单元之间的区间，光标或局部视频更新不再重写整行；一帧内复用颜色状态、按实际色板索引合并颜色设置，并省略纯背景空格的前景色指令。Pi／bridge 仍发送可独立绘制的完整字符行，同时使用上述颜色编码优化。四象限初始拟合使用整数子集求和计算同等精度的颜色和误差，边界连续性拟合复用加权样本，减少重复运算。
|
||||
|
||||
缩放在完整输出边界清理旧画面。上限 240 列、100 画面行；默认顶部预留一行状态栏、底部预留两行快捷键，另预留一列避免自动换行。顶部状态栏用反色区分，左侧为控制模式和连接进度，右侧为画面参数及网络信息；底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始，鼠标坐标按视频区域映射，点击状态栏不会传给远端。按键分组使用统一列宽和左右留白，窄窗口自动减少列数、合并按键组并省略次要参数，累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作，进入本地命令模式后才显示平移和缩放按键；终端高度不足时优先收起底部提示行。
|
||||
|
||||
快捷键行使用 `KEYS |`、`VIEW |` 等固定宽度标题，标题和按键组之间用竖线分隔；键名用 `<c>`、`<+/->` 等标签标识。宽窗口中的分组分隔线按列对齐，窄窗口保留标题和完整按键标签，按可用宽度合并或减少分组。
|
||||
|
||||
@@ -59,7 +63,11 @@ ANSI 在完整桌面缩略图中仍无法保留普通字号的文字。先增加
|
||||
|
||||
auto 模式发送 Kitty 查询与设备属性查询，等待最多 250ms；没有有效 Kitty 回应就用 ANSI。显式 kitty 模式探测失败时报错。tmux/screen 中 auto 使用 ANSI，显式 kitty 报不支持；不修改用户终端配置。
|
||||
|
||||
Kitty 通过终端字节流发送 zlib 压缩的内联 RGB/Base64（每块至多 4096 字符），不使用远端临时图片文件。图像固定 ID=31、placement=1，更新同一图像，退出删除。依据终端报告的像素尺寸生成画面（未知时按 10×20 像素字符估算），上限 1920×1080。保留黑边和宽高比；原始细节模式按像素视口裁取源区域，避免整桌面缩小。没有可靠终端像素尺寸时使用估算值，因此不承诺显示设备的物理像素一一对应。压缩后超出单次输出预算时逐级降低尺寸。高熵视频压缩率较低，同一输出预算下实际帧率可能低于目标。RGB 压缩无损，不代表上游视频编码无损。
|
||||
本地 macOS/Linux 的 Kitty 模式会另用一个 1×1 像素的共享内存对象探测 `t=s`，最多等待 250ms；探测成功后，将未压缩 RGB 写入权限为 0600 的 POSIX 共享内存，仅通过终端字节流发送名字和显示命令。每次使用唯一名字，最多保留一个待确认的共享缓冲区（1920×1080×3 字节）。收到上传成功确认后才显示新图并释放缓冲区；上传报错、确认超过 500ms 或本地分配失败时，本次会话退回内联传输，继续保留旧画面。终端和 CLI 都会清理共享对象，退出时也会释放未确认的对象。SSH 中不尝试共享内存；Windows 暂用内联传输。
|
||||
|
||||
内联 Kitty 通过终端字节流发送 zlib 压缩的 RGB/Base64（每块至多 4096 字符），不使用远端临时图片文件。两条路径都交替使用图像 ID=31/32：以 `a=t` 上传到未显示的 ID，传完后以 `a=p` 创建 placement=1，再删除旧图。仅在整帧显示命令输出完成后交替 ID，上传和短写期间保留旧画面，退出时清理两个 ID。这样避免重传同一 ID 时终端先移除旧图造成的帧间黑屏。
|
||||
|
||||
依据终端报告的像素尺寸生成画面（未知时按 10×20 像素字符估算），上限 1920×1080。保留面积滤波、黑边、宽高比和裁剪坐标；原始细节模式按像素视口裁取源区域。共享内存路径使用可复用水平行计算的整数面积权重，避免逐像素重复浮点运算，也不再因 Base64 字节预算额外降采样。没有可靠终端像素尺寸时使用估算值，因此不承诺显示设备的物理像素一一对应。内联压缩结果超出单次输出预算时仍逐级降低尺寸；高熵视频的实际帧率可能低于目标。RGB 传输和压缩不丢失像素，不代表面积采样或上游视频编码无损。
|
||||
|
||||
```sh
|
||||
ssh -t your-server
|
||||
@@ -70,7 +78,7 @@ crossdesk-cli 123456789 --renderer ansi --fps 8
|
||||
|
||||
## 预览与背压
|
||||
|
||||
`--fps 1..30` 是本地输出目标。对端声明 PreviewSubscription 时，映射到合法 1/8/15 FPS（15 需要对应能力）；普通编码设置仍使用合法 30 FPS。分辨率上限使用协商到的 v2，视频预算只在 bandwidth 能力存在时发送。`--quality low|balanced|high|ultra` 对应如下请求：
|
||||
`--fps 1..30` 是本地输出目标。未显式指定时，共享内存探测成功默认使用 30 FPS；ANSI、内联 Kitty 和 bridge 默认仍为 8 FPS。自动选择的共享内存路径运行中退回内联时，目标也恢复为 8；显式设置的 `--fps` 始终保留。对端声明 PreviewSubscription 时，目标不超过 15 映射到合法 1/8/15 FPS（15 需要对应能力）；超过 15 则请求项目已有的交互模式：协议 `fps=60` 是模式标记，编码设置 `frame_rate=30` 才是帧率，不再把 `--fps 30` 限制为 15 FPS。分辨率上限使用协商到的 v2，视频预算只在 bandwidth 能力存在时发送。`--quality low|balanced|high|ultra` 对应如下请求：
|
||||
|
||||
| 档位 | 分辨率上限 | 质量枚举 | 视频预算 |
|
||||
|---|---|---|---|
|
||||
@@ -83,9 +91,11 @@ crossdesk-cli 123456789 --renderer ansi --fps 8
|
||||
|
||||
媒体回调仅同步复制借用的 NV12 数据，校验尺寸、容量、native 描述符与 stride；不保留借用指针。上限单帧 16,777,216 像素、每边 8192、偶数尺寸。最多一个待转换帧，替换前先完成校验与复制，无效帧不会挤掉已有画面；播放器另保留最近转换的一帧用于静止桌面的本地放大/重绘，并在接收新帧时回收上一帧缓冲区供后续复制使用。只丢弃解码后图像。显示器切换/关闭提升 generation，旧帧失效。
|
||||
|
||||
转换与输出在拥有会话的事件循环，媒体回调独立。输出至多一个完整消息（2MiB），以 1MiB/s 令牌预算分块非阻塞写入；不能开始新画面时保留最新解码帧。ANSI 差分基准只在整帧输出完成后提交。Pi 桥接另要求消费帧 ACK，最多一帧等待 ACK，输入映射只在对应 ACK 后切换到新网格。状态栏优先在完整画面输出之间更新，避免大图持续刷新时状态饥饿；可写事件唤醒短写重试，仍遵守字节预算。控制事件至多 64 条，协议输入 64KiB/嵌套 32 层，输入粘贴缓冲 32KiB，文本单次最多 16KiB。
|
||||
转换与输出在拥有会话的事件循环，媒体回调独立。输出至多一个完整消息（2MiB），按令牌预算分块非阻塞写入，突发额度上限仍为 1MiB。本地 Kitty 使用 8MiB/s；存在 `SSH_CONNECTION`、`SSH_CLIENT` 或 `SSH_TTY` 时，Kitty 保持 1MiB/s，ANSI 和 Pi 桥接也保持 1MiB/s。不能开始新画面时保留最新解码帧。ANSI 差分基准只在整帧输出完成后提交。Pi 桥接另要求消费帧 ACK，最多一帧等待 ACK，输入映射只在对应 ACK 后切换到新网格。状态栏优先在完整画面输出之间更新，避免大图持续刷新时状态饥饿；可写事件唤醒短写重试，仍遵守字节预算。控制事件至多 64 条，协议输入 64KiB/嵌套 32 层，输入粘贴缓冲 32KiB，文本单次最多 16KiB。
|
||||
|
||||
统计包括收到帧数、被替换帧数、非法帧、等待帧数、实际完整提交次数、输出字节数、运行秒数、最近本地排队时间、网络码率/RTT/SRTP。通过计数差/时间差可得到区间 FPS；交互连接结束日志另给平均提交 FPS。`client.log` 摘要的 `startup_ms` 记录从发起连接到连接成功（`connected`）、首个取出用于显示的视频帧入队（`first_frame`）、首帧完整写入终端（`first_output`）的毫秒数；未到达的阶段为 `null`。写入 TTY 完成不等于屏幕显示完成，本地排队时间不是端到端延迟。
|
||||
测量像素渲染帧率时使用 release 构建；debug 的 `-O0` 面积采样开销明显更高。共享内存中的像素不经过终端字节预算，但仍受转换和终端消费速度限制；8MiB/s 是内联路径的输出上限，不是帧率保证。解码、像素转换、压缩、终端消费速度和帧大小都会影响结果。退出时，未完成画面最多继续写入 2s；取消图像传输及恢复屏幕的短写最多另重试 500ms。POSIX 立即恢复 termios 并清理输入，不等待输出排空；如果终端始终不消费输出，屏幕清理只能尽力发送。
|
||||
|
||||
统计包括收到帧数、被替换帧数、非法帧、等待帧数、实际完整提交次数、输出字节数、运行秒数、最近本地排队时间、网络码率/RTT/SRTP。通过计数差/时间差可得到区间 FPS；交互连接结束日志另给平均提交 FPS，以及实际 `renderer` 和 `output_bytes_per_second` 预算。`kitty_transport=shared_memory/inline/none` 标明生效路径，`target_fps` 是当前目标，`frame_processing_ms` 是平均本地布局／转换／传输准备耗时，`shared_ack_ms` 是平均上传命令写完到解析成功确认的耗时，`shared_fallbacks` 记录运行时回退次数。`client.log` 摘要的 `startup_ms` 记录从发起连接到连接成功（`connected`）、首个取出用于显示的视频帧入队（`first_frame`）、首帧显示命令完整写入终端（`first_output`）的毫秒数；未到达的阶段为 `null`。写入 TTY 完成不等于屏幕显示完成，本地排队时间不是端到端延迟。
|
||||
|
||||
## 输入与退出
|
||||
|
||||
|
||||
@@ -73,6 +73,45 @@
|
||||
- Pi 1.4.0 SDK 严格类型检查及模拟宿主运行通过：使用真实 SDK 分片组包，覆盖文本区成对回复、字符尺寸优先级、宿主缓存、字体／窗口变化、手动覆盖、旧比例帧丢弃及 ACK、控制键回归、启动失败／异常退出后的监听与定时器清理。
|
||||
- 新增验证代码和产物全部位于工程外系统临时目录；尚未在真实 Windows、GNOME／SSH 或 Pi TUI 进行视觉验收。
|
||||
|
||||
## ANSI 四象限接缝优化（2026-10-30）
|
||||
|
||||
- macOS arm64 的 CLI debug 构建和 `doctor --json` 通过。工程外 ASan/UBSan 检查覆盖低对比合并、横纵细线／斜线／双色图案、等亮度彩色边缘、随机 NV12 输入的失真上限、局部更新范围和重复转换确定性。
|
||||
- 合成低对比图案中，3,072 个非空白字形全部合并为纯背景。合成渐变纹理的加权接缝误差由约 207,373 降至 180,187（13.1%）；仅统计两端均未合并为纯色的边界时，由约 198,007 降至 169,621。指标衡量相邻样本的重建色差相对源图色差的偏离，并降低真实高对比边缘的权重；不代表实机条纹减少的百分比。
|
||||
- 8／24／256 色输出、half／ASCII 与原实现一致性、不同视口／比例／缩放的布局和鼠标映射、静止网格差分及 Kitty 输出检查通过。
|
||||
- 本机 `-O2`、1920×1080 输入、240×100 字符视口、每项 12 次转换的合成基准：平滑画面约 7.14 → 6.03 ms/帧，随机亮度噪声约 5.44 → 7.73 ms/帧。仅衡量转换，不包含视频解码、ANSI 输出和终端绘制。
|
||||
- 额外检查代码、基准和数据均位于工程外系统临时目录；未新增工程内测试，未连接真实远端或进行终端字形／行距的视觉验收。
|
||||
|
||||
## ANSI 转换与输出加速（2026-10-30）
|
||||
|
||||
- macOS arm64 CLI debug 构建和 `doctor --json` 通过。工程外 ASan/UBSan 与 `-O2` 检查均通过：加速后的转换与此前接缝优化版本逐格颜色／字形一致，覆盖随机 NV12、低对比及平滑纹理、half／quadrant／ASCII、缩放和细节模式；Kitty 数据保持一致。
|
||||
- ANSI 转义序列模拟器验证最终屏幕一致，覆盖全部四象限字形、8／24／256 色、纯背景及色板复用、首尾单元、局部修改、视口／偏移变化和可独立绘制的 bridge 字符行。未向工程新增测试文件或用例。
|
||||
- 本机 `-O2`、1920×1080 输入、240×100 字符视口、每项 24 次转换：平滑画面约 6.26 → 4.83 ms/帧，随机亮度噪声约 7.75 → 5.72 ms/帧。保留面积滤波、低对比合并及边界连续性约束。
|
||||
- 240×100 真彩色合成网格、每项 100 次输出生成：整帧约 2.03 → 1.43 ms；120×50 单元的局部修改约 1.02 → 0.36 ms，输出 461,743 → 231,026 字节；2×3 单元的光标修改约 0.072 → 0.013 ms，输出 27,763 → 262 字节。纯背景色块整帧输出 875,352 → 449,826 字节。
|
||||
- 这些结果衡量转换、编码及输出量，不包含实际终端绘制或网络传输。整帧高色彩变化图案仍需约 923 KB，仍受 1 MiB/s 输出预算限制；不能将转换耗时的倒数作为远程显示帧率。默认本地目标为 8 FPS，`--fps 15` 可请求更高目标，预览订阅还需要对端支持 15 FPS。
|
||||
- 额外检查代码、模拟器、基准和产物均位于工程外临时目录；未进行跨平台终端或真实远端帧率验收。
|
||||
|
||||
## Kitty 帧间黑屏与本地吞吐修复（2026-10-30）
|
||||
|
||||
- macOS arm64 debug、release 构建通过，release `doctor --json` 通过。release 使用工程外独立配置构建，保留工程原有 debug 配置；额外检查脚本、伪终端夹具和数据均位于系统临时目录。
|
||||
- ASan/UBSan 检查及 Kitty 协议模拟验证：ID 31/32 交替上传，所有上传分块完成前旧图保持显示；先创建新图 placement，再释放旧图；最多两个完整图像驻留，切换后只保留一个。新旧输出的解压 RGB 和尺寸逐字节一致，覆盖尺寸、偏移和缩放变化；单帧小于 2MiB，全部分块边界的中断清理通过。
|
||||
- ANSI 的 half／quadrant／ASCII 和 8／24／256 色共 12 种组合，转换单元及输出字节与此次修改前一致。
|
||||
- 使用生产 `View`、`Output` 和 Kitty 编码代码、`-O2`、合成 NV12 持续回调的本地 PTY 对比：目标 8 FPS，单帧约 1.25 MB，稳定完整接收速率约 0.84 → 6.71 FPS，帧间隔中位数约 1,192 → 148 ms。只衡量完整输出接收，不包含真实网络、视频解码和 Ghostty GPU 显示，不能作为实机帧率承诺。
|
||||
- 本地 Kitty 的日志预算为 8MiB/s；分别仅设置 `SSH_TTY`、`SSH_CONNECTION`、`SSH_CLIENT` 时均保持 1MiB/s，ANSI 也保持 1MiB/s。连续帧 ID 仅在完整写入后交替，退出清理两个 ID。
|
||||
- 慢速 PTY 在第二帧上传中退出：首帧保留、第二帧未提交，约 2.06s 后退出；两个 ID 清理、备用屏幕／光标控制序列和原 termios 均恢复。测试发现原单次清理写入会在满缓冲区时丢失，现增加至多 500ms 的非阻塞短写重试。
|
||||
- 模拟父 shell 保持会话、终端完全停止读取的 PTY：CLI 约 2.53s 后返回，termios 和非阻塞标志恢复。POSIX 恢复改为 `TCSANOW` 加输入清理，避免 `TCSAFLUSH` 等待输出排空。完全不读取时，屏幕清理字节不保证送达。
|
||||
- 本机安装 Ghostty 1.4.0，但自动化工具拒绝访问其窗口，未完成 Ghostty 实窗视觉验收。npm 安装的 CLI 不会随本地构建自动更新，复测需要直接运行本次 release 产物。
|
||||
|
||||
## 本地 Kitty 共享内存与 30 FPS 目标（2026-10-30）
|
||||
|
||||
- 本地 POSIX Kitty 增加共享内存能力探测、唯一名称、0600 权限、上传确认和 500ms 超时回退。单个待确认共享缓冲区不超过 1920×1080×3 字节；确认前保留旧画面，退出释放未确认对象。Windows 暂保留内联路径，SSH 不发共享内存探测。
|
||||
- 共享内存可用且未显式指定 `--fps` 时默认目标 30；自动回退内联恢复 8，显式帧率保持不变。目标大于 15 使用现有预览协议的交互模式 `fps=60, frame_rate=30`；没有新增协议取值或修改被控端。
|
||||
- 工程外 ASan/UBSan 检查通过：110 组随机帧／视口／宽高比／缩放／细节模式，整数面积滤波相对原浮点参考的每通道最大误差为 1/255；内联 Kitty 和 ANSI 保持原输出。覆盖共享内存权限、读取内容、命名、容量限制、双方 unlink、分片确认及非法／溢出／重复 ID 回应。
|
||||
- 使用实际 release CLI 对象代码、1920×1080 合成噪声 NV12 连续回调、1568×1088 报告视口的 PTY 测试：内联完整接收约 4.06 FPS，共享内存约 29.04 FPS；平均本地处理约 166.6 → 12.3 ms，终端字节流约 1,703,174 → 188 字节/帧。共享内存仍传输完整 RGB，并非总数据量只有 188 字节；数据复制、终端绘制和实际网络另有成本。
|
||||
- 显式 8 FPS、探测不支持／静默、第三帧上传报错／丢失确认、80ms 延迟确认、SSH 禁用探测均通过。回退前不显示未确认帧，运行时回退仅触发一次，并正确恢复自动目标和协议请求；所有共享对象均清理。第二帧上传后立即退出时，首帧仍是已提交画面，未确认对象及两个图像 ID 均清理。
|
||||
- 相同合成输入的 debug 构建约 5.91 FPS，本地处理约 163 ms，说明 `-O0` 仍明显影响像素转换；性能验收使用 release。上述精确数字是本地测试接收速率，不是 Ghostty GPU 显示帧率。
|
||||
- 用户在本机 Ghostty 运行新版 release 后反馈“明显流畅了”。额外测试源码、数据、构建夹具和产物均位于系统临时目录，未在工程或子模块新增测试。
|
||||
- 该次实机日志确认 `kitty_transport=shared_memory`、目标 30、预览请求已接受、运行时回退 0 次。会话约 13.61s 完整提交 273 帧，平均提交约 20.06 FPS（包含约 1.49s 首帧启动等待）；平均本地处理约 10.87ms、共享内存确认约 0.42ms。该日志记录提交速率，不测量终端 GPU 呈现频率。
|
||||
|
||||
## 实测边界
|
||||
|
||||
本地检查不涵盖 Linux / Windows 完整构建、跨平台远程控制、Kitty/SSH 实际终端显示、真实 Pi TUI 或长时间资源占用。五个平台 CI 配置与 npm 包名均不代表已发布或已验收。
|
||||
|
||||
+161
-20
@@ -31,6 +31,11 @@ public:
|
||||
explicit Output(int fd)
|
||||
: fd_(fd), nonblocking_(fd, true), last_(Clock::now()) {}
|
||||
bool Idle() const { return pending_.empty(); }
|
||||
size_t ByteRate() const { return byte_rate_; }
|
||||
void SetByteRate(size_t rate) {
|
||||
byte_rate_ = rate;
|
||||
last_ = Clock::now();
|
||||
}
|
||||
void Put(std::string text) {
|
||||
if (!Idle() || text.size() > 2 * 1024 * 1024)
|
||||
throw std::runtime_error("output budget exceeded");
|
||||
@@ -41,7 +46,7 @@ public:
|
||||
auto now = Clock::now();
|
||||
credit_ = std::min(
|
||||
1048576.0,
|
||||
credit_ + std::chrono::duration<double>(now - last_).count() * 1048576);
|
||||
credit_ + std::chrono::duration<double>(now - last_).count() * byte_rate_);
|
||||
last_ = now;
|
||||
if (Idle())
|
||||
return false;
|
||||
@@ -75,6 +80,7 @@ private:
|
||||
NonBlocking nonblocking_;
|
||||
std::string pending_;
|
||||
size_t offset_ = 0;
|
||||
size_t byte_rate_ = 1048576;
|
||||
double credit_ = 1048576;
|
||||
Clock::time_point last_;
|
||||
};
|
||||
@@ -93,7 +99,24 @@ struct Tty {
|
||||
"\033[0m\033[?1003l\033[?1002l\033[?1000l"
|
||||
"\033[?1006l\033[?1004l\033[?2004l\033["
|
||||
"?7h\033[?25h\033[?1049l";
|
||||
Write(out, end.data(), end.size());
|
||||
// A full PTY can reject even this small cleanup after an interrupted image.
|
||||
// Retry short writes without indefinitely blocking exit on a stalled reader.
|
||||
try {
|
||||
NonBlocking nonblocking(out, true);
|
||||
const auto until = Clock::now() + std::chrono::milliseconds(500);
|
||||
size_t offset = 0;
|
||||
while (offset < end.size() && Clock::now() < until) {
|
||||
auto n = Write(out, end.data() + offset, end.size() - offset);
|
||||
if (n > 0)
|
||||
offset += n;
|
||||
else if (n < 0 && errno != EAGAIN && errno != EINTR)
|
||||
break;
|
||||
else
|
||||
WaitIo(-1, out, 10);
|
||||
}
|
||||
} catch (...) {
|
||||
// A closed terminal must not prevent RawTerminal from restoring modes.
|
||||
}
|
||||
}
|
||||
};
|
||||
Viewport Size(int *status_rows = nullptr,
|
||||
@@ -242,10 +265,11 @@ std::string WaitingScreen(const Viewport &v, const std::string &title,
|
||||
}
|
||||
int View(SessionOptions options, const std::string &device,
|
||||
const std::string &renderer, bool control, int fd, AnsiDetail detail,
|
||||
double zoom, bool native, double cell_aspect) {
|
||||
double zoom, bool native, double cell_aspect, bool fps_explicit) {
|
||||
if (!IsTty(0) || !IsTty(fd) || Env("TERM") == "dumb")
|
||||
throw std::runtime_error("connecting needs an interactive ANSI terminal; use "
|
||||
"bridge --stdio for a host");
|
||||
throw std::runtime_error(
|
||||
"connecting needs an interactive ANSI terminal; use "
|
||||
"bridge --stdio for a host");
|
||||
auto password = Password();
|
||||
if (interrupted)
|
||||
return 130;
|
||||
@@ -269,6 +293,42 @@ int View(SessionOptions options, const std::string &device,
|
||||
if (renderer == "kitty" && !kitty)
|
||||
throw std::runtime_error(
|
||||
"Kitty probe failed or multiplexer active; use --renderer ansi");
|
||||
// Local pixel frames need more throughput than character output. Keep the
|
||||
// conservative budget for SSH and the existing one-message/burst limits.
|
||||
const bool local_terminal = Env("SSH_CONNECTION").empty() &&
|
||||
Env("SSH_CLIENT").empty() &&
|
||||
Env("SSH_TTY").empty();
|
||||
if (kitty && local_terminal)
|
||||
output.SetByteRate(8 * 1024 * 1024);
|
||||
SharedImage shared_image;
|
||||
bool shared_enabled = false;
|
||||
const uint8_t probe_pixel[] = {0, 0, 0};
|
||||
if (kitty && local_terminal && output.Idle() &&
|
||||
shared_image.Assign(probe_pixel, sizeof(probe_pixel))) {
|
||||
output.Put(KittySharedUpload(shared_image.Name(), 1, 1, false, true));
|
||||
std::string reply;
|
||||
const auto until = Clock::now() + std::chrono::milliseconds(250);
|
||||
while (Clock::now() < until && !interrupted) {
|
||||
output.Pump();
|
||||
WaitReadable(0, 10);
|
||||
char bytes[1024];
|
||||
const auto n = Read(0, bytes, sizeof(bytes));
|
||||
if (n > 0 && reply.size() < 8192)
|
||||
reply.append(bytes, n);
|
||||
const auto pos = reply.find("\033_Gi=33;");
|
||||
const auto end =
|
||||
pos == std::string::npos ? pos : reply.find("\033\\", pos);
|
||||
if (end != std::string::npos) {
|
||||
const auto ack = ParseKittyAck(reply.substr(pos, end + 2 - pos));
|
||||
shared_enabled = ack && ack->ok;
|
||||
break;
|
||||
}
|
||||
}
|
||||
shared_image.Reset();
|
||||
}
|
||||
bool automatic_fast_fps = shared_enabled && !fps_explicit;
|
||||
if (automatic_fast_fps)
|
||||
options.fps = 30;
|
||||
ControllerSession s;
|
||||
Input input;
|
||||
input.control = control;
|
||||
@@ -279,6 +339,11 @@ int View(SessionOptions options, const std::string &device,
|
||||
s.SetControl(control);
|
||||
std::optional<Grid> displayed, pending;
|
||||
bool pending_frame = false;
|
||||
bool shared_upload = false;
|
||||
std::optional<Clock::time_point> shared_deadline;
|
||||
std::string pending_clear;
|
||||
uint64_t shared_fallbacks = 0, rendered = 0, shared_acks = 0;
|
||||
double render_ms = 0, shared_ack_ms = 0;
|
||||
int last_status_rows = 3;
|
||||
Viewport last_view =
|
||||
Size(&last_status_rows, &input.terminal_metrics, cell_aspect);
|
||||
@@ -292,8 +357,24 @@ int View(SessionOptions options, const std::string &device,
|
||||
double queue_ms = 0;
|
||||
auto next = Clock::now(), status_tick = next;
|
||||
auto size_query_tick = next;
|
||||
const auto disable_shared = [&] {
|
||||
shared_enabled = shared_upload = false;
|
||||
shared_deadline.reset();
|
||||
shared_image.Reset();
|
||||
++shared_fallbacks;
|
||||
if (automatic_fast_fps) {
|
||||
automatic_fast_fps = false;
|
||||
options.fps = 8;
|
||||
s.SetPreview(options.fps, options.preview_width, options.preview_height,
|
||||
options.quality, options.max_bitrate_bps);
|
||||
}
|
||||
};
|
||||
const auto pump_output = [&] {
|
||||
if (output.Pump() && pending_frame) {
|
||||
if (!output.Pump())
|
||||
return;
|
||||
if (shared_upload) {
|
||||
shared_deadline = Clock::now() + std::chrono::milliseconds(500);
|
||||
} else if (pending_frame) {
|
||||
displayed = std::move(pending);
|
||||
pending.reset();
|
||||
pending_frame = false;
|
||||
@@ -317,6 +398,8 @@ int View(SessionOptions options, const std::string &device,
|
||||
const auto previous_error = input.error;
|
||||
input.Feed(n > 0 ? std::string(bytes, n) : "", s,
|
||||
displayed ? &*displayed : nullptr);
|
||||
if (input.exit || interrupted)
|
||||
break;
|
||||
if (previous_command != input.command_mode ||
|
||||
previous_control != input.control || previous_error != input.error)
|
||||
status_tick = Clock::now();
|
||||
@@ -324,6 +407,35 @@ int View(SessionOptions options, const std::string &device,
|
||||
if (s.Finished())
|
||||
break;
|
||||
auto now = Clock::now();
|
||||
if (shared_upload && shared_deadline) {
|
||||
const auto ack = input.kitty_ack;
|
||||
const bool replied = ack && ack->image_id == (submitted % 2 ? 32u : 31u);
|
||||
if (replied || now >= *shared_deadline) {
|
||||
if (replied && ack->ok) {
|
||||
shared_ack_ms +=
|
||||
std::chrono::duration<double, std::milli>(
|
||||
now - (*shared_deadline - std::chrono::milliseconds(500)))
|
||||
.count();
|
||||
++shared_acks;
|
||||
shared_upload = false;
|
||||
shared_deadline.reset();
|
||||
shared_image.Reset();
|
||||
// Display only an acknowledged upload. A missing/failed upload must
|
||||
// leave the previous image and pointer mapping intact.
|
||||
output.Put(pending_clear +
|
||||
KittyPresent(pending->viewport, submitted % 2 != 0));
|
||||
pending_frame = true;
|
||||
waiting_visible = false;
|
||||
} else {
|
||||
disable_shared();
|
||||
pending.reset();
|
||||
redraw = true;
|
||||
next = now;
|
||||
}
|
||||
pending_clear.clear();
|
||||
}
|
||||
}
|
||||
input.kitty_ack.reset();
|
||||
int status_rows;
|
||||
auto v = Size(&status_rows, &input.terminal_metrics, cell_aspect);
|
||||
v.detail = detail;
|
||||
@@ -338,7 +450,7 @@ int View(SessionOptions options, const std::string &device,
|
||||
}
|
||||
const bool grid_resized =
|
||||
v.columns != last_view.columns || v.rows != last_view.rows;
|
||||
if (output.Idle() &&
|
||||
if (output.Idle() && !shared_upload &&
|
||||
(grid_resized || v.cell_aspect != last_view.cell_aspect ||
|
||||
v.pixel_width != last_view.pixel_width ||
|
||||
v.pixel_height != last_view.pixel_height ||
|
||||
@@ -354,12 +466,12 @@ int View(SessionOptions options, const std::string &device,
|
||||
}
|
||||
// Query through the terminal byte stream so this also works over SSH.
|
||||
// Never wait for a reply or interrupt a partially written video frame.
|
||||
if (output.Idle() && now >= size_query_tick) {
|
||||
if (output.Idle() && !shared_upload && now >= size_query_tick) {
|
||||
input.terminal_metrics.BeginQuery();
|
||||
output.Put(kTerminalSizeQuery);
|
||||
size_query_tick = now + std::chrono::seconds(2);
|
||||
}
|
||||
if (output.Idle() && now >= status_tick) {
|
||||
if (output.Idle() && !shared_upload && now >= status_tick) {
|
||||
auto st = s.Status();
|
||||
const auto last_state = st["state"].get<std::string>();
|
||||
std::string progress =
|
||||
@@ -422,7 +534,7 @@ int View(SessionOptions options, const std::string &device,
|
||||
output.Put(std::move(screen));
|
||||
status_tick = now + std::chrono::seconds(1);
|
||||
}
|
||||
if (output.Idle() && now >= next) {
|
||||
if (output.Idle() && !shared_upload && now >= next) {
|
||||
next = now + std::chrono::milliseconds(1000 / options.fps);
|
||||
bool fresh = false;
|
||||
if (auto frame = s.TakeFrame(last_frame ? &*last_frame : nullptr);
|
||||
@@ -435,6 +547,7 @@ int View(SessionOptions options, const std::string &device,
|
||||
if (last_frame && last_frame->generation != s.Generation())
|
||||
last_frame.reset();
|
||||
if (last_frame && (fresh || redraw)) {
|
||||
const auto render_started = Clock::now();
|
||||
if (fresh)
|
||||
queue_ms = std::chrono::duration<double, std::milli>(
|
||||
now - last_frame->queued)
|
||||
@@ -446,12 +559,29 @@ int View(SessionOptions options, const std::string &device,
|
||||
for (int row = 0; row < v.rows; ++row)
|
||||
clear += "\033[" + std::to_string(v.row_offset + row + 1) +
|
||||
";1H\033[0m\033[2K";
|
||||
output.Put(clear + (kitty ? Kitty(*last_frame, v)
|
||||
: Difference(*pending, displayed)));
|
||||
waiting_visible = false;
|
||||
if (shared_enabled) {
|
||||
const auto pixels = KittyPixels(*last_frame, v);
|
||||
if (shared_image.Assign(pixels.rgb.data(), pixels.rgb.size())) {
|
||||
output.Put(KittySharedUpload(shared_image.Name(), pixels.width,
|
||||
pixels.height, submitted % 2 != 0));
|
||||
shared_upload = true;
|
||||
pending_clear = std::move(clear);
|
||||
} else {
|
||||
disable_shared();
|
||||
}
|
||||
}
|
||||
if (!shared_upload) {
|
||||
output.Put(clear + (kitty ? Kitty(*last_frame, v, submitted % 2 != 0)
|
||||
: Difference(*pending, displayed)));
|
||||
waiting_visible = false;
|
||||
pending_frame = true;
|
||||
}
|
||||
render_ms += std::chrono::duration<double, std::milli>(Clock::now() -
|
||||
render_started)
|
||||
.count();
|
||||
++rendered;
|
||||
redraw = false;
|
||||
pending_frame = true;
|
||||
if (output.Idle()) {
|
||||
if (output.Idle() && !shared_upload) {
|
||||
displayed = std::move(pending);
|
||||
pending.reset();
|
||||
pending_frame = false;
|
||||
@@ -492,6 +622,15 @@ int View(SessionOptions options, const std::string &device,
|
||||
{"reported", input.terminal_metrics.Known()},
|
||||
{"manual_aspect", cell_aspect > 0}}},
|
||||
{"submitted", submitted},
|
||||
{"renderer", kitty ? "kitty" : "ansi"},
|
||||
{"kitty_transport", !kitty ? "none"
|
||||
: shared_enabled ? "shared_memory"
|
||||
: "inline"},
|
||||
{"target_fps", options.fps},
|
||||
{"frame_processing_ms", rendered ? render_ms / rendered : 0},
|
||||
{"shared_ack_ms", shared_acks ? shared_ack_ms / shared_acks : 0},
|
||||
{"shared_fallbacks", shared_fallbacks},
|
||||
{"output_bytes_per_second", output.ByteRate()},
|
||||
{"output_bytes", output.bytes},
|
||||
{"last_queue_ms", queue_ms},
|
||||
{"elapsed_seconds",
|
||||
@@ -778,7 +917,8 @@ Ctrl+T and Ctrl+C pass through in control mode; another Ctrl+] leaves local mode
|
||||
Default ANSI detail: quadrant. Default quality: high (1920x1080); ultra requests up to 3840x2160.
|
||||
Cell aspect is character width/height: auto detects terminal geometry; use --cell-aspect to calibrate.
|
||||
Display index is zero-based. ANSI cannot reproduce full desktop text at fit scale.
|
||||
--fps controls local output; negotiated previews use supported 1/8/15 FPS.
|
||||
--fps controls local output. Local Kitty shared memory defaults to 30; other paths default to 8.
|
||||
Targets above 15 use the negotiated interactive 30 FPS stream; lower targets use 1/8/15 FPS previews.
|
||||
bridge: private NDJSON v1, credentials on a separate inherited descriptor.
|
||||
No IP direct connection, audio playback, clipboard or file transfer support.
|
||||
)";
|
||||
@@ -829,7 +969,7 @@ int main(int argc, char **argv) {
|
||||
std::string renderer = "auto", device;
|
||||
AnsiDetail detail = AnsiDetail::Quadrant;
|
||||
double zoom = 1, cell_aspect = 0;
|
||||
bool control = false, stdio = false, native = false;
|
||||
bool control = false, stdio = false, native = false, fps_explicit = false;
|
||||
int secret_fd = 3;
|
||||
if (command == "view") {
|
||||
if (i >= argc)
|
||||
@@ -860,9 +1000,10 @@ int main(int argc, char **argv) {
|
||||
options.host = value;
|
||||
else if (a == "--port")
|
||||
options.port = number(value, 1, 65535);
|
||||
else if (a == "--fps" && command == "view")
|
||||
else if (a == "--fps" && command == "view") {
|
||||
options.fps = number(value, 1, 30);
|
||||
else if (a == "--display" && command == "view")
|
||||
fps_explicit = true;
|
||||
} else if (a == "--display" && command == "view")
|
||||
options.display = number(value, 0, 63);
|
||||
else if (a == "--renderer" && command == "view")
|
||||
renderer = value;
|
||||
@@ -916,7 +1057,7 @@ int main(int argc, char **argv) {
|
||||
try {
|
||||
return command == "view" ? View(options, device, renderer, control,
|
||||
output.value, detail, zoom, native,
|
||||
cell_aspect)
|
||||
cell_aspect, fps_explicit)
|
||||
: Bridge(output.value, options, secret_fd);
|
||||
} catch (const std::exception &e) {
|
||||
if (command == "view") {
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
#include <atomic>
|
||||
#include <csignal>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
@@ -34,6 +35,21 @@ struct TerminalSize {
|
||||
};
|
||||
TerminalSize GetTerminalSize(int fd);
|
||||
|
||||
// One private, uniquely named pixel buffer. Keep it alive until the terminal
|
||||
// acknowledges reading it; Reset also unlinks objects the terminal left behind.
|
||||
class SharedImage {
|
||||
public:
|
||||
SharedImage();
|
||||
~SharedImage();
|
||||
bool Assign(const uint8_t *data, size_t size);
|
||||
void Reset();
|
||||
const std::string &Name() const;
|
||||
|
||||
private:
|
||||
struct State;
|
||||
std::unique_ptr<State> state_;
|
||||
};
|
||||
|
||||
class NonBlocking {
|
||||
public:
|
||||
explicit NonBlocking(int fd, bool output = false);
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
#ifndef _WIN32
|
||||
#include "terminal_io.h"
|
||||
#include <cerrno>
|
||||
#include <cstring>
|
||||
#include <fcntl.h>
|
||||
#include <poll.h>
|
||||
#include <stdexcept>
|
||||
#include <sys/ioctl.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/select.h>
|
||||
#include <termios.h>
|
||||
#include <unistd.h>
|
||||
@@ -78,6 +80,66 @@ TerminalSize GetTerminalSize(int fd) {
|
||||
}
|
||||
return size;
|
||||
}
|
||||
struct SharedImage::State {
|
||||
std::string name;
|
||||
void *data = MAP_FAILED;
|
||||
size_t size = 0;
|
||||
};
|
||||
SharedImage::SharedImage() : state_(std::make_unique<State>()) {}
|
||||
SharedImage::~SharedImage() { Reset(); }
|
||||
const std::string &SharedImage::Name() const { return state_->name; }
|
||||
void SharedImage::Reset() {
|
||||
if (state_->data != MAP_FAILED)
|
||||
munmap(state_->data, state_->size);
|
||||
if (!state_->name.empty())
|
||||
shm_unlink(state_->name.c_str());
|
||||
state_->data = MAP_FAILED;
|
||||
state_->size = 0;
|
||||
state_->name.clear();
|
||||
}
|
||||
bool SharedImage::Assign(const uint8_t *data, size_t size) {
|
||||
Reset();
|
||||
if (!data || !size || size > 1920 * 1080 * 3)
|
||||
return false;
|
||||
static std::atomic<uint32_t> sequence{0};
|
||||
int fd = -1;
|
||||
for (int attempt = 0; attempt < 16; ++attempt) {
|
||||
// Fits Darwin's 31-byte name limit. O_EXCL prevents opening another
|
||||
// process's object, including a stale name from a recycled process ID.
|
||||
auto name = "/cdx-" + std::to_string(getpid()) + "-" +
|
||||
std::to_string(++sequence);
|
||||
fd = shm_open(name.c_str(), O_RDWR | O_CREAT | O_EXCL, 0600);
|
||||
if (fd >= 0) {
|
||||
state_->name = std::move(name);
|
||||
break;
|
||||
}
|
||||
if (errno != EEXIST)
|
||||
return false;
|
||||
}
|
||||
if (fd < 0)
|
||||
return false;
|
||||
fcntl(fd, F_SETFD, FD_CLOEXEC);
|
||||
#ifdef __linux__
|
||||
// Reserve tmpfs space before touching the mapping; a successful ftruncate
|
||||
// alone can still SIGBUS later when /dev/shm is full.
|
||||
if (posix_fallocate(fd, 0, size) != 0) {
|
||||
close(fd);
|
||||
Reset();
|
||||
return false;
|
||||
}
|
||||
#endif
|
||||
if (ftruncate(fd, size) == 0) {
|
||||
state_->data = mmap(nullptr, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
|
||||
state_->size = size;
|
||||
}
|
||||
close(fd);
|
||||
if (state_->data == MAP_FAILED) {
|
||||
Reset();
|
||||
return false;
|
||||
}
|
||||
std::memcpy(state_->data, data, size);
|
||||
return true;
|
||||
}
|
||||
struct NonBlocking::State {
|
||||
int fd, flags;
|
||||
};
|
||||
@@ -107,7 +169,11 @@ RawTerminal::RawTerminal(int input, int) : state_(std::make_unique<State>()) {
|
||||
}
|
||||
}
|
||||
RawTerminal::~RawTerminal() {
|
||||
tcsetattr(state_->fd, TCSAFLUSH, &state_->saved);
|
||||
// TCSAFLUSH waits for queued output, which can hang exit when the terminal
|
||||
// stops consuming a partially written image. Restore immediately and flush
|
||||
// only pending input so protocol replies do not reach the next shell prompt.
|
||||
tcsetattr(state_->fd, TCSANOW, &state_->saved);
|
||||
tcflush(state_->fd, TCIFLUSH);
|
||||
fcntl(state_->fd, F_SETFL, state_->flags);
|
||||
}
|
||||
std::string Password() {
|
||||
|
||||
@@ -330,6 +330,15 @@ TerminalSize GetTerminalSize(int) {
|
||||
CloseHandle(handle);
|
||||
return size;
|
||||
}
|
||||
struct SharedImage::State {
|
||||
std::string name;
|
||||
};
|
||||
SharedImage::SharedImage() : state_(std::make_unique<State>()) {}
|
||||
SharedImage::~SharedImage() = default;
|
||||
bool SharedImage::Assign(const uint8_t *, size_t) { return false; }
|
||||
void SharedImage::Reset() {}
|
||||
const std::string &SharedImage::Name() const { return state_->name; }
|
||||
|
||||
struct NonBlocking::State {
|
||||
HANDLE handle;
|
||||
DWORD mode = 0;
|
||||
|
||||
@@ -44,5 +44,7 @@ target("crossdesk-cli")
|
||||
if is_plat("windows") then
|
||||
add_syslinks("advapi32", "user32")
|
||||
add_defines("NOMINMAX", "WIN32_LEAN_AND_MEAN")
|
||||
elseif is_plat("linux") then
|
||||
add_syslinks("rt")
|
||||
end
|
||||
target_end()
|
||||
|
||||
@@ -335,9 +335,13 @@ void ControllerSession::Preview() {
|
||||
s.request_id = ++preview_request_;
|
||||
preview_accepted_ = false;
|
||||
s.fps = options_.fps == 0 ? 0
|
||||
: options_.fps > 15 ? 60
|
||||
: options_.fps <= 1 ? 1
|
||||
: options_.fps <= 8 || !fps15_ ? 8
|
||||
: 15;
|
||||
// 60 is the existing interactive-mode sentinel, not a 60 FPS request.
|
||||
// The encoder cadence is independently specified by frame_rate.
|
||||
s.frame_rate = 30;
|
||||
s.audio = false;
|
||||
s.quality = options_.quality;
|
||||
if (resolution_) {
|
||||
|
||||
@@ -63,10 +63,27 @@ std::optional<std::pair<float, float>> MapPointer(const Grid &, int column,
|
||||
int row);
|
||||
std::vector<std::string> Lines(const Grid &);
|
||||
std::string Difference(const Grid &, const std::optional<Grid> &previous);
|
||||
std::string Kitty(const DecodedFrame &, const Viewport &);
|
||||
struct PixelFrame {
|
||||
int width = 0, height = 0;
|
||||
std::vector<uint8_t> rgb;
|
||||
};
|
||||
PixelFrame KittyPixels(const DecodedFrame &, const Viewport &);
|
||||
std::string KittySharedUpload(const std::string &name, int width, int height,
|
||||
bool alternate_buffer, bool query = false);
|
||||
std::string KittyPresent(const Viewport &, bool alternate_buffer);
|
||||
struct KittyAck {
|
||||
uint32_t image_id = 0;
|
||||
bool ok = false;
|
||||
};
|
||||
std::optional<KittyAck> ParseKittyAck(const std::string &);
|
||||
// Toggle alternate_buffer only after the previous output has been written in
|
||||
// full. Upload to the other image ID before replacing the visible placement.
|
||||
std::string Kitty(const DecodedFrame &, const Viewport &,
|
||||
bool alternate_buffer = false);
|
||||
inline constexpr char kKittyQuery[] =
|
||||
"\033_Gi=31,s=1,v=1,a=q,t=d,f=24;AAAA\033\\";
|
||||
inline constexpr char kKittyDelete[] = "\033_Ga=d,d=I,i=31,q=2\033\\";
|
||||
inline constexpr char kKittyDelete[] =
|
||||
"\033_Ga=d,d=I,i=31,q=2\033\\\033_Ga=d,d=I,i=32,q=2\033\\";
|
||||
bool KittyReply(const std::string &response);
|
||||
struct StatusBar {
|
||||
std::string progress, compact_progress, error, view, network;
|
||||
@@ -89,6 +106,7 @@ public:
|
||||
uint64_t mouse_legacy_reports = 0, mouse_forwarded = 0;
|
||||
uint64_t mouse_send_failures = 0;
|
||||
TerminalMetrics terminal_metrics;
|
||||
std::optional<KittyAck> kitty_ack;
|
||||
|
||||
private:
|
||||
std::string pending_;
|
||||
|
||||
@@ -209,6 +209,8 @@ void Input::Feed(const std::string &bytes, ControllerSession &s,
|
||||
auto end = pending_.find("\033\\", 2);
|
||||
if (end == std::string::npos)
|
||||
return;
|
||||
if (auto ack = ParseKittyAck(pending_.substr(0, end + 2)))
|
||||
kitty_ack = ack;
|
||||
pending_.erase(0, end + 2);
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#include "terminal_renderer.h"
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <charconv>
|
||||
#include <cmath>
|
||||
#include <stdexcept>
|
||||
#include <zlib.h>
|
||||
@@ -88,52 +89,163 @@ static uint32_t Sample(const DecodedFrame &f, double x0, double y0, double x1,
|
||||
};
|
||||
return (round(0) << 16) | (round(1) << 8) | round(2);
|
||||
}
|
||||
static Cell Quadrant(const std::array<uint32_t, 4> &pixels) {
|
||||
// Enumerate all seven two-colour partitions (complements are equivalent).
|
||||
static constexpr uint32_t glyphs[16] = {
|
||||
0x20, 0x2598, 0x259D, 0x2580, 0x2596, 0x258C, 0x259E, 0x259B,
|
||||
0x2597, 0x259A, 0x2590, 0x259C, 0x2584, 0x2599, 0x259F, 0x2588};
|
||||
Cell best{};
|
||||
double best_error = 1e30;
|
||||
for (unsigned mask = 1; mask < 8; ++mask) {
|
||||
double sums[2][3] = {};
|
||||
unsigned count[2] = {};
|
||||
using QuadrantPixels = std::array<uint32_t, 4>;
|
||||
struct QuadrantFit {
|
||||
Cell cell;
|
||||
QuadrantPixels pixels{};
|
||||
double error = 0;
|
||||
};
|
||||
struct QuadrantEdge {
|
||||
double weight = 0;
|
||||
std::array<double, 3> error{};
|
||||
};
|
||||
using QuadrantEdges = std::array<QuadrantEdge, 4>;
|
||||
static int Channel(uint32_t color, int channel) {
|
||||
return (color >> (16 - 8 * channel)) & 255;
|
||||
}
|
||||
// Seven partitions suffice; their complements produce the same pixel colours.
|
||||
static constexpr uint32_t kQuadrantGlyphs[] = {0x20, 0x2598, 0x259D, 0x2580,
|
||||
0x2596, 0x258C, 0x259E, 0x259B};
|
||||
static QuadrantFit Quadrant(const QuadrantPixels &pixels) {
|
||||
int channels[4][3], total[3] = {}, squares = 0;
|
||||
uint32_t mean = 0;
|
||||
bool flat = true;
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
int lo = 255, hi = 0;
|
||||
for (int i = 0; i < 4; ++i) {
|
||||
int g = (mask >> i) & 1;
|
||||
++count[g];
|
||||
sums[g][0] += pixels[i] >> 16;
|
||||
sums[g][1] += (pixels[i] >> 8) & 255;
|
||||
sums[g][2] += pixels[i] & 255;
|
||||
int value = channels[i][c] = Channel(pixels[i], c);
|
||||
lo = std::min(lo, value);
|
||||
hi = std::max(hi, value);
|
||||
total[c] += value;
|
||||
squares += value * value;
|
||||
}
|
||||
// Keep the per-channel threshold small to retain faint/coloured detail
|
||||
// above this range, rather than judging flatness by average luminance.
|
||||
flat = flat && hi - lo <= 6;
|
||||
mean |= uint32_t((total[c] + 2) / 4) << (16 - 8 * c);
|
||||
}
|
||||
if (flat) {
|
||||
QuadrantFit fit{
|
||||
{mean, mean, 0x20}, {mean, mean, mean, mean}, double(squares)};
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
int value = Channel(mean, c);
|
||||
fit.error += 4 * value * value - 2 * value * total[c];
|
||||
}
|
||||
return fit;
|
||||
}
|
||||
// Reuse subset sums. Integer rounded means and sum-of-squares give exactly
|
||||
// the original RGB squared error without rescanning four pixels per mask.
|
||||
static constexpr int counts[] = {0, 1, 1, 2, 1, 2, 2, 3};
|
||||
int sums[8][3] = {};
|
||||
QuadrantFit best;
|
||||
int best_error = INT32_MAX, best_mask = 0;
|
||||
for (int mask = 1; mask < 8; ++mask) {
|
||||
int bit = mask & 1 ? 0 : mask & 2 ? 1 : 2;
|
||||
int rest = mask & (mask - 1), count = counts[mask];
|
||||
uint32_t fg = 0, bg = 0;
|
||||
int error = squares;
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
int sum = sums[mask][c] = sums[rest][c] + channels[bit][c];
|
||||
int front = (sum + count / 2) / count;
|
||||
int back = (total[c] - sum + (4 - count) / 2) / (4 - count);
|
||||
fg |= uint32_t(front) << (16 - 8 * c);
|
||||
bg |= uint32_t(back) << (16 - 8 * c);
|
||||
error += count * front * front - 2 * front * sum +
|
||||
(4 - count) * back * back - 2 * back * (total[c] - sum);
|
||||
}
|
||||
uint32_t colors[2] = {};
|
||||
double error = 0;
|
||||
for (int g = 0; g < 2; ++g)
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
sums[g][c] = std::round(sums[g][c] / count[g]);
|
||||
colors[g] |= uint32_t(sums[g][c]) << (16 - 8 * c);
|
||||
}
|
||||
for (int i = 0; i < 4; ++i)
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
double d = double((pixels[i] >> (16 - 8 * c)) & 255) -
|
||||
sums[(mask >> i) & 1][c];
|
||||
error += d * d;
|
||||
}
|
||||
if (error < best_error) {
|
||||
best_error = error;
|
||||
best = {colors[1], colors[0], glyphs[mask]};
|
||||
best_mask = mask;
|
||||
best.cell = {fg, bg, kQuadrantGlyphs[mask]};
|
||||
}
|
||||
}
|
||||
if (best.top == best.bottom)
|
||||
best.glyph = 0x20;
|
||||
best.error = best_error;
|
||||
for (int i = 0; i < 4; ++i)
|
||||
best.pixels[i] = (best_mask >> i) & 1 ? best.cell.top : best.cell.bottom;
|
||||
if (best.cell.top == best.cell.bottom)
|
||||
best.cell.glyph = 0x20;
|
||||
return best;
|
||||
}
|
||||
static std::string EncodeUtf8(uint32_t c) {
|
||||
if (c < 128)
|
||||
return std::string(1, char(c));
|
||||
if (c < 2048)
|
||||
return {char(0xc0 | (c >> 6)), char(0x80 | (c & 63))};
|
||||
return {char(0xe0 | (c >> 12)), char(0x80 | ((c >> 6) & 63)),
|
||||
char(0x80 | (c & 63))};
|
||||
static QuadrantFit RefineQuadrant(const QuadrantPixels &pixels,
|
||||
const QuadrantEdges &edges,
|
||||
const QuadrantFit &initial) {
|
||||
int channels[4][3];
|
||||
double weighted[4][3];
|
||||
for (int i = 0; i < 4; ++i)
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
channels[i][c] = Channel(pixels[i], c);
|
||||
weighted[i][c] =
|
||||
(1 + edges[i].weight) * channels[i][c] + edges[i].error[c];
|
||||
}
|
||||
auto score = [&](const QuadrantFit &fit) {
|
||||
double value = fit.error;
|
||||
for (int i = 0; i < 4; ++i)
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
double d = Channel(fit.pixels[i], c) - channels[i][c];
|
||||
// Match neighbouring quantization errors, preserving the source
|
||||
// gradient. The neighbour-only constant does not affect selection.
|
||||
value += edges[i].weight * d * d - 2 * edges[i].error[c] * d;
|
||||
}
|
||||
return value;
|
||||
};
|
||||
QuadrantFit best = initial;
|
||||
double best_score = score(best);
|
||||
const double max_error = initial.error + std::min(48.0, initial.error * .05);
|
||||
for (unsigned mask = 1; mask < 8; ++mask) {
|
||||
double sums[2][3] = {};
|
||||
double weights[2] = {};
|
||||
for (int i = 0; i < 4; ++i) {
|
||||
int g = (mask >> i) & 1;
|
||||
double weight = 1 + edges[i].weight;
|
||||
weights[g] += weight;
|
||||
for (int c = 0; c < 3; ++c)
|
||||
sums[g][c] += weighted[i][c];
|
||||
}
|
||||
uint32_t colors[2] = {};
|
||||
for (int g = 0; g < 2; ++g)
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
sums[g][c] =
|
||||
std::clamp(std::round(sums[g][c] / weights[g]), 0.0, 255.0);
|
||||
colors[g] |= uint32_t(sums[g][c]) << (16 - 8 * c);
|
||||
}
|
||||
QuadrantFit fit{{colors[1], colors[0], kQuadrantGlyphs[mask]}};
|
||||
for (int i = 0; i < 4; ++i) {
|
||||
fit.pixels[i] = colors[(mask >> i) & 1];
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
double d = channels[i][c] - Channel(fit.pixels[i], c);
|
||||
fit.error += d * d;
|
||||
}
|
||||
}
|
||||
// Limit the cost of continuity to 5% of the independent fit's error,
|
||||
// capped at 2 RGB levels RMS over the cell's twelve channel samples.
|
||||
if (fit.error > max_error)
|
||||
continue;
|
||||
double value = score(fit);
|
||||
if (value < best_score) {
|
||||
best_score = value;
|
||||
best = fit;
|
||||
}
|
||||
}
|
||||
if (best.cell.top == best.cell.bottom)
|
||||
best.cell.glyph = 0x20;
|
||||
return best;
|
||||
}
|
||||
static void AppendUtf8(std::string &out, uint32_t c) {
|
||||
if (c < 128) {
|
||||
out += char(c);
|
||||
} else if (c < 2048) {
|
||||
out += char(0xc0 | (c >> 6));
|
||||
out += char(0x80 | (c & 63));
|
||||
} else {
|
||||
out += char(0xe0 | (c >> 12));
|
||||
out += char(0x80 | ((c >> 6) & 63));
|
||||
out += char(0x80 | (c & 63));
|
||||
}
|
||||
}
|
||||
static void AppendNumber(std::string &out, uint32_t value) {
|
||||
char digits[10];
|
||||
auto result = std::to_chars(digits, digits + sizeof(digits), value);
|
||||
out.append(digits, result.ptr);
|
||||
}
|
||||
Grid Layout(const DecodedFrame &f, Viewport v) {
|
||||
Validate(f, v);
|
||||
@@ -156,6 +268,8 @@ Grid Convert(const DecodedFrame &f, Viewport v) {
|
||||
const auto crop = g.source;
|
||||
const int sx = g.samples_x, sy = g.samples_y;
|
||||
g.cells.resize(size_t(v.columns) * v.rows);
|
||||
std::vector<QuadrantPixels> samples(sx == 2 ? g.cells.size() : 0);
|
||||
std::vector<QuadrantFit> fits(samples.size());
|
||||
auto pixel = [&](int x, int y) {
|
||||
const auto &r = g.content;
|
||||
if (x < r.x || y < r.y || x >= r.x + r.width || y >= r.y + r.height)
|
||||
@@ -167,15 +281,65 @@ Grid Convert(const DecodedFrame &f, Viewport v) {
|
||||
};
|
||||
for (int y = 0; y < v.rows; ++y)
|
||||
for (int x = 0; x < v.columns; ++x) {
|
||||
auto &c = g.cells[size_t(y) * v.columns + x];
|
||||
if (sx == 2)
|
||||
c = Quadrant({pixel(x * 2, y * 2), pixel(x * 2 + 1, y * 2),
|
||||
pixel(x * 2, y * 2 + 1), pixel(x * 2 + 1, y * 2 + 1)});
|
||||
else {
|
||||
const auto index = size_t(y) * v.columns + x;
|
||||
auto &c = g.cells[index];
|
||||
if (sx == 2) {
|
||||
samples[index] = {pixel(x * 2, y * 2), pixel(x * 2 + 1, y * 2),
|
||||
pixel(x * 2, y * 2 + 1), pixel(x * 2 + 1, y * 2 + 1)};
|
||||
fits[index] = Quadrant(samples[index]);
|
||||
c = fits[index].cell;
|
||||
} else {
|
||||
c.top = pixel(x, y * sy);
|
||||
c.bottom = sy == 1 ? c.top : pixel(x, y * sy + sy - 1);
|
||||
}
|
||||
}
|
||||
// Read only independent fits so refinement has no left-to-right/top-to-bottom
|
||||
// feedback, and a changed cell cannot propagate through the entire frame.
|
||||
if (sx == 2)
|
||||
for (int y = 0; y < v.rows; ++y)
|
||||
for (int x = 0; x < v.columns; ++x) {
|
||||
const auto index = size_t(y) * v.columns + x;
|
||||
if (fits[index].cell.glyph == 0x20 || fits[index].error == 0)
|
||||
continue;
|
||||
QuadrantEdges edges{};
|
||||
auto edge = [&](int i, size_t neighbour, int j) {
|
||||
int contrast = 0;
|
||||
for (int c = 0; c < 3; ++c)
|
||||
contrast =
|
||||
std::max(contrast, std::abs(Channel(samples[index][i], c) -
|
||||
Channel(samples[neighbour][j], c)));
|
||||
// Fade out across real edges instead of smoothing text into its
|
||||
// background. Equal source colours receive the strongest constraint.
|
||||
double strength = std::max(0.0, 1 - contrast / 24.0);
|
||||
double weight = .35 * strength * strength;
|
||||
edges[i].weight += weight;
|
||||
for (int c = 0; c < 3; ++c)
|
||||
edges[i].error[c] +=
|
||||
weight * (Channel(fits[neighbour].pixels[j], c) -
|
||||
Channel(samples[neighbour][j], c));
|
||||
};
|
||||
if (x > 0) {
|
||||
edge(0, index - 1, 1);
|
||||
edge(2, index - 1, 3);
|
||||
}
|
||||
if (x + 1 < v.columns) {
|
||||
edge(1, index + 1, 0);
|
||||
edge(3, index + 1, 2);
|
||||
}
|
||||
if (y > 0) {
|
||||
edge(0, index - v.columns, 2);
|
||||
edge(1, index - v.columns, 3);
|
||||
}
|
||||
if (y + 1 < v.rows) {
|
||||
edge(2, index + v.columns, 0);
|
||||
edge(3, index + v.columns, 1);
|
||||
}
|
||||
if (edges[0].weight + edges[1].weight + edges[2].weight +
|
||||
edges[3].weight >
|
||||
0)
|
||||
g.cells[index] =
|
||||
RefineQuadrant(samples[index], edges, fits[index]).cell;
|
||||
}
|
||||
return g;
|
||||
}
|
||||
std::optional<std::pair<float, float>> MapPointer(const Grid &g, int x, int y) {
|
||||
@@ -194,12 +358,10 @@ std::optional<std::pair<float, float>> MapPointer(const Grid &g, int x, int y) {
|
||||
(g.source.y + fy * std::max(0, g.source.height - 1)) /
|
||||
std::max(1u, g.source_height - 1)};
|
||||
}
|
||||
static std::string Color(uint32_t c, bool fg, int depth) {
|
||||
static uint32_t TerminalColor(uint32_t c, int depth) {
|
||||
const int r = c >> 16, g = (c >> 8) & 255, b = c & 255;
|
||||
const std::string prefix = fg ? "\033[38;" : "\033[48;";
|
||||
if (depth == 24)
|
||||
return prefix + "2;" + std::to_string(r) + ";" + std::to_string(g) + ";" +
|
||||
std::to_string(b) + "m";
|
||||
return c;
|
||||
if (depth == 256) {
|
||||
constexpr int levels[] = {0, 95, 135, 175, 215, 255};
|
||||
auto index = [&](int value) {
|
||||
@@ -218,55 +380,101 @@ static std::string Color(uint32_t c, bool fg, int depth) {
|
||||
gv = 8 + 10 * gray;
|
||||
int gray_error =
|
||||
(r - gv) * (r - gv) + (g - gv) * (g - gv) + (b - gv) * (b - gv);
|
||||
return prefix + "5;" +
|
||||
std::to_string(gray_error < cube ? 232 + gray
|
||||
: 16 + 36 * ri + 6 * gi + bi) +
|
||||
"m";
|
||||
return gray_error < cube ? 232 + gray : 16 + 36 * ri + 6 * gi + bi;
|
||||
}
|
||||
int i = (r >= 128 ? 1 : 0) + (g >= 128 ? 2 : 0) + (b >= 128 ? 4 : 0);
|
||||
return "\033[" + std::to_string((fg ? 30 : 40) + i) + "m";
|
||||
return (r >= 128 ? 1 : 0) + (g >= 128 ? 2 : 0) + (b >= 128 ? 4 : 0);
|
||||
}
|
||||
static std::string Line(const Grid &g, int y) {
|
||||
std::string line;
|
||||
uint32_t fg = UINT32_MAX, bg = UINT32_MAX;
|
||||
for (int x = 0; x < g.viewport.columns; ++x) {
|
||||
const auto &c = g.cells[size_t(y) * g.viewport.columns + x];
|
||||
if (g.viewport.utf8 && fg != c.top) {
|
||||
line += Color(c.top, true, g.viewport.colors);
|
||||
fg = c.top;
|
||||
static void AppendColor(std::string &out, uint32_t c, bool fg, int depth) {
|
||||
if (depth == 24 || depth == 256) {
|
||||
out += fg ? "\033[38;" : "\033[48;";
|
||||
if (depth == 24) {
|
||||
out += "2;";
|
||||
AppendNumber(out, c >> 16);
|
||||
out += ';';
|
||||
AppendNumber(out, (c >> 8) & 255);
|
||||
out += ';';
|
||||
AppendNumber(out, c & 255);
|
||||
} else {
|
||||
out += "5;";
|
||||
AppendNumber(out, c);
|
||||
}
|
||||
const auto bottom = g.viewport.utf8 ? c.bottom : c.top;
|
||||
if (bg != bottom) {
|
||||
line += Color(bottom, false, g.viewport.colors);
|
||||
bg = bottom;
|
||||
}
|
||||
line += g.viewport.utf8 ? EncodeUtf8(c.glyph) : " ";
|
||||
} else {
|
||||
out += "\033[";
|
||||
AppendNumber(out, (fg ? 30 : 40) + c);
|
||||
}
|
||||
out += 'm';
|
||||
}
|
||||
struct AnsiColors {
|
||||
uint32_t fg = UINT32_MAX, bg = UINT32_MAX;
|
||||
};
|
||||
static void AppendCells(std::string &out, const Grid &g, int y, int begin,
|
||||
int end, AnsiColors &colors) {
|
||||
for (int x = begin; x < end; ++x) {
|
||||
const auto &c = g.cells[size_t(y) * g.viewport.columns + x];
|
||||
// Spaces need no foreground SGR. Keep backgrounds for block glyphs too,
|
||||
// since font rasterization can expose them at the cell's edges.
|
||||
if (g.viewport.utf8 && c.glyph != 0x20) {
|
||||
auto fg = TerminalColor(c.top, g.viewport.colors);
|
||||
if (colors.fg != fg) {
|
||||
AppendColor(out, fg, true, g.viewport.colors);
|
||||
colors.fg = fg;
|
||||
}
|
||||
}
|
||||
// Track the emitted palette index so quantized colours do not resend SGR.
|
||||
auto bg =
|
||||
TerminalColor(g.viewport.utf8 ? c.bottom : c.top, g.viewport.colors);
|
||||
if (colors.bg != bg) {
|
||||
AppendColor(out, bg, false, g.viewport.colors);
|
||||
colors.bg = bg;
|
||||
}
|
||||
AppendUtf8(out, g.viewport.utf8 ? c.glyph : 0x20);
|
||||
}
|
||||
return line + "\033[0m";
|
||||
}
|
||||
std::vector<std::string> Lines(const Grid &g) {
|
||||
std::vector<std::string> lines;
|
||||
lines.reserve(g.viewport.rows);
|
||||
for (int y = 0; y < g.viewport.rows; ++y)
|
||||
lines.push_back(Line(g, y));
|
||||
for (int y = 0; y < g.viewport.rows; ++y) {
|
||||
std::string line;
|
||||
line.reserve(size_t(g.viewport.columns) * 40 + 4);
|
||||
AnsiColors colors;
|
||||
AppendCells(line, g, y, 0, g.viewport.columns, colors);
|
||||
line += "\033[0m";
|
||||
lines.push_back(std::move(line));
|
||||
}
|
||||
return lines;
|
||||
}
|
||||
std::string Difference(const Grid &g, const std::optional<Grid> &previous) {
|
||||
std::string out;
|
||||
AnsiColors colors;
|
||||
const bool same = previous &&
|
||||
previous->viewport.columns == g.viewport.columns &&
|
||||
previous->viewport.rows == g.viewport.rows &&
|
||||
previous->viewport.row_offset == g.viewport.row_offset &&
|
||||
previous->viewport.colors == g.viewport.colors &&
|
||||
previous->viewport.utf8 == g.viewport.utf8;
|
||||
previous->viewport.utf8 == g.viewport.utf8 &&
|
||||
previous->cells.size() == g.cells.size();
|
||||
for (int y = 0; y < g.viewport.rows; ++y) {
|
||||
auto start = g.cells.begin() + size_t(y) * g.viewport.columns;
|
||||
if (!same ||
|
||||
!std::equal(start, start + g.viewport.columns,
|
||||
previous->cells.begin() + size_t(y) * g.viewport.columns))
|
||||
out += "\033[" + std::to_string(g.viewport.row_offset + y + 1) +
|
||||
";1H" + Line(g, y);
|
||||
int begin = 0, end = g.viewport.columns;
|
||||
if (same) {
|
||||
const auto offset = size_t(y) * g.viewport.columns;
|
||||
while (begin < end &&
|
||||
g.cells[offset + begin] == previous->cells[offset + begin])
|
||||
++begin;
|
||||
while (end > begin &&
|
||||
g.cells[offset + end - 1] == previous->cells[offset + end - 1])
|
||||
--end;
|
||||
}
|
||||
if (begin == end)
|
||||
continue;
|
||||
out += "\033[";
|
||||
AppendNumber(out, g.viewport.row_offset + y + 1);
|
||||
out += ';';
|
||||
AppendNumber(out, begin + 1);
|
||||
out += 'H';
|
||||
AppendCells(out, g, y, begin, end, colors);
|
||||
}
|
||||
if (!out.empty())
|
||||
out += "\033[0m";
|
||||
return out;
|
||||
}
|
||||
static std::string Base64(const std::vector<uint8_t> &data) {
|
||||
@@ -287,7 +495,132 @@ static std::string Base64(const std::vector<uint8_t> &data) {
|
||||
}
|
||||
return out;
|
||||
}
|
||||
std::string Kitty(const DecodedFrame &f, const Viewport &requested) {
|
||||
PixelFrame KittyPixels(const DecodedFrame &f, const Viewport &requested) {
|
||||
auto v = requested;
|
||||
v.pixel_mode = true;
|
||||
Validate(f, v);
|
||||
PixelFrame pixels;
|
||||
pixels.width =
|
||||
std::clamp(v.pixel_width > 0 ? v.pixel_width : v.columns * 10, 2, 1920);
|
||||
pixels.height =
|
||||
std::clamp(v.pixel_height > 0 ? v.pixel_height : v.rows * 20, 2, 1080);
|
||||
pixels.rgb.resize(size_t(pixels.width) * pixels.height * 3, 0);
|
||||
const auto crop = Crop(f, v);
|
||||
const auto r = Fit(crop.width, crop.height, pixels.width, pixels.height,
|
||||
double(v.columns) * v.cell_aspect * pixels.height /
|
||||
(v.rows * pixels.width));
|
||||
if (r.width == crop.width && r.height == crop.height) {
|
||||
for (int y = 0; y < r.height; ++y)
|
||||
for (int x = 0; x < r.width; ++x) {
|
||||
const auto c = Pixel(f, crop.x + x, crop.y + y);
|
||||
auto *dst = &pixels.rgb[(size_t(y + r.y) * pixels.width + x + r.x) * 3];
|
||||
dst[0] = c >> 16;
|
||||
dst[1] = c >> 8;
|
||||
dst[2] = c;
|
||||
}
|
||||
return pixels;
|
||||
}
|
||||
// Separable area integration. Integer overlap weights avoid per-pixel
|
||||
// floor/ceil and repeated NV12 conversion in overlapping vertical samples.
|
||||
// Adjacent output rows can share only their boundary source row, so one
|
||||
// cached horizontal row suffices, even when upscaling.
|
||||
struct Span {
|
||||
int first, last, begin, end;
|
||||
};
|
||||
std::vector<Span> spans;
|
||||
for (int x = 0; x < r.width; ++x) {
|
||||
const int begin = x * crop.width, end = begin + crop.width;
|
||||
spans.push_back(
|
||||
{begin / r.width, (end + r.width - 1) / r.width, begin, end});
|
||||
}
|
||||
std::vector<uint32_t> source(crop.width), horizontal(size_t(r.width) * 3);
|
||||
std::vector<uint64_t> sums(size_t(r.width) * 3);
|
||||
const uint64_t area = uint64_t(crop.width) * crop.height;
|
||||
int cached_y = -1;
|
||||
for (int y = 0; y < r.height; ++y) {
|
||||
std::fill(sums.begin(), sums.end(), 0);
|
||||
const int begin = y * crop.height, end = begin + crop.height;
|
||||
for (int sy = begin / r.height; sy < (end + r.height - 1) / r.height;
|
||||
++sy) {
|
||||
if (cached_y != sy) {
|
||||
for (int x = 0; x < crop.width; ++x)
|
||||
source[x] = Pixel(f, crop.x + x, crop.y + sy);
|
||||
std::fill(horizontal.begin(), horizontal.end(), 0);
|
||||
for (int x = 0; x < r.width; ++x) {
|
||||
const auto &span = spans[x];
|
||||
for (int sx = span.first; sx < span.last; ++sx) {
|
||||
const unsigned weight = std::min(span.end, (sx + 1) * r.width) -
|
||||
std::max(span.begin, sx * r.width);
|
||||
const auto c = source[sx];
|
||||
horizontal[x * 3] += (c >> 16) * weight;
|
||||
horizontal[x * 3 + 1] += ((c >> 8) & 255) * weight;
|
||||
horizontal[x * 3 + 2] += (c & 255) * weight;
|
||||
}
|
||||
}
|
||||
cached_y = sy;
|
||||
}
|
||||
const unsigned weight =
|
||||
std::min(end, (sy + 1) * r.height) - std::max(begin, sy * r.height);
|
||||
for (size_t i = 0; i < sums.size(); ++i)
|
||||
sums[i] += uint64_t(horizontal[i]) * weight;
|
||||
}
|
||||
auto *dst = &pixels.rgb[(size_t(y + r.y) * pixels.width + r.x) * 3];
|
||||
for (size_t i = 0; i < sums.size(); ++i)
|
||||
dst[i] = (sums[i] + area / 2) / area;
|
||||
}
|
||||
return pixels;
|
||||
}
|
||||
std::string KittySharedUpload(const std::string &name, int width, int height,
|
||||
bool alternate_buffer, bool query) {
|
||||
if (name.empty() || name.size() > 128 || width < 1 || width > 1920 ||
|
||||
height < 1 || height > 1080)
|
||||
throw std::invalid_argument("invalid shared image");
|
||||
return "\033_Ga=" + std::string(query ? "q" : "t") + ",t=s,f=24,q=0,i=" +
|
||||
std::to_string(query ? 33
|
||||
: alternate_buffer ? 32
|
||||
: 31) +
|
||||
",s=" + std::to_string(width) + ",v=" + std::to_string(height) +
|
||||
",S=" + std::to_string(size_t(width) * height * 3) + ";" +
|
||||
Base64(std::vector<uint8_t>(name.begin(), name.end())) + "\033\\";
|
||||
}
|
||||
std::string KittyPresent(const Viewport &v, bool alternate_buffer) {
|
||||
return "\033[" + std::to_string(v.row_offset + 1) +
|
||||
";1H\033_Ga=p,i=" + std::to_string(alternate_buffer ? 32 : 31) +
|
||||
",p=1,C=1,q=2,c=" + std::to_string(v.columns) +
|
||||
",r=" + std::to_string(v.rows) +
|
||||
"\033\\\033_Ga=d,d=I,i=" + std::to_string(alternate_buffer ? 31 : 32) +
|
||||
",q=2\033\\";
|
||||
}
|
||||
std::optional<KittyAck> ParseKittyAck(const std::string &s) {
|
||||
if (s.size() > 1024 || s.size() < 10 || s.compare(0, 3, "\033_G") != 0 ||
|
||||
s.compare(s.size() - 2, 2, "\033\\") != 0)
|
||||
return {};
|
||||
const auto separator = s.find(';', 3);
|
||||
if (separator == std::string::npos)
|
||||
return {};
|
||||
uint32_t image_id = 0;
|
||||
for (size_t pos = 3; pos < separator;) {
|
||||
const auto end = std::min(s.find(',', pos), separator);
|
||||
if (s.compare(pos, 2, "i=") == 0) {
|
||||
if (image_id)
|
||||
return {};
|
||||
const auto parsed =
|
||||
std::from_chars(s.data() + pos + 2, s.data() + end, image_id);
|
||||
if (parsed.ec != std::errc{} || parsed.ptr != s.data() + end || !image_id)
|
||||
return {};
|
||||
} else if (s.compare(pos, 2, "p=") == 0 &&
|
||||
s.substr(pos, end - pos) != "p=0") {
|
||||
return {}; // Only upload/query acknowledgements are relevant here.
|
||||
}
|
||||
pos = end + 1;
|
||||
}
|
||||
if (!image_id)
|
||||
return {};
|
||||
return KittyAck{image_id,
|
||||
s.substr(separator + 1, s.size() - separator - 3) == "OK"};
|
||||
}
|
||||
std::string Kitty(const DecodedFrame &f, const Viewport &requested,
|
||||
bool alternate_buffer) {
|
||||
auto v = requested;
|
||||
v.pixel_mode = true;
|
||||
Validate(f, v);
|
||||
@@ -327,16 +660,22 @@ std::string Kitty(const DecodedFrame &f, const Viewport &requested) {
|
||||
w = std::max(2, w * 3 / 4);
|
||||
h = std::max(2, h * 3 / 4);
|
||||
}
|
||||
std::string out = "\033[" + std::to_string(v.row_offset + 1) + ";1H";
|
||||
const auto image_id = std::to_string(alternate_buffer ? 32 : 31);
|
||||
std::string out;
|
||||
out.reserve(b64.size() + (b64.size() / 4096 + 1) * 12 + 256);
|
||||
for (size_t i = 0; i < b64.size(); i += 4096) {
|
||||
out += "\033_G";
|
||||
if (i == 0)
|
||||
out += "a=T,t=d,f=24,o=z,q=2,i=31,p=1,C=1,s=" + std::to_string(w) +
|
||||
",v=" + std::to_string(h) + ",c=" + std::to_string(v.columns) +
|
||||
",r=" + std::to_string(v.rows) + ",";
|
||||
out += "m=" + std::to_string(i + 4096 < b64.size() ? 1 : 0) + ";" +
|
||||
b64.substr(i, 4096) + "\033\\";
|
||||
out += "a=t,t=d,f=24,o=z,q=2,i=" + image_id + ",s=" +
|
||||
std::to_string(w) + ",v=" + std::to_string(h) + ",";
|
||||
out += i + 4096 < b64.size() ? "m=1;" : "m=0;";
|
||||
out.append(b64, i, 4096);
|
||||
out += "\033\\";
|
||||
}
|
||||
// Re-transmitting a visible image ID can remove its placement as soon as
|
||||
// upload starts. Keep the old image throughout chunking/short writes, then
|
||||
// place the fully uploaded image before freeing the previous one.
|
||||
out += KittyPresent(v, alternate_buffer);
|
||||
return out;
|
||||
}
|
||||
bool KittyReply(const std::string &s) {
|
||||
|
||||
Reference in New Issue
Block a user