[fix] preserve CLI display aspect ratio across terminals

This commit is contained in:
dijunkun
2026-10-30 17:15:47 +08:00
parent 15092b16d6
commit cf039b8f91
10 changed files with 277 additions and 20 deletions
+5 -3
View File
@@ -8,7 +8,7 @@
| method | 字段 | 行为 |
|---|---|---|
| hello | version:1 | 协议版本握手 |
| viewport | columns:1..240, rows:1..100, colors:8/24/256, utf8:boolean, fps:1..30;可选 detail:quadrant/half, quality:low/balanced/high/ultra, zoom:1..32, native:boolean | 设置画面行数(不含状态栏)、本地目标与预览档位;默认 quadrant/high/1、native=false,保留未显式指定的 detail/quality/zoom/native;不探测宿主 TTY |
| viewport | columns:1..240, rows:1..100, colors:8/24/256, utf8:boolean, fps:1..30;可选 cell_aspect:0.2..2, detail:quadrant/half, quality:low/balanced/high/ultra, zoom:1..32, native:boolean | 设置画面行数(不含状态栏)、字符宽高比、本地目标与预览档位;默认 cell_aspect=0.5、quadrant/high/1、native=false,保留未显式指定的 cell_aspect/detail/quality/zoom/native;不探测宿主 TTY |
| connect | device:9位数字字符串, display:0..63(默认0) | 从专用 password fd 读取至换行/EOF(最多256字节、等待5秒),开始异步注册/连接;ok不是远端已连接 |
| status | 无 | 返回当前状态 |
| control | enabled:boolean | 显式启停控制;关闭时释放输入 |
@@ -23,10 +23,12 @@
帧事件:
```json
{"event":"frame","frame_id":42,"decoded_frame_id":38,"generation":4,"source_width":1280,"source_height":720,"zoom":2,"native":false,"source_rect":[320,180,640,360],"detail":"quadrant","columns":80,"rows":23,"lines":["..."],"queue_ms":3.2}
{"event":"frame","frame_id":42,"decoded_frame_id":38,"generation":4,"source_width":1280,"source_height":720,"zoom":2,"native":false,"source_rect":[320,180,640,360],"detail":"quadrant","columns":80,"rows":23,"cell_aspect":0.45,"lines":["..."],"queue_ms":3.2}
```
实际 lines 数为 rows,仅包含本地生成的 SGR 颜色序列、空格和 U+2580~U+259F 块字符,不含远端名称或任意终端指令。`frame_id` 是进程递增的输出序号,静止画面重绘仍产生新序号;`decoded_frame_id` 标识源解码帧,同源重绘可以相同。`source_width/height` 为实际解码尺寸,source_rect 为实际源裁剪矩形 [x,y,width,height]。native=true 表示原始细节模式,zoom 为当前模式下的倍率;generation 在关闭、重新连接和显示器切换时变化。viewport 变化可能与已写出的旧尺寸帧交错,宿主应丢弃不匹配尺寸/旧 generation 的帧并释放对应 ACK。宿主返回的 ACK 表示消费/布局,不表示屏幕物理显示完成。
实际 lines 数为 rows,仅包含本地生成的 SGR 颜色序列、空格和 U+2580~U+259F 块字符,不含远端名称或任意终端指令。`frame_id` 是进程递增的输出序号,静止画面重绘仍产生新序号;`decoded_frame_id` 标识源解码帧,同源重绘可以相同。`source_width/height` 为实际解码尺寸,source_rect 为实际源裁剪矩形 [x,y,width,height]。native=true 表示原始细节模式,zoom 为当前模式下的倍率;generation 在关闭、重新连接和显示器切换时变化。viewport 变化可能与已写出的旧尺寸帧交错,宿主应丢弃不匹配行列数/cell_aspect 或旧 generation 的帧并释放对应 ACK。宿主返回的 ACK 表示消费/布局,不表示屏幕物理显示完成。
`cell_aspect` 为单个字符的宽/高,必须是 0.2~2 范围内的有限 JSON 数值;例如 9×20 像素字符应传 0.45。宿主负责测量或提供手动设置,bridge 根据该比例计算图像布局与鼠标映射,响应及状态事件中的 `status.cell_aspect` 返回当前值。该字段为 v1 的可选扩展;省略时保留上次值,首次默认 0.5。旧版 bridge 的帧可能不含此字段。修改比例也会触发静止画面重绘,新的鼠标映射仍在帧 ACK 后生效。
状态事件每秒最多一次:`{"event":"state","status":{...},"output_bytes":123,"submitted_frames":3,"elapsed_seconds":1.5}`。status 含 registering/connecting/connected/failed/closed/idle、client_id(当前或刚结束会话的 `cli-xxxxxxxxx`,注册前为空)、error、control、generation、显示器、文本/预览能力、预览确认状态、请求的 preview_width/height/quality/bitrate_bps、帧计数与网络统计。接收/提交 FPS 由相应计数的增量除以时间增量计算。
+10 -1
View File
@@ -45,7 +45,13 @@ crossdesk-cli 123456789 --server your-server.example --port 9099
ANSI 默认使用四象限字符:每个字符从 2×2 样本选择误差最小的两个颜色和象限形状,比原先上下半方块多一倍水平采样点;每个字符仍只有前景/背景两色。使用 `--ansi-detail half` 可回到上下半方块以兼容字体。缩小时采用面积加权滤波,避免最近邻采样漏掉细线。支持 TrueColor、256 色和基础 8 色,256 色选择实际颜色立方体或灰阶条中更接近的颜色。环境的 COLORTERM/TERM 决定颜色模式;非 UTF-8 locale 使用背景色空格。TERM=dumb、非 TTY 输出明确拒绝交互连接,不输出全屏转义序列。
画面按字符单元比例等比缩放,留黑边及状态栏,默认假定字符宽高比 0.5;TTY 提供像素尺寸时使用其比例。每次只输出变化行,合并连续颜色设置;缩放在完整输出边界清理旧画面。上限 240 列、100 画面行;默认顶部预留一行状态栏、底部预留两行快捷键,另预留一列避免自动换行。顶部状态栏用反色区分,左侧为控制模式和连接进度,右侧为画面参数及网络信息;底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始,鼠标坐标按视频区域映射,点击状态栏不会传给远端。按键分组使用统一列宽和左右留白,窄窗口自动减少列数、合并按键组并省略次要参数,累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作,进入本地命令模式后才显示平移和缩放按键;终端高度不足时优先收起底部提示行。
画面按字符单元比例等比缩放,留黑边及状态栏。交互连接通过终端字节流查询字符尺寸,支持通过 SSH 返回的回复:优先使用 `CSI 16 t` 的字符像素尺寸,否则组合 `CSI 14 t` 的完整文本区像素尺寸和 `CSI 18 t` 的完整行列数计算,兼容 GNOME/VTE 的文本区报告。计算后再扣除状态栏及画面行列上限,不把预留行当作图像高度。窗口改变时重新查询,每两秒刷新一次以覆盖字体/行距变化;回复到达后会重绘静止画面和更新鼠标映射。查询不阻塞连接;没有有效回复时使用系统提供的像素尺寸,再缺失时回退为宽高比 0.5。曾收到有效回复时保留最近有效尺寸,直到下一次有效回复。字符栅格的整数舍入仍可能产生小幅比例误差。格式依据 [XTerm 窗口尺寸报告](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html) 和 [GNOME VTE 实现](https://github.com/GNOME/vte/blob/master/src/vteseq.cc)。
Windows 的可见传统控制台优先读取当前字体尺寸;Windows Terminal/ConPTY 使用显示终端的尺寸回复,不采用隐藏控制台宿主的字体值。不同终端版本可能不支持查询,或返回虚拟图形尺寸而非实际字体尺寸,此时需要手动校准。
默认 `--cell-aspect auto` 自动确定比例。如果终端屏蔽查询或需要手动校准,可设置 `--cell-aspect 0.45`,数值表示单个字符的**宽/高**,允许 0.2~2;例如字符为 9×20 像素时使用 0.45。画面被压窄时调小此值,被拉宽时调大。此参数优先于自动检测,同时用于全景、细节和鼠标坐标映射。
每次只输出变化行,合并连续颜色设置;缩放在完整输出边界清理旧画面。上限 240 列、100 画面行;默认顶部预留一行状态栏、底部预留两行快捷键,另预留一列避免自动换行。顶部状态栏用反色区分,左侧为控制模式和连接进度,右侧为画面参数及网络信息;底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始,鼠标坐标按视频区域映射,点击状态栏不会传给远端。按键分组使用统一列宽和左右留白,窄窗口自动减少列数、合并按键组并省略次要参数,累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作,进入本地命令模式后才显示平移和缩放按键;终端高度不足时优先收起底部提示行。
快捷键行使用 `KEYS |`、`VIEW |` 等固定宽度标题,标题和按键组之间用竖线分隔;键名用 `<c>`、`<+/->` 等标签标识。宽窗口中的分组分隔线按列对齐,窄窗口保留标题和完整按键标签,按可用宽度合并或减少分组。
@@ -123,6 +129,8 @@ pi -e ./apps/cli/plugins/pi/index.ts
在 Pi 输入 `/crossdesk` 或 `/crossdesk 123456789`。先输入 ID,再在自定义遮蔽组件输入密码。扩展使用独立 fd 3 写密码,随后关闭;不写配置、聊天、工具结果或模型上下文。查看组件与独立 CLI 使用相同的 `Ctrl+]` 命令前缀和上述模式按键,由 bridge 统一解析。插件默认请求 high 预览并显示 ANSI 象限字符行,支持同样的只读细节/放大/平移键,颜色深度依据终端环境回退,由 Pi 自身布局和刷新;不直接占用 TTY,不发送 Kitty 图形或自行打开鼠标报告。
Pi 扩展读取宿主的字符尺寸,并通过宿主终端写入/输入监听接口查询字符尺寸或完整文本区尺寸,每两秒及窗口变化时刷新;尺寸回复不会传给远端。得到的宽高比通过 bridge 的 `viewport.cell_aspect` 传入,比例变化时重绘静止画面,并丢弃仍在途的旧比例帧。自动检测不可用或不准确时,可在启动 Pi 前设置环境变量 `CROSSDESK_CELL_ASPECT=0.45`(宽/高,允许 0.2~2;`auto` 恢复自动检测)。视图关闭或 bridge 停止时移除查询定时器和输入监听。
需要全局发现时,把整个 `apps/cli/plugins/pi` 目录复制到 `~/.pi/agent/extensions/crossdesk/` 后重启 Pi 或执行 `/reload`。卸载时先关闭视图,删除该目录并 `/reload`;使用 `-e` 时从启动参数移除即可。无需修改包中的密码或设备配置。默认连接官方信令;自建信令当前通过独立 CLI 连接命令使用,插件尚无服务器设置界面。
仅在 `ctx.mode === "tui"`、hasUI 且进程双向 TTY 时打开组件。RPC/print/json 返回独立播放器提示,不声称内嵌视频。UI 关闭/会话退出先关闭 bridge stdin,超时 SIGTERM,再超时 SIGKILL。bridge 尚未就绪或已经停止时,组件可直接用 `q`、`Esc` 或 `Ctrl+C` 关闭。SDK 类型和模拟 UI 生命周期已验证,真实 Pi TUI 视觉刷新、增强按键与鼠标传递未实机验收。真实设备的 bridge 与独立 CLI 连接验证见 [验证记录](validation.md)。
@@ -144,6 +152,7 @@ pi -e ./apps/cli/plugins/pi/index.ts
- `connection_failed` 表示连接建立失败;`connection_lost` 表示连接成功后传输失效。xmake 随后的 `execv ... failed(1)` 表示程序以错误码 1 退出,本身不代表进程崩溃。遇到控制失效时,对照 transport.log 中的控制开关和连接状态;`mouse_reports` 为 0 表示终端没有向 CLI 提供可识别的鼠标事件。报告数增长但 `mouse_forwarded` 为 0 时,检查是否仍为只读/本地命令模式、是否已有画面,以及点击是否落在视频区域;`mouse_send_failures` 增长时检查连接状态。
- 如果整个 macOS Terminal 窗口退出且 CLI 记录 `output closed`,还应检查系统 DiagnosticReports 中的 Terminal 报告。已观察到 Terminal 2.15 在创建标签页并调整窗口尺寸时发生 `TTLogicalScreen expandUnicharOffsetCacheToLine:` 行缓存越界;这与 CLI 的 `connection_failed` 不同。临时可在独立窗口运行,避开该标签页操作,不能据此认为 Terminal 缺陷已修复。
- `doctor --json` 是环境报告,不会联网登录、主动探测 Kitty 或证明远端可连接。 `input_shortcuts=local_commands_v2` 表示支持本文的持续本地命令模式;启动日志同时记录实际执行路径,以便区分不同工程的构建产物。
- 画面比例异常时查看 `client.log` 摘要的 `terminal_geometry`:`cell_aspect` 为实际采用的字符宽高比,`reported` 表示收到有效的终端尺寸报告,`manual_aspect` 表示使用了手动覆盖。`doctor` 中的比例和像素尺寸只反映系统 TTY 信息,不主动查询终端,可能与交互连接自动检测后的值不同。
第一轮无音频播放或音频订阅 API,SSH 服务器上的声音不会传回本地。无后台守护进程、共享媒体 Broker、IP 直连、MCP 或自动截图。其他宿主可以使用 [私有桥接](bridge-protocol.md),没有可靠自定义 UI 时打开独立终端或给出播放器命令。
+15
View File
@@ -58,6 +58,21 @@
- macOS arm64 的 CLI debug 构建和 `doctor --json` 通过。生产 CLI 配合工程外假传输的 POSIX PTY 检查在 120/50 列窗口通过,验证逐字节鼠标输入、`Mouse RX`/`M:` 状态栏计数、日志统计、鼠标报告模式启停和原始终端模式恢复。
- 所有额外测试代码、脚本和产物均位于工程外临时目录。上述结果未覆盖真实 SSH 链路、GNOME Terminal 或 Linux 被控端实机,不能据此确认用户现场的问题已消失。
## 字符宽高比自动检测(2026-10-30)
- macOS arm64 CLI debug 构建通过。工程外 ASan/UBSan 检查覆盖字符/文本区尺寸报告、回复顺序与优先级、跨查询部分回复隔离、逐字节输入、本地命令/粘贴隔离、非法尺寸,以及 ANSI half/quadrant/ASCII 的画面比例与鼠标映射。
- 以 9×20 像素字符、120×40 字符视口和 1920×1080 源图模拟非 0.5 字符比例,旧默认值产生约 8%~9.3% 的宽高比误差;应用实际字符尺寸后为 1.25%,剩余为该网格的整数采样舍入。
- 生产 CLI 配合工程外假传输的 POSIX PTY 检查通过,覆盖 GNOME 风格的文本区像素/行列回复、直接字符尺寸、字体变化、窗口缩放、无回复/部分回复/非法回复、TTY 回退、手动覆盖、auto 和 Kitty。字体变化时没有新视频帧也能重绘,退出后终端模式恢复。
- 新增测试及产物均在工程外临时目录;尚未在真实 Linux GNOME Terminal 或 SSH 链路进行视觉验收。
## Windows 与 Pi/bridge 字符比例补全(2026-10-30)
- macOS arm64 CLI debug 构建通过;Windows x64 的主程序、平台 I/O、输入解析和尺寸解析通过交叉编译,平台 I/O 独立链接通过(含 user32/advapi32),不代表 Windows MiniRTC 完整构建或实机运行。
- 工程外 Win32 API 模拟检查直接使用生产尺寸读取函数,覆盖可见控制台字体尺寸、隐藏 ConPTY 回退、失败/非法字体尺寸与句柄释放,ASan/UBSan 通过。既有尺寸解析 ASan/UBSan 和 POSIX PTY 回归再次通过。
- 生产 bridge 配合工程外假传输验证可选比例字段保留、有限数值及范围检查、无效请求原子性、实际布局比例、静止画面重绘与帧比例元数据。
- Pi 1.4.0 SDK 严格类型检查及模拟宿主运行通过:使用真实 SDK 分片组包,覆盖文本区成对回复、字符尺寸优先级、宿主缓存、字体/窗口变化、手动覆盖、旧比例帧丢弃及 ACK、控制键回归、启动失败/异常退出后的监听与定时器清理。
- 新增验证代码和产物全部位于工程外系统临时目录;尚未在真实 Windows、GNOME/SSH 或 Pi TUI 进行视觉验收。
## 实测边界
本地检查不涵盖 Linux / Windows 完整构建、跨平台远程控制、Kitty/SSH 实际终端显示、真实 Pi TUI 或长时间资源占用。五个平台 CI 配置与 npm 包名均不代表已发布或已验收。
+72 -10
View File
@@ -2,6 +2,7 @@
#include <algorithm>
#include <cerrno>
#include <clocale>
#include <cmath>
#include <cstdlib>
#include <iostream>
#include <stdexcept>
@@ -95,7 +96,8 @@ struct Tty {
Write(out, end.data(), end.size());
}
};
Viewport Size(int *status_rows = nullptr) {
Viewport Size(int *status_rows = nullptr,
const TerminalMetrics *metrics = nullptr, double cell_aspect = 0) {
auto ws = GetTerminalSize(0);
const int reserved_rows = std::clamp(int(ws.rows) - 1, 0, 3);
if (status_rows)
@@ -112,6 +114,16 @@ Viewport Size(int *status_rows = nullptr) {
v.cell_aspect = std::clamp(double(ws.pixel_width) * ws.rows /
(double(ws.pixel_height) * ws.columns),
.2, 2.0);
// A visible Windows console's font metrics are authoritative. ConPTY has
// no local font geometry, so use the display terminal's replies there.
#ifdef _WIN32
if (metrics && (!ws.pixel_width || !ws.pixel_height))
#else
if (metrics)
#endif
metrics->Apply(v);
if (cell_aspect > 0)
v.cell_aspect = cell_aspect;
auto locale = Env("LC_ALL");
if (locale.empty())
locale = Env("LC_CTYPE");
@@ -172,6 +184,17 @@ bool Boolean(const json &j, const char *key, bool fallback) {
throw std::invalid_argument(std::string(key) + " must be boolean");
return j[key];
}
double Real(const json &j, const char *key, double lo, double hi,
double fallback) {
if (!j.contains(key))
return fallback;
if (!j[key].is_number())
throw std::invalid_argument(std::string(key) + " must be a number");
const double value = j[key].get<double>();
if (!std::isfinite(value) || value < lo || value > hi)
throw std::invalid_argument(std::string(key) + " out of range");
return value;
}
void Quality(SessionOptions &options, const std::string &name) {
if (name == "low") {
options.quality = 0;
@@ -219,7 +242,7 @@ std::string WaitingScreen(const Viewport &v, const std::string &title,
}
int View(SessionOptions options, const std::string &device,
const std::string &renderer, bool control, int fd, AnsiDetail detail,
double zoom, bool native) {
double zoom, bool native, double cell_aspect) {
if (!IsTty(0) || !IsTty(fd) || Env("TERM") == "dumb")
throw std::runtime_error("connecting needs an interactive ANSI terminal; use "
"bridge --stdio for a host");
@@ -257,7 +280,8 @@ int View(SessionOptions options, const std::string &device,
std::optional<Grid> displayed, pending;
bool pending_frame = false;
int last_status_rows = 3;
Viewport last_view = Size(&last_status_rows);
Viewport last_view =
Size(&last_status_rows, &input.terminal_metrics, cell_aspect);
std::optional<DecodedFrame> last_frame;
uint64_t view_revision = input.view_revision;
bool redraw = true;
@@ -267,6 +291,7 @@ int View(SessionOptions options, const std::string &device,
bool waiting_visible = false;
double queue_ms = 0;
auto next = Clock::now(), status_tick = next;
auto size_query_tick = next;
const auto pump_output = [&] {
if (output.Pump() && pending_frame) {
displayed = std::move(pending);
@@ -300,7 +325,7 @@ int View(SessionOptions options, const std::string &device,
break;
auto now = Clock::now();
int status_rows;
auto v = Size(&status_rows);
auto v = Size(&status_rows, &input.terminal_metrics, cell_aspect);
v.detail = detail;
v.native = input.native;
v.pixel_mode = kitty;
@@ -311,8 +336,10 @@ int View(SessionOptions options, const std::string &device,
view_revision = input.view_revision;
redraw = true;
}
const bool grid_resized =
v.columns != last_view.columns || v.rows != last_view.rows;
if (output.Idle() &&
(v.columns != last_view.columns || v.rows != last_view.rows ||
(grid_resized || v.cell_aspect != last_view.cell_aspect ||
v.pixel_width != last_view.pixel_width ||
v.pixel_height != last_view.pixel_height ||
status_rows != last_status_rows)) {
@@ -322,6 +349,15 @@ int View(SessionOptions options, const std::string &device,
last_status_rows = status_rows;
redraw = true;
status_tick = now;
if (grid_resized)
size_query_tick = now;
}
// Query through the terminal byte stream so this also works over SSH.
// Never wait for a reply or interrupt a partially written video frame.
if (output.Idle() && now >= size_query_tick) {
input.terminal_metrics.BeginQuery();
output.Put(kTerminalSizeQuery);
size_query_tick = now + std::chrono::seconds(2);
}
if (output.Idle() && now >= status_tick) {
auto st = s.Status();
@@ -449,6 +485,12 @@ int View(SessionOptions options, const std::string &device,
{"mouse_legacy_reports", input.mouse_legacy_reports},
{"mouse_forwarded", input.mouse_forwarded},
{"mouse_send_failures", input.mouse_send_failures},
{"terminal_geometry",
{{"cell_aspect", last_view.cell_aspect},
{"pixel_width", last_view.pixel_width},
{"pixel_height", last_view.pixel_height},
{"reported", input.terminal_metrics.Known()},
{"manual_aspect", cell_aspect > 0}}},
{"submitted", submitted},
{"output_bytes", output.bytes},
{"last_queue_ms", queue_ms},
@@ -468,15 +510,16 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
Output output(fd);
ControllerSession s;
Input input;
Viewport viewport;
viewport.detail = AnsiDetail::Quadrant;
auto status = [&] {
auto value = s.Status();
value["command_mode"] = input.command_mode;
value["resume_control"] = input.control;
value["exit_requested"] = input.exit;
value["cell_aspect"] = viewport.cell_aspect;
return value;
};
Viewport viewport;
viewport.detail = AnsiDetail::Quadrant;
NonBlocking input_mode(0);
std::deque<json> replies;
std::string buffer;
@@ -548,6 +591,8 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
next_view.colors != 256)
throw std::invalid_argument("unsupported_colors");
next_view.utf8 = Boolean(j, "utf8", true);
next_view.cell_aspect =
Real(j, "cell_aspect", .2, 2, viewport.cell_aspect);
if (j.contains("detail"))
next_view.detail = Detail(j.at("detail").get<std::string>());
next_options.fps = Integer(j, "fps", 1, 30, 8);
@@ -687,6 +732,7 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
{"generation", last_frame->generation},
{"columns", viewport.columns},
{"rows", viewport.rows},
{"cell_aspect", viewport.cell_aspect},
{"source_width", last_frame->width},
{"source_height", last_frame->height},
{"zoom", viewport.zoom},
@@ -718,6 +764,7 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
const char *help = R"(crossdesk-cli <9-digit-device-id> [--renderer auto|ansi|kitty] [--fps 1..30]
[--display 0..63] [--control] [--server host] [--port 9099]
[--quality low|balanced|high|ultra] [--ansi-detail quadrant|half] [--zoom 1..32|native]
[--cell-aspect auto|0.2..2]
crossdesk-cli view <9-digit-device-id> [options] (compatibility alias)
crossdesk-cli doctor [--json]
crossdesk-cli bridge --stdio [--password-fd 3] [--server host] [--port 9099]
@@ -729,6 +776,7 @@ View keys: click/d native detail, wheel/+/- zoom, arrows/ijkl pan (i up, j left,
Local mode stays active after commands, paste or focus loss; remote input remains paused.
Ctrl+T and Ctrl+C pass through in control mode; another Ctrl+] leaves local mode and forwards Ctrl+] if control is enabled.
Default ANSI detail: quadrant. Default quality: high (1920x1080); ultra requests up to 3840x2160.
Cell aspect is character width/height: auto detects terminal geometry; use --cell-aspect to calibrate.
Display index is zero-based. ANSI cannot reproduce full desktop text at fit scale.
--fps controls local output; negotiated previews use supported 1/8/15 FPS.
bridge: private NDJSON v1, credentials on a separate inherited descriptor.
@@ -758,6 +806,9 @@ int main(int argc, char **argv) {
{"transport_diagnostics", true},
{"utf8", v.utf8},
{"colors", v.colors},
{"cell_aspect", v.cell_aspect},
{"pixel_width", v.pixel_width},
{"pixel_height", v.pixel_height},
{"multiplexer", !Env("TMUX").empty() || !Env("STY").empty()},
{"software_decode", true},
{"kitty", "probe_on_view"},
@@ -777,7 +828,7 @@ int main(int argc, char **argv) {
SessionOptions options;
std::string renderer = "auto", device;
AnsiDetail detail = AnsiDetail::Quadrant;
double zoom = 1;
double zoom = 1, cell_aspect = 0;
bool control = false, stdio = false, native = false;
int secret_fd = 3;
if (command == "view") {
@@ -819,7 +870,17 @@ int main(int argc, char **argv) {
Quality(options, value);
else if (a == "--ansi-detail" && command == "view")
detail = Detail(value);
else if (a == "--zoom" && command == "view") {
else if (a == "--cell-aspect" && command == "view") {
if (value == "auto") {
cell_aspect = 0;
} else {
size_t used = 0;
cell_aspect = std::stod(value, &used);
if (used != value.size() || !std::isfinite(cell_aspect) ||
cell_aspect < .2 || cell_aspect > 2)
throw std::invalid_argument("cell aspect must be auto or 0.2..2");
}
} else if (a == "--zoom" && command == "view") {
native = value == "native";
zoom = native ? 1 : number(value, 1, 32);
} else if (a == "--password-fd" && command == "bridge")
@@ -854,7 +915,8 @@ int main(int argc, char **argv) {
<< "\n";
try {
return command == "view" ? View(options, device, renderer, control,
output.value, detail, zoom, native)
output.value, detail, zoom, native,
cell_aspect)
: Bridge(output.value, options, secret_fd);
} catch (const std::exception &e) {
if (command == "view") {
+73 -5
View File
@@ -1,9 +1,14 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { truncateToWidth, parseKey, decodeKittyPrintable, isKeyRelease } from "@earendil-works/pi-tui";
import { truncateToWidth, parseKey, decodeKittyPrintable, isKeyRelease, getCellDimensions } from "@earendil-works/pi-tui";
import { spawn, type ChildProcess } from "node:child_process";
import type { Writable } from "node:stream";
const clean = (s: unknown) => String(s ?? "").replace(/[^\x20-\x7e]/g, "?").slice(0, 180);
const validAspect = (n: number) => Number.isFinite(n) && n >= 0.2 && n <= 2;
function aspectOf(width: number, height: number): number | undefined {
const ratio = width / height;
return width >= 1 && width <= 1024 && height >= 1 && height <= 1024 && validAspect(ratio) ? ratio : undefined;
}
// Normalize enhanced key reports with the host's actual keyboard decoder.
function legacyInput(data: string): string {
@@ -29,6 +34,12 @@ export default function crossdesk(pi: ExtensionAPI) {
return;
}
if (stop) { ctx.ui.notify("A CrossDesk view is already open", "warning"); return; }
const configuredAspect = process.env.CROSSDESK_CELL_ASPECT?.trim();
const manualAspect = !configuredAspect || configuredAspect === "auto" ? undefined : Number(configuredAspect);
if (manualAspect !== undefined && !validAspect(manualAspect)) {
ctx.ui.notify("CROSSDESK_CELL_ASPECT must be auto or a width/height ratio from 0.2 to 2", "warning");
return;
}
const device = (args.trim() || await ctx.ui.input("CrossDesk device ID", "9 digits"))?.trim();
if (!device) return;
if (!/^\d{9}$/.test(device)) { ctx.ui.notify("Expected a 9-digit device ID", "error"); return; }
@@ -56,7 +67,9 @@ export default function crossdesk(pi: ExtensionAPI) {
if (password === undefined) return;
let child: ChildProcess | undefined;
let killTimer: ReturnType<typeof setTimeout> | undefined;
let stopGeometry: (() => void) | undefined;
const cleanup = () => {
stopGeometry?.(); stopGeometry = undefined;
if (!child) return;
const owned = child; child = undefined;
owned.stdin?.end(); // EOF asks the bridge to disconnect and release input.
@@ -75,6 +88,8 @@ export default function crossdesk(pi: ExtensionAPI) {
let frame: { frame_id: number; generation: number } | undefined;
let helloId = 0, zoom = 1, native = false;
let control = false, commandMode = false, bridgeStopped = false;
const hostCell = getCellDimensions();
let cellAspect = manualAspect ?? aspectOf(hostCell.widthPx, hostCell.heightPx) ?? 0.5;
const colorHint = process.env.COLORTERM || "";
const colors = /^(truecolor|24bit)$/.test(colorHint) ? 24 : process.env.TERM?.includes("256color") ? 256 : 8;
const finish = () => { if (closed) return; closed = true; cleanup(); stop = undefined; lines = []; done(); };
@@ -90,13 +105,21 @@ export default function crossdesk(pi: ExtensionAPI) {
const id = ++request;
child.stdin.write(JSON.stringify({ id, method, ...fields }) + "\n"); return id;
};
const sendViewport = () => send("viewport", {
columns: columns || 80, rows: rows || 20, cell_aspect: cellAspect,
colors, utf8: true, fps: 8, detail: "quadrant", quality: "high",
});
child.stdin?.on("error", () => fail("Bridge input closed"));
const secrets = child.stdio[3] as Writable;
secrets.on("error", () => fail("Credential channel closed"));
secrets.end(password + "\n"); password = undefined;
child.stderr?.resume(); // Never insert native diagnostics in the TUI or chat.
child.on("error", () => fail("Cannot start crossdesk-cli; check CROSSDESK_CLI or PATH"));
child.on("exit", (code) => { if (!closed) { bridgeStopped = true; error ||= `Bridge exited (${code ?? "signal"})`; tui.requestRender(); } });
child.on("exit", (code) => {
if (child === processChild) child = undefined;
stopGeometry?.(); stopGeometry = undefined;
if (!closed) { bridgeStopped = true; error ||= `Bridge exited (${code ?? "signal"})`; tui.requestRender(); }
});
child.stdout?.setEncoding("utf8");
child.stdout?.on("data", (data: string) => {
if (closed) return;
@@ -109,7 +132,7 @@ export default function crossdesk(pi: ExtensionAPI) {
const message = JSON.parse(raw);
if (message.id === helloId && message.ok && message.version === 1 && !ready) {
ready = true;
send("viewport", { columns: columns || 80, rows: rows || 20, colors, utf8: true, fps: 8, detail: "quadrant", quality: "high" });
sendViewport();
send("connect", { device });
}
if (message.ok === false) error = clean(message.error);
@@ -128,7 +151,8 @@ export default function crossdesk(pi: ExtensionAPI) {
if (!Number.isSafeInteger(message.frame_id) || !Number.isSafeInteger(message.generation) ||
!Array.isArray(message.lines) || message.lines.length > 100)
throw new Error("Invalid frame");
if (message.columns !== columns || message.rows !== rows || message.generation < generation) {
if (message.columns !== columns || message.rows !== rows || message.generation < generation ||
(message.cell_aspect !== undefined && message.cell_aspect !== cellAspect)) {
send("ack", {frame_id: message.frame_id, generation: message.generation});
continue; // A frame already in flight may precede the resize response.
}
@@ -148,13 +172,57 @@ export default function crossdesk(pi: ExtensionAPI) {
helloId = send("hello", { version: 1 });
const watchdog = setTimeout(() => { if (!ready) fail("Bridge handshake timeout"); }, 5000);
processChild.once("exit", () => clearTimeout(watchdog));
// Use the host's terminal writer and sequence listener; Pi remains
// the sole owner of raw input and screen rendering.
let pixelSize: [number, number] | undefined, cellCount: [number, number] | undefined;
let directCell = false, queried = false;
const updateAspect = (width: number, height: number) => {
const ratio = aspectOf(width, height);
if (manualAspect !== undefined || ratio === undefined || ratio === cellAspect) return;
cellAspect = ratio; lines = []; frame = undefined;
if (ready) sendViewport();
tui.requestRender();
};
const removeGeometryListener = tui.addInputListener(data => {
// Pi's StdinBuffer assembles split replies before listeners run.
const match = /^\x1b\[([468]);([0-9]+);([0-9]+)t$/.exec(data);
if (!queried || !match) return;
const type = Number(match[1]), height = Number(match[2]), width = Number(match[3]);
if (![height, width].every(n => Number.isSafeInteger(n) && n > 0 && n <= 1048576))
return { consume: true };
if (type === 6) {
if (aspectOf(width, height) === undefined) return { consume: true };
directCell = true;
updateAspect(width, height);
return; // Let Pi update its own shared cell-size cache as well.
}
if (type === 4) pixelSize = [width, height];
if (type === 8) cellCount = [width, height];
if (!directCell && pixelSize && cellCount)
updateAspect(pixelSize[0] / cellCount[0], pixelSize[1] / cellCount[1]);
return { consume: true };
});
let terminalColumns = tui.terminal.columns, terminalRows = tui.terminal.rows;
const queryGeometry = () => {
if (closed || bridgeStopped) return;
queried = true; directCell = false; pixelSize = cellCount = undefined;
tui.terminal.write("\x1b[16t\x1b[14t\x1b[18t");
};
const geometryTimer = setInterval(queryGeometry, 2000);
geometryTimer.unref();
stopGeometry = () => { clearInterval(geometryTimer); removeGeometryListener(); };
queryGeometry();
return {
render(width: number) {
const w = Math.max(1, Math.min(240, width - 1));
const h = Math.max(1, Math.min(100, tui.terminal.rows - 5));
if (w !== columns || h !== rows) {
columns = w; rows = h; lines = []; frame = undefined;
if (ready) send("viewport", { columns, rows, colors, utf8: true, fps: 8, detail: "quadrant", quality: "high" });
if (ready) sendViewport();
}
if (terminalColumns !== tui.terminal.columns || terminalRows !== tui.terminal.rows) {
terminalColumns = tui.terminal.columns; terminalRows = tui.terminal.rows;
queryGeometry();
}
const ack = frame; frame = undefined;
if (ack) queueMicrotask(() => send("ack", ack)); // Host consumed the frame; not a physical display timestamp.
+13
View File
@@ -312,6 +312,19 @@ TerminalSize GetTerminalSize(int) {
if (GetConsoleScreenBufferInfo(handle, &info)) {
size.columns = info.srWindow.Right - info.srWindow.Left + 1;
size.rows = info.srWindow.Bottom - info.srWindow.Top + 1;
// ConPTY's console window is only a message window, and its font API
// describes the hidden console host, not the user's display terminal.
// Only trust native font metrics for a visible classic console.
HWND window = GetConsoleWindow();
CONSOLE_FONT_INFOEX font{};
font.cbSize = sizeof(font);
if (window && IsWindowVisible(window) &&
GetCurrentConsoleFontEx(handle, FALSE, &font) &&
font.dwFontSize.X > 0 && font.dwFontSize.X <= 1024 &&
font.dwFontSize.Y > 0 && font.dwFontSize.Y <= 1024) {
size.pixel_width = size.columns * font.dwFontSize.X;
size.pixel_height = size.rows * font.dwFontSize.Y;
}
}
if (console_output < 0 && handle != INVALID_HANDLE_VALUE)
CloseHandle(handle);
+1 -1
View File
@@ -42,7 +42,7 @@ target("crossdesk-cli")
add_files("main.cpp")
add_files(is_plat("windows") and "terminal_io_windows.cpp" or "terminal_io_posix.cpp")
if is_plat("windows") then
add_syslinks("advapi32")
add_syslinks("advapi32", "user32")
add_defines("NOMINMAX", "WIN32_LEAN_AND_MEAN")
end
target_end()
+17
View File
@@ -21,6 +21,22 @@ struct Viewport {
bool native = false, pixel_mode = false;
int row_offset = 0; // Terminal rows reserved above the video area.
};
// Keep reported cell geometry across queries; never combine partial replies
// from different query cycles. Pixel/cell window sizes use the full text area.
class TerminalMetrics {
public:
void BeginQuery();
void Report(const std::string &sequence);
void Apply(Viewport &) const;
bool Known() const { return cell_width_ > 0; }
private:
bool SetCellSize(double width, double height);
double cell_width_ = 0, cell_height_ = 0;
int pixel_width_ = 0, pixel_height_ = 0, columns_ = 0, rows_ = 0;
bool queried_ = false, direct_reply_ = false;
};
inline constexpr char kTerminalSizeQuery[] = "\033[16t\033[14t\033[18t";
struct Rect {
int x, y, width, height;
};
@@ -72,6 +88,7 @@ public:
uint64_t mouse_reports = 0;
uint64_t mouse_legacy_reports = 0, mouse_forwarded = 0;
uint64_t mouse_send_failures = 0;
TerminalMetrics terminal_metrics;
private:
std::string pending_;
+4
View File
@@ -238,6 +238,10 @@ void Input::Feed(const std::string &bytes, ControllerSession &s,
return;
std::string seq = pending_.substr(0, end + 1);
pending_.erase(0, end + 1);
if (seq.back() == 't') {
terminal_metrics.Report(seq);
continue;
}
if (seq == "\033[O") {
control = false;
s.SetControl(false);
+67
View File
@@ -0,0 +1,67 @@
#include "terminal_renderer.h"
#include <charconv>
#include <cmath>
namespace crossdesk::terminal {
void TerminalMetrics::BeginQuery() {
queried_ = true;
direct_reply_ = false;
pixel_width_ = pixel_height_ = columns_ = rows_ = 0;
}
bool TerminalMetrics::SetCellSize(double width, double height) {
if (width < 1 || height < 1 || width > 1024 || height > 1024 ||
width / height < .2 || width / height > 2)
return false;
cell_width_ = width;
cell_height_ = height;
return true;
}
void TerminalMetrics::Report(const std::string &sequence) {
if (!queried_ || sequence.size() < 9 || sequence.compare(0, 2, "\033[") ||
sequence.back() != 't')
return;
int type = 0, height = 0, width = 0;
const char *cur = sequence.data() + 2;
const char *end = sequence.data() + sequence.size() - 1;
for (int *value : {&type, &height, &width}) {
auto result = std::from_chars(cur, end, *value);
if (result.ec != std::errc{} || *value <= 0 || *value > 1048576)
return;
cur = result.ptr;
if (value != &width) {
if (cur == end || *cur++ != ';')
return;
} else if (cur != end) {
return;
}
}
if (type == 6) {
// CSI 16 t replies with character height, then width (not width/height).
if (SetCellSize(width, height))
direct_reply_ = true;
} else if (type == 4) {
pixel_width_ = width;
pixel_height_ = height;
} else if (type == 8) {
columns_ = width;
rows_ = height;
} else {
return;
}
// VTE supports text-area pixels/cells even when CSI 16 t is unavailable.
// Use reported full rows/columns, not the capped video viewport or status
// rows, and retain fractional cell dimensions for scaled terminal fonts.
if (!direct_reply_ && pixel_width_ && pixel_height_ && columns_ && rows_)
SetCellSize(double(pixel_width_) / columns_, double(pixel_height_) / rows_);
}
void TerminalMetrics::Apply(Viewport &viewport) const {
if (!Known())
return;
viewport.cell_aspect = cell_width_ / cell_height_;
viewport.pixel_width = int(std::lround(cell_width_ * viewport.columns));
viewport.pixel_height = int(std::lround(cell_height_ * viewport.rows));
}
} // namespace crossdesk::terminal