mirror of
https://github.com/kunkundi/crossdesk.git
synced 2026-10-30 18:39:22 +08:00
[fix] preserve CLI display aspect ratio across terminals
This commit is contained in:
@@ -8,7 +8,7 @@
|
||||
| method | 字段 | 行为 |
|
||||
|---|---|---|
|
||||
| hello | version:1 | 协议版本握手 |
|
||||
| viewport | columns:1..240, rows:1..100, colors:8/24/256, utf8:boolean, fps:1..30;可选 detail:quadrant/half, quality:low/balanced/high/ultra, zoom:1..32, native:boolean | 设置画面行数(不含状态栏)、本地目标与预览档位;默认 quadrant/high/1、native=false,保留未显式指定的 detail/quality/zoom/native;不探测宿主 TTY |
|
||||
| viewport | columns:1..240, rows:1..100, colors:8/24/256, utf8:boolean, fps:1..30;可选 cell_aspect:0.2..2, detail:quadrant/half, quality:low/balanced/high/ultra, zoom:1..32, native:boolean | 设置画面行数(不含状态栏)、字符宽高比、本地目标与预览档位;默认 cell_aspect=0.5、quadrant/high/1、native=false,保留未显式指定的 cell_aspect/detail/quality/zoom/native;不探测宿主 TTY |
|
||||
| connect | device:9位数字字符串, display:0..63(默认0) | 从专用 password fd 读取至换行/EOF(最多256字节、等待5秒),开始异步注册/连接;ok不是远端已连接 |
|
||||
| status | 无 | 返回当前状态 |
|
||||
| control | enabled:boolean | 显式启停控制;关闭时释放输入 |
|
||||
@@ -23,10 +23,12 @@
|
||||
帧事件:
|
||||
|
||||
```json
|
||||
{"event":"frame","frame_id":42,"decoded_frame_id":38,"generation":4,"source_width":1280,"source_height":720,"zoom":2,"native":false,"source_rect":[320,180,640,360],"detail":"quadrant","columns":80,"rows":23,"lines":["..."],"queue_ms":3.2}
|
||||
{"event":"frame","frame_id":42,"decoded_frame_id":38,"generation":4,"source_width":1280,"source_height":720,"zoom":2,"native":false,"source_rect":[320,180,640,360],"detail":"quadrant","columns":80,"rows":23,"cell_aspect":0.45,"lines":["..."],"queue_ms":3.2}
|
||||
```
|
||||
|
||||
实际 lines 数为 rows,仅包含本地生成的 SGR 颜色序列、空格和 U+2580~U+259F 块字符,不含远端名称或任意终端指令。`frame_id` 是进程递增的输出序号,静止画面重绘仍产生新序号;`decoded_frame_id` 标识源解码帧,同源重绘可以相同。`source_width/height` 为实际解码尺寸,source_rect 为实际源裁剪矩形 [x,y,width,height]。native=true 表示原始细节模式,zoom 为当前模式下的倍率;generation 在关闭、重新连接和显示器切换时变化。viewport 变化可能与已写出的旧尺寸帧交错,宿主应丢弃不匹配尺寸/旧 generation 的帧并释放对应 ACK。宿主返回的 ACK 表示消费/布局,不表示屏幕物理显示完成。
|
||||
实际 lines 数为 rows,仅包含本地生成的 SGR 颜色序列、空格和 U+2580~U+259F 块字符,不含远端名称或任意终端指令。`frame_id` 是进程递增的输出序号,静止画面重绘仍产生新序号;`decoded_frame_id` 标识源解码帧,同源重绘可以相同。`source_width/height` 为实际解码尺寸,source_rect 为实际源裁剪矩形 [x,y,width,height]。native=true 表示原始细节模式,zoom 为当前模式下的倍率;generation 在关闭、重新连接和显示器切换时变化。viewport 变化可能与已写出的旧尺寸帧交错,宿主应丢弃不匹配行列数/cell_aspect 或旧 generation 的帧并释放对应 ACK。宿主返回的 ACK 表示消费/布局,不表示屏幕物理显示完成。
|
||||
|
||||
`cell_aspect` 为单个字符的宽/高,必须是 0.2~2 范围内的有限 JSON 数值;例如 9×20 像素字符应传 0.45。宿主负责测量或提供手动设置,bridge 根据该比例计算图像布局与鼠标映射,响应及状态事件中的 `status.cell_aspect` 返回当前值。该字段为 v1 的可选扩展;省略时保留上次值,首次默认 0.5。旧版 bridge 的帧可能不含此字段。修改比例也会触发静止画面重绘,新的鼠标映射仍在帧 ACK 后生效。
|
||||
|
||||
状态事件每秒最多一次:`{"event":"state","status":{...},"output_bytes":123,"submitted_frames":3,"elapsed_seconds":1.5}`。status 含 registering/connecting/connected/failed/closed/idle、client_id(当前或刚结束会话的 `cli-xxxxxxxxx`,注册前为空)、error、control、generation、显示器、文本/预览能力、预览确认状态、请求的 preview_width/height/quality/bitrate_bps、帧计数与网络统计。接收/提交 FPS 由相应计数的增量除以时间增量计算。
|
||||
|
||||
|
||||
@@ -45,7 +45,13 @@ crossdesk-cli 123456789 --server your-server.example --port 9099
|
||||
|
||||
ANSI 默认使用四象限字符:每个字符从 2×2 样本选择误差最小的两个颜色和象限形状,比原先上下半方块多一倍水平采样点;每个字符仍只有前景/背景两色。使用 `--ansi-detail half` 可回到上下半方块以兼容字体。缩小时采用面积加权滤波,避免最近邻采样漏掉细线。支持 TrueColor、256 色和基础 8 色,256 色选择实际颜色立方体或灰阶条中更接近的颜色。环境的 COLORTERM/TERM 决定颜色模式;非 UTF-8 locale 使用背景色空格。TERM=dumb、非 TTY 输出明确拒绝交互连接,不输出全屏转义序列。
|
||||
|
||||
画面按字符单元比例等比缩放,留黑边及状态栏,默认假定字符宽高比 0.5;TTY 提供像素尺寸时使用其比例。每次只输出变化行,合并连续颜色设置;缩放在完整输出边界清理旧画面。上限 240 列、100 画面行;默认顶部预留一行状态栏、底部预留两行快捷键,另预留一列避免自动换行。顶部状态栏用反色区分,左侧为控制模式和连接进度,右侧为画面参数及网络信息;底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始,鼠标坐标按视频区域映射,点击状态栏不会传给远端。按键分组使用统一列宽和左右留白,窄窗口自动减少列数、合并按键组并省略次要参数,累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作,进入本地命令模式后才显示平移和缩放按键;终端高度不足时优先收起底部提示行。
|
||||
画面按字符单元比例等比缩放,留黑边及状态栏。交互连接通过终端字节流查询字符尺寸,支持通过 SSH 返回的回复:优先使用 `CSI 16 t` 的字符像素尺寸,否则组合 `CSI 14 t` 的完整文本区像素尺寸和 `CSI 18 t` 的完整行列数计算,兼容 GNOME/VTE 的文本区报告。计算后再扣除状态栏及画面行列上限,不把预留行当作图像高度。窗口改变时重新查询,每两秒刷新一次以覆盖字体/行距变化;回复到达后会重绘静止画面和更新鼠标映射。查询不阻塞连接;没有有效回复时使用系统提供的像素尺寸,再缺失时回退为宽高比 0.5。曾收到有效回复时保留最近有效尺寸,直到下一次有效回复。字符栅格的整数舍入仍可能产生小幅比例误差。格式依据 [XTerm 窗口尺寸报告](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html) 和 [GNOME VTE 实现](https://github.com/GNOME/vte/blob/master/src/vteseq.cc)。
|
||||
|
||||
Windows 的可见传统控制台优先读取当前字体尺寸;Windows Terminal/ConPTY 使用显示终端的尺寸回复,不采用隐藏控制台宿主的字体值。不同终端版本可能不支持查询,或返回虚拟图形尺寸而非实际字体尺寸,此时需要手动校准。
|
||||
|
||||
默认 `--cell-aspect auto` 自动确定比例。如果终端屏蔽查询或需要手动校准,可设置 `--cell-aspect 0.45`,数值表示单个字符的**宽/高**,允许 0.2~2;例如字符为 9×20 像素时使用 0.45。画面被压窄时调小此值,被拉宽时调大。此参数优先于自动检测,同时用于全景、细节和鼠标坐标映射。
|
||||
|
||||
每次只输出变化行,合并连续颜色设置;缩放在完整输出边界清理旧画面。上限 240 列、100 画面行;默认顶部预留一行状态栏、底部预留两行快捷键,另预留一列避免自动换行。顶部状态栏用反色区分,左侧为控制模式和连接进度,右侧为画面参数及网络信息;底部两行分别为会话和视图快捷键。视频和等待提示从顶部状态栏下方开始,鼠标坐标按视频区域映射,点击状态栏不会传给远端。按键分组使用统一列宽和左右留白,窄窗口自动减少列数、合并按键组并省略次要参数,累计帧数和输出字节数保留在日志中。控制模式下仅提示本地命令入口及后续操作,进入本地命令模式后才显示平移和缩放按键;终端高度不足时优先收起底部提示行。
|
||||
|
||||
快捷键行使用 `KEYS |`、`VIEW |` 等固定宽度标题,标题和按键组之间用竖线分隔;键名用 `<c>`、`<+/->` 等标签标识。宽窗口中的分组分隔线按列对齐,窄窗口保留标题和完整按键标签,按可用宽度合并或减少分组。
|
||||
|
||||
@@ -123,6 +129,8 @@ pi -e ./apps/cli/plugins/pi/index.ts
|
||||
|
||||
在 Pi 输入 `/crossdesk` 或 `/crossdesk 123456789`。先输入 ID,再在自定义遮蔽组件输入密码。扩展使用独立 fd 3 写密码,随后关闭;不写配置、聊天、工具结果或模型上下文。查看组件与独立 CLI 使用相同的 `Ctrl+]` 命令前缀和上述模式按键,由 bridge 统一解析。插件默认请求 high 预览并显示 ANSI 象限字符行,支持同样的只读细节/放大/平移键,颜色深度依据终端环境回退,由 Pi 自身布局和刷新;不直接占用 TTY,不发送 Kitty 图形或自行打开鼠标报告。
|
||||
|
||||
Pi 扩展读取宿主的字符尺寸,并通过宿主终端写入/输入监听接口查询字符尺寸或完整文本区尺寸,每两秒及窗口变化时刷新;尺寸回复不会传给远端。得到的宽高比通过 bridge 的 `viewport.cell_aspect` 传入,比例变化时重绘静止画面,并丢弃仍在途的旧比例帧。自动检测不可用或不准确时,可在启动 Pi 前设置环境变量 `CROSSDESK_CELL_ASPECT=0.45`(宽/高,允许 0.2~2;`auto` 恢复自动检测)。视图关闭或 bridge 停止时移除查询定时器和输入监听。
|
||||
|
||||
需要全局发现时,把整个 `apps/cli/plugins/pi` 目录复制到 `~/.pi/agent/extensions/crossdesk/` 后重启 Pi 或执行 `/reload`。卸载时先关闭视图,删除该目录并 `/reload`;使用 `-e` 时从启动参数移除即可。无需修改包中的密码或设备配置。默认连接官方信令;自建信令当前通过独立 CLI 连接命令使用,插件尚无服务器设置界面。
|
||||
|
||||
仅在 `ctx.mode === "tui"`、hasUI 且进程双向 TTY 时打开组件。RPC/print/json 返回独立播放器提示,不声称内嵌视频。UI 关闭/会话退出先关闭 bridge stdin,超时 SIGTERM,再超时 SIGKILL。bridge 尚未就绪或已经停止时,组件可直接用 `q`、`Esc` 或 `Ctrl+C` 关闭。SDK 类型和模拟 UI 生命周期已验证,真实 Pi TUI 视觉刷新、增强按键与鼠标传递未实机验收。真实设备的 bridge 与独立 CLI 连接验证见 [验证记录](validation.md)。
|
||||
@@ -144,6 +152,7 @@ pi -e ./apps/cli/plugins/pi/index.ts
|
||||
- `connection_failed` 表示连接建立失败;`connection_lost` 表示连接成功后传输失效。xmake 随后的 `execv ... failed(1)` 表示程序以错误码 1 退出,本身不代表进程崩溃。遇到控制失效时,对照 transport.log 中的控制开关和连接状态;`mouse_reports` 为 0 表示终端没有向 CLI 提供可识别的鼠标事件。报告数增长但 `mouse_forwarded` 为 0 时,检查是否仍为只读/本地命令模式、是否已有画面,以及点击是否落在视频区域;`mouse_send_failures` 增长时检查连接状态。
|
||||
- 如果整个 macOS Terminal 窗口退出且 CLI 记录 `output closed`,还应检查系统 DiagnosticReports 中的 Terminal 报告。已观察到 Terminal 2.15 在创建标签页并调整窗口尺寸时发生 `TTLogicalScreen expandUnicharOffsetCacheToLine:` 行缓存越界;这与 CLI 的 `connection_failed` 不同。临时可在独立窗口运行,避开该标签页操作,不能据此认为 Terminal 缺陷已修复。
|
||||
- `doctor --json` 是环境报告,不会联网登录、主动探测 Kitty 或证明远端可连接。 `input_shortcuts=local_commands_v2` 表示支持本文的持续本地命令模式;启动日志同时记录实际执行路径,以便区分不同工程的构建产物。
|
||||
- 画面比例异常时查看 `client.log` 摘要的 `terminal_geometry`:`cell_aspect` 为实际采用的字符宽高比,`reported` 表示收到有效的终端尺寸报告,`manual_aspect` 表示使用了手动覆盖。`doctor` 中的比例和像素尺寸只反映系统 TTY 信息,不主动查询终端,可能与交互连接自动检测后的值不同。
|
||||
|
||||
第一轮无音频播放或音频订阅 API,SSH 服务器上的声音不会传回本地。无后台守护进程、共享媒体 Broker、IP 直连、MCP 或自动截图。其他宿主可以使用 [私有桥接](bridge-protocol.md),没有可靠自定义 UI 时打开独立终端或给出播放器命令。
|
||||
|
||||
|
||||
@@ -58,6 +58,21 @@
|
||||
- macOS arm64 的 CLI debug 构建和 `doctor --json` 通过。生产 CLI 配合工程外假传输的 POSIX PTY 检查在 120/50 列窗口通过,验证逐字节鼠标输入、`Mouse RX`/`M:` 状态栏计数、日志统计、鼠标报告模式启停和原始终端模式恢复。
|
||||
- 所有额外测试代码、脚本和产物均位于工程外临时目录。上述结果未覆盖真实 SSH 链路、GNOME Terminal 或 Linux 被控端实机,不能据此确认用户现场的问题已消失。
|
||||
|
||||
## 字符宽高比自动检测(2026-10-30)
|
||||
|
||||
- macOS arm64 CLI debug 构建通过。工程外 ASan/UBSan 检查覆盖字符/文本区尺寸报告、回复顺序与优先级、跨查询部分回复隔离、逐字节输入、本地命令/粘贴隔离、非法尺寸,以及 ANSI half/quadrant/ASCII 的画面比例与鼠标映射。
|
||||
- 以 9×20 像素字符、120×40 字符视口和 1920×1080 源图模拟非 0.5 字符比例,旧默认值产生约 8%~9.3% 的宽高比误差;应用实际字符尺寸后为 1.25%,剩余为该网格的整数采样舍入。
|
||||
- 生产 CLI 配合工程外假传输的 POSIX PTY 检查通过,覆盖 GNOME 风格的文本区像素/行列回复、直接字符尺寸、字体变化、窗口缩放、无回复/部分回复/非法回复、TTY 回退、手动覆盖、auto 和 Kitty。字体变化时没有新视频帧也能重绘,退出后终端模式恢复。
|
||||
- 新增测试及产物均在工程外临时目录;尚未在真实 Linux GNOME Terminal 或 SSH 链路进行视觉验收。
|
||||
|
||||
## Windows 与 Pi/bridge 字符比例补全(2026-10-30)
|
||||
|
||||
- macOS arm64 CLI debug 构建通过;Windows x64 的主程序、平台 I/O、输入解析和尺寸解析通过交叉编译,平台 I/O 独立链接通过(含 user32/advapi32),不代表 Windows MiniRTC 完整构建或实机运行。
|
||||
- 工程外 Win32 API 模拟检查直接使用生产尺寸读取函数,覆盖可见控制台字体尺寸、隐藏 ConPTY 回退、失败/非法字体尺寸与句柄释放,ASan/UBSan 通过。既有尺寸解析 ASan/UBSan 和 POSIX PTY 回归再次通过。
|
||||
- 生产 bridge 配合工程外假传输验证可选比例字段保留、有限数值及范围检查、无效请求原子性、实际布局比例、静止画面重绘与帧比例元数据。
|
||||
- Pi 1.4.0 SDK 严格类型检查及模拟宿主运行通过:使用真实 SDK 分片组包,覆盖文本区成对回复、字符尺寸优先级、宿主缓存、字体/窗口变化、手动覆盖、旧比例帧丢弃及 ACK、控制键回归、启动失败/异常退出后的监听与定时器清理。
|
||||
- 新增验证代码和产物全部位于工程外系统临时目录;尚未在真实 Windows、GNOME/SSH 或 Pi TUI 进行视觉验收。
|
||||
|
||||
## 实测边界
|
||||
|
||||
本地检查不涵盖 Linux / Windows 完整构建、跨平台远程控制、Kitty/SSH 实际终端显示、真实 Pi TUI 或长时间资源占用。五个平台 CI 配置与 npm 包名均不代表已发布或已验收。
|
||||
|
||||
+72
-10
@@ -2,6 +2,7 @@
|
||||
#include <algorithm>
|
||||
#include <cerrno>
|
||||
#include <clocale>
|
||||
#include <cmath>
|
||||
#include <cstdlib>
|
||||
#include <iostream>
|
||||
#include <stdexcept>
|
||||
@@ -95,7 +96,8 @@ struct Tty {
|
||||
Write(out, end.data(), end.size());
|
||||
}
|
||||
};
|
||||
Viewport Size(int *status_rows = nullptr) {
|
||||
Viewport Size(int *status_rows = nullptr,
|
||||
const TerminalMetrics *metrics = nullptr, double cell_aspect = 0) {
|
||||
auto ws = GetTerminalSize(0);
|
||||
const int reserved_rows = std::clamp(int(ws.rows) - 1, 0, 3);
|
||||
if (status_rows)
|
||||
@@ -112,6 +114,16 @@ Viewport Size(int *status_rows = nullptr) {
|
||||
v.cell_aspect = std::clamp(double(ws.pixel_width) * ws.rows /
|
||||
(double(ws.pixel_height) * ws.columns),
|
||||
.2, 2.0);
|
||||
// A visible Windows console's font metrics are authoritative. ConPTY has
|
||||
// no local font geometry, so use the display terminal's replies there.
|
||||
#ifdef _WIN32
|
||||
if (metrics && (!ws.pixel_width || !ws.pixel_height))
|
||||
#else
|
||||
if (metrics)
|
||||
#endif
|
||||
metrics->Apply(v);
|
||||
if (cell_aspect > 0)
|
||||
v.cell_aspect = cell_aspect;
|
||||
auto locale = Env("LC_ALL");
|
||||
if (locale.empty())
|
||||
locale = Env("LC_CTYPE");
|
||||
@@ -172,6 +184,17 @@ bool Boolean(const json &j, const char *key, bool fallback) {
|
||||
throw std::invalid_argument(std::string(key) + " must be boolean");
|
||||
return j[key];
|
||||
}
|
||||
double Real(const json &j, const char *key, double lo, double hi,
|
||||
double fallback) {
|
||||
if (!j.contains(key))
|
||||
return fallback;
|
||||
if (!j[key].is_number())
|
||||
throw std::invalid_argument(std::string(key) + " must be a number");
|
||||
const double value = j[key].get<double>();
|
||||
if (!std::isfinite(value) || value < lo || value > hi)
|
||||
throw std::invalid_argument(std::string(key) + " out of range");
|
||||
return value;
|
||||
}
|
||||
void Quality(SessionOptions &options, const std::string &name) {
|
||||
if (name == "low") {
|
||||
options.quality = 0;
|
||||
@@ -219,7 +242,7 @@ std::string WaitingScreen(const Viewport &v, const std::string &title,
|
||||
}
|
||||
int View(SessionOptions options, const std::string &device,
|
||||
const std::string &renderer, bool control, int fd, AnsiDetail detail,
|
||||
double zoom, bool native) {
|
||||
double zoom, bool native, double cell_aspect) {
|
||||
if (!IsTty(0) || !IsTty(fd) || Env("TERM") == "dumb")
|
||||
throw std::runtime_error("connecting needs an interactive ANSI terminal; use "
|
||||
"bridge --stdio for a host");
|
||||
@@ -257,7 +280,8 @@ int View(SessionOptions options, const std::string &device,
|
||||
std::optional<Grid> displayed, pending;
|
||||
bool pending_frame = false;
|
||||
int last_status_rows = 3;
|
||||
Viewport last_view = Size(&last_status_rows);
|
||||
Viewport last_view =
|
||||
Size(&last_status_rows, &input.terminal_metrics, cell_aspect);
|
||||
std::optional<DecodedFrame> last_frame;
|
||||
uint64_t view_revision = input.view_revision;
|
||||
bool redraw = true;
|
||||
@@ -267,6 +291,7 @@ int View(SessionOptions options, const std::string &device,
|
||||
bool waiting_visible = false;
|
||||
double queue_ms = 0;
|
||||
auto next = Clock::now(), status_tick = next;
|
||||
auto size_query_tick = next;
|
||||
const auto pump_output = [&] {
|
||||
if (output.Pump() && pending_frame) {
|
||||
displayed = std::move(pending);
|
||||
@@ -300,7 +325,7 @@ int View(SessionOptions options, const std::string &device,
|
||||
break;
|
||||
auto now = Clock::now();
|
||||
int status_rows;
|
||||
auto v = Size(&status_rows);
|
||||
auto v = Size(&status_rows, &input.terminal_metrics, cell_aspect);
|
||||
v.detail = detail;
|
||||
v.native = input.native;
|
||||
v.pixel_mode = kitty;
|
||||
@@ -311,8 +336,10 @@ int View(SessionOptions options, const std::string &device,
|
||||
view_revision = input.view_revision;
|
||||
redraw = true;
|
||||
}
|
||||
const bool grid_resized =
|
||||
v.columns != last_view.columns || v.rows != last_view.rows;
|
||||
if (output.Idle() &&
|
||||
(v.columns != last_view.columns || v.rows != last_view.rows ||
|
||||
(grid_resized || v.cell_aspect != last_view.cell_aspect ||
|
||||
v.pixel_width != last_view.pixel_width ||
|
||||
v.pixel_height != last_view.pixel_height ||
|
||||
status_rows != last_status_rows)) {
|
||||
@@ -322,6 +349,15 @@ int View(SessionOptions options, const std::string &device,
|
||||
last_status_rows = status_rows;
|
||||
redraw = true;
|
||||
status_tick = now;
|
||||
if (grid_resized)
|
||||
size_query_tick = now;
|
||||
}
|
||||
// Query through the terminal byte stream so this also works over SSH.
|
||||
// Never wait for a reply or interrupt a partially written video frame.
|
||||
if (output.Idle() && now >= size_query_tick) {
|
||||
input.terminal_metrics.BeginQuery();
|
||||
output.Put(kTerminalSizeQuery);
|
||||
size_query_tick = now + std::chrono::seconds(2);
|
||||
}
|
||||
if (output.Idle() && now >= status_tick) {
|
||||
auto st = s.Status();
|
||||
@@ -449,6 +485,12 @@ int View(SessionOptions options, const std::string &device,
|
||||
{"mouse_legacy_reports", input.mouse_legacy_reports},
|
||||
{"mouse_forwarded", input.mouse_forwarded},
|
||||
{"mouse_send_failures", input.mouse_send_failures},
|
||||
{"terminal_geometry",
|
||||
{{"cell_aspect", last_view.cell_aspect},
|
||||
{"pixel_width", last_view.pixel_width},
|
||||
{"pixel_height", last_view.pixel_height},
|
||||
{"reported", input.terminal_metrics.Known()},
|
||||
{"manual_aspect", cell_aspect > 0}}},
|
||||
{"submitted", submitted},
|
||||
{"output_bytes", output.bytes},
|
||||
{"last_queue_ms", queue_ms},
|
||||
@@ -468,15 +510,16 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
|
||||
Output output(fd);
|
||||
ControllerSession s;
|
||||
Input input;
|
||||
Viewport viewport;
|
||||
viewport.detail = AnsiDetail::Quadrant;
|
||||
auto status = [&] {
|
||||
auto value = s.Status();
|
||||
value["command_mode"] = input.command_mode;
|
||||
value["resume_control"] = input.control;
|
||||
value["exit_requested"] = input.exit;
|
||||
value["cell_aspect"] = viewport.cell_aspect;
|
||||
return value;
|
||||
};
|
||||
Viewport viewport;
|
||||
viewport.detail = AnsiDetail::Quadrant;
|
||||
NonBlocking input_mode(0);
|
||||
std::deque<json> replies;
|
||||
std::string buffer;
|
||||
@@ -548,6 +591,8 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
|
||||
next_view.colors != 256)
|
||||
throw std::invalid_argument("unsupported_colors");
|
||||
next_view.utf8 = Boolean(j, "utf8", true);
|
||||
next_view.cell_aspect =
|
||||
Real(j, "cell_aspect", .2, 2, viewport.cell_aspect);
|
||||
if (j.contains("detail"))
|
||||
next_view.detail = Detail(j.at("detail").get<std::string>());
|
||||
next_options.fps = Integer(j, "fps", 1, 30, 8);
|
||||
@@ -687,6 +732,7 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
|
||||
{"generation", last_frame->generation},
|
||||
{"columns", viewport.columns},
|
||||
{"rows", viewport.rows},
|
||||
{"cell_aspect", viewport.cell_aspect},
|
||||
{"source_width", last_frame->width},
|
||||
{"source_height", last_frame->height},
|
||||
{"zoom", viewport.zoom},
|
||||
@@ -718,6 +764,7 @@ int Bridge(int fd, SessionOptions options, int password_fd) {
|
||||
const char *help = R"(crossdesk-cli <9-digit-device-id> [--renderer auto|ansi|kitty] [--fps 1..30]
|
||||
[--display 0..63] [--control] [--server host] [--port 9099]
|
||||
[--quality low|balanced|high|ultra] [--ansi-detail quadrant|half] [--zoom 1..32|native]
|
||||
[--cell-aspect auto|0.2..2]
|
||||
crossdesk-cli view <9-digit-device-id> [options] (compatibility alias)
|
||||
crossdesk-cli doctor [--json]
|
||||
crossdesk-cli bridge --stdio [--password-fd 3] [--server host] [--port 9099]
|
||||
@@ -729,6 +776,7 @@ View keys: click/d native detail, wheel/+/- zoom, arrows/ijkl pan (i up, j left,
|
||||
Local mode stays active after commands, paste or focus loss; remote input remains paused.
|
||||
Ctrl+T and Ctrl+C pass through in control mode; another Ctrl+] leaves local mode and forwards Ctrl+] if control is enabled.
|
||||
Default ANSI detail: quadrant. Default quality: high (1920x1080); ultra requests up to 3840x2160.
|
||||
Cell aspect is character width/height: auto detects terminal geometry; use --cell-aspect to calibrate.
|
||||
Display index is zero-based. ANSI cannot reproduce full desktop text at fit scale.
|
||||
--fps controls local output; negotiated previews use supported 1/8/15 FPS.
|
||||
bridge: private NDJSON v1, credentials on a separate inherited descriptor.
|
||||
@@ -758,6 +806,9 @@ int main(int argc, char **argv) {
|
||||
{"transport_diagnostics", true},
|
||||
{"utf8", v.utf8},
|
||||
{"colors", v.colors},
|
||||
{"cell_aspect", v.cell_aspect},
|
||||
{"pixel_width", v.pixel_width},
|
||||
{"pixel_height", v.pixel_height},
|
||||
{"multiplexer", !Env("TMUX").empty() || !Env("STY").empty()},
|
||||
{"software_decode", true},
|
||||
{"kitty", "probe_on_view"},
|
||||
@@ -777,7 +828,7 @@ int main(int argc, char **argv) {
|
||||
SessionOptions options;
|
||||
std::string renderer = "auto", device;
|
||||
AnsiDetail detail = AnsiDetail::Quadrant;
|
||||
double zoom = 1;
|
||||
double zoom = 1, cell_aspect = 0;
|
||||
bool control = false, stdio = false, native = false;
|
||||
int secret_fd = 3;
|
||||
if (command == "view") {
|
||||
@@ -819,7 +870,17 @@ int main(int argc, char **argv) {
|
||||
Quality(options, value);
|
||||
else if (a == "--ansi-detail" && command == "view")
|
||||
detail = Detail(value);
|
||||
else if (a == "--zoom" && command == "view") {
|
||||
else if (a == "--cell-aspect" && command == "view") {
|
||||
if (value == "auto") {
|
||||
cell_aspect = 0;
|
||||
} else {
|
||||
size_t used = 0;
|
||||
cell_aspect = std::stod(value, &used);
|
||||
if (used != value.size() || !std::isfinite(cell_aspect) ||
|
||||
cell_aspect < .2 || cell_aspect > 2)
|
||||
throw std::invalid_argument("cell aspect must be auto or 0.2..2");
|
||||
}
|
||||
} else if (a == "--zoom" && command == "view") {
|
||||
native = value == "native";
|
||||
zoom = native ? 1 : number(value, 1, 32);
|
||||
} else if (a == "--password-fd" && command == "bridge")
|
||||
@@ -854,7 +915,8 @@ int main(int argc, char **argv) {
|
||||
<< "\n";
|
||||
try {
|
||||
return command == "view" ? View(options, device, renderer, control,
|
||||
output.value, detail, zoom, native)
|
||||
output.value, detail, zoom, native,
|
||||
cell_aspect)
|
||||
: Bridge(output.value, options, secret_fd);
|
||||
} catch (const std::exception &e) {
|
||||
if (command == "view") {
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||
import { truncateToWidth, parseKey, decodeKittyPrintable, isKeyRelease } from "@earendil-works/pi-tui";
|
||||
import { truncateToWidth, parseKey, decodeKittyPrintable, isKeyRelease, getCellDimensions } from "@earendil-works/pi-tui";
|
||||
import { spawn, type ChildProcess } from "node:child_process";
|
||||
import type { Writable } from "node:stream";
|
||||
|
||||
const clean = (s: unknown) => String(s ?? "").replace(/[^\x20-\x7e]/g, "?").slice(0, 180);
|
||||
const validAspect = (n: number) => Number.isFinite(n) && n >= 0.2 && n <= 2;
|
||||
function aspectOf(width: number, height: number): number | undefined {
|
||||
const ratio = width / height;
|
||||
return width >= 1 && width <= 1024 && height >= 1 && height <= 1024 && validAspect(ratio) ? ratio : undefined;
|
||||
}
|
||||
|
||||
// Normalize enhanced key reports with the host's actual keyboard decoder.
|
||||
function legacyInput(data: string): string {
|
||||
@@ -29,6 +34,12 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
return;
|
||||
}
|
||||
if (stop) { ctx.ui.notify("A CrossDesk view is already open", "warning"); return; }
|
||||
const configuredAspect = process.env.CROSSDESK_CELL_ASPECT?.trim();
|
||||
const manualAspect = !configuredAspect || configuredAspect === "auto" ? undefined : Number(configuredAspect);
|
||||
if (manualAspect !== undefined && !validAspect(manualAspect)) {
|
||||
ctx.ui.notify("CROSSDESK_CELL_ASPECT must be auto or a width/height ratio from 0.2 to 2", "warning");
|
||||
return;
|
||||
}
|
||||
const device = (args.trim() || await ctx.ui.input("CrossDesk device ID", "9 digits"))?.trim();
|
||||
if (!device) return;
|
||||
if (!/^\d{9}$/.test(device)) { ctx.ui.notify("Expected a 9-digit device ID", "error"); return; }
|
||||
@@ -56,7 +67,9 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
if (password === undefined) return;
|
||||
let child: ChildProcess | undefined;
|
||||
let killTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let stopGeometry: (() => void) | undefined;
|
||||
const cleanup = () => {
|
||||
stopGeometry?.(); stopGeometry = undefined;
|
||||
if (!child) return;
|
||||
const owned = child; child = undefined;
|
||||
owned.stdin?.end(); // EOF asks the bridge to disconnect and release input.
|
||||
@@ -75,6 +88,8 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
let frame: { frame_id: number; generation: number } | undefined;
|
||||
let helloId = 0, zoom = 1, native = false;
|
||||
let control = false, commandMode = false, bridgeStopped = false;
|
||||
const hostCell = getCellDimensions();
|
||||
let cellAspect = manualAspect ?? aspectOf(hostCell.widthPx, hostCell.heightPx) ?? 0.5;
|
||||
const colorHint = process.env.COLORTERM || "";
|
||||
const colors = /^(truecolor|24bit)$/.test(colorHint) ? 24 : process.env.TERM?.includes("256color") ? 256 : 8;
|
||||
const finish = () => { if (closed) return; closed = true; cleanup(); stop = undefined; lines = []; done(); };
|
||||
@@ -90,13 +105,21 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
const id = ++request;
|
||||
child.stdin.write(JSON.stringify({ id, method, ...fields }) + "\n"); return id;
|
||||
};
|
||||
const sendViewport = () => send("viewport", {
|
||||
columns: columns || 80, rows: rows || 20, cell_aspect: cellAspect,
|
||||
colors, utf8: true, fps: 8, detail: "quadrant", quality: "high",
|
||||
});
|
||||
child.stdin?.on("error", () => fail("Bridge input closed"));
|
||||
const secrets = child.stdio[3] as Writable;
|
||||
secrets.on("error", () => fail("Credential channel closed"));
|
||||
secrets.end(password + "\n"); password = undefined;
|
||||
child.stderr?.resume(); // Never insert native diagnostics in the TUI or chat.
|
||||
child.on("error", () => fail("Cannot start crossdesk-cli; check CROSSDESK_CLI or PATH"));
|
||||
child.on("exit", (code) => { if (!closed) { bridgeStopped = true; error ||= `Bridge exited (${code ?? "signal"})`; tui.requestRender(); } });
|
||||
child.on("exit", (code) => {
|
||||
if (child === processChild) child = undefined;
|
||||
stopGeometry?.(); stopGeometry = undefined;
|
||||
if (!closed) { bridgeStopped = true; error ||= `Bridge exited (${code ?? "signal"})`; tui.requestRender(); }
|
||||
});
|
||||
child.stdout?.setEncoding("utf8");
|
||||
child.stdout?.on("data", (data: string) => {
|
||||
if (closed) return;
|
||||
@@ -109,7 +132,7 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
const message = JSON.parse(raw);
|
||||
if (message.id === helloId && message.ok && message.version === 1 && !ready) {
|
||||
ready = true;
|
||||
send("viewport", { columns: columns || 80, rows: rows || 20, colors, utf8: true, fps: 8, detail: "quadrant", quality: "high" });
|
||||
sendViewport();
|
||||
send("connect", { device });
|
||||
}
|
||||
if (message.ok === false) error = clean(message.error);
|
||||
@@ -128,7 +151,8 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
if (!Number.isSafeInteger(message.frame_id) || !Number.isSafeInteger(message.generation) ||
|
||||
!Array.isArray(message.lines) || message.lines.length > 100)
|
||||
throw new Error("Invalid frame");
|
||||
if (message.columns !== columns || message.rows !== rows || message.generation < generation) {
|
||||
if (message.columns !== columns || message.rows !== rows || message.generation < generation ||
|
||||
(message.cell_aspect !== undefined && message.cell_aspect !== cellAspect)) {
|
||||
send("ack", {frame_id: message.frame_id, generation: message.generation});
|
||||
continue; // A frame already in flight may precede the resize response.
|
||||
}
|
||||
@@ -148,13 +172,57 @@ export default function crossdesk(pi: ExtensionAPI) {
|
||||
helloId = send("hello", { version: 1 });
|
||||
const watchdog = setTimeout(() => { if (!ready) fail("Bridge handshake timeout"); }, 5000);
|
||||
processChild.once("exit", () => clearTimeout(watchdog));
|
||||
// Use the host's terminal writer and sequence listener; Pi remains
|
||||
// the sole owner of raw input and screen rendering.
|
||||
let pixelSize: [number, number] | undefined, cellCount: [number, number] | undefined;
|
||||
let directCell = false, queried = false;
|
||||
const updateAspect = (width: number, height: number) => {
|
||||
const ratio = aspectOf(width, height);
|
||||
if (manualAspect !== undefined || ratio === undefined || ratio === cellAspect) return;
|
||||
cellAspect = ratio; lines = []; frame = undefined;
|
||||
if (ready) sendViewport();
|
||||
tui.requestRender();
|
||||
};
|
||||
const removeGeometryListener = tui.addInputListener(data => {
|
||||
// Pi's StdinBuffer assembles split replies before listeners run.
|
||||
const match = /^\x1b\[([468]);([0-9]+);([0-9]+)t$/.exec(data);
|
||||
if (!queried || !match) return;
|
||||
const type = Number(match[1]), height = Number(match[2]), width = Number(match[3]);
|
||||
if (![height, width].every(n => Number.isSafeInteger(n) && n > 0 && n <= 1048576))
|
||||
return { consume: true };
|
||||
if (type === 6) {
|
||||
if (aspectOf(width, height) === undefined) return { consume: true };
|
||||
directCell = true;
|
||||
updateAspect(width, height);
|
||||
return; // Let Pi update its own shared cell-size cache as well.
|
||||
}
|
||||
if (type === 4) pixelSize = [width, height];
|
||||
if (type === 8) cellCount = [width, height];
|
||||
if (!directCell && pixelSize && cellCount)
|
||||
updateAspect(pixelSize[0] / cellCount[0], pixelSize[1] / cellCount[1]);
|
||||
return { consume: true };
|
||||
});
|
||||
let terminalColumns = tui.terminal.columns, terminalRows = tui.terminal.rows;
|
||||
const queryGeometry = () => {
|
||||
if (closed || bridgeStopped) return;
|
||||
queried = true; directCell = false; pixelSize = cellCount = undefined;
|
||||
tui.terminal.write("\x1b[16t\x1b[14t\x1b[18t");
|
||||
};
|
||||
const geometryTimer = setInterval(queryGeometry, 2000);
|
||||
geometryTimer.unref();
|
||||
stopGeometry = () => { clearInterval(geometryTimer); removeGeometryListener(); };
|
||||
queryGeometry();
|
||||
return {
|
||||
render(width: number) {
|
||||
const w = Math.max(1, Math.min(240, width - 1));
|
||||
const h = Math.max(1, Math.min(100, tui.terminal.rows - 5));
|
||||
if (w !== columns || h !== rows) {
|
||||
columns = w; rows = h; lines = []; frame = undefined;
|
||||
if (ready) send("viewport", { columns, rows, colors, utf8: true, fps: 8, detail: "quadrant", quality: "high" });
|
||||
if (ready) sendViewport();
|
||||
}
|
||||
if (terminalColumns !== tui.terminal.columns || terminalRows !== tui.terminal.rows) {
|
||||
terminalColumns = tui.terminal.columns; terminalRows = tui.terminal.rows;
|
||||
queryGeometry();
|
||||
}
|
||||
const ack = frame; frame = undefined;
|
||||
if (ack) queueMicrotask(() => send("ack", ack)); // Host consumed the frame; not a physical display timestamp.
|
||||
|
||||
@@ -312,6 +312,19 @@ TerminalSize GetTerminalSize(int) {
|
||||
if (GetConsoleScreenBufferInfo(handle, &info)) {
|
||||
size.columns = info.srWindow.Right - info.srWindow.Left + 1;
|
||||
size.rows = info.srWindow.Bottom - info.srWindow.Top + 1;
|
||||
// ConPTY's console window is only a message window, and its font API
|
||||
// describes the hidden console host, not the user's display terminal.
|
||||
// Only trust native font metrics for a visible classic console.
|
||||
HWND window = GetConsoleWindow();
|
||||
CONSOLE_FONT_INFOEX font{};
|
||||
font.cbSize = sizeof(font);
|
||||
if (window && IsWindowVisible(window) &&
|
||||
GetCurrentConsoleFontEx(handle, FALSE, &font) &&
|
||||
font.dwFontSize.X > 0 && font.dwFontSize.X <= 1024 &&
|
||||
font.dwFontSize.Y > 0 && font.dwFontSize.Y <= 1024) {
|
||||
size.pixel_width = size.columns * font.dwFontSize.X;
|
||||
size.pixel_height = size.rows * font.dwFontSize.Y;
|
||||
}
|
||||
}
|
||||
if (console_output < 0 && handle != INVALID_HANDLE_VALUE)
|
||||
CloseHandle(handle);
|
||||
|
||||
+1
-1
@@ -42,7 +42,7 @@ target("crossdesk-cli")
|
||||
add_files("main.cpp")
|
||||
add_files(is_plat("windows") and "terminal_io_windows.cpp" or "terminal_io_posix.cpp")
|
||||
if is_plat("windows") then
|
||||
add_syslinks("advapi32")
|
||||
add_syslinks("advapi32", "user32")
|
||||
add_defines("NOMINMAX", "WIN32_LEAN_AND_MEAN")
|
||||
end
|
||||
target_end()
|
||||
|
||||
@@ -21,6 +21,22 @@ struct Viewport {
|
||||
bool native = false, pixel_mode = false;
|
||||
int row_offset = 0; // Terminal rows reserved above the video area.
|
||||
};
|
||||
// Keep reported cell geometry across queries; never combine partial replies
|
||||
// from different query cycles. Pixel/cell window sizes use the full text area.
|
||||
class TerminalMetrics {
|
||||
public:
|
||||
void BeginQuery();
|
||||
void Report(const std::string &sequence);
|
||||
void Apply(Viewport &) const;
|
||||
bool Known() const { return cell_width_ > 0; }
|
||||
|
||||
private:
|
||||
bool SetCellSize(double width, double height);
|
||||
double cell_width_ = 0, cell_height_ = 0;
|
||||
int pixel_width_ = 0, pixel_height_ = 0, columns_ = 0, rows_ = 0;
|
||||
bool queried_ = false, direct_reply_ = false;
|
||||
};
|
||||
inline constexpr char kTerminalSizeQuery[] = "\033[16t\033[14t\033[18t";
|
||||
struct Rect {
|
||||
int x, y, width, height;
|
||||
};
|
||||
@@ -72,6 +88,7 @@ public:
|
||||
uint64_t mouse_reports = 0;
|
||||
uint64_t mouse_legacy_reports = 0, mouse_forwarded = 0;
|
||||
uint64_t mouse_send_failures = 0;
|
||||
TerminalMetrics terminal_metrics;
|
||||
|
||||
private:
|
||||
std::string pending_;
|
||||
|
||||
@@ -238,6 +238,10 @@ void Input::Feed(const std::string &bytes, ControllerSession &s,
|
||||
return;
|
||||
std::string seq = pending_.substr(0, end + 1);
|
||||
pending_.erase(0, end + 1);
|
||||
if (seq.back() == 't') {
|
||||
terminal_metrics.Report(seq);
|
||||
continue;
|
||||
}
|
||||
if (seq == "\033[O") {
|
||||
control = false;
|
||||
s.SetControl(false);
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
#include "terminal_renderer.h"
|
||||
#include <charconv>
|
||||
#include <cmath>
|
||||
|
||||
namespace crossdesk::terminal {
|
||||
void TerminalMetrics::BeginQuery() {
|
||||
queried_ = true;
|
||||
direct_reply_ = false;
|
||||
pixel_width_ = pixel_height_ = columns_ = rows_ = 0;
|
||||
}
|
||||
|
||||
bool TerminalMetrics::SetCellSize(double width, double height) {
|
||||
if (width < 1 || height < 1 || width > 1024 || height > 1024 ||
|
||||
width / height < .2 || width / height > 2)
|
||||
return false;
|
||||
cell_width_ = width;
|
||||
cell_height_ = height;
|
||||
return true;
|
||||
}
|
||||
|
||||
void TerminalMetrics::Report(const std::string &sequence) {
|
||||
if (!queried_ || sequence.size() < 9 || sequence.compare(0, 2, "\033[") ||
|
||||
sequence.back() != 't')
|
||||
return;
|
||||
int type = 0, height = 0, width = 0;
|
||||
const char *cur = sequence.data() + 2;
|
||||
const char *end = sequence.data() + sequence.size() - 1;
|
||||
for (int *value : {&type, &height, &width}) {
|
||||
auto result = std::from_chars(cur, end, *value);
|
||||
if (result.ec != std::errc{} || *value <= 0 || *value > 1048576)
|
||||
return;
|
||||
cur = result.ptr;
|
||||
if (value != &width) {
|
||||
if (cur == end || *cur++ != ';')
|
||||
return;
|
||||
} else if (cur != end) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (type == 6) {
|
||||
// CSI 16 t replies with character height, then width (not width/height).
|
||||
if (SetCellSize(width, height))
|
||||
direct_reply_ = true;
|
||||
} else if (type == 4) {
|
||||
pixel_width_ = width;
|
||||
pixel_height_ = height;
|
||||
} else if (type == 8) {
|
||||
columns_ = width;
|
||||
rows_ = height;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
// VTE supports text-area pixels/cells even when CSI 16 t is unavailable.
|
||||
// Use reported full rows/columns, not the capped video viewport or status
|
||||
// rows, and retain fractional cell dimensions for scaled terminal fonts.
|
||||
if (!direct_reply_ && pixel_width_ && pixel_height_ && columns_ && rows_)
|
||||
SetCellSize(double(pixel_width_) / columns_, double(pixel_height_) / rows_);
|
||||
}
|
||||
|
||||
void TerminalMetrics::Apply(Viewport &viewport) const {
|
||||
if (!Known())
|
||||
return;
|
||||
viewport.cell_aspect = cell_width_ / cell_height_;
|
||||
viewport.pixel_width = int(std::lround(cell_width_ * viewport.columns));
|
||||
viewport.pixel_height = int(std::lround(cell_height_ * viewport.rows));
|
||||
}
|
||||
} // namespace crossdesk::terminal
|
||||
Reference in New Issue
Block a user