Compare commits

...
48 Commits
Author SHA1 Message Date
dijunkun 8ea4668305 [refactor] modularize Slint video rendering backends 2026-08-25 11:45:38 +08:00
dijunkun 21eec76008 [feat] add OpenGL NV12 underlay rendering for Slint on Linux 2026-08-25 10:26:05 +08:00
dijunkun 3e63675787 [fix] update minirtc NVDEC resolution handling 2026-08-25 05:28:39 +08:00
dijunkun 439f137764 [fix] update minirtc signal join sequencing 2026-08-25 04:40:37 +08:00
dijunkun f537147f54 [fix] update minirtc ICE state handling 2026-08-25 04:28:34 +08:00
dijunkun 2186ebd7eb [fix] restore Linux GCC build compatibility 2026-08-25 04:05:11 +08:00
dijunkun c11d48fc1f [fix] correct controlled-side status window behavior 2026-08-25 03:54:51 +08:00
dijunkun 1690fc92a3 [fix] prevent green tint during NV12 scaling 2026-08-25 03:37:02 +08:00
dijunkun 0d9d385d10 [feat] add OpenGL NV12 underlay rendering for Slint on Windows 2026-08-25 02:16:14 +08:00
dijunkun 20f8137df2 [fix] update minirtc video bitrate serialization 2026-08-25 01:34:03 +08:00
dijunkun 29aae747c6 [feat] add native Metal video rendering for Slint on macOS 2026-08-25 00:41:18 +08:00
dijunkun 3f93dc05aa [fix] eliminate Slint UI frame rate cap on macOS and Linux 2026-08-24 16:35:09 +08:00
dijunkun 05885ecd48 [fix] eliminate 30 fps cap in remote desktop rendering pipeline 2026-08-24 16:22:32 +08:00
dijunkun ee92c4ce5f [fix] update minirtc NVDEC teardown, fixes #95 2026-08-24 14:46:58 +08:00
dijunkun d94443330c [feat] prioritize frame rate for remote desktop 2026-08-24 14:00:40 +08:00
dijunkun df72204619 [fix] update minirtc OpenH264 bitrate handling 2026-08-24 10:33:56 +08:00
dijunkun 3678271577 [feat] report client version and platform 2026-08-24 09:46:51 +08:00
dijunkun 6b5535ab3d [fix] keep display stream IDs stable after monitor changes 2026-08-24 00:32:43 +08:00
dijunkun 078bd1e520 [fix] recover interrupted password changes 2026-08-23 21:59:20 +08:00
dijunkun 718705c352 [fix] prevent NVCodec unload crashes, fixes #95 2026-08-23 21:16:41 +08:00
dijunkun 1ca5852280 [fix] update minirtc reliable transmission 2026-08-23 19:04:43 +08:00
dijunkun 1eda6cc183 [fix] update minirtc feedback handling 2026-08-21 16:19:49 +08:00
dijunkun 8d37f8f5fb [fix] update minirtc CBR encoder configuration 2026-08-21 15:51:20 +08:00
dijunkun 05b8aa629e [fix] update minirtc for Windows build 2026-08-21 14:36:21 +08:00
dijunkun 2cbceba953 [fix] use screen content defaults for desktop streaming 2026-08-21 14:29:44 +08:00
dijunkun 89d3c8b53f [fix] prevent expired probe cluster reuse 2026-08-20 18:30:04 +08:00
dijunkun 51a4190e97 [fix] propagate encoder bitrate update failures 2026-08-20 18:20:02 +08:00
dijunkun 3737cd0dd5 [fix] align encoder and network bitrate limits 2026-08-20 18:05:45 +08:00
dijunkun 00957d6740 [fix] serialize pacer tasks and harden probing 2026-08-20 17:42:50 +08:00
dijunkun 6ec86322b2 [fix] abort probing when ICE becomes unavailable 2026-08-20 17:12:45 +08:00
dijunkun ecc131b5ee [fix] gate media-less probing by transport capability 2026-08-20 16:56:49 +08:00
dijunkun 1aa50e9ffa [fix] schedule pacer with high precision tasks 2026-08-20 16:22:01 +08:00
dijunkun d49f5bc520 [fix] handle task queue wakeups correctly 2026-08-20 14:45:18 +08:00
dijunkun 8818b9b8be [fix] ensure video bitrate updates reach encoders 2026-08-19 16:33:47 +08:00
dijunkun 3be283238c [fix] prevent screen capture without connected peers 2026-08-19 15:51:37 +08:00
dijunkun 5d0453fff3 [fix] prevent ICE callback self-join during teardown 2026-08-18 17:10:03 +08:00
dijunkun 8a9775a571 [fix] adapt VideoToolbox encoder resolution 2026-08-18 16:42:42 +08:00
dijunkun 7f4f1882ae [fix] sync signal status after server switch 2026-08-17 00:07:01 +08:00
dijunkun 23e24e3d47 [fix] prevent websocket reconnect during shutdown 2026-08-16 23:07:02 +08:00
dijunkun 7e1b9cb00e [fix] use -ld_classic only on macOS Intel 2026-08-16 22:34:58 +08:00
dijunkun 0c70acbdbf [fix] eliminate macOS build warnings 2026-08-16 22:18:30 +08:00
dijunkun fcbbb0f51c [fix] check log path before logger initialization 2026-08-16 22:16:26 +08:00
dijunkun f4be0700bf [fix] remove deprecated -ld_classic linker flag 2026-08-16 22:05:22 +08:00
dijunkun a45670b04d [fix] prevent invalid file descriptors during ICE teardown 2026-08-16 21:54:02 +08:00
dijunkun ff329cae81 [fix] use libsrtp v2.7.0 release tag 2026-08-16 17:51:08 +08:00
dijunkun 092664784e [fix] synchronize authenticated password changes 2026-08-16 16:57:36 +08:00
dijunkun 1709eb0efb [feat] use server-issued TURN credentials 2026-08-16 16:41:26 +08:00
dijunkun d27bb5d7a4 [docs] add CrossDesk privacy policy 2026-08-08 03:26:02 +08:00
59 changed files with 5612 additions and 842 deletions
+1 -1
View File
@@ -260,7 +260,7 @@ jobs:
- name: Build CrossDesk
run: |
xmake f --target_minver=${MACOSX_DEPLOYMENT_TARGET} --CROSSDESK_VERSION=${VERSION_NUM} --USE_CUDA=true -y
xmake f --target_minver=${MACOSX_DEPLOYMENT_TARGET} --CROSSDESK_VERSION=${VERSION_NUM} --USE_CUDA=false -y
xmake b -vy crossdesk
- name: Package CrossDesk app
+99
View File
@@ -0,0 +1,99 @@
# CrossDesk Privacy Policy
Last updated: August 8, 2026
[中文](#中文) | [English](#english)
## 中文
### 适用范围
本隐私政策适用于 CrossDesk 桌面客户端及项目提供的默认网络服务。用户自行部署或指定的服务器由相应服务器运营者负责,其数据处理方式不在本政策控制范围内。
### 网络通信
CrossDesk 是远程桌面软件,需要通过网络完成设备发现、连接协商及远程会话。客户端可能进行以下网络通信:
- **默认服务:** 未启用自托管时,客户端会连接预配置的 CrossDesk 服务(`api.crossdesk.cn`),用于设备注册、信令、网络地址转换(NAT)穿透,并在无法建立直接连接时提供中继。
- **版本检查:** 客户端启动时会从 CrossDesk 版本服务(`version.crossdesk.cn`)获取最新版本信息。当前即使启用自托管,版本检查仍会访问 CrossDesk 版本服务。
- **远程会话:** 用户主动发起或接受远程连接后,CrossDesk 会根据使用的功能在设备之间传输屏幕画面、音频、键盘和鼠标输入、剪贴板文本以及用户选择传输的文件。剪贴板文本可能在远程会话期间自动同步。这些数据可能通过直接连接传输,也可能在必要时通过配置的中继服务器转发。
- **自托管:** 用户可以在设置中指定自己的信令及中继服务器。自托管服务器的日志、数据保存和访问规则由其运营者决定。
### 处理的信息
为提供上述功能,CrossDesk 可能处理:
- 连接所需的设备标识及认证信息;
- IP 地址、端口、连接时间、连接状态等必要的网络元数据;
- 版本检查请求产生的标准网络请求信息;
- 用户授权的远程会话中处理的屏幕、音频、控制指令、剪贴板及文件内容。
### 信息的使用与保存
- 设备标识、连接设置及用户偏好会保存在本地设备上,以便后续使用。
- 通过官方信令和中继服务处理的信息仅用于建立、维持和转发 CrossDesk 连接,以及服务运行、安全防护和故障排查。
- 中继传输的远程会话内容仅用于实时转发,不用于广告或用户画像。
- CrossDesk 桌面客户端不包含广告功能,也不会主动上传诊断日志、崩溃报告或用户行为分析数据。
- 自托管服务器可能保存的日志和数据由其运营者控制,请向相应运营者了解其隐私与保存政策。
### 用户选择
用户可以:
- 配置自托管服务器,以替代默认信令和中继服务;
- 决定是否发起或接受远程会话;
- 决定是否选择并传输文件;
- 结束远程会话,以停止会话数据传输;
- 删除 CrossDesk 的本地配置、缓存及日志文件。
### 政策更新与联系方式
本政策可能随着 CrossDesk 功能或网络服务的变化而更新。更新内容将在本文件中发布。如有隐私相关问题,请联系 [junkun.di@hotmail.com](mailto:junkun.di@hotmail.com)。
---
## English
### Scope
This Privacy Policy applies to the CrossDesk desktop client and the default network services provided by the project. Servers deployed or selected by users are operated by their respective administrators, and their data-processing practices are outside the control of this policy.
### Network communications
CrossDesk is remote desktop software and requires network communications for device discovery, connection negotiation, and remote sessions. The client may perform the following communications:
- **Default services:** Unless self-hosting is enabled, the client connects to preconfigured CrossDesk services (`api.crossdesk.cn`) for device registration, signaling, network address translation (NAT) traversal, and relay when a direct connection cannot be established.
- **Update checks:** On startup, the client retrieves current release information from the CrossDesk version service (`version.crossdesk.cn`). Update checks currently continue to use the CrossDesk version service even when self-hosting is enabled.
- **Remote sessions:** After a user initiates or accepts a remote connection, CrossDesk transmits the data required by enabled features between devices. This may include screen content, audio, keyboard and mouse input, clipboard text, and files selected by the user. Clipboard text may be synchronized automatically during a remote session. Data may travel over a direct connection or through the configured relay server when necessary.
- **Self-hosting:** Users may configure their own signaling and relay servers. Logging, retention, and access policies for a self-hosted server are determined by its operator.
### Information processed
To provide these functions, CrossDesk may process:
- device identifiers and authentication information required for connections;
- necessary network metadata, such as IP addresses, ports, connection times, and connection status;
- standard network request information generated by update checks; and
- screen, audio, control, clipboard, and file content processed during a remote session authorized by the user.
### Use and storage of information
- Device identifiers, connection settings, and user preferences are stored locally for subsequent use.
- Information processed by the official signaling and relay services is used only to establish, maintain, and relay CrossDesk connections and for service operation, security, and troubleshooting.
- Remote-session content passing through a relay is used only for real-time forwarding and is not used for advertising or user profiling.
- The CrossDesk desktop client does not contain advertising functionality and does not intentionally upload diagnostic logs, crash reports, or behavioral analytics.
- Logs and data that may be retained by self-hosted servers are controlled by their operators. Contact the relevant operator for its privacy and retention practices.
### User choices
Users may:
- configure a self-hosted server instead of the default signaling and relay services;
- choose whether to initiate or accept a remote session;
- choose whether to select and transfer files;
- end a remote session to stop session-data transmission; and
- remove local CrossDesk configuration, cache, and log files.
### Changes and contact
This policy may be updated as CrossDesk features or network services change. Updates will be published in this file. For privacy-related questions, contact [junkun.di@hotmail.com](mailto:junkun.di@hotmail.com).
+3 -2
View File
@@ -117,7 +117,8 @@ sudo apt-get install -y \
libxinerama-dev libxcursor-dev libxi-dev libxfixes-dev libxv-dev \
libxtst-dev libxcb-randr0-dev libxcb-xtest0-dev libxcb-xinerama0-dev \
libxcb-shape0-dev libxcb-xkb-dev libxcb-xfixes0-dev libxcb-shm0-dev \
libasound2-dev libsndio-dev libpulse-dev
libasound2-dev libsndio-dev libpulse-dev \
libgl1-mesa-dev
```
Ubuntu 20.04 自带的 CMake 3.16 不满足要求时,xmake 会自动下载并使用
@@ -430,4 +431,4 @@ CrossDesk 使用 SignPath.io 为本仓库构建的官方 Windows 发布包提供
- **Committers and reviewers(代码提交与审核):** [kunkundi](https://github.com/kunkundi)
- **Approvers(签名审批):** [kunkundi](https://github.com/kunkundi)
- **Privacy policy隐私政策** 除非用户或安装、运行 CrossDesk 的人员明确要求,否则 CrossDesk 不会向其他联网系统传输任何信息。
- **隐私政策:** [查看 CrossDesk 隐私政策](PRIVACY.md#中文)
+3 -2
View File
@@ -120,7 +120,8 @@ sudo apt-get install -y \
libxinerama-dev libxcursor-dev libxi-dev libxfixes-dev libxv-dev \
libxtst-dev libxcb-randr0-dev libxcb-xtest0-dev libxcb-xinerama0-dev \
libxcb-shape0-dev libxcb-xkb-dev libxcb-xfixes0-dev libxcb-shm0-dev \
libasound2-dev libsndio-dev libpulse-dev
libasound2-dev libsndio-dev libpulse-dev \
libgl1-mesa-dev
```
When Ubuntu 20.04's bundled CMake 3.16 is too old, xmake automatically
@@ -447,4 +448,4 @@ CrossDesk uses SignPath.io to code-sign official Windows releases built from thi
- **Committers and reviewers:** [kunkundi](https://github.com/kunkundi)
- **Approvers:** [kunkundi](https://github.com/kunkundi)
- **Privacy policy:** CrossDesk will not transfer any information to other networked systems unless specifically requested by the user or the person installing or operating it.
- **Privacy policy:** [Read the CrossDesk Privacy Policy](PRIVACY.md#english)
+1 -1
View File
@@ -160,7 +160,7 @@ Depends: libc6 (>= 2.31), libstdc++6 (>= 10), libx11-6, libxext6,
libxrender1, libxft2, libxrandr2, libxfixes3, libxcb1, libxcb-randr0,
libxcb-xtest0, libxcb-xinerama0, libxcb-shape0, libxcb-xkb1,
libxcb-xfixes0, libxv1, libxtst6, $ALSA_RUNTIME_DEP, libsndio7.0,
libxcb-shm0, libpulse0, libdrm2, libdbus-1-3
libxcb-shm0, libpulse0, libdrm2, libdbus-1-3, libgl1
Recommends: $RECOMMENDS
Priority: optional
Section: utils
+59
View File
@@ -0,0 +1,59 @@
#ifndef CROSSDESK_DISPLAY_STREAM_ID_H_
#define CROSSDESK_DISPLAY_STREAM_ID_H_
#include <cstddef>
#include <string>
namespace crossdesk {
// MiniRTC stream identifiers are protocol-facing logical identifiers. Keep
// them independent from platform display handles and user-visible names, both
// of which may change or contain duplicate/Unicode text.
inline std::string MakeDisplayStreamId(size_t display_index) {
return "Display" + std::to_string(display_index + 1);
}
inline bool IsRegisteredDisplayStreamId(const std::string& stream_id,
size_t display_count) {
if (stream_id.empty()) {
return false;
}
for (size_t index = 0; index < display_count; ++index) {
if (stream_id == MakeDisplayStreamId(index)) {
return true;
}
}
return false;
}
// Resolves a backend-reported identifier to a stream registered with MiniRTC.
// Backends should report MakeDisplayStreamId(index); the fallbacks keep older
// capture plugins and hotplug transitions safe.
inline std::string ResolveDisplayStreamId(
const char* reported_id, size_t display_count, int preferred_index = -1,
const std::string& previous_id = {}) {
const std::string candidate = reported_id ? reported_id : "";
if (IsRegisteredDisplayStreamId(candidate, display_count)) {
return candidate;
}
if (preferred_index >= 0 &&
preferred_index < static_cast<int>(display_count)) {
return MakeDisplayStreamId(static_cast<size_t>(preferred_index));
}
if (IsRegisteredDisplayStreamId(previous_id, display_count)) {
return previous_id;
}
if (display_count == 1) {
return MakeDisplayStreamId(0);
}
return {};
}
} // namespace crossdesk
#endif // CROSSDESK_DISPLAY_STREAM_ID_H_
+62
View File
@@ -0,0 +1,62 @@
#include "nv12_scaler.h"
#include <cstddef>
#include <limits>
#include "libyuv/planar_functions.h"
namespace crossdesk {
int ScaleNv12ViaI420(const uint8_t* src_y, int src_stride_y,
const uint8_t* src_uv, int src_stride_uv, int src_width,
int src_height, uint8_t* dst_y, int dst_stride_y,
uint8_t* dst_uv, int dst_stride_uv, int dst_width,
int dst_height, libyuv::FilterMode filtering,
std::vector<uint8_t>* scratch_buffer) {
if (!src_y || !src_uv || !dst_y || !dst_uv || !scratch_buffer ||
src_width <= 0 || src_height <= 0 || dst_width <= 0 || dst_height <= 0 ||
(src_width & 1) != 0 || (src_height & 1) != 0 || (dst_width & 1) != 0 ||
(dst_height & 1) != 0) {
return -1;
}
const int src_chroma_width = src_width / 2;
const int src_chroma_height = src_height / 2;
const int dst_chroma_width = dst_width / 2;
const int dst_chroma_height = dst_height / 2;
const size_t src_chroma_size =
static_cast<size_t>(src_chroma_width) * src_chroma_height;
const size_t dst_chroma_size =
static_cast<size_t>(dst_chroma_width) * dst_chroma_height;
const size_t max_size = std::numeric_limits<size_t>::max();
if (src_chroma_size > max_size / 2 || dst_chroma_size > max_size / 2) {
return -1;
}
const size_t src_planes_size = 2 * src_chroma_size;
const size_t dst_planes_size = 2 * dst_chroma_size;
if (src_planes_size > max_size - dst_planes_size) {
return -1;
}
scratch_buffer->resize(src_planes_size + dst_planes_size);
uint8_t* src_u = scratch_buffer->data();
uint8_t* src_v = src_u + src_chroma_size;
uint8_t* dst_u = src_v + src_chroma_size;
uint8_t* dst_v = dst_u + dst_chroma_size;
libyuv::SplitUVPlane(src_uv, src_stride_uv, src_u, src_chroma_width, src_v,
src_chroma_width, src_chroma_width, src_chroma_height);
const int result = libyuv::I420Scale(
src_y, src_stride_y, src_u, src_chroma_width, src_v, src_chroma_width,
src_width, src_height, dst_y, dst_stride_y, dst_u, dst_chroma_width,
dst_v, dst_chroma_width, dst_width, dst_height, filtering);
if (result != 0) {
return result;
}
libyuv::MergeUVPlane(dst_u, dst_chroma_width, dst_v, dst_chroma_width, dst_uv,
dst_stride_uv, dst_chroma_width, dst_chroma_height);
return 0;
}
} // namespace crossdesk
+23
View File
@@ -0,0 +1,23 @@
#ifndef CROSSDESK_COMMON_NV12_SCALER_H_
#define CROSSDESK_COMMON_NV12_SCALER_H_
#include <cstdint>
#include <vector>
#include "libyuv/scale.h"
namespace crossdesk {
// Scales NV12 by separating the interleaved chroma plane before filtering.
// This avoids the downward chroma rounding bias in libyuv::NV12Scale while
// preserving the requested filtering quality.
int ScaleNv12ViaI420(const uint8_t* src_y, int src_stride_y,
const uint8_t* src_uv, int src_stride_uv, int src_width,
int src_height, uint8_t* dst_y, int dst_stride_y,
uint8_t* dst_uv, int dst_stride_uv, int dst_width,
int dst_height, libyuv::FilterMode filtering,
std::vector<uint8_t>* scratch_buffer);
} // namespace crossdesk
#endif // CROSSDESK_COMMON_NV12_SCALER_H_
+7 -5
View File
@@ -29,7 +29,11 @@ int ConfigCenter::Load() {
return -1;
}
bool persist_turn_mode_migration = false;
bool persist_config_migration = false;
persist_config_migration |= ini_.Delete(section_, "video_content_type");
persist_config_migration |= ini_.Delete(section_, "screen_content");
persist_config_migration |=
ini_.Delete(section_, "enable_desktop_quality_optimization");
const long language_value =
ini_.GetLongValue(section_, "language", static_cast<long>(language_));
@@ -52,7 +56,6 @@ int ConfigCenter::Load() {
hardware_video_codec_ = ini_.GetBoolValue(section_, "hardware_video_codec",
hardware_video_codec_);
const char* turn_mode_value = ini_.GetValue(section_, "turn_mode", nullptr);
if (turn_mode_value != nullptr && strlen(turn_mode_value) > 0) {
const long parsed_turn_mode = ini_.GetLongValue(
@@ -70,7 +73,7 @@ int ConfigCenter::Load() {
turn_mode_ = legacy_enable_turn ? TURN_MODE::AUTO_UDP_TCP
: TURN_MODE::DISABLED;
ini_.SetLongValue(section_, "turn_mode", static_cast<long>(turn_mode_));
persist_turn_mode_migration = true;
persist_config_migration = true;
}
enable_srtp_ = ini_.GetBoolValue(section_, "enable_srtp", enable_srtp_);
enable_self_hosted_ =
@@ -121,8 +124,7 @@ int ConfigCenter::Load() {
file_transfer_save_path_ = "";
}
if (persist_turn_mode_migration &&
ini_.SaveFile(config_path_.c_str()) < 0) {
if (persist_config_migration && ini_.SaveFile(config_path_.c_str()) < 0) {
return -1;
}
File diff suppressed because it is too large Load Diff
+7
View File
@@ -1,6 +1,7 @@
#ifndef CROSSDESK_GUI_APPLICATION_H_
#define CROSSDESK_GUI_APPLICATION_H_
#include <chrono>
#include <memory>
#include <string>
@@ -31,10 +32,15 @@ private:
void BindStreamCallbacks();
void BindServerCallbacks();
void Tick();
void HandlePasswordChangeResult();
void HandleCredentialRecovery();
void SyncMainWindow();
void SyncConnectionDialog();
void SyncPlatformDialogs();
void SyncStreamWindow();
void SyncStreamVideoFrame();
void ScheduleNextVideoFrame();
void ConfigureStreamVideoRenderer();
void SyncStreamKeyboardFocus();
void SetStreamKeyboardFocus(bool focused);
void SyncServerWindow();
@@ -64,6 +70,7 @@ private:
bool OpenUrl(const std::string &url);
std::unique_ptr<SlintUi> ui_;
std::chrono::steady_clock::time_point next_video_frame_time_{};
#if defined(__linux__) && !defined(__APPLE__)
bool use_xwayland_gui_ = false;
bool use_x11_custom_titlebar_ = false;
+28
View File
@@ -0,0 +1,28 @@
#pragma once
#include <algorithm>
#include <iterator>
#include <string>
#include <vector>
namespace crossdesk::server_window_state {
inline int ReconcileSelectedController(const std::vector<std::string>& ids,
std::string* selected_id) {
if (!selected_id) {
return 0;
}
if (ids.empty()) {
selected_id->clear();
return 0;
}
const auto selected = std::find(ids.begin(), ids.end(), *selected_id);
if (selected == ids.end()) {
*selected_id = ids.front();
return 0;
}
return static_cast<int>(std::distance(ids.begin(), selected));
}
} // namespace crossdesk::server_window_state
+40
View File
@@ -0,0 +1,40 @@
#pragma once
#include <algorithm>
namespace crossdesk::window_geometry {
struct PhysicalRect {
int x = 0;
int y = 0;
int width = 0;
int height = 0;
};
struct PhysicalSize {
int width = 0;
int height = 0;
};
struct PhysicalPosition {
int x = 0;
int y = 0;
};
constexpr PhysicalPosition CenteredPosition(const PhysicalRect& bounds,
const PhysicalSize& size) {
return {
bounds.x + (bounds.width - std::min(bounds.width, size.width)) / 2,
bounds.y + (bounds.height - std::min(bounds.height, size.height)) / 2,
};
}
constexpr PhysicalPosition BottomRightPosition(const PhysicalRect& bounds,
const PhysicalSize& size) {
return {
bounds.x + std::max(0, bounds.width - size.width),
bounds.y + std::max(0, bounds.height - size.height),
};
}
} // namespace crossdesk::window_geometry
@@ -6,6 +6,7 @@
#ifndef _LOCALIZATION_H_
#define _LOCALIZATION_H_
#include <cstddef>
#include <string>
#include <unordered_map>
#include <vector>
@@ -115,6 +116,20 @@ inline const std::string& LocalizedString::operator[](
return detail::GetTranslatedText(key_, language_index);
}
inline std::string FormatDisplayLabel(size_t display_index,
const std::string& display_name,
int language_index) {
const std::string number = std::to_string(display_index + 1);
std::string label =
detail::GetTranslatedText("display_screen", language_index) + " " +
number;
const std::string fallback_name = "Display" + number;
if (!display_name.empty() && display_name != fallback_name) {
label += " (" + display_name + ")";
}
return label;
}
#define CROSSDESK_DECLARE_LOCALIZED_STRING(name, zh, en, ru) \
inline const LocalizedString name(#name);
CROSSDESK_LOCALIZATION_ALL(CROSSDESK_DECLARE_LOCALIZED_STRING)
@@ -35,6 +35,7 @@ struct TranslationRow {
u8"Недавние подключения") \
X(disconnect, u8"断开连接", "Disconnect", u8"Отключить") \
X(select_display, u8"选择显示器", "Select Display", u8"Выбрать дисплей") \
X(display_screen, u8"显示屏", "Display", u8"Экран") \
X(expand_control_bar, u8"展开控制栏", "Expand Control Bar", \
u8"Развернуть панель управления") \
X(collapse_control_bar, u8"收起控制栏", "Collapse Control Bar", \
@@ -3,6 +3,7 @@
#include <chrono>
#include <cstdlib>
#include "display_stream_id.h"
#include "platform.h"
#include "rd_log.h"
#include "runtime/gui_runtime.h"
@@ -12,11 +13,29 @@ namespace {
constexpr uint64_t kCaptureResumeKeyFrameGapMs = 500;
constexpr size_t kMaxCapturedKeyboardInputs = 512;
constexpr auto kFrameDeadlineTolerance = std::chrono::milliseconds(1);
} // namespace
SessionDeviceManager::SessionDeviceManager(GuiRuntime &owner) : owner_(owner) {}
bool SessionDeviceManager::ShouldSendCapturedFrame(
std::chrono::steady_clock::time_point now, int fps) {
const auto interval =
std::chrono::nanoseconds(std::chrono::seconds(1)) / (fps > 0 ? fps : 1);
if (next_frame_deadline_ == std::chrono::steady_clock::time_point{}) {
next_frame_deadline_ = now;
}
if (now + kFrameDeadlineTolerance < next_frame_deadline_) {
return false;
}
next_frame_deadline_ += interval;
if (next_frame_deadline_ <= now) {
next_frame_deadline_ = now + interval;
}
return true;
}
void SessionDeviceManager::Initialize() {
InitializeAudioOutput();
screen_capturer_factory_ = new ScreenCapturerFactory();
@@ -39,37 +58,79 @@ int SessionDeviceManager::InitializeScreenCapturer() {
static_cast<ScreenCapturer *>(screen_capturer_factory_->Create());
}
last_frame_time_ = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now().time_since_epoch())
.count();
last_frame_time_ = {};
next_frame_deadline_ = {};
const int fps = owner_.config_center_->GetVideoFrameRate() ==
ConfigCenter::VIDEO_FRAME_RATE::FPS_30
? 30
: 60;
LOG_INFO("Init screen capturer with {} fps", fps);
display_info_list_.clear();
registered_display_stream_count_ = 0;
last_video_frame_stream_id_.clear();
invalid_video_stream_id_logged_ = false;
const int init_ret = screen_capturer_->Init(
fps, [this, fps](unsigned char *data, int size, int width, int height,
const char *display_name) {
const auto now_time = static_cast<uint64_t>(
std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now().time_since_epoch())
.count());
const auto duration = now_time - last_frame_time_;
if (duration * fps < 1000) {
const auto now_time = std::chrono::steady_clock::now();
if (!ShouldSendCapturedFrame(now_time, fps)) {
return;
}
const std::string stream_id = display_name ? display_name : "";
const bool has_previous_frame =
last_frame_time_.time_since_epoch().count() != 0;
const auto duration_ms =
has_previous_frame
? std::chrono::duration_cast<std::chrono::milliseconds>(
now_time - last_frame_time_)
.count()
: 0;
std::vector<std::string> connected_remote_ids;
{
std::shared_lock lock(owner_.connection_status_mutex_);
connected_remote_ids.reserve(owner_.connection_status_.size());
for (const auto &[remote_id, status] : owner_.connection_status_) {
if (status == ConnectionStatus::Connected) {
connected_remote_ids.push_back(remote_id);
}
}
}
// Capture can still deliver frames while ICE is gathering or after
// the final controller disconnects. Do not broadcast those frames to
// MiniRTC: a broadcast also reaches newly joining peers whose ICE
// transport is not ready yet.
if (connected_remote_ids.empty()) {
last_frame_time_ = now_time;
return;
}
const std::string stream_id = ResolveDisplayStreamId(
display_name, registered_display_stream_count_, -1,
last_video_frame_stream_id_);
if (stream_id.empty()) {
if (!invalid_video_stream_id_logged_) {
LOG_ERROR(
"Drop captured frames with an empty or unregistered video "
"stream id, reported='{}', registered_streams={}",
display_name ? display_name : "",
registered_display_stream_count_);
invalid_video_stream_id_logged_ = true;
}
return;
}
invalid_video_stream_id_logged_ = false;
const bool resumed_after_gap =
last_frame_time_ != 0 && duration >= kCaptureResumeKeyFrameGapMs;
has_previous_frame && duration_ms >= kCaptureResumeKeyFrameGapMs;
const bool stream_changed = !last_video_frame_stream_id_.empty() &&
last_video_frame_stream_id_ != stream_id;
if (resumed_after_gap || stream_changed) {
if (RequestVideoKeyFrame(owner_.peer_, stream_id.c_str()) == 0) {
LOG_INFO("Request video key frame before sending captured frame, "
"stream='{}', gap_ms={}, stream_changed={}",
stream_id, duration, stream_changed);
stream_id, duration_ms, stream_changed);
}
}
@@ -79,7 +140,10 @@ int SessionDeviceManager::InitializeScreenCapturer() {
frame.width = width;
frame.height = height;
frame.captured_timestamp = GetSystemTimeMicros(owner_.peer_);
SendVideoFrame(owner_.peer_, &frame, stream_id.c_str());
for (const std::string &remote_id : connected_remote_ids) {
SendVideoFrameToPeer(owner_.peer_, &frame, stream_id.c_str(),
remote_id.data(), remote_id.size());
}
last_video_frame_stream_id_ = stream_id;
last_frame_time_ = now_time;
});
@@ -90,6 +154,7 @@ int SessionDeviceManager::InitializeScreenCapturer() {
if (!latest_display_info.empty()) {
display_info_list_ = latest_display_info;
}
registered_display_stream_count_ = display_info_list_.size();
return 0;
}
@@ -3,6 +3,7 @@
#include <SDL3/SDL.h>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <deque>
@@ -66,6 +67,8 @@ private:
uint32_t scan_code, bool extended);
void DrainCapturedKeyboardInput();
void ClearCapturedKeyboardInput();
bool ShouldSendCapturedFrame(std::chrono::steady_clock::time_point now,
int fps);
GuiRuntime &owner_;
SDL_AudioStream *output_stream_ = nullptr;
@@ -77,10 +80,13 @@ private:
MouseController *mouse_controller_ = nullptr;
KeyboardCapturer *keyboard_capturer_ = nullptr;
std::vector<DisplayInfo> display_info_list_;
size_t registered_display_stream_count_ = 0;
std::deque<CapturedKeyboardInput> captured_keyboard_inputs_;
std::mutex captured_keyboard_inputs_mutex_;
uint64_t last_frame_time_ = 0;
std::chrono::steady_clock::time_point last_frame_time_{};
std::chrono::steady_clock::time_point next_frame_deadline_{};
std::string last_video_frame_stream_id_;
bool invalid_video_stream_id_logged_ = false;
};
} // namespace crossdesk
+319 -40
View File
@@ -1,11 +1,24 @@
#include "features/settings/settings_manager.h"
#include <chrono>
#include <cstddef>
#include <cstdio>
#include <cstring>
#include <exception>
#include <filesystem>
#include <fstream>
#include <functional>
#include <memory>
#include <string>
#include <thread>
#if defined(_WIN32)
#include <io.h>
#include <windows.h>
#else
#include <fcntl.h>
#include <unistd.h>
#endif
#include "localization.h"
#include "rd_log.h"
@@ -14,6 +27,9 @@
namespace crossdesk {
namespace {
constexpr uint32_t kCacheV3Magic = 0x33444358; // "XCD3"
constexpr uint32_t kCacheV3Version = 3;
template <size_t Size>
void CopyString(char (&destination)[Size], const char *source) {
static_assert(Size > 0);
@@ -23,6 +39,99 @@ void CopyString(char (&destination)[Size], const char *source) {
}
}
uint32_t CacheChecksum(const void *data, size_t size) {
const auto *bytes = static_cast<const unsigned char *>(data);
uint32_t hash = 2166136261u;
for (size_t i = 0; i < size; ++i) {
hash ^= bytes[i];
hash *= 16777619u;
}
return hash;
}
std::filesystem::path TemporaryPathFor(
const std::filesystem::path &target) {
const auto timestamp = std::chrono::steady_clock::now()
.time_since_epoch()
.count();
const auto thread_id =
std::hash<std::thread::id>{}(std::this_thread::get_id());
std::filesystem::path temporary = target;
temporary += ".tmp-" + std::to_string(timestamp) + "-" +
std::to_string(thread_id);
return temporary;
}
bool FlushFileToDisk(FILE *file) {
if (!file || std::fflush(file) != 0) {
return false;
}
#if defined(_WIN32)
return _commit(_fileno(file)) == 0;
#else
return fsync(fileno(file)) == 0;
#endif
}
bool ReplaceFileAtomically(const std::filesystem::path &temporary,
const std::filesystem::path &target) {
#if defined(_WIN32)
return MoveFileExW(temporary.c_str(), target.c_str(),
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH) != 0;
#else
if (::rename(temporary.c_str(), target.c_str()) != 0) {
return false;
}
const std::filesystem::path parent = target.parent_path().empty()
? std::filesystem::path(".")
: target.parent_path();
int directory_flags = O_RDONLY;
#if defined(O_DIRECTORY)
directory_flags |= O_DIRECTORY;
#endif
const int directory = open(parent.c_str(), directory_flags);
if (directory >= 0) {
const bool synced = fsync(directory) == 0;
close(directory);
return synced;
}
return false;
#endif
}
bool WriteFileAtomically(const std::filesystem::path &target,
const void *data, size_t size) {
std::error_code ec;
const std::filesystem::path parent = target.parent_path();
if (!parent.empty()) {
std::filesystem::create_directories(parent, ec);
if (ec) {
return false;
}
}
const std::filesystem::path temporary = TemporaryPathFor(target);
#if defined(_WIN32)
FILE *file = _wfopen(temporary.c_str(), L"wb");
#else
FILE *file = std::fopen(temporary.c_str(), "wb");
#endif
if (!file) {
return false;
}
const bool written = std::fwrite(data, 1, size, file) == size;
const bool flushed = written && FlushFileToDisk(file);
const bool closed = std::fclose(file) == 0;
if (!written || !flushed || !closed ||
!ReplaceFileAtomically(temporary, target)) {
std::filesystem::remove(temporary, ec);
return false;
}
return true;
}
} // namespace
SettingsManager::SettingsManager(GuiRuntime &owner) : owner_(owner) {}
@@ -33,36 +142,74 @@ int SettingsManager::Save() {
}
int SettingsManager::SaveLocked() {
std::ofstream cache_v2_file(owner_.cache_path_ + "/secure_cache_v2.enc",
std::ios::binary);
if (!cache_v2_file) {
return -1;
}
CopyString(cache_v2_.client_id_with_password,
owner_.client_id_with_password_);
std::memcpy(cache_v2_.key, owner_.aes128_key_, sizeof(owner_.aes128_key_));
std::memcpy(cache_v2_.iv, owner_.aes128_iv_, sizeof(owner_.aes128_iv_));
CopyString(cache_v2_.self_hosted_id, owner_.self_hosted_id_);
cache_v2_file.write(reinterpret_cast<const char *>(&cache_v2_),
sizeof(cache_v2_));
cache_v3_.magic = kCacheV3Magic;
cache_v3_.version = kCacheV3Version;
cache_v3_.base = cache_v2_;
cache_v3_.pending_identity[sizeof(cache_v3_.pending_identity) - 1] = '\0';
cache_v3_.pending_server_host[sizeof(cache_v3_.pending_server_host) - 1] =
'\0';
cache_v3_.pending_request_id[sizeof(cache_v3_.pending_request_id) - 1] =
'\0';
cache_v3_.checksum =
CacheChecksum(&cache_v3_, offsetof(CacheV3, checksum));
if (!WriteFileAtomically(owner_.cache_path_ + "/secure_cache_v3.enc",
&cache_v3_, sizeof(cache_v3_))) {
return -1;
}
if (!WriteFileAtomically(owner_.cache_path_ + "/secure_cache_v2.enc",
&cache_v2_, sizeof(cache_v2_))) {
LOG_WARN("Failed to update legacy v2 credential cache");
}
// Keep writing the legacy cache while older installations may still read it.
std::ofstream cache_v1_file(owner_.cache_path_ + "/secure_cache.enc",
std::ios::binary);
if (cache_v1_file) {
CopyString(cache_v1_.client_id_with_password,
owner_.client_id_with_password_);
std::memcpy(cache_v1_.key, owner_.aes128_key_, sizeof(owner_.aes128_key_));
std::memcpy(cache_v1_.iv, owner_.aes128_iv_, sizeof(owner_.aes128_iv_));
cache_v1_file.write(reinterpret_cast<const char *>(&cache_v1_),
sizeof(cache_v1_));
if (!WriteFileAtomically(owner_.cache_path_ + "/secure_cache.enc",
&cache_v1_, sizeof(cache_v1_))) {
LOG_WARN("Failed to update legacy v1 credential cache");
}
return 0;
}
bool SettingsManager::ReadV3Locked() {
std::ifstream cache_file(owner_.cache_path_ + "/secure_cache_v3.enc",
std::ios::binary);
if (!cache_file) {
return false;
}
CacheV3 loaded{};
cache_file.read(reinterpret_cast<char *>(&loaded), sizeof(loaded));
if (cache_file.gcount() != static_cast<std::streamsize>(sizeof(loaded)) ||
loaded.magic != kCacheV3Magic ||
loaded.version != kCacheV3Version ||
loaded.checksum != CacheChecksum(&loaded, offsetof(CacheV3, checksum))) {
LOG_WARN("Ignore invalid v3 credential cache");
return false;
}
loaded.base.client_id_with_password
[sizeof(loaded.base.client_id_with_password) - 1] = '\0';
loaded.base.self_hosted_id[sizeof(loaded.base.self_hosted_id) - 1] = '\0';
loaded.pending_identity[sizeof(loaded.pending_identity) - 1] = '\0';
loaded.pending_server_host[sizeof(loaded.pending_server_host) - 1] = '\0';
loaded.pending_request_id[sizeof(loaded.pending_request_id) - 1] = '\0';
cache_v3_ = loaded;
cache_v2_ = loaded.base;
return true;
}
bool SettingsManager::ReadV2Locked() {
std::ifstream cache_v2_file(owner_.cache_path_ + "/secure_cache_v2.enc",
std::ios::binary);
@@ -84,13 +231,20 @@ bool SettingsManager::ReadV2Locked() {
int SettingsManager::Load() {
std::unique_lock<std::mutex> lock(cache_mutex_);
if (ReadV2Locked()) {
const bool loaded_v3 = ReadV3Locked();
if (loaded_v3 || ReadV2Locked()) {
CopyString(owner_.client_id_with_password_,
cache_v2_.client_id_with_password);
CopyString(owner_.self_hosted_id_, cache_v2_.self_hosted_id);
std::memcpy(owner_.aes128_key_, cache_v2_.key, sizeof(cache_v2_.key));
std::memcpy(owner_.aes128_iv_, cache_v2_.iv, sizeof(cache_v2_.iv));
LOG_INFO("Load settings from v2 cache file");
if (loaded_v3) {
LOG_INFO("Load settings from v3 cache file");
} else {
cache_v3_ = {};
SaveLocked();
LOG_INFO("Migrated settings from v2 to v3 cache file");
}
} else {
std::ifstream cache_v1_file(owner_.cache_path_ + "/secure_cache.enc",
std::ios::binary);
@@ -128,20 +282,14 @@ int SettingsManager::Load() {
std::memcpy(owner_.aes128_key_, cache_v1_.key, sizeof(cache_v1_.key));
std::memcpy(owner_.aes128_iv_, cache_v1_.iv, sizeof(cache_v1_.iv));
cache_v3_ = {};
SaveLocked();
LOG_INFO("Migrated settings from v1 to v2 cache file");
LOG_INFO("Migrated settings from v1 to v3 cache file");
}
lock.unlock();
const char *at_pos = std::strchr(owner_.client_id_with_password_, '@');
if (at_pos != nullptr) {
const std::string id(owner_.client_id_with_password_,
at_pos - owner_.client_id_with_password_);
const std::string password(at_pos + 1);
CopyString(owner_.client_id_, id.c_str());
CopyString(owner_.password_saved_, password.c_str());
}
ActivateCachedPublicIdentity();
owner_.thumbnail_ =
std::make_shared<Thumbnail>(owner_.cache_path_ + "/thumbnails/",
@@ -193,8 +341,11 @@ int SettingsManager::Load() {
bool SettingsManager::LoadCachedSelfHostedIdentity() {
std::lock_guard<std::mutex> lock(cache_mutex_);
if (!ReadV2Locked() || cache_v2_.self_hosted_id[0] == '\0') {
if ((!ReadV3Locked() && !ReadV2Locked()) ||
cache_v2_.self_hosted_id[0] == '\0') {
std::memset(owner_.self_hosted_id_, 0, sizeof(owner_.self_hosted_id_));
std::memset(owner_.client_id_, 0, sizeof(owner_.client_id_));
std::memset(owner_.password_saved_, 0, sizeof(owner_.password_saved_));
return false;
}
@@ -212,27 +363,155 @@ bool SettingsManager::LoadCachedSelfHostedIdentity() {
return true;
}
bool SettingsManager::ActivateCachedPublicIdentity() {
if (owner_.client_id_with_password_[0] == '\0') {
std::memset(owner_.client_id_, 0, sizeof(owner_.client_id_));
std::memset(owner_.password_saved_, 0, sizeof(owner_.password_saved_));
return false;
}
const char *at_pos = std::strchr(owner_.client_id_with_password_, '@');
if (at_pos == nullptr) {
CopyString(owner_.client_id_, owner_.client_id_with_password_);
std::memset(owner_.password_saved_, 0, sizeof(owner_.password_saved_));
} else {
const std::string id(owner_.client_id_with_password_,
at_pos - owner_.client_id_with_password_);
CopyString(owner_.client_id_, id.c_str());
CopyString(owner_.password_saved_, at_pos + 1);
}
return owner_.client_id_[0] != '\0';
}
void SettingsManager::ActivateIdentity(const char *identity,
bool self_hosted) {
if (!identity) {
return;
}
if (self_hosted) {
CopyString(owner_.self_hosted_id_, identity);
} else {
CopyString(owner_.client_id_with_password_, identity);
}
const char *at_pos = std::strchr(identity, '@');
if (at_pos == nullptr) {
CopyString(owner_.client_id_, identity);
std::memset(owner_.password_saved_, 0, sizeof(owner_.password_saved_));
return;
}
const std::string id(identity, at_pos - identity);
CopyString(owner_.client_id_, id.c_str());
CopyString(owner_.password_saved_, at_pos + 1);
}
bool SettingsManager::StagePendingPasswordChange(
const std::string &identity, bool self_hosted,
const std::string &server_host, int server_port,
const std::string &request_id) {
if (identity.empty() || identity.size() >= sizeof(cache_v3_.pending_identity) ||
server_host.empty() ||
server_host.size() >= sizeof(cache_v3_.pending_server_host) ||
server_port <= 0 || request_id.empty() ||
request_id.size() >= sizeof(cache_v3_.pending_request_id)) {
return false;
}
std::lock_guard<std::mutex> lock(cache_mutex_);
if (cache_v3_.pending_identity[0] != '\0' &&
(cache_v3_.pending_self_hosted != self_hosted ||
cache_v3_.pending_server_port != server_port ||
server_host != cache_v3_.pending_server_host ||
request_id != cache_v3_.pending_request_id)) {
LOG_WARN("Refuse to overwrite an unresolved password change");
return false;
}
const CacheV3 previous = cache_v3_;
CopyString(cache_v3_.pending_identity, identity.c_str());
CopyString(cache_v3_.pending_server_host, server_host.c_str());
cache_v3_.pending_server_port = server_port;
cache_v3_.pending_self_hosted = self_hosted;
CopyString(cache_v3_.pending_request_id, request_id.c_str());
if (SaveLocked() != 0) {
cache_v3_ = previous;
return false;
}
return true;
}
std::string SettingsManager::PendingPasswordChangeIdentity(
bool self_hosted, const std::string &server_host, int server_port) const {
std::lock_guard<std::mutex> lock(cache_mutex_);
if (cache_v3_.pending_identity[0] == '\0' ||
cache_v3_.pending_self_hosted != self_hosted ||
cache_v3_.pending_server_port != server_port ||
server_host != cache_v3_.pending_server_host) {
return {};
}
return cache_v3_.pending_identity;
}
bool SettingsManager::PromotePendingPasswordChange() {
std::lock_guard<std::mutex> lock(cache_mutex_);
if (cache_v3_.pending_identity[0] == '\0') {
return true;
}
const CacheV3 previous = cache_v3_;
const std::string identity = cache_v3_.pending_identity;
const bool self_hosted = cache_v3_.pending_self_hosted;
ActivateIdentity(identity.c_str(), self_hosted);
std::memset(cache_v3_.pending_identity, 0,
sizeof(cache_v3_.pending_identity));
std::memset(cache_v3_.pending_server_host, 0,
sizeof(cache_v3_.pending_server_host));
cache_v3_.pending_server_port = 0;
cache_v3_.pending_self_hosted = false;
std::memset(cache_v3_.pending_request_id, 0,
sizeof(cache_v3_.pending_request_id));
if (SaveLocked() != 0) {
// The already-durable pending credential remains the recovery source on
// disk. Keep it in memory as well so a reconnect in this process retries
// the credential that the server has accepted.
cache_v3_ = previous;
return false;
}
return true;
}
bool SettingsManager::ClearPendingPasswordChange() {
std::lock_guard<std::mutex> lock(cache_mutex_);
if (cache_v3_.pending_identity[0] == '\0') {
return true;
}
const CacheV3 previous = cache_v3_;
std::memset(cache_v3_.pending_identity, 0,
sizeof(cache_v3_.pending_identity));
std::memset(cache_v3_.pending_server_host, 0,
sizeof(cache_v3_.pending_server_host));
cache_v3_.pending_server_port = 0;
cache_v3_.pending_self_hosted = false;
std::memset(cache_v3_.pending_request_id, 0,
sizeof(cache_v3_.pending_request_id));
if (SaveLocked() != 0) {
cache_v3_ = previous;
return false;
}
return true;
}
void SettingsManager::PersistSelfHostedIdentity(const char *client_id) {
if (!client_id) {
return;
}
std::lock_guard<std::mutex> lock(cache_mutex_);
if (!ReadV2Locked()) {
cache_v2_ = {};
}
CopyString(cache_v2_.self_hosted_id, client_id);
CopyString(cache_v2_.client_id_with_password,
owner_.client_id_with_password_);
std::memcpy(cache_v2_.key, owner_.aes128_key_, sizeof(owner_.aes128_key_));
std::memcpy(cache_v2_.iv, owner_.aes128_iv_, sizeof(owner_.aes128_iv_));
std::ofstream cache_v2_file(owner_.cache_path_ + "/secure_cache_v2.enc",
std::ios::binary);
if (cache_v2_file) {
cache_v2_file.write(reinterpret_cast<const char *>(&cache_v2_),
sizeof(cache_v2_));
CopyString(owner_.self_hosted_id_, client_id);
if (SaveLocked() != 0) {
LOG_ERROR("Failed to persist self-hosted identity atomically");
}
}
@@ -1,6 +1,7 @@
#ifndef CROSSDESK_GUI_SETTINGS_MANAGER_H_
#define CROSSDESK_GUI_SETTINGS_MANAGER_H_
#include <cstdint>
#include <mutex>
#include <string>
#include <unordered_map>
@@ -33,8 +34,28 @@ public:
// Loads the cached self-hosted identity into the owner's active connection
// fields. Returns true only when a non-empty identity was restored.
bool LoadCachedSelfHostedIdentity();
// Restores the public-server identity kept in the in-memory cache fields.
// This must run when switching away from a self-hosted server so signal
// callbacks are matched against the identity used by the replacement peer.
bool ActivateCachedPublicIdentity();
void PersistSelfHostedIdentity(const char *client_id);
// Password rotation is deliberately persisted in two phases. The active
// credential remains usable while the pending credential records the value
// that may already have been committed by the server. On the next launch the
// caller can try the pending credential first and safely fall back to the
// active credential when the request never reached the server.
bool StagePendingPasswordChange(const std::string &identity,
bool self_hosted,
const std::string &server_host,
int server_port,
const std::string &request_id);
std::string PendingPasswordChangeIdentity(
bool self_hosted, const std::string &server_host,
int server_port) const;
bool PromotePendingPasswordChange();
bool ClearPendingPasswordChange();
private:
struct CacheV1 {
char client_id_with_password[17];
@@ -63,12 +84,27 @@ private:
char self_hosted_id[17];
};
struct CacheV3 {
uint32_t magic;
uint32_t version;
CacheV2 base;
char pending_identity[17];
char pending_server_host[256];
int pending_server_port;
bool pending_self_hosted;
char pending_request_id[64];
uint32_t checksum;
};
int SaveLocked();
bool ReadV3Locked();
bool ReadV2Locked();
void ActivateIdentity(const char *identity, bool self_hosted);
GuiRuntime &owner_;
CacheV1 cache_v1_{};
CacheV2 cache_v2_{};
CacheV3 cache_v3_{};
mutable std::mutex cache_mutex_;
std::unordered_map<std::string, std::string> recent_connection_aliases_;
};
+82
View File
@@ -0,0 +1,82 @@
#ifndef CROSSDESK_GUI_PLATFORM_METAL_VIDEO_RENDERER_H_
#define CROSSDESK_GUI_PLATFORM_METAL_VIDEO_RENDERER_H_
#include <cstddef>
#include <cstdint>
#include <memory>
#include <string>
#include <string_view>
#include <vector>
#include "platform/video_renderer.h"
namespace crossdesk {
// macOS video compositor used by CrossDesk only. MiniRTC continues to expose
// tightly packed CPU NV12 frames; this renderer performs the single required
// copy into shared Metal buffers and converts NV12 to RGB in the fragment
// shader. AppKit attachment and rendering are main-thread operations, while
// SubmitNv12() is safe to call from MiniRTC's decode callback thread.
class MacMetalVideoRenderer final : public VideoRenderer {
public:
MacMetalVideoRenderer();
~MacMetalVideoRenderer() override;
MacMetalVideoRenderer(const MacMetalVideoRenderer&) = delete;
MacMetalVideoRenderer& operator=(const MacMetalVideoRenderer&) = delete;
bool IsReady() const override;
bool IsActive() const override;
// The selected stream is the only stream uploaded. Frames for background
// tabs are intentionally dropped to avoid wasting memory bandwidth.
bool SetSelectedStream(std::string remote_id) override;
void DiscardStream(std::string_view remote_id) override;
// Copies one tightly packed NV12 frame into a free shared Metal slot.
SubmitResult SubmitNv12(std::string_view remote_id, const uint8_t* data,
size_t size, int width, int height) override;
// Seeds a newly selected stream from its retained CPU snapshot without
// replacing a newer decoded frame that is already queued or rendering.
SubmitResult SubmitCachedNv12(std::string_view remote_id,
const uint8_t* data, size_t size, int width,
int height) override;
// Attaches a native CAMetalLayer-backed sibling below Slint's NSView and
// restores the containing AppKit window's ordinary opaque titlebar.
// |slint_view| is an NSView* kept opaque so this header remains valid C++.
bool Attach(void* slint_view);
void Detach();
bool IsAttached() const;
// Cheap main-thread geometry check used by the UI frame timer. New decoded
// frames are tracked by GuiRuntime's dirty flag; this covers native resize,
// scale, stream selection, and retry work without a full session sync.
bool NeedsSurfaceRedraw() const;
// Draws the newest queued frame for |remote_id|. top_inset_points reserves
// Slint's tab strip above the native video surface. A rendered outcome
// carries metadata from the exact slot submitted to the command buffer.
RenderOutcome RenderLatest(std::string_view remote_id,
double top_inset_points,
bool native_titlebar_visible);
// Used only when a session closes, so thumbnail generation does not require
// a second per-frame CPU copy during normal playback.
bool CopyLatestNv12(std::string_view remote_id,
std::vector<unsigned char>* output, int* width,
int* height) const override;
private:
SubmitResult SubmitNv12Internal(std::string_view remote_id,
const uint8_t* data, size_t size, int width,
int height, bool replace_pending);
struct Impl;
std::unique_ptr<Impl> impl_;
};
} // namespace crossdesk
#endif // CROSSDESK_GUI_PLATFORM_METAL_VIDEO_RENDERER_H_
File diff suppressed because it is too large Load Diff
+928
View File
@@ -0,0 +1,928 @@
#include "platform/opengl_video_renderer.h"
#if !defined(_WIN32) && !defined(__linux__)
#error "OpenGlVideoRenderer is only available on Windows and Linux"
#endif
#if defined(_WIN32)
#include <windows.h>
#elif defined(__linux__)
#include <dlfcn.h>
#endif
#include <GL/gl.h>
#include <GL/glext.h>
#if defined(__linux__)
#include <GL/glx.h>
#endif
#include <algorithm>
#include <array>
#include <atomic>
#include <cstring>
#include <limits>
#include <mutex>
#include <string>
#include <utility>
#include "rd_log.h"
namespace crossdesk {
namespace {
constexpr size_t kFrameSlotCount = 3;
enum class SlotUse {
available,
pending,
uploading,
};
struct FrameSlot {
std::vector<unsigned char> bytes;
int width = 0;
int height = 0;
std::string remote_id;
uint64_t sequence = 0;
SlotUse use = SlotUse::available;
bool valid = false;
};
struct SharedFrameState {
mutable std::mutex mutex;
std::array<FrameSlot, kFrameSlotCount> slots;
std::string selected_stream;
uint64_t next_sequence = 1;
};
template <typename T> bool LoadOpenGlFunction(T *function, const char *name) {
#if defined(_WIN32)
PROC address = wglGetProcAddress(name);
if (address == nullptr || address == reinterpret_cast<PROC>(1) ||
address == reinterpret_cast<PROC>(2) ||
address == reinterpret_cast<PROC>(3) ||
address == reinterpret_cast<PROC>(-1)) {
HMODULE module = GetModuleHandleW(L"opengl32.dll");
address = module ? GetProcAddress(module, name) : nullptr;
}
*function = reinterpret_cast<T>(address);
#elif defined(__linux__)
void *address = dlsym(RTLD_DEFAULT, name);
if (address == nullptr) {
address = reinterpret_cast<void *>(
glXGetProcAddressARB(reinterpret_cast<const GLubyte *>(name)));
}
*function = reinterpret_cast<T>(address);
#endif
return *function != nullptr;
}
struct OpenGlFunctions {
PFNGLACTIVETEXTUREPROC active_texture = nullptr;
PFNGLATTACHSHADERPROC attach_shader = nullptr;
PFNGLBINDBUFFERPROC bind_buffer = nullptr;
PFNGLBUFFERDATAPROC buffer_data = nullptr;
PFNGLCOMPILESHADERPROC compile_shader = nullptr;
PFNGLCREATEPROGRAMPROC create_program = nullptr;
PFNGLCREATESHADERPROC create_shader = nullptr;
PFNGLDELETEBUFFERSPROC delete_buffers = nullptr;
PFNGLDELETEPROGRAMPROC delete_program = nullptr;
PFNGLDELETESHADERPROC delete_shader = nullptr;
PFNGLDISABLEVERTEXATTRIBARRAYPROC disable_vertex_attrib_array = nullptr;
PFNGLENABLEVERTEXATTRIBARRAYPROC enable_vertex_attrib_array = nullptr;
PFNGLGENBUFFERSPROC gen_buffers = nullptr;
PFNGLGETATTRIBLOCATIONPROC get_attrib_location = nullptr;
PFNGLGETPROGRAMINFOLOGPROC get_program_info_log = nullptr;
PFNGLGETPROGRAMIVPROC get_program_iv = nullptr;
PFNGLGETSHADERINFOLOGPROC get_shader_info_log = nullptr;
PFNGLGETSHADERIVPROC get_shader_iv = nullptr;
PFNGLGETUNIFORMLOCATIONPROC get_uniform_location = nullptr;
PFNGLGETVERTEXATTRIBIVPROC get_vertex_attrib_iv = nullptr;
PFNGLGETVERTEXATTRIBPOINTERVPROC get_vertex_attrib_pointer_v = nullptr;
PFNGLLINKPROGRAMPROC link_program = nullptr;
PFNGLSHADERSOURCEPROC shader_source = nullptr;
PFNGLUNIFORM1FPROC uniform_1f = nullptr;
PFNGLUNIFORM1IPROC uniform_1i = nullptr;
PFNGLUNIFORM2FPROC uniform_2f = nullptr;
PFNGLUNIFORM4FPROC uniform_4f = nullptr;
PFNGLUSEPROGRAMPROC use_program = nullptr;
PFNGLVERTEXATTRIBPOINTERPROC vertex_attrib_pointer = nullptr;
bool Load() {
return LoadOpenGlFunction(&active_texture, "glActiveTexture") &&
LoadOpenGlFunction(&attach_shader, "glAttachShader") &&
LoadOpenGlFunction(&bind_buffer, "glBindBuffer") &&
LoadOpenGlFunction(&buffer_data, "glBufferData") &&
LoadOpenGlFunction(&compile_shader, "glCompileShader") &&
LoadOpenGlFunction(&create_program, "glCreateProgram") &&
LoadOpenGlFunction(&create_shader, "glCreateShader") &&
LoadOpenGlFunction(&delete_buffers, "glDeleteBuffers") &&
LoadOpenGlFunction(&delete_program, "glDeleteProgram") &&
LoadOpenGlFunction(&delete_shader, "glDeleteShader") &&
LoadOpenGlFunction(&disable_vertex_attrib_array,
"glDisableVertexAttribArray") &&
LoadOpenGlFunction(&enable_vertex_attrib_array,
"glEnableVertexAttribArray") &&
LoadOpenGlFunction(&gen_buffers, "glGenBuffers") &&
LoadOpenGlFunction(&get_attrib_location, "glGetAttribLocation") &&
LoadOpenGlFunction(&get_program_info_log, "glGetProgramInfoLog") &&
LoadOpenGlFunction(&get_program_iv, "glGetProgramiv") &&
LoadOpenGlFunction(&get_shader_info_log, "glGetShaderInfoLog") &&
LoadOpenGlFunction(&get_shader_iv, "glGetShaderiv") &&
LoadOpenGlFunction(&get_uniform_location, "glGetUniformLocation") &&
LoadOpenGlFunction(&get_vertex_attrib_iv, "glGetVertexAttribiv") &&
LoadOpenGlFunction(&get_vertex_attrib_pointer_v,
"glGetVertexAttribPointerv") &&
LoadOpenGlFunction(&link_program, "glLinkProgram") &&
LoadOpenGlFunction(&shader_source, "glShaderSource") &&
LoadOpenGlFunction(&uniform_1f, "glUniform1f") &&
LoadOpenGlFunction(&uniform_1i, "glUniform1i") &&
LoadOpenGlFunction(&uniform_2f, "glUniform2f") &&
LoadOpenGlFunction(&uniform_4f, "glUniform4f") &&
LoadOpenGlFunction(&use_program, "glUseProgram") &&
LoadOpenGlFunction(&vertex_attrib_pointer, "glVertexAttribPointer");
}
};
GLuint CompileShader(const OpenGlFunctions &gl, GLenum type,
const char *source) {
const GLuint shader = gl.create_shader(type);
if (shader == 0) {
return 0;
}
gl.shader_source(shader, 1, &source, nullptr);
gl.compile_shader(shader);
GLint compiled = GL_FALSE;
gl.get_shader_iv(shader, GL_COMPILE_STATUS, &compiled);
if (compiled == GL_TRUE) {
return shader;
}
GLint log_length = 0;
gl.get_shader_iv(shader, GL_INFO_LOG_LENGTH, &log_length);
std::string log(static_cast<size_t>(std::max(log_length, 1)), '\0');
GLsizei written = 0;
gl.get_shader_info_log(shader, static_cast<GLsizei>(log.size()), &written,
log.data());
if (written >= 0 && static_cast<size_t>(written) < log.size()) {
log.resize(static_cast<size_t>(written));
}
LOG_ERROR("OpenGL NV12 {} shader compilation failed: {}",
type == GL_VERTEX_SHADER ? "vertex" : "fragment", log);
gl.delete_shader(shader);
return 0;
}
bool IsEnabled(GLenum capability) { return glIsEnabled(capability) == GL_TRUE; }
void RestoreCapability(GLenum capability, bool enabled) {
if (enabled) {
glEnable(capability);
} else {
glDisable(capability);
}
}
struct VertexAttribState {
GLint enabled = GL_FALSE;
GLint size = 4;
GLint stride = 0;
GLint type = GL_FLOAT;
GLint normalized = GL_FALSE;
GLint buffer = 0;
void *pointer = nullptr;
};
VertexAttribState CaptureVertexAttrib(const OpenGlFunctions &gl, GLuint index) {
VertexAttribState state;
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_ENABLED,
&state.enabled);
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_SIZE, &state.size);
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_STRIDE, &state.stride);
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_TYPE, &state.type);
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_NORMALIZED,
&state.normalized);
gl.get_vertex_attrib_iv(index, GL_VERTEX_ATTRIB_ARRAY_BUFFER_BINDING,
&state.buffer);
gl.get_vertex_attrib_pointer_v(index, GL_VERTEX_ATTRIB_ARRAY_POINTER,
&state.pointer);
return state;
}
void RestoreVertexAttrib(const OpenGlFunctions &gl, GLuint index,
const VertexAttribState &state) {
if (state.buffer != 0 || state.pointer != nullptr) {
gl.bind_buffer(GL_ARRAY_BUFFER, static_cast<GLuint>(state.buffer));
gl.vertex_attrib_pointer(index, state.size, static_cast<GLenum>(state.type),
static_cast<GLboolean>(state.normalized),
state.stride, state.pointer);
}
if (state.enabled == GL_TRUE) {
gl.enable_vertex_attrib_array(index);
} else {
gl.disable_vertex_attrib_array(index);
}
}
} // namespace
struct OpenGlVideoRenderer::Impl {
OpenGlFunctions gl;
std::shared_ptr<SharedFrameState> frames =
std::make_shared<SharedFrameState>();
std::atomic<bool> ready{false};
GLuint program = 0;
GLuint vertex_buffer = 0;
GLuint y_texture = 0;
GLuint uv_texture = 0;
GLint position_location = -1;
GLint texcoord_location = -1;
GLint y_texture_location = -1;
GLint uv_texture_location = -1;
GLint target_size_location = -1;
GLint video_rect_location = -1;
GLint corner_radius_location = -1;
GLint video_enabled_location = -1;
int texture_width = 0;
int texture_height = 0;
std::string uploaded_stream;
uint64_t uploaded_sequence = 0;
void ResetGlHandles() {
program = 0;
vertex_buffer = 0;
y_texture = 0;
uv_texture = 0;
position_location = -1;
texcoord_location = -1;
y_texture_location = -1;
uv_texture_location = -1;
target_size_location = -1;
video_rect_location = -1;
corner_radius_location = -1;
video_enabled_location = -1;
texture_width = 0;
texture_height = 0;
uploaded_stream.clear();
uploaded_sequence = 0;
}
};
OpenGlVideoRenderer::OpenGlVideoRenderer()
: impl_(std::make_unique<Impl>()) {}
OpenGlVideoRenderer::~OpenGlVideoRenderer() = default;
bool OpenGlVideoRenderer::Setup() {
if (IsReady()) {
return true;
}
impl_->ready.store(false, std::memory_order_release);
impl_->ResetGlHandles();
if (!impl_->gl.Load()) {
LOG_WARN("OpenGL NV12 underlay unavailable: required functions missing");
return false;
}
static constexpr char kGlesVertexShader[] = R"glsl(
#version 100
attribute vec2 position;
attribute vec2 texcoord;
varying vec2 video_texcoord;
void main() {
video_texcoord = texcoord;
gl_Position = vec4(position, 0.0, 1.0);
}
)glsl";
static constexpr char kGlesFragmentShader[] = R"glsl(
#version 100
#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif
uniform sampler2D y_texture;
uniform sampler2D uv_texture;
uniform vec2 target_size;
uniform vec4 video_rect;
uniform float corner_radius;
uniform float video_enabled;
varying vec2 video_texcoord;
void main() {
vec2 surface_point = video_texcoord * target_size;
vec3 rgb = vec3(0.0);
if (video_enabled > 0.5 &&
surface_point.x >= video_rect.x &&
surface_point.x <= video_rect.x + video_rect.z &&
surface_point.y >= video_rect.y &&
surface_point.y <= video_rect.y + video_rect.w) {
vec2 frame_texcoord =
(surface_point - video_rect.xy) / video_rect.zw;
float y = 1.16438356 *
(texture2D(y_texture, frame_texcoord).r - 16.0 / 255.0);
vec2 uv =
texture2D(uv_texture, frame_texcoord).ra - vec2(0.5, 0.5);
rgb = clamp(vec3(y + 1.59602678 * uv.y,
y - 0.39176229 * uv.x - 0.81296764 * uv.y,
y + 2.01723214 * uv.x),
0.0, 1.0);
}
float coverage = 1.0;
if (corner_radius > 0.0) {
vec2 half_size = target_size * 0.5;
vec2 corner = abs(surface_point - half_size) -
(half_size - vec2(corner_radius));
float distance_to_edge =
length(max(corner, vec2(0.0))) +
min(max(corner.x, corner.y), 0.0) - corner_radius;
coverage = clamp(0.5 - distance_to_edge, 0.0, 1.0);
}
gl_FragColor = vec4(rgb * coverage, coverage);
}
)glsl";
static constexpr char kDesktopVertexShader[] = R"glsl(
#version 110
attribute vec2 position;
attribute vec2 texcoord;
varying vec2 video_texcoord;
void main() {
video_texcoord = texcoord;
gl_Position = vec4(position, 0.0, 1.0);
}
)glsl";
static constexpr char kDesktopFragmentShader[] = R"glsl(
#version 110
uniform sampler2D y_texture;
uniform sampler2D uv_texture;
uniform vec2 target_size;
uniform vec4 video_rect;
uniform float corner_radius;
uniform float video_enabled;
varying vec2 video_texcoord;
void main() {
vec2 surface_point = video_texcoord * target_size;
vec3 rgb = vec3(0.0);
if (video_enabled > 0.5 &&
surface_point.x >= video_rect.x &&
surface_point.x <= video_rect.x + video_rect.z &&
surface_point.y >= video_rect.y &&
surface_point.y <= video_rect.y + video_rect.w) {
vec2 frame_texcoord =
(surface_point - video_rect.xy) / video_rect.zw;
float y = 1.16438356 *
(texture2D(y_texture, frame_texcoord).r - 16.0 / 255.0);
vec2 uv =
texture2D(uv_texture, frame_texcoord).ra - vec2(0.5, 0.5);
rgb = clamp(vec3(y + 1.59602678 * uv.y,
y - 0.39176229 * uv.x - 0.81296764 * uv.y,
y + 2.01723214 * uv.x),
0.0, 1.0);
}
float coverage = 1.0;
if (corner_radius > 0.0) {
vec2 half_size = target_size * 0.5;
vec2 corner = abs(surface_point - half_size) -
(half_size - vec2(corner_radius));
float distance_to_edge =
length(max(corner, vec2(0.0))) +
min(max(corner.x, corner.y), 0.0) - corner_radius;
coverage = clamp(0.5 - distance_to_edge, 0.0, 1.0);
}
gl_FragColor = vec4(rgb * coverage, coverage);
}
)glsl";
const auto *version = reinterpret_cast<const char *>(glGetString(GL_VERSION));
const bool is_gles = version && std::strstr(version, "OpenGL ES") != nullptr;
const char *vertex_source =
is_gles ? kGlesVertexShader : kDesktopVertexShader;
const char *fragment_source =
is_gles ? kGlesFragmentShader : kDesktopFragmentShader;
const GLuint vertex_shader =
CompileShader(impl_->gl, GL_VERTEX_SHADER, vertex_source);
const GLuint fragment_shader =
CompileShader(impl_->gl, GL_FRAGMENT_SHADER, fragment_source);
if (vertex_shader == 0 || fragment_shader == 0) {
if (vertex_shader != 0)
impl_->gl.delete_shader(vertex_shader);
if (fragment_shader != 0)
impl_->gl.delete_shader(fragment_shader);
return false;
}
impl_->program = impl_->gl.create_program();
impl_->gl.attach_shader(impl_->program, vertex_shader);
impl_->gl.attach_shader(impl_->program, fragment_shader);
impl_->gl.link_program(impl_->program);
impl_->gl.delete_shader(vertex_shader);
impl_->gl.delete_shader(fragment_shader);
GLint linked = GL_FALSE;
impl_->gl.get_program_iv(impl_->program, GL_LINK_STATUS, &linked);
if (linked != GL_TRUE) {
GLint log_length = 0;
impl_->gl.get_program_iv(impl_->program, GL_INFO_LOG_LENGTH, &log_length);
std::string log(static_cast<size_t>(std::max(log_length, 1)), '\0');
GLsizei written = 0;
impl_->gl.get_program_info_log(
impl_->program, static_cast<GLsizei>(log.size()), &written, log.data());
if (written >= 0 && static_cast<size_t>(written) < log.size()) {
log.resize(static_cast<size_t>(written));
}
LOG_ERROR("OpenGL NV12 program link failed: {}", log);
impl_->gl.delete_program(impl_->program);
impl_->ResetGlHandles();
return false;
}
impl_->position_location =
impl_->gl.get_attrib_location(impl_->program, "position");
impl_->texcoord_location =
impl_->gl.get_attrib_location(impl_->program, "texcoord");
impl_->y_texture_location =
impl_->gl.get_uniform_location(impl_->program, "y_texture");
impl_->uv_texture_location =
impl_->gl.get_uniform_location(impl_->program, "uv_texture");
impl_->target_size_location =
impl_->gl.get_uniform_location(impl_->program, "target_size");
impl_->video_rect_location =
impl_->gl.get_uniform_location(impl_->program, "video_rect");
impl_->corner_radius_location =
impl_->gl.get_uniform_location(impl_->program, "corner_radius");
impl_->video_enabled_location =
impl_->gl.get_uniform_location(impl_->program, "video_enabled");
if (impl_->position_location < 0 || impl_->texcoord_location < 0 ||
impl_->y_texture_location < 0 || impl_->uv_texture_location < 0 ||
impl_->target_size_location < 0 || impl_->video_rect_location < 0 ||
impl_->corner_radius_location < 0 ||
impl_->video_enabled_location < 0) {
LOG_ERROR("OpenGL NV12 program is missing required shader bindings");
impl_->gl.delete_program(impl_->program);
impl_->ResetGlHandles();
return false;
}
static constexpr GLfloat kVertices[] = {
-1.0f, 1.0f, 0.0f, 0.0f, // top-left
-1.0f, -1.0f, 0.0f, 1.0f, // bottom-left
1.0f, 1.0f, 1.0f, 0.0f, // top-right
1.0f, -1.0f, 1.0f, 1.0f, // bottom-right
};
GLint previous_array_buffer = 0;
glGetIntegerv(GL_ARRAY_BUFFER_BINDING, &previous_array_buffer);
impl_->gl.gen_buffers(1, &impl_->vertex_buffer);
impl_->gl.bind_buffer(GL_ARRAY_BUFFER, impl_->vertex_buffer);
impl_->gl.buffer_data(GL_ARRAY_BUFFER, sizeof(kVertices), kVertices,
GL_STATIC_DRAW);
impl_->gl.bind_buffer(GL_ARRAY_BUFFER,
static_cast<GLuint>(previous_array_buffer));
GLint previous_active_texture = GL_TEXTURE0;
glGetIntegerv(GL_ACTIVE_TEXTURE, &previous_active_texture);
impl_->gl.active_texture(GL_TEXTURE0);
GLint previous_texture_0 = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture_0);
impl_->gl.active_texture(GL_TEXTURE1);
GLint previous_texture_1 = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture_1);
while (glGetError() != GL_NO_ERROR) {
}
glGenTextures(1, &impl_->y_texture);
glGenTextures(1, &impl_->uv_texture);
for (const auto [unit, texture] :
{std::pair{GL_TEXTURE0, impl_->y_texture},
std::pair{GL_TEXTURE1, impl_->uv_texture}}) {
impl_->gl.active_texture(unit);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}
impl_->gl.active_texture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture_0));
impl_->gl.active_texture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture_1));
impl_->gl.active_texture(static_cast<GLenum>(previous_active_texture));
const GLenum setup_error = glGetError();
if (impl_->program == 0 || impl_->vertex_buffer == 0 ||
impl_->y_texture == 0 || impl_->uv_texture == 0 ||
setup_error != GL_NO_ERROR) {
LOG_ERROR("OpenGL NV12 resource setup failed, error={}", setup_error);
Teardown();
return false;
}
impl_->ready.store(true, std::memory_order_release);
LOG_INFO("OpenGL NV12 underlay initialized ({})",
version ? version : "unknown OpenGL version");
return true;
}
void OpenGlVideoRenderer::Teardown() {
impl_->ready.store(false, std::memory_order_release);
if (impl_->y_texture != 0)
glDeleteTextures(1, &impl_->y_texture);
if (impl_->uv_texture != 0)
glDeleteTextures(1, &impl_->uv_texture);
if (impl_->vertex_buffer != 0 && impl_->gl.delete_buffers) {
impl_->gl.delete_buffers(1, &impl_->vertex_buffer);
}
if (impl_->program != 0 && impl_->gl.delete_program) {
impl_->gl.delete_program(impl_->program);
}
impl_->ResetGlHandles();
}
bool OpenGlVideoRenderer::IsReady() const {
return impl_->ready.load(std::memory_order_acquire);
}
bool OpenGlVideoRenderer::IsActive() const { return IsReady(); }
bool OpenGlVideoRenderer::SetSelectedStream(std::string remote_id) {
std::lock_guard lock(impl_->frames->mutex);
if (impl_->frames->selected_stream == remote_id) {
return false;
}
impl_->frames->selected_stream = std::move(remote_id);
for (auto &slot : impl_->frames->slots) {
if (slot.use == SlotUse::pending &&
slot.remote_id != impl_->frames->selected_stream) {
slot.use = SlotUse::available;
}
}
return true;
}
void OpenGlVideoRenderer::DiscardStream(std::string_view remote_id) {
std::lock_guard lock(impl_->frames->mutex);
for (auto &slot : impl_->frames->slots) {
if (slot.remote_id == remote_id && slot.use != SlotUse::uploading) {
slot.valid = false;
slot.use = SlotUse::available;
slot.remote_id.clear();
}
}
if (impl_->frames->selected_stream == remote_id) {
impl_->frames->selected_stream.clear();
}
}
OpenGlVideoRenderer::SubmitResult
OpenGlVideoRenderer::SubmitNv12(std::string_view remote_id,
const uint8_t *data, size_t size,
int width, int height) {
return SubmitNv12Internal(remote_id, data, size, width, height, true);
}
OpenGlVideoRenderer::SubmitResult
OpenGlVideoRenderer::SubmitCachedNv12(std::string_view remote_id,
const uint8_t *data, size_t size,
int width, int height) {
return SubmitNv12Internal(remote_id, data, size, width, height, false);
}
OpenGlVideoRenderer::SubmitResult
OpenGlVideoRenderer::SubmitNv12Internal(std::string_view remote_id,
const uint8_t *data, size_t size,
int width, int height,
bool replace_pending) {
if (!IsReady()) {
return SubmitResult::failed;
}
if (data == nullptr || width <= 0 || height <= 0 || (width & 1) != 0 ||
(height & 1) != 0) {
return SubmitResult::failed;
}
const size_t y_size = static_cast<size_t>(width) * height;
const size_t required_size = y_size + y_size / 2;
if (size < required_size) {
return SubmitResult::failed;
}
auto &frames = *impl_->frames;
std::lock_guard lock(frames.mutex);
if (frames.selected_stream != remote_id) {
return SubmitResult::not_selected;
}
FrameSlot *target = nullptr;
if (replace_pending) {
for (auto &slot : frames.slots) {
if (slot.use == SlotUse::pending) {
target = &slot;
break;
}
}
} else {
for (const auto &slot : frames.slots) {
if (slot.valid && slot.remote_id == remote_id &&
(slot.use == SlotUse::pending || slot.use == SlotUse::uploading)) {
return SubmitResult::dropped;
}
}
}
if (target == nullptr) {
uint64_t oldest_sequence = std::numeric_limits<uint64_t>::max();
for (auto &slot : frames.slots) {
if (slot.use != SlotUse::available) {
continue;
}
if (!slot.valid) {
target = &slot;
break;
}
if (slot.sequence < oldest_sequence) {
oldest_sequence = slot.sequence;
target = &slot;
}
}
}
if (target == nullptr) {
return SubmitResult::dropped;
}
target->bytes.resize(required_size);
std::memcpy(target->bytes.data(), data, required_size);
target->width = width;
target->height = height;
target->remote_id.assign(remote_id);
target->sequence = frames.next_sequence++;
target->use = SlotUse::pending;
target->valid = true;
return SubmitResult::submitted;
}
OpenGlVideoRenderer::RenderOutcome
OpenGlVideoRenderer::RenderLatest(std::string_view remote_id,
int target_width, int target_height,
int top_inset_pixels,
int corner_radius_pixels) {
if (!IsReady() || target_width <= 0 || target_height <= 0) {
return {RenderResult::failed};
}
size_t slot_index = kFrameSlotCount;
uint64_t sequence = 0;
int source_width = 0;
int source_height = 0;
const unsigned char *frame_data = nullptr;
{
std::lock_guard lock(impl_->frames->mutex);
for (size_t index = 0; index < impl_->frames->slots.size(); ++index) {
const auto &slot = impl_->frames->slots[index];
if (slot.valid && slot.use == SlotUse::pending &&
slot.remote_id == remote_id &&
(slot_index == kFrameSlotCount || slot.sequence > sequence)) {
slot_index = index;
sequence = slot.sequence;
}
}
if (slot_index != kFrameSlotCount) {
auto &selected = impl_->frames->slots[slot_index];
selected.use = SlotUse::uploading;
source_width = selected.width;
source_height = selected.height;
frame_data = selected.bytes.data();
for (size_t index = 0; index < impl_->frames->slots.size(); ++index) {
auto &slot = impl_->frames->slots[index];
if (index != slot_index && slot.use == SlotUse::pending &&
slot.remote_id == remote_id) {
slot.use = SlotUse::available;
}
}
}
}
GLint previous_program = 0;
GLint previous_active_texture = GL_TEXTURE0;
GLint previous_array_buffer = 0;
GLint previous_unpack_alignment = 4;
GLint previous_viewport[4] = {};
GLint previous_scissor_box[4] = {};
GLfloat previous_clear_color[4] = {};
GLboolean previous_color_mask[4] = {};
glGetIntegerv(GL_CURRENT_PROGRAM, &previous_program);
glGetIntegerv(GL_ACTIVE_TEXTURE, &previous_active_texture);
glGetIntegerv(GL_ARRAY_BUFFER_BINDING, &previous_array_buffer);
glGetIntegerv(GL_UNPACK_ALIGNMENT, &previous_unpack_alignment);
glGetIntegerv(GL_VIEWPORT, previous_viewport);
glGetIntegerv(GL_SCISSOR_BOX, previous_scissor_box);
glGetFloatv(GL_COLOR_CLEAR_VALUE, previous_clear_color);
glGetBooleanv(GL_COLOR_WRITEMASK, previous_color_mask);
const bool blend_enabled = IsEnabled(GL_BLEND);
const bool cull_enabled = IsEnabled(GL_CULL_FACE);
const bool depth_enabled = IsEnabled(GL_DEPTH_TEST);
const bool scissor_enabled = IsEnabled(GL_SCISSOR_TEST);
const bool stencil_enabled = IsEnabled(GL_STENCIL_TEST);
impl_->gl.active_texture(GL_TEXTURE0);
GLint previous_texture_0 = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture_0);
impl_->gl.active_texture(GL_TEXTURE1);
GLint previous_texture_1 = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture_1);
const auto position_state = CaptureVertexAttrib(
impl_->gl, static_cast<GLuint>(impl_->position_location));
const auto texcoord_state = CaptureVertexAttrib(
impl_->gl, static_cast<GLuint>(impl_->texcoord_location));
glDisable(GL_BLEND);
glDisable(GL_CULL_FACE);
glDisable(GL_DEPTH_TEST);
glDisable(GL_SCISSOR_TEST);
glDisable(GL_STENCIL_TEST);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glViewport(0, 0, target_width, target_height);
// The Slint items above this underlay are transparent around the custom
// window corners. Keep those pixels transparent so the compositor can show
// the desktop instead of an opaque black framebuffer clear.
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT);
bool upload_succeeded = true;
if (slot_index != kFrameSlotCount) {
while (glGetError() != GL_NO_ERROR) {
}
const size_t y_size = static_cast<size_t>(source_width) * source_height;
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
impl_->gl.active_texture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, impl_->y_texture);
impl_->gl.active_texture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, impl_->uv_texture);
if (impl_->texture_width != source_width ||
impl_->texture_height != source_height) {
impl_->gl.active_texture(GL_TEXTURE0);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, source_width, source_height,
0, GL_LUMINANCE, GL_UNSIGNED_BYTE, frame_data);
impl_->gl.active_texture(GL_TEXTURE1);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE_ALPHA, source_width / 2,
source_height / 2, 0, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE,
frame_data + y_size);
} else {
impl_->gl.active_texture(GL_TEXTURE0);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, source_width, source_height,
GL_LUMINANCE, GL_UNSIGNED_BYTE, frame_data);
impl_->gl.active_texture(GL_TEXTURE1);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, source_width / 2,
source_height / 2, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE,
frame_data + y_size);
}
const GLenum upload_error = glGetError();
upload_succeeded = upload_error == GL_NO_ERROR;
if (upload_succeeded) {
impl_->texture_width = source_width;
impl_->texture_height = source_height;
impl_->uploaded_stream.assign(remote_id);
impl_->uploaded_sequence = sequence;
} else {
LOG_WARN("OpenGL NV12 texture upload failed, error={}", upload_error);
}
}
{
std::lock_guard lock(impl_->frames->mutex);
if (slot_index != kFrameSlotCount) {
auto &slot = impl_->frames->slots[slot_index];
if (slot.sequence == sequence && slot.use == SlotUse::uploading) {
slot.use = SlotUse::available;
}
}
}
const int content_y = std::clamp(top_inset_pixels, 0, target_height);
const int content_height = target_height - content_y;
const bool has_video =
upload_succeeded && impl_->uploaded_stream == remote_id &&
impl_->texture_width > 0 && impl_->texture_height > 0 &&
content_height > 0;
int video_x = 0;
int video_y = content_y;
int video_width = target_width;
int video_height = content_height;
if (has_video) {
source_width = impl_->texture_width;
source_height = impl_->texture_height;
sequence = impl_->uploaded_sequence;
const double source_aspect =
static_cast<double>(source_width) / source_height;
const double target_aspect =
static_cast<double>(target_width) / content_height;
if (source_aspect > target_aspect) {
video_height =
std::max(1, static_cast<int>(target_width / source_aspect + 0.5));
video_y = content_y + (content_height - video_height) / 2;
} else {
video_width =
std::max(1, static_cast<int>(content_height * source_aspect + 0.5));
video_x = (target_width - video_width) / 2;
}
}
while (glGetError() != GL_NO_ERROR) {
}
impl_->gl.use_program(impl_->program);
impl_->gl.active_texture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, impl_->y_texture);
impl_->gl.uniform_1i(impl_->y_texture_location, 0);
impl_->gl.active_texture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, impl_->uv_texture);
impl_->gl.uniform_1i(impl_->uv_texture_location, 1);
impl_->gl.uniform_2f(impl_->target_size_location,
static_cast<GLfloat>(target_width),
static_cast<GLfloat>(target_height));
impl_->gl.uniform_4f(
impl_->video_rect_location, static_cast<GLfloat>(video_x),
static_cast<GLfloat>(video_y), static_cast<GLfloat>(video_width),
static_cast<GLfloat>(video_height));
impl_->gl.uniform_1f(
impl_->corner_radius_location,
static_cast<GLfloat>(std::clamp(
corner_radius_pixels, 0, std::min(target_width, target_height) / 2)));
impl_->gl.uniform_1f(impl_->video_enabled_location, has_video ? 1.0f : 0.0f);
impl_->gl.bind_buffer(GL_ARRAY_BUFFER, impl_->vertex_buffer);
impl_->gl.vertex_attrib_pointer(
static_cast<GLuint>(impl_->position_location), 2, GL_FLOAT, GL_FALSE,
4 * static_cast<GLsizei>(sizeof(GLfloat)), nullptr);
impl_->gl.vertex_attrib_pointer(
static_cast<GLuint>(impl_->texcoord_location), 2, GL_FLOAT, GL_FALSE,
4 * static_cast<GLsizei>(sizeof(GLfloat)),
reinterpret_cast<const void *>(2 * sizeof(GLfloat)));
impl_->gl.enable_vertex_attrib_array(
static_cast<GLuint>(impl_->position_location));
impl_->gl.enable_vertex_attrib_array(
static_cast<GLuint>(impl_->texcoord_location));
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
const GLenum draw_error = glGetError();
RenderOutcome outcome{
!upload_succeeded || draw_error != GL_NO_ERROR
? RenderResult::failed
: has_video ? RenderResult::rendered : RenderResult::empty,
has_video ? source_width : 0, has_video ? source_height : 0,
has_video ? sequence : 0};
RestoreVertexAttrib(impl_->gl, static_cast<GLuint>(impl_->position_location),
position_state);
RestoreVertexAttrib(impl_->gl, static_cast<GLuint>(impl_->texcoord_location),
texcoord_state);
impl_->gl.bind_buffer(GL_ARRAY_BUFFER,
static_cast<GLuint>(previous_array_buffer));
impl_->gl.active_texture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture_0));
impl_->gl.active_texture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture_1));
impl_->gl.active_texture(static_cast<GLenum>(previous_active_texture));
impl_->gl.use_program(static_cast<GLuint>(previous_program));
glPixelStorei(GL_UNPACK_ALIGNMENT, previous_unpack_alignment);
glViewport(previous_viewport[0], previous_viewport[1], previous_viewport[2],
previous_viewport[3]);
glScissor(previous_scissor_box[0], previous_scissor_box[1],
previous_scissor_box[2], previous_scissor_box[3]);
glClearColor(previous_clear_color[0], previous_clear_color[1],
previous_clear_color[2], previous_clear_color[3]);
glColorMask(previous_color_mask[0], previous_color_mask[1],
previous_color_mask[2], previous_color_mask[3]);
RestoreCapability(GL_BLEND, blend_enabled);
RestoreCapability(GL_CULL_FACE, cull_enabled);
RestoreCapability(GL_DEPTH_TEST, depth_enabled);
RestoreCapability(GL_SCISSOR_TEST, scissor_enabled);
RestoreCapability(GL_STENCIL_TEST, stencil_enabled);
const GLenum restore_error = glGetError();
if (restore_error != GL_NO_ERROR) {
LOG_WARN("OpenGL NV12 state restoration failed, error={}", restore_error);
if (outcome.result == RenderResult::rendered) {
outcome.result = RenderResult::failed;
}
}
return outcome;
}
bool OpenGlVideoRenderer::CopyLatestNv12(
std::string_view remote_id, std::vector<unsigned char> *output, int *width,
int *height) const {
if (!output || !width || !height) {
return false;
}
std::lock_guard lock(impl_->frames->mutex);
const FrameSlot *newest = nullptr;
for (const auto &slot : impl_->frames->slots) {
if (slot.valid && slot.remote_id == remote_id &&
(!newest || slot.sequence > newest->sequence)) {
newest = &slot;
}
}
if (!newest || newest->bytes.empty()) {
return false;
}
*output = newest->bytes;
*width = newest->width;
*height = newest->height;
return true;
}
} // namespace crossdesk
+70
View File
@@ -0,0 +1,70 @@
#ifndef CROSSDESK_GUI_PLATFORM_OPENGL_VIDEO_RENDERER_H_
#define CROSSDESK_GUI_PLATFORM_OPENGL_VIDEO_RENDERER_H_
#include <cstddef>
#include <cstdint>
#include <memory>
#include <string>
#include <string_view>
#include <vector>
#include "platform/video_renderer.h"
namespace crossdesk {
// NV12 underlay for Slint's FemtoVG OpenGL renderer on Windows and Linux.
// MiniRTC submits tightly packed CPU NV12 frames from its decode callback
// thread. The Slint UI thread uploads the newest frame as Y and UV textures and
// performs color conversion and scaling in a fragment shader before Slint
// draws its overlay.
class OpenGlVideoRenderer final : public VideoRenderer {
public:
OpenGlVideoRenderer();
~OpenGlVideoRenderer() override;
OpenGlVideoRenderer(const OpenGlVideoRenderer &) = delete;
OpenGlVideoRenderer &operator=(const OpenGlVideoRenderer &) = delete;
// These methods must run while Slint's OpenGL context is current, from the
// RenderingSetup and RenderingTeardown notifier states respectively.
bool Setup();
void Teardown();
bool IsReady() const override;
bool IsActive() const override;
// Only the selected stream is queued for upload. Frames for other tabs are
// retained by GuiRuntime at a throttled rate for thumbnail/tab restoration.
bool SetSelectedStream(std::string remote_id) override;
void DiscardStream(std::string_view remote_id) override;
// Thread-safe; called from MiniRTC's decode callback thread.
SubmitResult SubmitNv12(std::string_view remote_id, const uint8_t *data,
size_t size, int width, int height) override;
SubmitResult SubmitCachedNv12(std::string_view remote_id, const uint8_t *data,
size_t size, int width, int height) override;
// Draws below Slint while its OpenGL context is current. All dimensions are
// physical pixels in the window client area. corner_radius_pixels clips the
// underlay to Slint's custom rounded window surface.
RenderOutcome RenderLatest(std::string_view remote_id, int target_width,
int target_height, int top_inset_pixels,
int corner_radius_pixels);
// Used during disconnect cleanup so thumbnail generation does not require a
// second per-frame CPU copy during normal playback.
bool CopyLatestNv12(std::string_view remote_id,
std::vector<unsigned char> *output, int *width,
int *height) const override;
private:
SubmitResult SubmitNv12Internal(std::string_view remote_id,
const uint8_t *data, size_t size, int width,
int height, bool replace_pending);
struct Impl;
std::unique_ptr<Impl> impl_;
};
} // namespace crossdesk
#endif // CROSSDESK_GUI_PLATFORM_OPENGL_VIDEO_RENDERER_H_
+70
View File
@@ -0,0 +1,70 @@
#ifndef CROSSDESK_GUI_PLATFORM_VIDEO_RENDERER_H_
#define CROSSDESK_GUI_PLATFORM_VIDEO_RENDERER_H_
#include <cstddef>
#include <cstdint>
#include <memory>
#include <string>
#include <string_view>
#include <vector>
namespace crossdesk {
// Platform-independent contract for the native stream video renderer.
//
// Frame submission and snapshot methods may be called from MiniRTC callback
// threads. Surface setup, rendering and teardown remain backend-specific and
// are driven by the Slint presentation module on the UI thread.
class VideoRenderer {
public:
enum class SubmitResult {
submitted,
not_selected,
dropped,
failed,
};
enum class RenderResult {
rendered,
idle,
empty,
failed,
};
struct RenderOutcome {
RenderResult result = RenderResult::failed;
int width = 0;
int height = 0;
uint64_t sequence = 0;
};
virtual ~VideoRenderer() = default;
virtual bool IsReady() const = 0;
// True only while the renderer can accept and present native frames. Metal
// additionally requires attachment to an AppKit view; OpenGL requires an
// active Slint rendering context.
virtual bool IsActive() const = 0;
virtual bool SetSelectedStream(std::string remote_id) = 0;
virtual void DiscardStream(std::string_view remote_id) = 0;
virtual SubmitResult SubmitNv12(std::string_view remote_id,
const uint8_t* data, size_t size, int width,
int height) = 0;
virtual SubmitResult SubmitCachedNv12(std::string_view remote_id,
const uint8_t* data, size_t size,
int width, int height) = 0;
virtual bool CopyLatestNv12(std::string_view remote_id,
std::vector<unsigned char>* output, int* width,
int* height) const = 0;
};
// Exactly one factory implementation is linked for the target platform.
std::unique_ptr<VideoRenderer> CreateVideoRenderer();
} // namespace crossdesk
#endif // CROSSDESK_GUI_PLATFORM_VIDEO_RENDERER_H_
@@ -0,0 +1,13 @@
#include "platform/video_renderer.h"
#include <memory>
#include "platform/metal_video_renderer.h"
namespace crossdesk {
std::unique_ptr<VideoRenderer> CreateVideoRenderer() {
return std::make_unique<MacMetalVideoRenderer>();
}
} // namespace crossdesk
@@ -0,0 +1,12 @@
#include <memory>
#include "platform/opengl_video_renderer.h"
#include "platform/video_renderer.h"
namespace crossdesk {
std::unique_ptr<VideoRenderer> CreateVideoRenderer() {
return std::make_unique<OpenGlVideoRenderer>();
}
} // namespace crossdesk
+8 -1
View File
@@ -14,7 +14,14 @@ bool HideDisabledMainWindowZoomButton();
// Configures live resize and replaces the stream window's native Space
// fullscreen action with an immediate, single-window fullscreen transition.
bool ConfigureStreamWindowLiveResize();
// |slint_view| is the current stream window's AppKit NSView*. Passing the
// identity explicitly prevents a closing window with the same title from
// being selected during a later connection.
bool ConfigureStreamWindowLiveResize(void* slint_view);
// Drops the native stream-window reference if it belongs to |slint_view|.
// This must run before the corresponding Slint component is destroyed.
void UnregisterStreamWindow(void* slint_view);
// Returns whether the configured stream window is currently the active AppKit
// key window.
+29 -13
View File
@@ -17,7 +17,8 @@
namespace crossdesk {
namespace {
NSWindow *stream_window = nil;
__weak NSWindow *stream_window = nil;
__weak NSView *stream_slint_view = nil;
CrossDeskStreamFullscreenTarget *stream_fullscreen_target = nil;
bool stream_fullscreen = false;
NSRect stream_restore_frame = NSZeroRect;
@@ -101,23 +102,18 @@ bool HideDisabledMainWindowZoomButton() {
}
}
bool ConfigureStreamWindowLiveResize() {
bool ConfigureStreamWindowLiveResize(void *opaque_slint_view) {
@autoreleasepool {
for (NSWindow *window in [NSApp windows]) {
if (![window.title isEqualToString:@"CrossDesk"]) {
continue;
NSView *slint_view = (__bridge NSView *)opaque_slint_view;
NSWindow *window = slint_view.window;
if (window == nil) {
return false;
}
NSButton *zoom_button =
[window standardWindowButton:NSWindowZoomButton];
// The stream window is resizable and therefore keeps an enabled native
// zoom button. The fixed-size main window's zoom button is disabled.
if (zoom_button == nil || !zoom_button.enabled) {
continue;
}
NSView *content_view = window.contentView;
if (content_view == nil) {
if (zoom_button == nil || !zoom_button.enabled || content_view == nil) {
return false;
}
@@ -129,6 +125,7 @@ bool ConfigureStreamWindowLiveResize() {
content_view.layerContentsPlacement =
NSViewLayerContentsPlacementScaleProportionallyToFit;
stream_window = window;
stream_slint_view = slint_view;
// Native Space fullscreen cross-fades an AppKit source snapshot over a
// separately rendered destination window. Disable that path for the
// stream window and route the green button to the immediate transition.
@@ -142,7 +139,26 @@ bool ConfigureStreamWindowLiveResize() {
content_view.needsDisplay = YES;
return true;
}
return false;
}
void UnregisterStreamWindow(void *opaque_slint_view) {
@autoreleasepool {
NSView *slint_view = (__bridge NSView *)opaque_slint_view;
if (slint_view == nil || slint_view != stream_slint_view) {
return;
}
stream_window = nil;
stream_slint_view = nil;
stream_fullscreen = false;
stream_restore_frame = NSZeroRect;
stream_restore_style_mask = NSWindowStyleMaskTitled;
stream_restore_title_visibility = NSWindowTitleVisible;
stream_restore_titlebar_transparent = NO;
stream_restore_movable = YES;
stream_restore_close_hidden = NO;
stream_restore_miniaturize_hidden = NO;
stream_restore_zoom_hidden = NO;
stream_restore_presentation_options = NSApplicationPresentationDefault;
}
}
+502
View File
@@ -0,0 +1,502 @@
#include "rendering/slint_video_presenter.h"
#include <libyuv.h>
#include <slint.h>
#include <algorithm>
#include <cmath>
#include <mutex>
#include <unordered_map>
#include <utility>
#include "nv12_scaler.h"
#include "platform/video_renderer.h"
#include "rd_log.h"
#if defined(__APPLE__)
#include "platform/metal_video_renderer.h"
#else
#if defined(_WIN32)
#include <windows.h>
#endif
#include <GL/gl.h>
#include "platform/opengl_video_renderer.h"
#endif
namespace crossdesk {
namespace {
#if !defined(__APPLE__)
constexpr GLint kGlClampToEdge = 0x812F;
constexpr float kStreamWindowCornerRadius = 12.0f;
#else
constexpr int kMetalAttachmentAttemptLimit = 30;
#endif
struct VideoRenderSize {
int width = 0;
int height = 0;
bool operator==(const VideoRenderSize&) const = default;
};
VideoRenderSize FitVideoToRenderArea(int source_width, int source_height,
int area_width, int area_height) {
if (source_width <= 0 || source_height <= 0 || area_width <= 0 ||
area_height <= 0) {
return {};
}
const double scale =
std::min({1.0, static_cast<double>(area_width) / source_width,
static_cast<double>(area_height) / source_height});
int width = std::max(2, static_cast<int>(std::floor(source_width * scale)));
int height = std::max(2, static_cast<int>(std::floor(source_height * scale)));
width &= ~1;
height &= ~1;
return {std::min(width, source_width & ~1),
std::min(height, source_height & ~1)};
}
} // namespace
struct SlintVideoPresenter::Impl {
explicit Impl(VideoRenderer& renderer) : renderer(renderer) {}
VideoRenderer& renderer;
slint::ComponentHandle<ui::StreamWindow>* stream = nullptr;
StateProvider state_provider;
DirtyCallback dirty_callback;
std::unordered_map<std::string, uint64_t> displayed_frame_sequence;
std::unordered_map<std::string, uint64_t> seeded_frame_sequence;
std::vector<uint8_t> scaled_video_frame;
std::vector<uint8_t> scaled_video_frame_scratch;
#if defined(__APPLE__)
int attachment_attempts = 0;
bool skip_attachment_once = false;
#else
std::mutex gl_mutex;
std::vector<uint8_t> gl_conversion_frame;
std::vector<uint8_t> gl_pending_frame;
uint32_t gl_texture = 0;
VideoRenderSize gl_texture_size;
VideoRenderSize gl_image_size;
VideoRenderSize gl_pending_size;
bool gl_pending_dirty = false;
#endif
SurfaceState CurrentState() const {
return state_provider ? state_provider() : SurfaceState{};
}
void MarkDirty() const {
if (dirty_callback) {
dirty_callback();
}
}
bool MarkPresented(const std::string& remote_id, uint64_t sequence) {
if (remote_id.empty() || sequence == 0 ||
displayed_frame_sequence[remote_id] == sequence) {
return false;
}
displayed_frame_sequence[remote_id] = sequence;
return true;
}
#if !defined(__APPLE__)
void ConfigureOpenGlNotifier() {
if (!stream) {
return;
}
const auto error = (*stream)->window().set_rendering_notifier(
[this](slint::RenderingState state, slint::GraphicsAPI graphics_api) {
if (graphics_api != slint::GraphicsAPI::NativeOpenGL) {
return;
}
std::lock_guard lock(gl_mutex);
auto* opengl_renderer =
dynamic_cast<OpenGlVideoRenderer*>(&renderer);
if (state == slint::RenderingState::RenderingSetup) {
GLuint texture = 0;
glGenTextures(1, &texture);
if (texture == 0) {
return;
}
GLint previous_texture = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, kGlClampToEdge);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, kGlClampToEdge);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture));
gl_texture = texture;
if (opengl_renderer && opengl_renderer->Setup()) {
const std::string selected_stream =
CurrentState().selected_stream;
opengl_renderer->SetSelectedStream(selected_stream);
seeded_frame_sequence.erase(selected_stream);
MarkDirty();
}
return;
}
if (state == slint::RenderingState::BeforeRendering && stream &&
opengl_renderer && opengl_renderer->IsReady() &&
(*stream)->get_native_video_enabled()) {
const SurfaceState surface = CurrentState();
const auto window_size = (*stream)->window().size();
const float scale_factor =
std::max(1.0f, (*stream)->window().scale_factor());
const int top_inset =
!surface.fullscreen && surface.tab_count > 1
? static_cast<int>(std::lround(30.0f * scale_factor))
: 0;
const bool rounded_window = (*stream)->get_custom_titlebar() &&
!surface.fullscreen &&
!(*stream)->window().is_maximized();
const int corner_radius =
rounded_window ? static_cast<int>(std::lround(
kStreamWindowCornerRadius * scale_factor))
: 0;
opengl_renderer->RenderLatest(
surface.selected_stream, static_cast<int>(window_size.width),
static_cast<int>(window_size.height), top_inset, corner_radius);
return;
}
if (state == slint::RenderingState::BeforeRendering &&
gl_texture != 0 && gl_pending_dirty &&
!gl_pending_frame.empty()) {
GLint previous_texture = 0;
GLint previous_unpack_alignment = 0;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &previous_texture);
glGetIntegerv(GL_UNPACK_ALIGNMENT, &previous_unpack_alignment);
glBindTexture(GL_TEXTURE_2D, gl_texture);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
if (gl_texture_size != gl_pending_size) {
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, gl_pending_size.width,
gl_pending_size.height, 0, GL_RGBA, GL_UNSIGNED_BYTE,
gl_pending_frame.data());
gl_texture_size = gl_pending_size;
} else {
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, gl_pending_size.width,
gl_pending_size.height, GL_RGBA, GL_UNSIGNED_BYTE,
gl_pending_frame.data());
}
glPixelStorei(GL_UNPACK_ALIGNMENT, previous_unpack_alignment);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(previous_texture));
gl_pending_dirty = false;
return;
}
if (state == slint::RenderingState::RenderingTeardown) {
if (opengl_renderer) {
opengl_renderer->Teardown();
}
if (gl_texture != 0) {
const GLuint texture = gl_texture;
glDeleteTextures(1, &texture);
}
gl_texture = 0;
gl_texture_size = {};
gl_image_size = {};
gl_pending_size = {};
gl_pending_dirty = false;
gl_pending_frame.clear();
}
});
if (error.has_value()) {
LOG_WARN("Slint OpenGL video renderer unavailable, using pixel buffers");
} else {
renderer.SetSelectedStream(CurrentState().selected_stream);
}
}
#endif
};
SlintVideoPresenter::SlintVideoPresenter(VideoRenderer& renderer)
: impl_(std::make_unique<Impl>(renderer)) {}
SlintVideoPresenter::~SlintVideoPresenter() = default;
void SlintVideoPresenter::PrepareWindow(
slint::ComponentHandle<ui::StreamWindow>& stream,
StateProvider state_provider, DirtyCallback dirty_callback) {
impl_->stream = &stream;
impl_->state_provider = std::move(state_provider);
impl_->dirty_callback = std::move(dirty_callback);
#if defined(__APPLE__)
stream->set_native_video_enabled(false);
impl_->attachment_attempts = kMetalAttachmentAttemptLimit;
impl_->skip_attachment_once = true;
#else
impl_->ConfigureOpenGlNotifier();
#endif
}
bool SlintVideoPresenter::EnsureAttached() {
#if defined(__APPLE__)
if (!impl_->stream || impl_->attachment_attempts <= 0) {
return false;
}
if (impl_->skip_attachment_once) {
impl_->skip_attachment_once = false;
return false;
}
auto* metal_renderer =
dynamic_cast<MacMetalVideoRenderer*>(&impl_->renderer);
if (!metal_renderer) {
impl_->attachment_attempts = 0;
(*impl_->stream)->set_native_video_enabled(false);
return false;
}
if (!metal_renderer->IsReady()) {
impl_->attachment_attempts = 0;
(*impl_->stream)->set_native_video_enabled(false);
return false;
}
if (metal_renderer->IsAttached()) {
impl_->attachment_attempts = 0;
return false;
}
if (void* view = (*impl_->stream)->window().appkit_view()) {
(*impl_->stream)->set_native_video_enabled(true);
if (metal_renderer->Attach(view)) {
impl_->attachment_attempts = 0;
const SurfaceState surface = impl_->CurrentState();
impl_->displayed_frame_sequence.erase(surface.selected_stream);
impl_->seeded_frame_sequence.erase(surface.selected_stream);
if (metal_renderer->SetSelectedStream(surface.selected_stream)) {
impl_->MarkDirty();
}
return true;
}
(*impl_->stream)->set_native_video_enabled(false);
}
if (--impl_->attachment_attempts == 0) {
(*impl_->stream)->set_native_video_enabled(false);
LOG_WARN("Unable to attach the Metal video surface; using CPU rendering");
}
#endif
return false;
}
bool SlintVideoPresenter::NeedsRedraw() const {
#if defined(__APPLE__)
const auto* metal_renderer =
dynamic_cast<const MacMetalVideoRenderer*>(&impl_->renderer);
return metal_renderer && metal_renderer->IsActive() &&
metal_renderer->NeedsSurfaceRedraw();
#else
return false;
#endif
}
bool SlintVideoPresenter::SelectStream(std::string remote_id) {
const bool changed = impl_->renderer.SetSelectedStream(remote_id);
if (changed && !remote_id.empty()) {
impl_->seeded_frame_sequence.erase(remote_id);
}
return changed;
}
SlintVideoPresenter::PresentResult SlintVideoPresenter::Present(
const Frame& frame) {
PresentResult result;
if (!impl_->stream || frame.remote_id.empty()) {
return result;
}
if (impl_->renderer.IsActive() && frame.nv12 && frame.width > 0 &&
frame.height > 0 && frame.sequence > 0 &&
impl_->seeded_frame_sequence[frame.remote_id] != frame.sequence) {
const auto submit_result = impl_->renderer.SubmitCachedNv12(
frame.remote_id, frame.nv12->data(), frame.nv12->size(), frame.width,
frame.height);
if (submit_result == VideoRenderer::SubmitResult::submitted ||
submit_result == VideoRenderer::SubmitResult::dropped) {
impl_->seeded_frame_sequence[frame.remote_id] = frame.sequence;
}
}
#if defined(__APPLE__)
auto* metal_renderer =
dynamic_cast<MacMetalVideoRenderer*>(&impl_->renderer);
if (metal_renderer && metal_renderer->IsActive()) {
const SurfaceState surface = impl_->CurrentState();
const double top_inset =
!surface.fullscreen && surface.tab_count > 1 ? 30.0 : 0.0;
const auto outcome = metal_renderer->RenderLatest(
frame.remote_id, top_inset, !surface.fullscreen);
if (outcome.result == VideoRenderer::RenderResult::rendered) {
(*impl_->stream)->set_has_frame(true);
(*impl_->stream)->set_receiving_text("");
result.width = outcome.width;
result.height = outcome.height;
result.new_frame =
impl_->MarkPresented(frame.remote_id, outcome.sequence);
} else if (outcome.result == VideoRenderer::RenderResult::empty) {
(*impl_->stream)->set_has_frame(false);
}
return result;
}
#else
auto* opengl_renderer =
dynamic_cast<OpenGlVideoRenderer*>(&impl_->renderer);
if (opengl_renderer && opengl_renderer->IsReady()) {
opengl_renderer->SetSelectedStream(frame.remote_id);
if (!(*impl_->stream)->get_native_video_enabled()) {
(*impl_->stream)->set_native_video_enabled(true);
}
const bool has_frame =
frame.width > 0 && frame.height > 0 && frame.sequence > 0;
(*impl_->stream)->set_has_frame(has_frame);
if (has_frame) {
(*impl_->stream)->set_receiving_text("");
result.width = frame.width;
result.height = frame.height;
result.new_frame = impl_->MarkPresented(frame.remote_id, frame.sequence);
}
(*impl_->stream)->window().request_redraw();
return result;
}
if ((*impl_->stream)->get_native_video_enabled()) {
(*impl_->stream)->set_native_video_enabled(false);
}
#endif
const size_t nv12_size =
static_cast<size_t>(frame.width) * frame.height * 3 / 2;
if (!frame.nv12 || frame.width <= 0 || frame.height <= 0 ||
frame.nv12->size() < nv12_size) {
(*impl_->stream)->set_has_frame(false);
return result;
}
if (impl_->displayed_frame_sequence[frame.remote_id] == frame.sequence) {
return result;
}
const auto window_size = (*impl_->stream)->window().size();
const VideoRenderSize render_size = FitVideoToRenderArea(
frame.width, frame.height, static_cast<int>(window_size.width),
static_cast<int>(window_size.height));
const uint8_t* y_plane = frame.nv12->data();
const uint8_t* uv_plane =
frame.nv12->data() + static_cast<size_t>(frame.width) * frame.height;
int output_width = frame.width;
int output_height = frame.height;
if (render_size.width > 0 && render_size.height > 0 &&
(render_size.width != frame.width ||
render_size.height != frame.height)) {
const size_t scaled_nv12_size =
static_cast<size_t>(render_size.width) * render_size.height * 3 / 2;
impl_->scaled_video_frame.resize(scaled_nv12_size);
uint8_t* scaled_y = impl_->scaled_video_frame.data();
uint8_t* scaled_uv =
scaled_y + static_cast<size_t>(render_size.width) * render_size.height;
if (ScaleNv12ViaI420(y_plane, frame.width, uv_plane, frame.width,
frame.width, frame.height, scaled_y, render_size.width,
scaled_uv, render_size.width, render_size.width,
render_size.height, libyuv::kFilterBox,
&impl_->scaled_video_frame_scratch) == 0) {
y_plane = scaled_y;
uv_plane = scaled_uv;
output_width = render_size.width;
output_height = render_size.height;
}
}
#if !defined(__APPLE__)
uint32_t texture_id = 0;
{
std::lock_guard lock(impl_->gl_mutex);
texture_id = impl_->gl_texture;
}
if (texture_id != 0) {
impl_->gl_conversion_frame.resize(static_cast<size_t>(output_width) *
output_height * 4);
if (libyuv::NV12ToABGR(y_plane, output_width, uv_plane, output_width,
impl_->gl_conversion_frame.data(), output_width * 4,
output_width, output_height) != 0) {
return result;
}
{
std::lock_guard lock(impl_->gl_mutex);
if (impl_->gl_texture == 0) {
return result;
}
texture_id = impl_->gl_texture;
impl_->gl_pending_frame.swap(impl_->gl_conversion_frame);
impl_->gl_pending_size = {output_width, output_height};
impl_->gl_pending_dirty = true;
}
const VideoRenderSize output_size{output_width, output_height};
if (impl_->gl_image_size != output_size) {
(*impl_->stream)
->set_frame(slint::Image::create_from_borrowed_gl_2d_rgba_texture(
texture_id,
slint::Size<uint32_t>{static_cast<uint32_t>(output_width),
static_cast<uint32_t>(output_height)}));
impl_->gl_image_size = output_size;
}
(*impl_->stream)->set_has_frame(true);
(*impl_->stream)->set_receiving_text("");
(*impl_->stream)->window().request_redraw();
result.width = frame.width;
result.height = frame.height;
result.new_frame = impl_->MarkPresented(frame.remote_id, frame.sequence);
return result;
}
#endif
slint::SharedPixelBuffer<slint::Rgb8Pixel> pixels(output_width,
output_height);
if (libyuv::NV12ToRAW(y_plane, output_width, uv_plane, output_width,
reinterpret_cast<uint8_t*>(pixels.begin()),
output_width * 3, output_width, output_height) != 0) {
return result;
}
(*impl_->stream)->set_frame(slint::Image(std::move(pixels)));
(*impl_->stream)->set_has_frame(true);
(*impl_->stream)->set_receiving_text("");
result.width = frame.width;
result.height = frame.height;
result.new_frame = impl_->MarkPresented(frame.remote_id, frame.sequence);
return result;
}
void SlintVideoPresenter::ForgetStream(const std::string& remote_id) {
impl_->displayed_frame_sequence.erase(remote_id);
impl_->seeded_frame_sequence.erase(remote_id);
}
void SlintVideoPresenter::Detach() {
impl_->renderer.SetSelectedStream({});
#if defined(__APPLE__)
if (auto* metal_renderer =
dynamic_cast<MacMetalVideoRenderer*>(&impl_->renderer)) {
metal_renderer->Detach();
}
impl_->attachment_attempts = 0;
impl_->skip_attachment_once = false;
#endif
impl_->stream = nullptr;
impl_->state_provider = {};
impl_->dirty_callback = {};
}
} // namespace crossdesk
+75
View File
@@ -0,0 +1,75 @@
#ifndef CROSSDESK_GUI_RENDERING_SLINT_VIDEO_PRESENTER_H_
#define CROSSDESK_GUI_RENDERING_SLINT_VIDEO_PRESENTER_H_
#include <cstddef>
#include <cstdint>
#include <functional>
#include <memory>
#include <string>
#include <vector>
#include "crossdesk_ui.h"
namespace crossdesk {
class VideoRenderer;
// Owns the bridge between decoded NV12 frames, the selected native graphics
// backend and Slint's StreamWindow. Platform macros and graphics-API lifecycle
// details are intentionally contained in the implementation file.
class SlintVideoPresenter {
public:
struct SurfaceState {
std::string selected_stream;
bool fullscreen = false;
size_t tab_count = 0;
};
struct Frame {
std::string remote_id;
std::shared_ptr<std::vector<unsigned char>> nv12;
int width = 0;
int height = 0;
uint64_t sequence = 0;
};
struct PresentResult {
int width = 0;
int height = 0;
bool new_frame = false;
};
using StateProvider = std::function<SurfaceState()>;
using DirtyCallback = std::function<void()>;
explicit SlintVideoPresenter(VideoRenderer& renderer);
~SlintVideoPresenter();
SlintVideoPresenter(const SlintVideoPresenter&) = delete;
SlintVideoPresenter& operator=(const SlintVideoPresenter&) = delete;
// Called before the window is shown. OpenGL installs Slint's rendering
// notifier here; macOS deliberately defers native view attachment until a
// later UI pass, after AppKit has committed the ordinary window style.
void PrepareWindow(slint::ComponentHandle<ui::StreamWindow>& stream,
StateProvider state_provider,
DirtyCallback dirty_callback);
// Returns true only when a deferred native surface became attached during
// this call, allowing the caller to seed it from a cached frame once.
bool EnsureAttached();
bool NeedsRedraw() const;
bool SelectStream(std::string remote_id);
PresentResult Present(const Frame& frame);
void ForgetStream(const std::string& remote_id);
void Detach();
private:
struct Impl;
std::unique_ptr<Impl> impl_;
};
} // namespace crossdesk
#endif // CROSSDESK_GUI_RENDERING_SLINT_VIDEO_PRESENTER_H_
+30 -2
View File
@@ -8,8 +8,10 @@
#include <thread>
#include <vector>
#include "display_stream_id.h"
#include "localization.h"
#include "platform.h"
#include "platform/video_renderer.h"
#include "rd_log.h"
#include "runtime/gui_runtime.h"
@@ -222,6 +224,21 @@ void GuiRuntime::CloseRemoteSession(std::shared_ptr<RemoteSession> props) {
frame_snapshot = props->front_frame_;
video_width = props->video_width_;
video_height = props->video_height_;
if ((!frame_snapshot || frame_snapshot->empty()) &&
props->thumbnail_frame_ && !props->thumbnail_frame_->empty()) {
frame_snapshot = props->thumbnail_frame_;
video_width = props->thumbnail_width_;
video_height = props->thumbnail_height_;
}
}
auto* native_renderer = video_renderer_.get();
if ((!frame_snapshot || frame_snapshot->empty()) && native_renderer) {
auto native_snapshot = std::make_shared<std::vector<unsigned char>>();
if (native_renderer->CopyLatestNv12(props->remote_id_,
native_snapshot.get(), &video_width,
&video_height)) {
frame_snapshot = std::move(native_snapshot);
}
}
if (frame_snapshot && !frame_snapshot->empty() && video_width > 0 &&
@@ -246,6 +263,11 @@ void GuiRuntime::CloseRemoteSession(std::shared_ptr<RemoteSession> props) {
}
}
if (native_renderer) {
native_renderer->DiscardStream(props->remote_id_);
video_frame_dirty_.store(true, std::memory_order_release);
}
if (props->peer_) {
LOG_INFO("[{}] Leave connection [{}]", props->local_id_, props->remote_id_);
LeaveConnection(props->peer_, props->remote_id_.c_str());
@@ -306,6 +328,10 @@ void GuiRuntime::ResetRemoteSessionResources(
std::lock_guard<std::mutex> lock(props->video_frame_mutex_);
props->front_frame_.reset();
props->back_frame_.reset();
props->thumbnail_frame_.reset();
props->thumbnail_width_ = 0;
props->thumbnail_height_ = 0;
props->background_snapshot_time_ = {};
props->video_width_ = 0;
props->video_height_ = 0;
props->video_size_ = 0;
@@ -381,8 +407,10 @@ int GuiRuntime::ConnectTo(const std::string& remote_id, const char* password,
return -1;
}
for (const auto& display_info : devices_.display_info_list()) {
AddVideoStream(props->peer_, display_info.name.c_str());
const auto& displays = devices_.display_info_list();
for (size_t index = 0; index < displays.size(); ++index) {
const std::string stream_id = MakeDisplayStreamId(index);
AddVideoStream(props->peer_, stream_id.c_str());
}
AddAudioStream(props->peer_, props->audio_label_.c_str());
AddDataStream(props->peer_, props->data_label_.c_str(), false);
+59 -8
View File
@@ -10,7 +10,9 @@
#include <thread>
#include <vector>
#include "display_stream_id.h"
#include "localization.h"
#include "platform/video_renderer.h"
#include "rd_log.h"
namespace crossdesk {
@@ -57,9 +59,44 @@ int GuiRuntime::CreateConnectionPeer() {
signal_server_ip = config_center_->GetDefaultServerHost();
signal_server_port = config_center_->GetDefaultSignalServerPort();
coturn_server_port = config_center_->GetDefaultCoturnServerPort();
settings_.ActivateCachedPublicIdentity();
params_.user_id = client_id_with_password_;
}
const bool self_hosted = config_center_->IsSelfHosted();
const std::string pending_identity =
settings_.PendingPasswordChangeIdentity(
self_hosted, signal_server_ip, signal_server_port);
bool try_pending_identity = false;
{
std::lock_guard<std::mutex> lock(password_change_mutex_);
if (pending_identity.empty()) {
credential_recovery_in_progress_ = false;
credential_recovery_attempt_pending_ = false;
credential_recovery_retry_active_ = false;
credential_recovery_promote_pending_ = false;
credential_recovery_clear_pending_ = false;
} else if (!credential_recovery_in_progress_) {
credential_recovery_in_progress_ = true;
credential_recovery_attempt_pending_ = true;
LOG_INFO("Recovering an interrupted password change for [{}]",
client_id_);
}
try_pending_identity = credential_recovery_in_progress_ &&
credential_recovery_attempt_pending_ &&
!pending_identity.empty();
}
const char *active_identity =
self_hosted ? self_hosted_user_id_ : client_id_with_password_;
const std::string login_identity =
try_pending_identity ? pending_identity : std::string(active_identity);
std::memset(connection_login_identity_, 0,
sizeof(connection_login_identity_));
std::strncpy(connection_login_identity_, login_identity.c_str(),
sizeof(connection_login_identity_) - 1);
params_.user_id = connection_login_identity_;
// self hosted server config
strncpy(signal_server_ip_self_, config_center_->GetSignalServerHost().c_str(),
sizeof(signal_server_ip_self_) - 1);
@@ -94,12 +131,11 @@ int GuiRuntime::CreateConnectionPeer() {
sizeof(params_.turn_server_ip) - 1);
params_.turn_server_ip[sizeof(params_.turn_server_ip) - 1] = '\0';
params_.turn_server_port = coturn_server_port;
strncpy((char *)params_.turn_server_username, "crossdesk",
sizeof(params_.turn_server_username) - 1);
params_.turn_server_username[sizeof(params_.turn_server_username) - 1] = '\0';
strncpy((char *)params_.turn_server_password, "crossdeskpw",
sizeof(params_.turn_server_password) - 1);
params_.turn_server_password[sizeof(params_.turn_server_password) - 1] = '\0';
// TURN credentials are issued by the signaling server after login. Keep the
// initial values empty so a reusable static password is never embedded in
// the client binary.
params_.turn_server_username[0] = '\0';
params_.turn_server_password[0] = '\0';
strncpy(params_.log_path, dll_log_path_.c_str(),
sizeof(params_.log_path) - 1);
@@ -111,8 +147,16 @@ int GuiRuntime::CreateConnectionPeer() {
: false;
params_.turn_mode = static_cast<TurnMode>(config_center_->GetTurnMode());
params_.enable_srtp = config_center_->IsEnableSrtp();
params_.video_content_type = VideoContentType::ScreenContent;
params_.video_quality =
static_cast<VideoQuality>(config_center_->GetVideoQuality());
params_.video_frame_rate =
config_center_->GetVideoFrameRate() ==
ConfigCenter::VIDEO_FRAME_RATE::FPS_30
? 30
: 60;
params_.video_degradation_preference =
VideoDegradationPreference::MaintainFrameRate;
params_.on_receive_video_buffer = nullptr;
params_.on_receive_audio_buffer = PeerEventHandler::OnReceiveAudioBuffer;
params_.on_receive_data_buffer = PeerEventHandler::OnReceiveDataBuffer;
@@ -126,6 +170,11 @@ int GuiRuntime::CreateConnectionPeer() {
params_.user_data = &peer_events_;
// The previous peer may have left a terminal status behind. Reset it before
// Init() starts emitting callbacks for the newly selected server.
signal_connected_ = false;
signal_status_ = SignalStatus::SignalConnecting;
peer_ = CreatePeer(&params_);
if (peer_) {
LOG_INFO("Create peer instance [{}] successful", client_id_);
@@ -136,8 +185,10 @@ int GuiRuntime::CreateConnectionPeer() {
}
if (0 == devices_.InitializeScreenCapturer()) {
for (const auto &display_info : devices_.display_info_list()) {
AddVideoStream(peer_, display_info.name.c_str());
const auto &displays = devices_.display_info_list();
for (size_t index = 0; index < displays.size(); ++index) {
const std::string stream_id = MakeDisplayStreamId(index);
AddVideoStream(peer_, stream_id.c_str());
}
AddAudioStream(peer_, audio_label_.c_str());
+5
View File
@@ -1,6 +1,7 @@
#ifndef CROSSDESK_GUI_RUNTIME_H_
#define CROSSDESK_GUI_RUNTIME_H_
#include <atomic>
#include <memory>
#include <string>
@@ -14,6 +15,8 @@
namespace crossdesk {
class VideoRenderer;
// Shared GUI runtime. It owns subsystem controllers and cross-cutting session
// state, but no window lifecycle, ImGui view, or transport callback methods.
class GuiRuntime : protected gui_detail::GuiState {
@@ -81,6 +84,8 @@ class GuiRuntime : protected gui_detail::GuiState {
SettingsManager settings_;
KeyboardController keyboard_;
PeerEventHandler peer_events_;
std::atomic<bool> video_frame_dirty_{false};
std::unique_ptr<VideoRenderer> video_renderer_;
private:
friend class ClipboardController;
+105
View File
@@ -10,11 +10,13 @@
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include "device_controller.h"
#include "file_transfer.h"
#include "localization.h"
#include "platform.h"
#include "platform/video_renderer.h"
#include "rd_log.h"
#include "runtime/gui_runtime.h"
#include "runtime/remote_action_codec.h"
@@ -26,6 +28,31 @@
namespace crossdesk {
void PeerEventHandler::SendClientInfo(PeerPtr* peer,
const std::string& client_id) {
if (!peer) {
return;
}
#if defined(_WIN32)
constexpr const char* kClientPlatform = "windows";
#elif defined(__APPLE__)
constexpr const char* kClientPlatform = "macos";
#elif defined(__linux__)
constexpr const char* kClientPlatform = "linux";
#else
constexpr const char* kClientPlatform = "unknown";
#endif
const nlohmann::json message = {{"type", "client_info"},
{"version", CROSSDESK_VERSION},
{"platform", kClientPlatform}};
const std::string payload = message.dump();
if (SendSignalMessage(peer, payload.data(), payload.size()) != 0) {
LOG_WARN("[{}] failed to report client information", client_id);
}
}
PeerEventHandler::PeerEventHandler(GuiRuntime& owner) : owner_(owner) {}
void PeerEventHandler::OnSignalMessage(const char* message, size_t size,
@@ -85,6 +112,39 @@ void PeerEventHandler::OnSignalMessage(const char* message, size_t size,
}
}
}
} else if (type == "change_password") {
std::lock_guard<std::mutex> lock(runtime->password_change_mutex_);
if (!runtime->password_change_pending_) {
LOG_WARN("Ignore unexpected password change response");
return;
}
if (!j.contains("request_id") || !j["request_id"].is_string() ||
j["request_id"].get<std::string>() !=
runtime->pending_password_change_request_id_) {
LOG_WARN("Ignore password change response with unexpected request id");
return;
}
if (j.contains("user_id") && j["user_id"].is_string()) {
const std::string response_user_id = j["user_id"].get<std::string>();
if (!response_user_id.empty() &&
response_user_id != runtime->client_id_) {
LOG_WARN("Ignore password change response for unexpected id [{}]",
response_user_id);
return;
}
}
runtime->password_change_succeeded_ =
j.contains("status") && j["status"].is_string() &&
j["status"].get<std::string>() == "success";
runtime->password_change_error_ =
j.contains("reason") && j["reason"].is_string()
? j["reason"].get<std::string>()
: "Password change failed";
runtime->password_change_result_uncertain_ = false;
runtime->password_change_result_ready_ = true;
}
}
@@ -105,8 +165,22 @@ void PeerEventHandler::OnSignalStatus(SignalStatus status, const char* user_id,
runtime->signal_connected_ = true;
runtime->need_to_send_recent_connections_ = true;
LOG_INFO("[{}] connected to signal server", client_id);
SendClientInfo(runtime->peer_, client_id);
std::lock_guard<std::mutex> lock(runtime->password_change_mutex_);
if (runtime->credential_recovery_in_progress_) {
if (runtime->credential_recovery_attempt_pending_) {
runtime->credential_recovery_promote_pending_ = true;
} else {
runtime->credential_recovery_clear_pending_ = true;
}
}
} else if (SignalStatus::SignalFailed == status) {
runtime->signal_connected_ = false;
std::lock_guard<std::mutex> lock(runtime->password_change_mutex_);
if (runtime->credential_recovery_in_progress_ &&
runtime->credential_recovery_attempt_pending_) {
runtime->credential_recovery_retry_active_ = true;
}
} else if (SignalStatus::SignalClosed == status) {
runtime->signal_connected_ = false;
} else if (SignalStatus::SignalReconnecting == status) {
@@ -134,6 +208,7 @@ void PeerEventHandler::OnSignalStatus(SignalStatus status, const char* user_id,
} else if (SignalStatus::SignalConnected == status) {
props->signal_connected_ = true;
LOG_INFO("[{}] connected to signal server", remote_id);
SendClientInfo(props->peer_, client_id);
} else if (SignalStatus::SignalFailed == status) {
props->signal_connected_ = false;
} else if (SignalStatus::SignalClosed == status) {
@@ -239,16 +314,46 @@ void PeerEventHandler::OnConnectionStatus(ConnectionStatus status,
props->enable_mouse_control_ = false;
runtime->ResetRemoteServiceStatus(*props);
std::shared_ptr<std::vector<unsigned char>> native_snapshot;
int native_snapshot_width = 0;
int native_snapshot_height = 0;
bool needs_native_snapshot = false;
{
std::lock_guard<std::mutex> lock(props->video_frame_mutex_);
needs_native_snapshot =
!props->thumbnail_frame_ || props->thumbnail_frame_->empty();
}
if (needs_native_snapshot) {
auto* native_renderer = runtime->video_renderer_.get();
auto snapshot = std::make_shared<std::vector<unsigned char>>();
if (native_renderer &&
native_renderer->CopyLatestNv12(remote_id, snapshot.get(),
&native_snapshot_width,
&native_snapshot_height)) {
native_snapshot = std::move(snapshot);
}
}
{
std::lock_guard<std::mutex> lock(props->video_frame_mutex_);
props->front_frame_.reset();
props->back_frame_.reset();
if (native_snapshot &&
(!props->thumbnail_frame_ || props->thumbnail_frame_->empty())) {
props->thumbnail_frame_ = std::move(native_snapshot);
props->thumbnail_width_ = native_snapshot_width;
props->thumbnail_height_ = native_snapshot_height;
}
props->video_width_ = 0;
props->video_height_ = 0;
props->video_size_ = 0;
props->render_rect_dirty_ = true;
props->stream_cleanup_pending_ = true;
}
auto* native_renderer = runtime->video_renderer_.get();
if (native_renderer) {
native_renderer->DiscardStream(remote_id);
runtime->video_frame_dirty_.store(true, std::memory_order_release);
}
runtime->focus_on_stream_window_ = false;
+3
View File
@@ -2,6 +2,7 @@
#define CROSSDESK_GUI_PEER_EVENT_HANDLER_H_
#include <cstddef>
#include <string>
#include "minirtc.h"
@@ -39,6 +40,8 @@ public:
void *user_data);
private:
static void SendClientInfo(PeerPtr *peer, const std::string &client_id);
GuiRuntime &owner_;
};
+69 -32
View File
@@ -1,30 +1,20 @@
#include "runtime/peer_event_handler.h"
#include <algorithm>
#include <chrono>
#include <cstring>
#include <filesystem>
#include <memory>
#include <mutex>
#include <shared_mutex>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include "device_controller.h"
#include "file_transfer.h"
#include "localization.h"
#include "platform.h"
#include "rd_log.h"
#include "platform/video_renderer.h"
#include "runtime/gui_runtime.h"
#include "runtime/remote_action_codec.h"
#if _WIN32
#include "interactive_state.h"
#include "service_host.h"
#endif
#include "runtime/peer_event_handler.h"
namespace crossdesk {
namespace {
constexpr auto kBackgroundSnapshotInterval = std::chrono::seconds(1);
} // namespace
void PeerEventHandler::OnReceiveVideoBuffer(
const XVideoFrame* video_frame, const char* user_id, size_t user_id_size,
@@ -45,10 +35,61 @@ void PeerEventHandler::OnReceiveVideoBuffer(
runtime->remote_sessions_.find(remote_id)->second.get();
if (props->connection_established_) {
bool background_snapshot_only = false;
auto* native_renderer = runtime->video_renderer_.get();
if (native_renderer && native_renderer->IsActive()) {
const auto submit_result = native_renderer->SubmitNv12(
remote_id, reinterpret_cast<const uint8_t*>(video_frame->data),
video_frame->size, video_frame->width, video_frame->height);
if (submit_result == VideoRenderer::SubmitResult::submitted) {
std::lock_guard<std::mutex> lock(props->video_frame_mutex_);
const bool size_changed = (props->video_width_ != video_frame->width) ||
(props->video_height_ != video_frame->height);
if (size_changed) {
props->render_rect_dirty_ = true;
}
props->video_width_ = video_frame->width;
props->video_height_ = video_frame->height;
props->video_size_ = video_frame->size;
// Once the native renderer owns the current stream, do not leave an
// older CPU frame ahead of the renderer's close snapshot.
props->front_frame_.reset();
props->back_frame_.reset();
props->thumbnail_frame_.reset();
props->thumbnail_width_ = 0;
props->thumbnail_height_ = 0;
props->background_snapshot_time_ = {};
++props->video_frame_sequence_;
props->streaming_ = true;
runtime->video_frame_dirty_.store(true, std::memory_order_release);
return;
}
if (submit_result == VideoRenderer::SubmitResult::dropped ||
submit_result == VideoRenderer::SubmitResult::failed) {
// Keep presenting the last native frame. A later decoded frame can
// reuse the renderer without changing ownership mid-window.
props->streaming_ = true;
return;
}
if (submit_result == VideoRenderer::SubmitResult::not_selected) {
background_snapshot_only = true;
}
}
{
std::lock_guard<std::mutex> lock(props->video_frame_mutex_);
const auto now = std::chrono::steady_clock::now();
if (background_snapshot_only && props->thumbnail_frame_ &&
!props->thumbnail_frame_->empty() &&
props->background_snapshot_time_ !=
std::chrono::steady_clock::time_point{} &&
now - props->background_snapshot_time_ <
kBackgroundSnapshotInterval) {
props->streaming_ = true;
return;
}
if (!props->back_frame_) {
// Allocate a third buffer only while the UI still owns the old snapshot.
if (!props->back_frame_ || props->back_frame_.use_count() != 1) {
props->back_frame_ =
std::make_shared<std::vector<unsigned char>>(video_frame->size);
}
@@ -70,24 +111,20 @@ void PeerEventHandler::OnReceiveVideoBuffer(
props->video_size_ = video_frame->size;
props->front_frame_.swap(props->back_frame_);
props->thumbnail_frame_ = props->front_frame_;
props->thumbnail_width_ = video_frame->width;
props->thumbnail_height_ = video_frame->height;
if (background_snapshot_only) {
props->background_snapshot_time_ = now;
}
++props->video_frame_sequence_;
}
props->streaming_ = true;
if (props->net_traffic_stats_button_pressed_) {
props->frame_count_++;
auto now = std::chrono::steady_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
now - props->last_time_)
.count();
if (elapsed >= 1000) {
props->fps_ = props->frame_count_ * 1000 / elapsed;
props->frame_count_ = 0;
props->last_time_ = now;
}
if (background_snapshot_only) {
return;
}
runtime->video_frame_dirty_.store(true, std::memory_order_release);
}
}
+6
View File
@@ -110,6 +110,12 @@ struct RemoteSession {
std::mutex video_frame_mutex_;
std::shared_ptr<std::vector<unsigned char>> front_frame_;
std::shared_ptr<std::vector<unsigned char>> back_frame_;
// Retains the most recent background CPU frame across disconnect cleanup
// so closing an unselected tab can still update its recent-item thumbnail.
std::shared_ptr<std::vector<unsigned char>> thumbnail_frame_;
int thumbnail_width_ = 0;
int thumbnail_height_ = 0;
std::chrono::steady_clock::time_point background_snapshot_time_;
bool render_rect_dirty_ = false;
bool stream_cleanup_pending_ = false;
float mouse_pos_x_ = 0;
+20
View File
@@ -116,6 +116,7 @@ struct UserSettingsState {
char password_saved_[7] = "";
char self_hosted_id_[17] = "";
char self_hosted_user_id_[17] = "";
char connection_login_identity_[17] = "";
int language_button_value_ = 0;
int video_quality_button_value_ = 2;
int video_frame_rate_button_value_ = 1;
@@ -152,6 +153,24 @@ struct UserSettingsState {
bool show_file_browser_ = true;
};
struct PasswordChangeState {
std::mutex password_change_mutex_;
uint64_t next_password_change_request_id_ = 0;
bool password_change_pending_ = false;
bool password_change_result_ready_ = false;
bool password_change_succeeded_ = false;
bool password_change_result_uncertain_ = false;
std::chrono::steady_clock::time_point password_change_requested_at_;
std::string pending_password_change_request_id_;
std::string pending_local_password_;
std::string password_change_error_;
bool credential_recovery_in_progress_ = false;
bool credential_recovery_attempt_pending_ = false;
bool credential_recovery_retry_active_ = false;
bool credential_recovery_promote_pending_ = false;
bool credential_recovery_clear_pending_ = false;
};
struct ConnectionState {
using RemoteSessionMap =
std::unordered_map<std::string, RemoteSessionPtr>;
@@ -180,6 +199,7 @@ struct RuntimeState : InfrastructureState,
PeerState,
PlatformIntegrationState,
UserSettingsState,
PasswordChangeState,
ConnectionState {};
} // namespace crossdesk::gui_detail
+141 -20
View File
@@ -116,10 +116,12 @@ export component StreamWindow inherits Window {
min-width: 640px;
min-height: 360px + (root.custom-titlebar ? 32px : 0px);
no-frame: root.custom-titlebar;
background: root.custom-titlebar ? transparent : black;
background: root.native-video-enabled
? transparent : root.custom-titlebar ? transparent : black;
default-font-size: ImGuiFontStyle.base;
in property <bool> custom-titlebar: false;
in property <bool> native-video-enabled: false;
in property <bool> window-active: true;
in property <bool> window-maximized: false;
in property <[StreamTab]> tabs;
@@ -167,6 +169,8 @@ export component StreamWindow inherits Window {
private property <bool> control-expanded: true;
private property <bool> display-menu-open: false;
private property <bool> display-menu-animation-enabled: false;
private property <float> display-menu-progress: 0.0;
private property <bool> shortcut-menu-open: false;
private property <bool> control-docked-left: true;
private property <bool> control-dragging: false;
@@ -183,6 +187,48 @@ export component StreamWindow inherits Window {
callback begin-control-drag(length, length);
callback move-control-drag(length, length);
callback end-control-drag;
pure callback is-local-control-area(length, length) -> bool;
animate display-menu-progress {
duration: 120ms;
easing: ease-out;
enabled: root.display-menu-animation-enabled;
}
display-menu-open-timer := Timer {
interval: 16ms;
running: false;
triggered => {
self.running = false;
if root.display-menu-open {
root.display-menu-animation-enabled = true;
root.display-menu-progress = 1.0;
}
}
}
is-local-control-area(pointer-x, pointer-y) => {
let over-control = pointer-x >= control.x
&& pointer-x <= control.x + control.width
&& pointer-y >= control.y
&& pointer-y <= control.y + control.height;
let display-menu-x = control.x + (root.control-docked-left ? 9px : 42px);
let display-menu-y = control.y + control.height - 1px;
let display-menu-height = min(180px, root.displays.length * 30px + 8px);
let over-display-menu = root.display-menu-open
&& pointer-x >= display-menu-x
&& pointer-x <= display-menu-x + 180px
&& pointer-y >= display-menu-y
&& pointer-y <= display-menu-y + display-menu-height;
let shortcut-menu-x = control.x + (root.control-docked-left ? 41px : 74px);
let shortcut-menu-y = control.y + 40px;
let over-shortcut-menu = root.shortcut-menu-open
&& pointer-x >= shortcut-menu-x
&& pointer-x <= shortcut-menu-x + 150px
&& pointer-y >= shortcut-menu-y
&& pointer-y <= shortcut-menu-y + 68px;
return over-control || over-display-menu || over-shortcut-menu;
}
begin-control-drag(press-x, press-y) => {
// Capture the snapped position before switching x to control-drag-x.
@@ -195,6 +241,9 @@ export component StreamWindow inherits Window {
// pointer positions remain relative to the stationary video surface.
root.control-press-x = press-x - current-x;
root.control-press-y = press-y - current-y;
display-menu-open-timer.running = false;
root.display-menu-animation-enabled = false;
root.display-menu-progress = 0.0;
root.display-menu-open = false;
root.shortcut-menu-open = false;
root.control-dragging = true;
@@ -216,7 +265,7 @@ export component StreamWindow inherits Window {
window-surface := Rectangle {
width: root.width;
height: root.height;
background: black;
background: root.native-video-enabled ? transparent : black;
border-radius:
root.custom-titlebar && !root.fullscreen-enabled && !root.window-maximized
? root.window-corner-radius : 0px;
@@ -372,7 +421,7 @@ export component StreamWindow inherits Window {
y: root.titlebar-height;
width: parent.width;
height: parent.height - self.y;
background: black;
background: root.native-video-enabled ? transparent : black;
clip: true;
// Keep the ImGui tab strip behavior, including pointer-drag reordering.
@@ -452,7 +501,7 @@ export component StreamWindow inherits Window {
video := Rectangle {
y: root.fullscreen-enabled ? 0px : (root.tabs.length > 1 ? 30px : 0px);
height: parent.height - self.y;
background: black;
background: root.native-video-enabled ? transparent : black;
clip: true;
frame-image := Image {
@@ -460,7 +509,7 @@ export component StreamWindow inherits Window {
height: parent.height;
source: root.frame;
image-fit: contain;
visible: root.has-frame;
visible: root.has-frame && !root.native-video-enabled;
}
Text {
@@ -503,7 +552,9 @@ export component StreamWindow inherits Window {
&& self.mouse-x <= control.x + control.width
&& self.mouse-y >= control.y
&& self.mouse-y <= control.y + control.height;
if event.kind == PointerEventKind.down && !over-control {
let over-local-ui = root.is-local-control-area(
self.mouse-x, self.mouse-y);
if event.kind == PointerEventKind.down && !over-local-ui {
input-focus.focus();
}
if event.kind == PointerEventKind.down
@@ -515,9 +566,9 @@ export component StreamWindow inherits Window {
|| event.kind == PointerEventKind.cancel) {
root.end-control-drag();
// The control bar overlays the remote video. Events in
// its full rectangle (including gaps between buttons)
// it and its attached menus (including padding and gaps)
// are local UI input and must never reach the peer.
} else if !root.control-dragging && !over-control {
} else if !root.control-dragging && !over-local-ui {
root.pointer-input(event.button, event.kind, self.mouse-x, self.mouse-y);
}
}
@@ -527,10 +578,7 @@ export component StreamWindow inherits Window {
}
}
scroll-event(event) => {
if self.mouse-x < control.x
|| self.mouse-x > control.x + control.width
|| self.mouse-y < control.y
|| self.mouse-y > control.y + control.height {
if !root.is-local-control-area(self.mouse-x, self.mouse-y) {
root.scroll-input(event.delta-x, event.delta-y, self.mouse-x, self.mouse-y);
}
accept
@@ -565,12 +613,27 @@ export component StreamWindow inherits Window {
icon: FontAwesomeIcons.display;
badge: root.selected-display + 1;
tooltip: StreamStrings.select-display;
clicked => { root.display-menu-open = !root.display-menu-open; root.shortcut-menu-open = false; }
clicked => {
display-menu-open-timer.running = false;
root.display-menu-animation-enabled = false;
root.display-menu-progress = 0.0;
root.display-menu-open = !root.display-menu-open;
root.shortcut-menu-open = false;
if root.display-menu-open {
display-menu-open-timer.running = true;
}
}
}
if root.control-expanded: shortcut-button := ControlBarButton {
x: root.control-docked-left ? 41px : 74px; y: 7px;
icon: FontAwesomeIcons.keyboard; tooltip: StreamStrings.send-shortcut;
clicked => { root.shortcut-menu-open = !root.shortcut-menu-open; root.display-menu-open = false; }
clicked => {
display-menu-open-timer.running = false;
root.display-menu-animation-enabled = false;
root.display-menu-progress = 0.0;
root.shortcut-menu-open = !root.shortcut-menu-open;
root.display-menu-open = false;
}
}
if root.control-expanded: mouse-button := ControlBarButton {
x: root.control-docked-left ? 73px : 106px; y: 7px;
@@ -753,17 +816,75 @@ export component StreamWindow inherits Window {
}
}
if root.display-menu-open: Rectangle {
x: control.x + (root.control-docked-left ? 9px : 42px); y: control.y + 40px; width: 180px; height: min(180px, root.displays.length * 30px + 8px);
background: white; border-width: 1px; border-color: ImGuiLineStyle.border; border-radius: 5px; z: 11;
if root.display-menu-open: display-menu := Rectangle {
private property <length> expanded-height: min(180px, root.displays.length * 30px + 8px);
private property <int> hovered-index:
display-touch.has-hover && display-touch.mouse-y >= 4px
&& floor((display-touch.mouse-y - 4px) / 30px) < root.displays.length
? floor((display-touch.mouse-y - 4px) / 30px) : -1;
x: control.x + (root.control-docked-left ? 9px : 42px);
// Overlap the control border by one pixel so the menu reads as
// an attached panel growing downward from the display button.
y: control.y + control.height - 1px;
width: 180px;
height: self.expanded-height * root.display-menu-progress;
opacity: root.display-menu-progress;
background: white;
border-width: 1px;
border-color: ImGuiLineStyle.border;
border-radius: 5px;
clip: true;
z: 11;
VerticalLayout {
padding: 4px; spacing: 1px;
for display[index] in root.displays: Rectangle {
height: 29px;
background: display-touch.has-hover || index == root.selected-display ? #e8eef9 : transparent;
Text { x: 8px; text: display; color: #30343b; font-size: ImGuiFontStyle.body; vertical-alignment: center; }
display-touch := TouchArea { clicked => { root.selected-display = index; root.switch-display(index); root.display-menu-open = false; } }
background: index == display-menu.hovered-index && display-touch.pressed ? #bfd4f2
: index == display-menu.hovered-index ? #d8e6f8
: index == root.selected-display ? #e8eef9
: transparent;
border-radius: 3px;
Text {
x: 8px;
width: parent.width - 34px;
text: display;
color: #30343b;
font-size: ImGuiFontStyle.body;
overflow: elide;
vertical-alignment: center;
}
if index == root.selected-display: Text {
x: parent.width - 24px;
width: 16px;
text: FontAwesomeIcons.check;
color: #2463c7;
font-family: "Font Awesome 6 Free";
font-weight: 900;
font-size: 10px;
horizontal-alignment: center;
vertical-alignment: center;
}
}
}
// Use one hit target for the whole popup. A single owner keeps
// hover stable while still consuming padding, gaps and scroll
// events locally instead of forwarding them to the peer.
display-touch := TouchArea {
clicked => {
let index = floor((self.mouse-y - 4px) / 30px);
let row-y = self.mouse-y - 4px - index * 30px;
if self.mouse-y >= 4px && row-y < 29px
&& index >= 0 && index < root.displays.length {
display-menu-open-timer.running = false;
root.display-menu-animation-enabled = false;
root.display-menu-progress = 0.0;
root.selected-display = index;
root.switch-display(index);
root.display-menu-open = false;
}
}
scroll-event(event) => { accept }
}
}
+159
View File
@@ -0,0 +1,159 @@
#ifndef CROSSDESK_GUI_UI_UI_LOCALIZATION_H_
#define CROSSDESK_GUI_UI_UI_LOCALIZATION_H_
#include <slint.h>
#include <string>
#include "crossdesk_ui.h"
#include "localization.h"
namespace crossdesk::ui_localization {
inline slint::SharedString Text(const std::string& value) {
return slint::SharedString(value);
}
inline int ApplyMainWindowStrings(
const slint::ComponentHandle<ui::MainWindow>& window,
int language_index) {
const int language =
localization::detail::ClampLanguageIndex(language_index);
auto& strings = window->global<ui::UiStrings>();
strings.set_local_desktop(Text(localization::local_desktop[language]));
strings.set_remote_desktop(Text(localization::remote_desktop[language]));
strings.set_recent_connections(
Text(localization::recent_connections[language]));
strings.set_local_id(Text(localization::local_id[language]));
strings.set_device_name(Text(localization::device_name[language]));
strings.set_remote_id(Text(localization::remote_id[language]));
strings.set_online(Text(localization::online[language]));
strings.set_offline(Text(localization::offline[language]));
strings.set_password(Text(localization::password[language]));
strings.set_copied(
Text(localization::local_id_copied_to_clipboard[language]));
strings.set_connect(Text(localization::connect[language]));
strings.set_settings(Text(localization::settings[language]));
strings.set_about(Text(localization::about[language]));
strings.set_ok(Text(localization::ok[language]));
strings.set_cancel(Text(localization::cancel[language]));
strings.set_new_password(Text(localization::new_password[language]));
strings.set_invalid_password(Text(localization::max_password_len[language]));
strings.set_edit_alias(
Text(localization::input_connection_alias[language]));
strings.set_delete_connection(
Text(localization::delete_connection[language]));
strings.set_confirm_delete(
Text(localization::confirm_delete_connection[language]));
strings.set_language(Text(localization::language[language]));
strings.set_video_quality(Text(localization::video_quality[language]));
strings.set_frame_rate(Text(localization::video_frame_rate[language]));
strings.set_codec(Text(localization::video_encode_format[language]));
strings.set_hardware_codec(
Text(localization::enable_hardware_video_codec[language]));
strings.set_turn_relay(Text(localization::enable_turn[language]));
strings.set_srtp(Text(localization::enable_srtp[language]));
strings.set_self_hosted(
Text(localization::self_hosted_server_config[language]));
strings.set_autostart(Text(localization::enable_autostart[language]));
strings.set_daemon(Text(localization::enable_daemon[language]));
strings.set_minimize_to_tray(
Text(localization::minimize_to_tray[language]));
strings.set_file_save_path(
Text(localization::file_transfer_save_path[language]));
strings.set_default_desktop(Text(localization::default_desktop[language]));
strings.set_server_host(
Text(localization::self_hosted_server_address[language]));
strings.set_server_port(
Text(localization::self_hosted_server_port[language]));
strings.set_coturn_port(
Text(localization::self_hosted_server_coturn_server_port[language]));
strings.set_version(Text(localization::version[language]));
strings.set_signal_connected(Text(localization::signal_connected[language]));
strings.set_signal_disconnected(
Text(localization::signal_disconnected[language]));
strings.set_tls_error(Text(localization::signal_tls_cert_error[language]));
strings.set_update_available(
Text(localization::new_version_available[language]));
strings.set_release_notes(Text(localization::release_notes[language]));
strings.set_download(Text(localization::update[language]));
strings.set_input_password(Text(localization::input_password[language]));
strings.set_reinput_password(Text(localization::reinput_password[language]));
strings.set_remember_password(
Text(localization::remember_password[language]));
strings.set_validate_password(
Text(localization::validate_password[language]));
strings.set_request_permissions(
Text(localization::request_permissions[language]));
strings.set_permission_required(
Text(localization::permission_required_message[language]));
strings.set_screen_recording_permission(
Text(localization::screen_recording_permission[language]));
strings.set_accessibility_permission(
Text(localization::accessibility_permission[language]));
strings.set_service_setup_title(
Text(localization::windows_service_setup_title[language]));
strings.set_service_setup_message(
Text(localization::windows_service_setup_message[language]));
strings.set_service_settings_label(
Text(localization::windows_service_settings_label[language]));
strings.set_install_service(
Text(localization::install_windows_service[language]));
strings.set_service_installed(
Text(localization::windows_service_installed[language]));
strings.set_do_not_remind(
Text(localization::do_not_remind_again[language]));
strings.set_notification(Text(localization::notification[language]));
strings.set_service_suppressed_message(
Text(localization::windows_service_prompt_suppressed_message[language]));
strings.set_quality_low(Text(localization::video_quality_low[language]));
strings.set_quality_medium(
Text(localization::video_quality_medium[language]));
strings.set_quality_high(Text(localization::video_quality_high[language]));
strings.set_codec_h264(Text(localization::h264[language]));
strings.set_codec_av1(Text(localization::av1[language]));
strings.set_self_hosted_settings(
Text(localization::self_hosted_server_settings[language]));
strings.set_access_website(Text(localization::access_website[language]));
strings.set_release_date_label(Text(localization::release_date[language]));
strings.set_later(Text(localization::cancel[language]));
return language;
}
template <typename Window>
inline int ApplyStreamWindowStrings(
const slint::ComponentHandle<Window>& window, int language_index) {
const int language =
localization::detail::ClampLanguageIndex(language_index);
auto& strings = window->template global<ui::StreamStrings>();
strings.set_select_display(Text(localization::select_display[language]));
strings.set_send_shortcut(Text(localization::send_shortcut[language]));
strings.set_control_mouse(Text(localization::control_mouse[language]));
strings.set_release_mouse(Text(localization::release_mouse[language]));
strings.set_audio(Text(localization::audio_capture[language]));
strings.set_mute(Text(localization::mute[language]));
strings.set_select_file(Text(localization::select_file[language]));
strings.set_show_stats(
Text(localization::show_net_traffic_stats[language]));
strings.set_hide_stats(
Text(localization::hide_net_traffic_stats[language]));
strings.set_fullscreen(Text(localization::fullscreen[language]));
strings.set_exit_fullscreen(Text(localization::exit_fullscreen[language]));
strings.set_disconnect(Text(localization::disconnect[language]));
strings.set_file_transfer(
Text(localization::file_transfer_progress[language]));
strings.set_expand_control(Text(localization::expand_control_bar[language]));
strings.set_collapse_control(
Text(localization::collapse_control_bar[language]));
strings.set_stats_in(Text(localization::in[language]));
strings.set_stats_out(Text(localization::out[language]));
strings.set_stats_loss_rate(Text(localization::loss_rate[language]));
strings.set_stats_resolution(Text(localization::resolution[language]));
strings.set_stats_connection_mode(
Text(localization::connection_mode[language]));
return language;
}
} // namespace crossdesk::ui_localization
#endif // CROSSDESK_GUI_UI_UI_LOCALIZATION_H_
+4 -1
View File
@@ -108,7 +108,10 @@ int GuiApplication::ControlBar(
if (ImGui::BeginPopup("display")) {
ImGui::SetWindowFontScale(0.5f);
for (int i = 0; i < props->display_info_list_.size(); i++) {
if (ImGui::Selectable(props->display_info_list_[i].name.c_str())) {
const std::string display_label = localization::FormatDisplayLabel(
static_cast<size_t>(i), props->display_info_list_[i].name,
localization_language_index_);
if (ImGui::Selectable(display_label.c_str())) {
props->selected_display_ = i;
RemoteAction remote_action;
@@ -26,6 +26,7 @@
#include <chrono>
#include <thread>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -262,6 +263,7 @@ bool ScreenCapturerDrm::DiscoverOutputs() {
output.height = static_cast<int>(crtc->height);
output.name = std::string(ConnectorTypeName(connector->connector_type)) +
std::to_string(connector->connector_type_id);
output.stream_id = MakeDisplayStreamId(outputs_.size());
outputs_.push_back(output);
display_info_list_.push_back(
@@ -427,7 +429,7 @@ bool ScreenCapturerDrm::CaptureOutputFrame(const DrmOutput& output,
if (emit_callback && callback_) {
callback_(nv12.data(), static_cast<int>(nv12.size()), capture_width,
capture_height, output.name.c_str());
capture_height, output.stream_id.c_str());
}
UnmapFramebuffer(mapped_ptr, mapped_size, prime_fd);
@@ -47,6 +47,7 @@ class ScreenCapturerDrm : public ScreenCapturer {
uint32_t connector_id = 0;
uint32_t crtc_id = 0;
std::string name;
std::string stream_id;
int left = 0;
int top = 0;
int width = 0;
@@ -6,6 +6,7 @@
#include <string>
#include <utility>
#include "display_stream_id.h"
#include "platform.h"
#include "rd_log.h"
#if defined(CROSSDESK_HAS_DRM) && CROSSDESK_HAS_DRM
@@ -49,10 +50,21 @@ int ScreenCapturerLinux::Init(const int fps, cb_desktop_data cb) {
fps_ = fps;
callback_orig_ = std::move(cb);
callback_ = [this](unsigned char* data, int size, int width, int height,
const char* display_name) {
const std::string mapped_name = MapDisplayName(display_name);
const char* reported_stream_id) {
const std::string mapped_stream_id = MapStreamId(reported_stream_id);
if (mapped_stream_id.empty()) {
if (!invalid_stream_id_logged_.exchange(true,
std::memory_order_relaxed)) {
LOG_WARN("Linux capturer dropping frame without a registered stream "
"id: reported='{}', size={}x{}, bytes={}",
reported_stream_id ? reported_stream_id : "", width, height,
size);
}
return;
}
invalid_stream_id_logged_.store(false, std::memory_order_relaxed);
if (callback_orig_) {
callback_orig_(data, size, width, height, mapped_name.c_str());
callback_orig_(data, size, width, height, mapped_stream_id.c_str());
}
};
@@ -139,10 +151,12 @@ int ScreenCapturerLinux::Destroy() {
backend_ = BackendType::kNone;
callback_ = nullptr;
callback_orig_ = nullptr;
current_monitor_index_.store(0, std::memory_order_relaxed);
invalid_stream_id_logged_.store(false, std::memory_order_relaxed);
{
std::lock_guard<std::mutex> lock(alias_mutex_);
canonical_displays_.clear();
label_alias_.clear();
stream_id_alias_.clear();
}
return 0;
}
@@ -153,15 +167,12 @@ int ScreenCapturerLinux::Start(bool show_cursor) {
return -1;
}
#if defined(CROSSDESK_HAS_WAYLAND_CAPTURER) && CROSSDESK_HAS_WAYLAND_CAPTURER
if (backend_ == BackendType::kWayland) {
const int refresh_ret = RefreshWaylandBackend();
// X11 output names, DRM connectors and Wayland stream handles may all change
// after an HDMI hotplug, so rebuild the current backend before each session.
const int refresh_ret = RefreshCurrentBackend();
if (refresh_ret != 0) {
LOG_WARN("Linux screen capturer Wayland backend refresh failed: {}",
refresh_ret);
LOG_WARN("Linux screen capturer backend refresh failed: {}", refresh_ret);
}
}
#endif
const int ret = impl_->Start(show_cursor);
if (ret == 0) {
@@ -235,14 +246,23 @@ int ScreenCapturerLinux::SwitchTo(int monitor_index) {
if (!impl_) {
return -1;
}
return impl_->SwitchTo(monitor_index);
const int ret = impl_->SwitchTo(monitor_index);
if (ret == 0) {
current_monitor_index_.store(monitor_index, std::memory_order_relaxed);
}
return ret;
}
int ScreenCapturerLinux::ResetToInitialMonitor() {
if (!impl_) {
return -1;
}
return impl_->ResetToInitialMonitor();
const int ret = impl_->ResetToInitialMonitor();
if (ret == 0) {
current_monitor_index_.store(initial_monitor_index_,
std::memory_order_relaxed);
}
return ret;
}
std::vector<DisplayInfo> ScreenCapturerLinux::GetDisplayInfoList() {
@@ -320,27 +340,55 @@ int ScreenCapturerLinux::InitWayland() {
#endif
}
int ScreenCapturerLinux::RefreshWaylandBackend() {
int ScreenCapturerLinux::RefreshCurrentBackend() {
std::unique_ptr<ScreenCapturer> backend;
const char* backend_name = "unknown";
switch (backend_) {
case BackendType::kX11:
backend = std::make_unique<ScreenCapturerX11>();
backend_name = "X11";
break;
case BackendType::kDrm:
#if defined(CROSSDESK_HAS_DRM) && CROSSDESK_HAS_DRM
backend = std::make_unique<ScreenCapturerDrm>();
backend_name = "DRM";
break;
#else
return -1;
#endif
case BackendType::kWayland:
#if defined(CROSSDESK_HAS_WAYLAND_CAPTURER) && CROSSDESK_HAS_WAYLAND_CAPTURER
auto backend = std::make_unique<ScreenCapturerWayland>();
backend = std::make_unique<ScreenCapturerWayland>();
backend_name = "Wayland";
break;
#else
return -1;
#endif
case BackendType::kNone:
return -1;
}
const int ret = backend->Init(fps_, callback_);
if (ret != 0) {
backend->Destroy();
return ret;
}
const int requested_monitor =
current_monitor_index_.load(std::memory_order_relaxed);
UpdateAliasesFromBackend(backend.get());
if (requested_monitor > 0 && backend->SwitchTo(requested_monitor) != 0) {
current_monitor_index_.store(0, std::memory_order_relaxed);
}
if (impl_) {
impl_->Destroy();
}
UpdateAliasesFromBackend(backend.get());
impl_ = std::move(backend);
backend_ = BackendType::kWayland;
LOG_INFO("Linux screen capturer Wayland backend refreshed before start");
LOG_INFO("Linux screen capturer {} backend refreshed before start",
backend_name);
return 0;
#else
return -1;
#endif
}
bool ScreenCapturerLinux::TryFallbackToDrm(bool show_cursor) {
@@ -447,61 +495,82 @@ void ScreenCapturerLinux::UpdateAliasesFromBackend(ScreenCapturer* backend) {
}
std::lock_guard<std::mutex> lock(alias_mutex_);
label_alias_.clear();
stream_id_alias_.clear();
if (canonical_displays_.empty()) {
canonical_displays_ = backend_displays;
for (const auto& display : backend_displays) {
label_alias_[display.name] = display.name;
for (size_t i = 0; i < backend_displays.size(); ++i) {
auto& canonical = canonical_displays_[i];
const std::string stream_id = MakeDisplayStreamId(i);
if (canonical.name.empty()) {
canonical.name = stream_id;
}
stream_id_alias_[stream_id] = stream_id;
}
return;
}
if (canonical_displays_.size() < backend_displays.size()) {
for (size_t i = canonical_displays_.size(); i < backend_displays.size();
++i) {
canonical_displays_.push_back(backend_displays[i]);
}
LOG_WARN("Linux capturer detected {} additional display(s); they remain "
"unavailable until peer streams are reinitialized",
backend_displays.size() - canonical_displays_.size());
}
for (size_t i = 0; i < backend_displays.size(); ++i) {
const std::string mapped_name = i < canonical_displays_.size()
? canonical_displays_[i].name
: backend_displays[i].name;
label_alias_[backend_displays[i].name] = mapped_name;
auto similar = [](const DisplayInfo& current, const DisplayInfo& canonical) {
return std::abs(current.left - canonical.left) <= 10 &&
std::abs(current.top - canonical.top) <= 10 &&
std::abs(current.width - canonical.width) <= 20 &&
std::abs(current.height - canonical.height) <= 20;
};
std::vector<bool> used(canonical_displays_.size(), false);
for (size_t current_index = 0; current_index < backend_displays.size();
++current_index) {
const auto& backend_display = backend_displays[current_index];
int canonical_index = -1;
for (size_t i = 0; i < canonical_displays_.size(); ++i) {
if (!used[i] && similar(backend_display, canonical_displays_[i])) {
canonical_index = static_cast<int>(i);
break;
}
}
if (canonical_index < 0 && current_index < canonical_displays_.size() &&
!used[current_index]) {
canonical_index = static_cast<int>(current_index);
}
if (canonical_index < 0) {
continue;
}
if (i < canonical_displays_.size()) {
// Keep original stable names, but refresh geometry from active backend.
canonical_displays_[i].handle = backend_displays[i].handle;
canonical_displays_[i].is_primary = backend_displays[i].is_primary;
canonical_displays_[i].left = backend_displays[i].left;
canonical_displays_[i].top = backend_displays[i].top;
canonical_displays_[i].right = backend_displays[i].right;
canonical_displays_[i].bottom = backend_displays[i].bottom;
canonical_displays_[i].width = backend_displays[i].width;
canonical_displays_[i].height = backend_displays[i].height;
used[canonical_index] = true;
const std::string backend_stream_id =
MakeDisplayStreamId(current_index);
const std::string stable_stream_id =
MakeDisplayStreamId(static_cast<size_t>(canonical_index));
stream_id_alias_[backend_stream_id] = stable_stream_id;
// Refresh the user-visible label and geometry. The MiniRTC stream ID is
// derived independently from this stable logical index.
canonical_displays_[canonical_index] = backend_display;
if (canonical_displays_[canonical_index].name.empty()) {
canonical_displays_[canonical_index].name = stable_stream_id;
}
}
}
std::string ScreenCapturerLinux::MapDisplayName(
const char* display_name) const {
std::string input_name = display_name ? display_name : "";
if (input_name.empty()) {
return input_name;
}
std::string ScreenCapturerLinux::MapStreamId(
const char* reported_stream_id) const {
std::string input_id = reported_stream_id ? reported_stream_id : "";
std::lock_guard<std::mutex> lock(alias_mutex_);
auto it = label_alias_.find(input_name);
if (it != label_alias_.end()) {
return it->second;
auto it = stream_id_alias_.find(input_id);
if (it != stream_id_alias_.end()) {
input_id = it->second;
} else {
input_id.clear();
}
if (canonical_displays_.size() == 1) {
return canonical_displays_[0].name;
}
return input_name;
return ResolveDisplayStreamId(
input_id.c_str(), canonical_displays_.size(),
current_monitor_index_.load(std::memory_order_relaxed));
}
} // namespace crossdesk
@@ -7,6 +7,7 @@
#ifndef _SCREEN_CAPTURER_LINUX_H_
#define _SCREEN_CAPTURER_LINUX_H_
#include <atomic>
#include <memory>
#include <mutex>
#include <string>
@@ -43,12 +44,12 @@ class ScreenCapturerLinux : public ScreenCapturer {
int InitX11();
int InitDrm();
int InitWayland();
int RefreshWaylandBackend();
int RefreshCurrentBackend();
bool TryFallbackToDrm(bool show_cursor);
bool TryFallbackToX11(bool show_cursor);
bool TryFallbackToWayland(bool show_cursor);
void UpdateAliasesFromBackend(ScreenCapturer* backend);
std::string MapDisplayName(const char* display_name) const;
std::string MapStreamId(const char* reported_stream_id) const;
private:
std::unique_ptr<ScreenCapturer> impl_;
@@ -58,7 +59,10 @@ class ScreenCapturerLinux : public ScreenCapturer {
cb_desktop_data callback_orig_;
std::vector<DisplayInfo> canonical_displays_;
mutable std::mutex alias_mutex_;
std::unordered_map<std::string, std::string> label_alias_;
std::unordered_map<std::string, std::string> stream_id_alias_;
std::atomic<int> current_monitor_index_{0};
int initial_monitor_index_ = 0;
mutable std::atomic<bool> invalid_stream_id_logged_{false};
};
} // namespace crossdesk
@@ -15,6 +15,7 @@
#include <thread>
#include <vector>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -831,8 +832,9 @@ void ScreenCapturerWayland::HandlePipeWireBuffer() {
nv12.insert(nv12.end(), uv_plane_.begin(), uv_plane_.end());
if (callback_) {
const std::string stream_id = MakeDisplayStreamId(0);
callback_(nv12.data(), static_cast<int>(nv12.size()), even_width,
even_height, display_name_.c_str());
even_height, stream_id.c_str());
}
pipewire_last_frame_ms_.store(NowMs());
@@ -10,6 +10,7 @@
#include <mutex>
#include <thread>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -97,12 +98,6 @@ int ScreenCapturerX11::Init(const int fps, cb_desktop_data cb) {
name = "Display" + std::to_string(i + 1);
}
// clean display name, remove non-alphanumeric characters
name.erase(
std::remove_if(name.begin(), name.end(),
[](unsigned char c) { return !std::isalnum(c); }),
name.end());
display_info_list_.push_back(DisplayInfo(
(void*)display_, name, true, crtc_info->x, crtc_info->y,
crtc_info->x + crtc_info->width, crtc_info->y + crtc_info->height));
@@ -324,8 +319,9 @@ void ScreenCapturerX11::OnFrame() {
nv12.insert(nv12.end(), uv_plane_.begin(), uv_plane_.end());
if (callback_) {
const std::string stream_id = MakeDisplayStreamId(monitor_index);
callback_(nv12.data(), width_ * height_ * 3 / 2, width_, height_,
display_info_list_[monitor_index].name.c_str());
stream_id.c_str());
}
XDestroyImage(image);
@@ -10,21 +10,21 @@
#include "screen_capturer_sck.h"
#include <AppKit/AppKit.h>
#include <ApplicationServices/ApplicationServices.h>
#include <CoreGraphics/CoreGraphics.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/graphics/IOGraphicsLib.h>
#include <IOSurface/IOSurface.h>
#include <ScreenCaptureKit/ScreenCaptureKit.h>
#include <algorithm>
#include <atomic>
#include <cctype>
#include <cstring>
#include <limits>
#include <map>
#include <mutex>
#include <vector>
#include "display_info.h"
#include "display_stream_id.h"
#include "rd_log.h"
using namespace crossdesk;
@@ -83,20 +83,23 @@ class API_AVAILABLE(macos(14.0)) ScreenCapturerSckImpl : public ScreenCapturer {
int Resume(int monitor_index) override { return 0; }
std::vector<DisplayInfo> GetDisplayInfoList() override { return display_info_list_; }
std::vector<DisplayInfo> GetDisplayInfoList() override {
std::lock_guard<std::mutex> lock(lock_);
return display_info_list_;
}
int ResetToInitialMonitor() override;
private:
std::vector<DisplayInfo> display_info_list_;
std::map<int, CGDirectDisplayID> display_id_map_;
std::map<CGDirectDisplayID, int> display_id_map_reverse_;
std::map<CGDirectDisplayID, std::string> display_id_name_map_;
unsigned char *nv12_frame_ = nullptr;
size_t nv12_frame_size_ = 0;
int width_ = 0;
int height_ = 0;
int fps_ = 60;
bool show_cursor_ = false;
bool capture_requested_ = false;
bool invalid_stream_id_logged_ = false;
public:
// Called by SckHelper when shareable content is returned by ScreenCaptureKit. `content` will be
@@ -130,9 +133,38 @@ class API_AVAILABLE(macos(14.0)) ScreenCapturerSckImpl : public ScreenCapturer {
// support full-desktop capture, and will fall back to the first display.
CGDirectDisplayID current_display_ = 0;
int initial_monitor_index_ = 0;
int current_monitor_index_ = 0;
std::string current_stream_id_;
};
static std::string NSStringToUtf8(NSString *value) {
if (!value || value.length == 0) return "";
const char *utf8 = value.UTF8String;
return utf8 ? utf8 : "";
}
static std::string GetAppKitDisplayName(CGDirectDisplayID display_id) {
if (@available(macOS 10.15, *)) {
@autoreleasepool {
for (NSScreen *screen in NSScreen.screens) {
NSNumber *screen_number = screen.deviceDescription[@"NSScreenNumber"];
if (screen_number && screen_number.unsignedIntValue == display_id) {
return NSStringToUtf8(screen.localizedName);
}
}
}
}
return "";
}
std::string GetDisplayName(CGDirectDisplayID display_id) {
// IODisplayConnect is no longer exposed for some external displays on
// modern Apple Silicon Macs. NSScreen is the system-owned source for the
// user-visible name and maps directly to CGDirectDisplayID.
std::string appkit_name = GetAppKitDisplayName(display_id);
if (!appkit_name.empty()) return appkit_name;
// Keep the IOKit path for older macOS versions and display drivers.
io_iterator_t iter;
io_service_t serv = 0, matched_serv = 0;
@@ -212,8 +244,6 @@ ScreenCapturerSckImpl::~ScreenCapturerSckImpl() {
display_info_list_.clear();
display_id_map_.clear();
display_id_map_reverse_.clear();
display_id_name_map_.clear();
if (nv12_frame_) {
delete[] nv12_frame_;
@@ -232,8 +262,6 @@ int ScreenCapturerSckImpl::Init(const int fps, cb_desktop_data cb) {
fps_ = fps > 0 ? fps : 60;
display_info_list_.clear();
display_id_map_.clear();
display_id_map_reverse_.clear();
display_id_name_map_.clear();
if (@available(macOS 10.15, *)) {
bool has_permission = CGPreflightScreenCaptureAccess();
@@ -265,26 +293,18 @@ int ScreenCapturerSckImpl::Init(const int fps, cb_desktop_data cb) {
return -1;
}
CGDirectDisplayID displays[10];
uint32_t count;
CGGetActiveDisplayList(10, displays, &count);
int unnamed_count = 1;
for (SCDisplay *display in content.displays) {
CGDirectDisplayID display_id = display.displayID;
CGRect bounds = CGDisplayBounds(display_id);
bool is_primary = CGDisplayIsMain(display_id);
std::string name = GetDisplayName(display_id);
if (name.empty()) {
name = "Display" + std::to_string(unnamed_count++);
name = MakeDisplayStreamId(display_info_list_.size());
}
// clean display name, remove non-alphanumeric characters
name.erase(
std::remove_if(name.begin(), name.end(), [](unsigned char c) { return !std::isalnum(c); }),
name.end());
LOG_INFO("macOS display discovered: index={}, display_id={}, name='{}'",
display_info_list_.size(), display_id, name);
DisplayInfo info((void *)(uintptr_t)display_id, name, is_primary,
static_cast<int>(bounds.origin.x), static_cast<int>(bounds.origin.y),
@@ -293,11 +313,12 @@ int ScreenCapturerSckImpl::Init(const int fps, cb_desktop_data cb) {
display_info_list_.push_back(info);
display_id_map_[display_info_list_.size() - 1] = display_id;
display_id_map_reverse_[display_id] = display_info_list_.size() - 1;
display_id_name_map_[display_id] = name;
}
initial_monitor_index_ = 0;
current_monitor_index_ = initial_monitor_index_;
current_display_ = display_id_map_[current_monitor_index_];
current_stream_id_ = MakeDisplayStreamId(current_monitor_index_);
return 0;
}
@@ -307,49 +328,69 @@ int ScreenCapturerSckImpl::Start(bool show_cursor) {
return -1;
}
{
std::lock_guard<std::mutex> lock(lock_);
if (display_info_list_.empty()) {
LOG_ERROR("Cannot start capturer: display info not initialized");
return -1;
}
show_cursor_ = show_cursor;
capture_requested_ = true;
invalid_stream_id_logged_ = false;
}
StartOrReconfigureCapturer();
return 0;
}
int ScreenCapturerSckImpl::SwitchTo(int monitor_index) {
bool should_reconfigure = false;
{
std::lock_guard<std::mutex> lock(lock_);
auto display_it = display_id_map_.find(monitor_index);
if (display_it == display_id_map_.end()) {
if (display_it == display_id_map_.end() || monitor_index < 0 ||
monitor_index >= static_cast<int>(display_info_list_.size())) {
LOG_WARN("SwitchTo skipped, invalid monitor_index={}, displays={}",
monitor_index, display_id_map_.size());
return -1;
}
const CGDirectDisplayID target_display = display_it->second;
{
std::lock_guard<std::mutex> lock(lock_);
current_display_ = target_display;
current_monitor_index_ = monitor_index;
current_display_ = display_it->second;
current_stream_id_ = MakeDisplayStreamId(monitor_index);
should_reconfigure = capture_requested_;
}
if (should_reconfigure) {
StartOrReconfigureCapturer();
}
return 0;
}
int ScreenCapturerSckImpl::ResetToInitialMonitor() {
int target = initial_monitor_index_;
const int target = initial_monitor_index_;
bool should_reconfigure = false;
{
std::lock_guard<std::mutex> lock(lock_);
if (display_info_list_.empty()) return -1;
auto display_it = display_id_map_.find(target);
if (display_it == display_id_map_.end()) {
LOG_WARN("ResetToInitialMonitor skipped, invalid monitor_index={}", target);
return -1;
}
CGDirectDisplayID target_display = display_it->second;
if (current_display_ == target_display) return 0;
{
std::lock_guard<std::mutex> lock(lock_);
current_display_ = target_display;
const CGDirectDisplayID target_display = display_it->second;
if (current_display_ == target_display &&
current_monitor_index_ == target) {
return 0;
}
current_monitor_index_ = target;
current_display_ = target_display;
current_stream_id_ = MakeDisplayStreamId(target);
should_reconfigure = capture_requested_ && stream_ != nil;
}
// Resetting session state must not create a capture stream. Preserve the
// selected monitor for the next Start(), and only reconfigure an active one.
if (should_reconfigure) {
StartOrReconfigureCapturer();
}
return 0;
}
@@ -362,7 +403,9 @@ int ScreenCapturerSckImpl::Destroy() {
[stream_ stopCaptureWithCompletionHandler:nil];
stream_ = nil;
}
capture_requested_ = false;
current_display_ = 0;
current_stream_id_.clear();
permanent_error_ = false;
_on_data = nullptr;
helper_to_release = helper_;
@@ -376,6 +419,7 @@ int ScreenCapturerSckImpl::Destroy() {
int ScreenCapturerSckImpl::Stop() {
std::lock_guard<std::mutex> lock(lock_);
capture_requested_ = false;
if (stream_) {
LOG_INFO("Stopping stream");
[stream_ stopCaptureWithCompletionHandler:nil];
@@ -387,6 +431,14 @@ int ScreenCapturerSckImpl::Stop() {
}
void ScreenCapturerSckImpl::OnShareableContentCreated(SCShareableContent *content) {
{
std::lock_guard<std::mutex> lock(lock_);
if (!capture_requested_) {
LOG_INFO("Ignoring stale ScreenCaptureKit display refresh after stop");
return;
}
}
if (!content) {
LOG_ERROR("getShareableContent failed");
permanent_error_ = true;
@@ -400,22 +452,71 @@ void ScreenCapturerSckImpl::OnShareableContentCreated(SCShareableContent *conten
}
SCDisplay *captured_display = nil;
bool show_cursor = false;
{
std::lock_guard<std::mutex> lock(lock_);
if (!capture_requested_) return;
int logical_index = current_monitor_index_;
if (logical_index < 0 ||
logical_index >= static_cast<int>(display_info_list_.size())) {
logical_index = 0;
}
SCDisplay *logical_fallback = nil;
int display_index = 0;
for (SCDisplay *display in content.displays) {
if (display_index == logical_index) logical_fallback = display;
if (current_display_ != 0 && current_display_ == display.displayID) {
LOG_WARN("current display: {}, name: {}", current_display_,
display_id_name_map_[current_display_]);
captured_display = display;
break;
}
++display_index;
}
if (!captured_display) {
captured_display = content.displays.firstObject;
captured_display = logical_fallback ? logical_fallback
: content.displays.firstObject;
if (!logical_fallback) logical_index = 0;
}
if (captured_display) {
current_display_ = captured_display.displayID;
const CGDirectDisplayID old_display = current_display_;
const CGDirectDisplayID new_display = captured_display.displayID;
const std::string stable_stream_id = ResolveDisplayStreamId(
current_stream_id_.c_str(), display_info_list_.size(), logical_index);
if (stable_stream_id.empty()) {
LOG_ERROR("Cannot map macOS display {} to a registered video stream",
new_display);
return;
}
current_monitor_index_ = logical_index;
current_display_ = new_display;
current_stream_id_ = stable_stream_id;
display_id_map_[logical_index] = new_display;
CGRect bounds = CGDisplayBounds(new_display);
auto& info = display_info_list_[logical_index];
info.handle = (void *)(uintptr_t)new_display;
info.is_primary = CGDisplayIsMain(new_display);
info.left = static_cast<int>(bounds.origin.x);
info.top = static_cast<int>(bounds.origin.y);
info.right = static_cast<int>(bounds.origin.x + bounds.size.width);
info.bottom = static_cast<int>(bounds.origin.y + bounds.size.height);
info.width = info.right - info.left;
info.height = info.bottom - info.top;
const std::string refreshed_name = GetDisplayName(new_display);
if (!refreshed_name.empty()) {
info.name = refreshed_name;
}
if (old_display != new_display) {
LOG_INFO("macOS display mapping refreshed: slot={}, old_id={}, "
"new_id={}, stream='{}'",
logical_index, old_display, new_display, stable_stream_id);
}
}
show_cursor = show_cursor_;
}
if (!captured_display) {
@@ -440,13 +541,14 @@ void ScreenCapturerSckImpl::OnShareableContentCreated(SCShareableContent *conten
}
config.pixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange;
config.showsCursor = show_cursor_;
config.showsCursor = show_cursor;
config.width = filter.contentRect.size.width * filter.pointPixelScale;
config.height = filter.contentRect.size.height * filter.pointPixelScale;
config.captureResolution = SCCaptureResolutionAutomatic;
config.minimumFrameInterval = CMTimeMake(1, fps_);
std::lock_guard<std::mutex> lock(lock_);
if (!capture_requested_) return;
if (stream_) {
LOG_INFO("Updating stream configuration");
@@ -516,7 +618,7 @@ void ScreenCapturerSckImpl::OnNewCVPixelBuffer(CVPixelBufferRef pixelBuffer,
}
std::lock_guard<std::mutex> lock(lock_);
if (!_on_data) {
if (!_on_data || !capture_requested_) {
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
return;
}
@@ -551,9 +653,22 @@ void ScreenCapturerSckImpl::OnNewCVPixelBuffer(CVPixelBufferRef pixelBuffer,
memcpy(dst_uv + row * width, static_cast<unsigned char *>(base_uv) + row * stride_uv, width);
}
const std::string stream_id = ResolveDisplayStreamId(
current_stream_id_.c_str(), display_info_list_.size(),
current_monitor_index_);
if (stream_id.empty()) {
if (!invalid_stream_id_logged_) {
LOG_ERROR("Dropping macOS frames without a registered stream id, "
"display_id={}",
current_display_);
invalid_stream_id_logged_ = true;
}
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
return;
}
invalid_stream_id_logged_ = false;
_on_data(nv12_frame_, static_cast<int>(required_size), static_cast<int>(width),
static_cast<int>(height),
display_id_name_map_[current_display_].c_str());
static_cast<int>(height), stream_id.c_str());
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
}
+2
View File
@@ -15,6 +15,8 @@ namespace crossdesk {
class ScreenCapturer {
public:
// The final callback argument is a logical MiniRTC stream ID (DisplayN),
// not a platform display name or physical handle.
typedef std::function<void(unsigned char*, int, int, int, const char*)>
cb_desktop_data;
@@ -5,6 +5,7 @@
#include <string>
#include <vector>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -21,11 +22,12 @@ std::string WideToUtf8(const std::wstring& wstr) {
return result;
}
std::string CleanDisplayName(const std::wstring& wide_name) {
std::string GetDisplayLabel(const std::wstring& wide_name) {
std::string name = WideToUtf8(wide_name);
name.erase(std::remove_if(name.begin(), name.end(),
[](unsigned char c) { return !std::isalnum(c); }),
name.end());
constexpr char kDevicePrefix[] = "\\\\.\\";
if (name.rfind(kDevicePrefix, 0) == 0) {
name.erase(0, sizeof(kDevicePrefix) - 1);
}
return name;
}
} // namespace
@@ -207,7 +209,7 @@ void ScreenCapturerDxgi::EnumerateDisplays() {
if (FAILED(output->GetDesc(&desc))) {
continue;
}
std::string name = CleanDisplayName(desc.DeviceName);
std::string name = GetDisplayLabel(desc.DeviceName);
MONITORINFOEX mi{};
mi.cbSize = sizeof(MONITORINFOEX);
if (GetMonitorInfo(desc.Monitor, &mi)) {
@@ -380,8 +382,9 @@ void ScreenCapturerDxgi::CaptureLoop() {
if (callback_) {
int idx = monitor_index_.load();
if (idx >= 0 && idx < static_cast<int>(display_info_list_.size())) {
const std::string stream_id = MakeDisplayStreamId(idx);
callback_(nv12_frame_, nv12_size, even_width, even_height,
display_info_list_[idx].name.c_str());
stream_id.c_str());
} else {
LOG_ERROR("DXGI: CaptureLoop invalid monitor_index {} (list size {})",
idx, display_info_list_.size());
@@ -5,6 +5,7 @@
#include <string>
#include <vector>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -21,11 +22,12 @@ std::string WideToUtf8(const std::wstring& wstr) {
return result;
}
std::string CleanDisplayName(const std::wstring& wide_name) {
std::string GetDisplayLabel(const std::wstring& wide_name) {
std::string name = WideToUtf8(wide_name);
name.erase(std::remove_if(name.begin(), name.end(),
[](unsigned char c) { return !std::isalnum(c); }),
name.end());
constexpr char kDevicePrefix[] = "\\\\.\\";
if (name.rfind(kDevicePrefix, 0) == 0) {
name.erase(0, sizeof(kDevicePrefix) - 1);
}
return name;
}
} // namespace
@@ -42,7 +44,7 @@ BOOL CALLBACK ScreenCapturerGdi::EnumMonitorProc(HMONITOR hMonitor, HDC, LPRECT,
MONITORINFOEX mi{};
mi.cbSize = sizeof(MONITORINFOEX);
if (GetMonitorInfo(hMonitor, &mi)) {
std::string name = CleanDisplayName(mi.szDevice);
std::string name = GetDisplayLabel(mi.szDevice);
bool is_primary = (mi.dwFlags & MONITORINFOF_PRIMARY) ? true : false;
DisplayInfo info((void*)hMonitor, name, is_primary, mi.rcMonitor.left,
mi.rcMonitor.top, mi.rcMonitor.right, mi.rcMonitor.bottom);
@@ -209,7 +211,8 @@ void ScreenCapturerGdi::CaptureLoop() {
width, height);
if (callback_) {
callback_(nv12_frame_, nv12_size, width, height, di.name.c_str());
const std::string stream_id = MakeDisplayStreamId(idx);
callback_(nv12_frame_, nv12_size, width, height, stream_id.c_str());
}
SelectObject(mem_dc, old);
@@ -7,6 +7,7 @@
#include <iostream>
#include "display_stream_id.h"
#include "libyuv.h"
#include "rd_log.h"
@@ -24,11 +25,12 @@ std::string WideToUtf8(const std::wstring& wstr) {
return result;
}
std::string CleanDisplayName(const std::wstring& wide_name) {
std::string GetDisplayLabel(const std::wstring& wide_name) {
std::string name = WideToUtf8(wide_name);
name.erase(std::remove_if(name.begin(), name.end(),
[](unsigned char c) { return !std::isalnum(c); }),
name.end());
constexpr char kDevicePrefix[] = "\\\\.\\";
if (name.rfind(kDevicePrefix, 0) == 0) {
name.erase(0, sizeof(kDevicePrefix) - 1);
}
return name;
}
@@ -38,7 +40,7 @@ BOOL WINAPI EnumMonitorProc(HMONITOR hmonitor, [[maybe_unused]] HDC hdc,
monitor_info_.cbSize = sizeof(MONITORINFOEX);
if (GetMonitorInfo(hmonitor, &monitor_info_)) {
std::string display_name = CleanDisplayName(monitor_info_.szDevice);
std::string display_name = GetDisplayLabel(monitor_info_.szDevice);
if (monitor_info_.dwFlags & MONITORINFOF_PRIMARY) {
gs_display_list.insert(
gs_display_list.begin(),
@@ -402,8 +404,9 @@ void ScreenCapturerWgc::OnFrame(const WgcSession::wgc_session_frame& frame,
(uint8_t*)(nv12_frame_ + even_width * even_height),
even_width, even_width, even_height);
const std::string stream_id = MakeDisplayStreamId(id);
on_data_(nv12_frame_, nv12_size, even_width, even_height,
display_info_list_[id].name.c_str());
stream_id.c_str());
}
}
@@ -14,6 +14,7 @@
#include <utility>
#include <vector>
#include "display_stream_id.h"
#include "interactive_state.h"
#include "rd_log.h"
#include "screen_capturer_dxgi.h"
@@ -403,37 +404,37 @@ int ScreenCapturerWin::Init(const int fps, cb_desktop_data cb) {
fps_ = fps;
cb_orig_ = cb;
cb_ = [this](unsigned char* data, int size, int w, int h,
const char* display_name) {
const char* reported_stream_id) {
if (secure_desktop_capture_active_.load(std::memory_order_relaxed)) {
return;
}
const char* raw_display_name = display_name ? display_name : "";
std::string mapped_name;
const char* raw_stream_id = reported_stream_id ? reported_stream_id : "";
std::string mapped_stream_id;
{
std::lock_guard<std::mutex> lock(alias_mutex_);
auto it = label_alias_.find(raw_display_name);
if (it != label_alias_.end())
mapped_name = it->second;
else
mapped_name = raw_display_name;
auto it = stream_id_alias_.find(raw_stream_id);
if (it != stream_id_alias_.end()) {
mapped_stream_id = it->second;
} else {
// Unknown backend labels are presentation data, not protocol IDs.
// Resolve them through the selected logical display instead.
mapped_stream_id.clear();
}
{
std::lock_guard<std::mutex> lock(alias_mutex_);
if (canonical_labels_.find(mapped_name) == canonical_labels_.end()) {
if (post_secure_desktop_waiting_for_frame_.load(
std::memory_order_relaxed) &&
!post_secure_desktop_drop_logged_.exchange(
true, std::memory_order_relaxed)) {
LOG_WARN(
"Windows capturer dropping post-secure-desktop frame from "
"unknown display: display='{}', mapped='{}', size={}x{}, "
"bytes={}",
raw_display_name, mapped_name, w, h, size);
mapped_stream_id = ResolveDisplayStreamId(
mapped_stream_id.c_str(), canonical_displays_.size(),
monitor_index_.load(std::memory_order_relaxed));
}
if (mapped_stream_id.empty()) {
if (!invalid_stream_id_logged_.exchange(true,
std::memory_order_relaxed)) {
LOG_WARN("Windows capturer dropping frame without a registered stream "
"id: reported='{}', size={}x{}, bytes={}",
raw_stream_id, w, h, size);
}
return;
}
}
invalid_stream_id_logged_.store(false, std::memory_order_relaxed);
if (post_secure_desktop_waiting_for_frame_.exchange(
false, std::memory_order_relaxed)) {
const ULONGLONG start_tick =
@@ -445,10 +446,11 @@ int ScreenCapturerWin::Init(const int fps, cb_desktop_data cb) {
std::memory_order_relaxed);
LOG_INFO(
"Windows capturer first normal frame after secure desktop: "
"display='{}', mapped='{}', size={}x{}, bytes={}, elapsed_ms={}",
raw_display_name, mapped_name, w, h, size, elapsed_ms);
"reported_stream='{}', mapped_stream='{}', size={}x{}, bytes={}, "
"elapsed_ms={}",
raw_stream_id, mapped_stream_id, w, h, size, elapsed_ms);
}
if (cb_orig_) cb_orig_(data, size, w, h, mapped_name.c_str());
if (cb_orig_) cb_orig_(data, size, w, h, mapped_stream_id.c_str());
};
int ret = -1;
@@ -508,9 +510,8 @@ int ScreenCapturerWin::Destroy() {
}
{
std::lock_guard<std::mutex> lock(alias_mutex_);
label_alias_.clear();
handle_to_canonical_.clear();
canonical_labels_.clear();
stream_id_alias_.clear();
handle_to_canonical_index_.clear();
}
return 0;
}
@@ -523,10 +524,26 @@ int ScreenCapturerWin::Start(bool show_cursor) {
show_cursor_.store(show_cursor, std::memory_order_relaxed);
paused_.store(false, std::memory_order_relaxed);
invalid_stream_id_logged_.store(false, std::memory_order_relaxed);
int ret = impl_->Start(show_cursor);
// Refresh physical monitor identities before every session. HMONITOR and
// DXGI output handles may change after an HDMI hotplug while CrossDesk stays
// open; logical stream IDs must remain stable.
const int requested_monitor =
monitor_index_.load(std::memory_order_relaxed);
impl_->Destroy();
int ret = impl_->Init(fps_, cb_);
if (ret == 0) {
RebuildAliasesFromImpl();
ret = impl_->Start(show_cursor);
if (ret == 0 && requested_monitor > 0 &&
impl_->SwitchTo(requested_monitor) != 0) {
monitor_index_.store(0, std::memory_order_relaxed);
}
}
if (ret != 0) {
LOG_WARN("Windows capturer: Start failed (ret={}), trying fallback", ret);
LOG_WARN("Windows capturer: refresh/start failed (ret={}), trying fallback",
ret);
auto try_init_start = [&](std::unique_ptr<ScreenCapturer> cand) -> bool {
int r = cand->Init(fps_, cb_);
@@ -536,6 +553,10 @@ int ScreenCapturerWin::Start(bool show_cursor) {
impl_ = std::move(cand);
impl_is_wgc_plugin_ = false;
RebuildAliasesFromImpl();
if (requested_monitor > 0 &&
impl_->SwitchTo(requested_monitor) != 0) {
monitor_index_.store(0, std::memory_order_relaxed);
}
return true;
}
return false;
@@ -624,25 +645,49 @@ int ScreenCapturerWin::ResetToInitialMonitor() {
std::vector<DisplayInfo> ScreenCapturerWin::GetDisplayInfoList() {
if (!impl_) return {};
return impl_->GetDisplayInfoList();
std::lock_guard<std::mutex> lock(alias_mutex_);
return canonical_displays_;
}
void ScreenCapturerWin::BuildCanonicalFromImpl() {
std::lock_guard<std::mutex> lock(alias_mutex_);
handle_to_canonical_.clear();
label_alias_.clear();
handle_to_canonical_index_.clear();
stream_id_alias_.clear();
canonical_displays_ = impl_->GetDisplayInfoList();
canonical_labels_.clear();
for (const auto& di : canonical_displays_) {
handle_to_canonical_[di.handle] = di.name;
canonical_labels_.insert(di.name);
std::unordered_map<std::string, size_t> name_counts;
for (const auto& display : canonical_displays_) {
if (!display.name.empty()) {
++name_counts[display.name];
}
}
for (size_t i = 0; i < canonical_displays_.size(); ++i) {
auto& di = canonical_displays_[i];
const std::string stream_id = MakeDisplayStreamId(i);
if (di.name.empty()) {
di.name = stream_id;
}
handle_to_canonical_index_[di.handle] = i;
stream_id_alias_[stream_id] = stream_id;
if (name_counts[di.name] == 1) {
// Compatibility with an older WGC plugin that reports display names.
stream_id_alias_[di.name] = stream_id;
}
}
}
void ScreenCapturerWin::RebuildAliasesFromImpl() {
std::lock_guard<std::mutex> lock(alias_mutex_);
label_alias_.clear();
stream_id_alias_.clear();
auto current = impl_->GetDisplayInfoList();
if (current.empty() || canonical_displays_.empty()) return;
const auto previous_handles = handle_to_canonical_index_;
handle_to_canonical_index_.clear();
std::unordered_map<std::string, size_t> name_counts;
for (const auto& display : current) {
if (!display.name.empty()) {
++name_counts[display.name];
}
}
auto similar = [&](const DisplayInfo& a, const DisplayInfo& b) {
int dl = std::abs(a.left - b.left);
int dt = std::abs(a.top - b.top);
@@ -650,21 +695,52 @@ void ScreenCapturerWin::RebuildAliasesFromImpl() {
int dh = std::abs(a.height - b.height);
return dl <= 10 && dt <= 10 && dw <= 20 && dh <= 20;
};
for (const auto& di : current) {
std::string canonical;
auto it = handle_to_canonical_.find(di.handle);
if (it != handle_to_canonical_.end()) {
canonical = it->second;
} else {
for (const auto& c : canonical_displays_) {
if (similar(di, c) || (di.is_primary && c.is_primary)) {
canonical = c.name;
std::vector<bool> used(canonical_displays_.size(), false);
for (size_t current_index = 0; current_index < current.size();
++current_index) {
const auto& di = current[current_index];
int canonical_index = -1;
auto old_handle = previous_handles.find(di.handle);
if (old_handle != previous_handles.end() &&
old_handle->second < canonical_displays_.size() &&
!used[old_handle->second]) {
canonical_index = static_cast<int>(old_handle->second);
}
if (canonical_index < 0) {
for (size_t i = 0; i < canonical_displays_.size(); ++i) {
if (!used[i] && (similar(di, canonical_displays_[i]) ||
(di.is_primary && canonical_displays_[i].is_primary))) {
canonical_index = static_cast<int>(i);
break;
}
}
}
if (!canonical.empty() && canonical != di.name) {
label_alias_[di.name] = canonical;
if (canonical_index < 0 && current_index < canonical_displays_.size() &&
!used[current_index]) {
canonical_index = static_cast<int>(current_index);
}
if (canonical_index < 0) {
LOG_WARN("Windows capturer ignoring unregistered display after topology "
"change: display='{}', index={}",
di.name, current_index);
continue;
}
used[canonical_index] = true;
auto& canonical = canonical_displays_[canonical_index];
const std::string backend_stream_id =
MakeDisplayStreamId(current_index);
const std::string stable_stream_id =
MakeDisplayStreamId(static_cast<size_t>(canonical_index));
stream_id_alias_[backend_stream_id] = stable_stream_id;
if (!di.name.empty() && name_counts[di.name] == 1) {
stream_id_alias_[di.name] = stable_stream_id;
}
handle_to_canonical_index_[di.handle] =
static_cast<size_t>(canonical_index);
canonical = di;
if (canonical.name.empty()) {
canonical.name = stable_stream_id;
}
}
}
@@ -792,7 +868,7 @@ bool ScreenCapturerWin::GetCurrentCaptureRegion(int* left, int* top, int* width,
*top = display.top;
*width = capture_width;
*height = capture_height;
*display_name = display.name;
*display_name = MakeDisplayStreamId(static_cast<size_t>(current_monitor));
return true;
}
@@ -10,12 +10,12 @@
#include <Windows.h>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <memory>
#include <mutex>
#include <thread>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include "screen_capturer.h"
@@ -48,11 +48,11 @@ class ScreenCapturerWin : public ScreenCapturer {
cb_desktop_data cb_;
cb_desktop_data cb_orig_;
std::unordered_map<void*, std::string> handle_to_canonical_;
std::unordered_map<std::string, std::string> label_alias_;
std::unordered_map<void*, size_t> handle_to_canonical_index_;
std::unordered_map<std::string, std::string> stream_id_alias_;
std::mutex alias_mutex_;
std::vector<DisplayInfo> canonical_displays_;
std::unordered_set<std::string> canonical_labels_;
std::atomic<bool> invalid_stream_id_logged_{false};
std::atomic<bool> running_{false};
std::atomic<bool> paused_{false};
std::atomic<bool> show_cursor_{true};
+312
View File
@@ -1,11 +1,14 @@
#include "crossdesk_ui.h"
#include "fa_solid_900.h"
#include "ui/ui_localization.h"
#include <cassert>
#include <charconv>
#include <cstdlib>
#include <fstream>
#include <memory>
#include <string>
#include <string_view>
#include <vector>
namespace {
@@ -36,9 +39,309 @@ bool RegisterFontAwesome(slint::Window &window) {
.has_value();
}
struct CaptureOptions {
bool enabled = false;
std::string page;
int language = 0;
std::string snapshot_path;
};
// Compatibility entry point for the former application-level debug capture
// mode. CROSSDESK_UI_CAPTURE keeps the interactive workflow, while
// CROSSDESK_UI_CAPTURE_SNAPSHOT writes a deterministic PPM and exits so every
// page can be exercised in automation. The legacy /tmp overrides still win.
std::string ReadFirstLine(const char *path) {
std::ifstream input(path);
std::string value;
std::getline(input, value);
return value;
}
CaptureOptions LoadCaptureOptions() {
CaptureOptions options;
if (const char *capture = std::getenv("CROSSDESK_UI_CAPTURE")) {
options.enabled = std::string_view(capture) != "0";
}
if (!options.enabled) {
return options;
}
if (const char *page = std::getenv("CROSSDESK_UI_CAPTURE_PAGE")) {
options.page = page;
}
if (const std::string page_override =
ReadFirstLine("/tmp/crossdesk-ui-capture-page");
!page_override.empty()) {
options.page = page_override;
}
std::string language_value =
ReadFirstLine("/tmp/crossdesk-ui-capture-language");
if (language_value.empty()) {
if (const char *language =
std::getenv("CROSSDESK_UI_CAPTURE_LANGUAGE")) {
language_value = language;
}
}
if (!language_value.empty()) {
int language = 0;
const auto [end, error] = std::from_chars(
language_value.data(), language_value.data() + language_value.size(),
language);
if (error == std::errc{} &&
end == language_value.data() + language_value.size()) {
options.language =
crossdesk::localization::detail::ClampLanguageIndex(language);
}
}
if (const char *snapshot =
std::getenv("CROSSDESK_UI_CAPTURE_SNAPSHOT")) {
options.snapshot_path = snapshot;
}
return options;
}
bool HasNonEmptyEnvironmentVariable(const char *name) {
const char *value = std::getenv(name);
return value != nullptr && value[0] != '\0';
}
void ResetMainCaptureState(
const slint::ComponentHandle<crossdesk::ui::MainWindow> &window) {
window->set_settings_open(false);
window->set_self_host_settings_open(false);
window->set_about_open(false);
window->set_update_open(false);
window->set_update_available(false);
window->set_reset_password_open(false);
window->set_reset_password_invalid(false);
window->set_new_password_input("");
window->set_alias_open(false);
window->set_alias_input("");
window->set_delete_open(false);
window->set_offline_warning("");
window->set_connection_dialog_open(false);
window->set_connection_status_text("");
window->set_connection_pending(false);
window->set_connection_password_required(false);
window->set_connection_validating(false);
window->set_permission_dialog_open(false);
window->set_screen_recording_granted(true);
window->set_accessibility_granted(true);
window->set_settings_session_active(false);
window->set_portable_service_settings_visible(false);
window->set_portable_service_dialog_open(false);
window->set_portable_service_suppressed_notice_open(false);
}
void ConfigureMainCapturePage(
const slint::ComponentHandle<crossdesk::ui::MainWindow> &window,
const CaptureOptions &options) {
const int language = crossdesk::ui_localization::ApplyMainWindowStrings(
window, options.language);
crossdesk::ui_localization::ApplyStreamWindowStrings(window, language);
window->set_language_index(language);
ResetMainCaptureState(window);
#if _WIN32
window->set_custom_titlebar(true);
window->set_wayland_titlebar(false);
#if CROSSDESK_PORTABLE
window->set_portable_service_settings_visible(true);
#endif
#elif defined(__linux__)
const bool has_x11 = HasNonEmptyEnvironmentVariable("DISPLAY");
const bool use_xwayland =
has_x11 && HasNonEmptyEnvironmentVariable("WAYLAND_DISPLAY");
window->set_custom_titlebar(has_x11);
window->set_wayland_titlebar(use_xwayland);
#else
window->set_custom_titlebar(false);
window->set_wayland_titlebar(false);
#endif
if (options.page == "settings") {
window->set_settings_open(true);
} else if (options.page == "self-hosted") {
window->set_settings_open(true);
window->set_self_host_settings_open(true);
} else if (options.page == "about") {
window->set_current_version("0.0.0");
window->set_about_open(true);
} else if (options.page == "update") {
window->set_update_available(true);
window->set_latest_version("v0.0.1");
window->set_release_name("CrossDesk Preview");
window->set_release_date("2026-08-25");
window->set_update_open(true);
} else if (options.page == "reset-password") {
window->set_reset_password_open(true);
} else if (options.page == "reset-password-invalid") {
window->set_reset_password_open(true);
window->set_reset_password_invalid(true);
window->set_new_password_input("123");
} else if (options.page == "alias") {
window->set_alias_input("Office workstation");
window->set_alias_open(true);
} else if (options.page == "delete") {
window->set_delete_open(true);
} else if (options.page == "offline") {
window->set_offline_warning(crossdesk::ui_localization::Text(
crossdesk::localization::device_offline[language]));
} else if (options.page == "connection") {
window->set_connection_status_text(crossdesk::ui_localization::Text(
crossdesk::localization::p2p_connecting[language]));
window->set_connection_pending(true);
window->set_connection_dialog_open(true);
} else if (options.page == "connection-password") {
window->set_connection_status_text(crossdesk::ui_localization::Text(
crossdesk::localization::reinput_password[language]));
window->set_connection_password_required(true);
window->set_connection_dialog_open(true);
} else if (options.page == "service") {
window->set_portable_service_settings_visible(true);
window->set_portable_service_dialog_open(true);
} else if (options.page == "service-notice") {
window->set_portable_service_settings_visible(true);
window->set_portable_service_suppressed_notice_open(true);
} else if (options.page == "permission") {
window->set_permission_dialog_open(true);
window->set_screen_recording_granted(false);
window->set_accessibility_granted(false);
}
}
slint::Image CreateStreamPreviewImage() {
constexpr int preview_width = 640;
constexpr int preview_height = 360;
slint::SharedPixelBuffer<slint::Rgb8Pixel> pixels(preview_width,
preview_height);
auto *data = reinterpret_cast<uint8_t *>(pixels.begin());
for (int y = 0; y < preview_height; ++y) {
for (int x = 0; x < preview_width; ++x) {
const bool border = x < 8 || y < 8 || x >= preview_width - 8 ||
y >= preview_height - 8;
const size_t offset = (static_cast<size_t>(y) * preview_width + x) * 3;
data[offset] =
border ? 240 : static_cast<uint8_t>(35 + 150 * x / preview_width);
data[offset + 1] =
border ? 70 : static_cast<uint8_t>(35 + 150 * y / preview_height);
data[offset + 2] = border ? 70 : 90;
}
}
return slint::Image(std::move(pixels));
}
void ConfigureStreamCapturePage(
const slint::ComponentHandle<crossdesk::ui::StreamWindow> &stream,
int language) {
language = crossdesk::ui_localization::ApplyStreamWindowStrings(
stream, language);
#if defined(__linux__)
stream->set_custom_titlebar(
HasNonEmptyEnvironmentVariable("DISPLAY") &&
HasNonEmptyEnvironmentVariable("WAYLAND_DISPLAY"));
#else
stream->set_custom_titlebar(false);
#endif
stream->set_native_video_enabled(false);
stream->set_window_maximized(false);
stream->set_fullscreen_enabled(false);
stream->set_stats_visible(false);
stream->set_file_transfer_visible(false);
stream->set_receiving_text("");
stream->set_status_text("");
crossdesk::ui::StreamTab tab;
tab.remote_id = "589173341";
tab.title = "Mac";
tab.connected = true;
stream->set_tabs(
std::make_shared<slint::VectorModel<crossdesk::ui::StreamTab>>(
std::vector{tab}));
stream->set_displays(
std::make_shared<slint::VectorModel<slint::SharedString>>(
std::vector{slint::SharedString("Display 1")}));
stream->set_frame(CreateStreamPreviewImage());
stream->set_has_frame(true);
const float height = stream->get_custom_titlebar() ? 752.0f : 720.0f;
stream->window().set_size(
slint::LogicalSize(slint::Size<float>{1280.0f, height}));
}
void ConfigureServerCapturePage(
const slint::ComponentHandle<crossdesk::ui::ServerWindow> &server,
int language) {
language = crossdesk::localization::detail::ClampLanguageIndex(language);
crossdesk::ui::ControllerEntry controller;
controller.remote_id = "589173341";
controller.display_name = "Mac";
server->set_controllers(
std::make_shared<slint::VectorModel<crossdesk::ui::ControllerEntry>>(
std::vector{controller}));
server->set_controller_names(
std::make_shared<slint::VectorModel<slint::SharedString>>(
std::vector{slint::SharedString("Mac")}));
server->set_language_index(language);
server->set_controller_label(crossdesk::ui_localization::Text(
crossdesk::localization::controller[language]));
server->set_connection_label(crossdesk::ui_localization::Text(
crossdesk::localization::connection_status[language]));
server->set_connection_status(crossdesk::ui_localization::Text(
crossdesk::localization::p2p_connected[language]));
server->set_file_transfer_label(crossdesk::ui_localization::Text(
crossdesk::localization::file_transfer[language]));
server->set_select_file_label(crossdesk::ui_localization::Text(
crossdesk::localization::select_file[language]));
server->set_file_transfer_visible(false);
server->set_sending_file(false);
const float width = language == 0 ? 250.0f : language == 1 ? 330.0f : 430.0f;
server->window().set_size(
slint::LogicalSize(slint::Size<float>{width, 150.0f}));
}
int RunCaptureMode(
const CaptureOptions &options,
slint::ComponentHandle<crossdesk::ui::MainWindow> &window,
slint::ComponentHandle<crossdesk::ui::StreamWindow> &stream,
slint::ComponentHandle<crossdesk::ui::ServerWindow> &server) {
ConfigureMainCapturePage(window, options);
stream->hide();
server->hide();
slint::Window *capture_window = &window->window();
if (options.page == "stream") {
ConfigureStreamCapturePage(stream, options.language);
window->hide();
stream->show();
capture_window = &stream->window();
} else if (options.page == "server") {
ConfigureServerCapturePage(server, options.language);
window->hide();
server->show();
capture_window = &server->window();
} else {
window->show();
}
if (!options.snapshot_path.empty()) {
return WriteWindowSnapshot(*capture_window, options.snapshot_path.c_str())
? 0
: 7;
}
if (options.page == "stream" || options.page == "server") {
slint::run_event_loop();
} else {
window->run();
}
return 0;
}
} // namespace
int main() {
const CaptureOptions capture_options = LoadCaptureOptions();
auto window = crossdesk::ui::MainWindow::create();
auto stream = crossdesk::ui::StreamWindow::create();
auto server = crossdesk::ui::ServerWindow::create();
@@ -143,10 +446,15 @@ int main() {
std::make_shared<slint::VectorModel<crossdesk::ui::StreamTab>>(
std::vector{tab}));
stream->set_custom_titlebar(true);
stream->set_native_video_enabled(true);
stream->set_window_maximized(true);
assert(stream->get_tabs()->row_count() == 1);
assert(stream->get_custom_titlebar());
assert(stream->get_native_video_enabled());
assert(stream->get_window_maximized());
// Keep snapshots on the CPU-rendered UI path; the native Metal video view
// is intentionally outside Slint's component snapshot.
stream->set_native_video_enabled(false);
if (const char *snapshot_path =
std::getenv("CROSSDESK_STREAM_UI_SNAPSHOT")) {
stream->set_window_maximized(false);
@@ -276,5 +584,9 @@ int main() {
server->invoke_controller_selected(0);
assert(controller_selected);
if (capture_options.enabled) {
return RunCaptureMode(capture_options, window, stream, server);
}
return 0;
}
+4 -2
View File
@@ -32,6 +32,7 @@ package("slint")
break
end
end
local use_macos_metal = package:is_plat("macosx")
local configs = {
"-DSLINT_BUILD_TESTING=OFF",
"-DSLINT_BUILD_EXAMPLES=OFF",
@@ -41,10 +42,11 @@ package("slint")
"-DSLINT_FEATURE_SYSTEM_TESTING=OFF",
"-DSLINT_FEATURE_MCP=OFF",
"-DSLINT_FEATURE_BACKEND_QT=OFF",
"-DSLINT_FEATURE_RENDERER_SKIA=OFF",
"-DSLINT_FEATURE_RENDERER_SKIA=" .. (use_macos_metal and "ON" or "OFF"),
"-DSLINT_FEATURE_RENDERER_SKIA_OPENGL=OFF",
"-DSLINT_FEATURE_RENDERER_SKIA_VULKAN=OFF",
"-DSLINT_FEATURE_RENDERER_FEMTOVG=ON",
"-DSLINT_FEATURE_RENDERER_FEMTOVG=" .. (use_macos_metal and "OFF" or "ON"),
"-DSLINT_FEATURE_RENDERER_FEMTOVG_WGPU=OFF",
"-DSLINT_FEATURE_RENDERER_SOFTWARE=ON",
"-DSLINT_STYLE=fluent",
"-DBUILD_SHARED_LIBS=ON"
+5 -3
View File
@@ -80,10 +80,12 @@ function setup_platform_settings()
add_cxflags("-Wno-unused-variable")
elseif is_os("macosx") then
if is_arch("x86_64") then
add_ldflags("-Wl,-ld_classic")
end
add_cxflags("-Wno-unused-variable")
add_frameworks("Cocoa", "OpenGL", "IOSurface", "ScreenCaptureKit",
"AVFoundation", "CoreMedia", "CoreVideo", "CoreAudio",
"AudioToolbox")
add_frameworks("Cocoa", "Metal", "QuartzCore", "IOSurface",
"ScreenCaptureKit", "AVFoundation", "CoreMedia", "CoreVideo",
"CoreAudio", "AudioToolbox")
end
end
+15 -4
View File
@@ -33,6 +33,7 @@ function setup_targets()
target("common")
set_kind("object")
add_deps("rd_log")
add_packages("libyuv")
add_files("src/common/*.cpp")
remove_files("src/common/rounded_corner_button.cpp")
if is_os("macosx") then
@@ -112,7 +113,8 @@ function setup_targets()
add_cxxflags("/bigobj")
end
add_packages("slint")
add_includedirs("src/gui/assets/fonts")
add_includedirs("src/gui", "src/gui/assets/fonts",
"src/gui/assets/localization")
add_rules("slint")
add_files("src/gui/ui/crossdesk_ui.slint")
add_files("tests/slint_ui_smoke_test.cpp")
@@ -268,6 +270,7 @@ function setup_targets()
"device_controller", "thumbnail", "version_checker", "tools")
add_files("src/gui/render.cpp", "src/gui/application/gui_application.cpp",
"src/gui/application/portable_service_integration.cpp",
"src/gui/rendering/*.cpp",
"src/gui/runtime/*.cpp",
"src/gui/features/devices/*.cpp", "src/gui/features/input/*.cpp",
"src/gui/features/clipboard/*.cpp", "src/gui/features/file_transfer/*.cpp",
@@ -275,13 +278,21 @@ function setup_targets()
add_includedirs("src/gui", {public = true})
if is_os("windows") then
add_cxxflags("/bigobj")
add_files("src/gui/platform/tray/win_tray.cpp")
add_links("opengl32")
add_files("src/gui/platform/tray/win_tray.cpp",
"src/gui/platform/opengl_video_renderer.cpp",
"src/gui/platform/video_renderer_factory_opengl.cpp")
add_includedirs("src/service/windows", {public = true})
elseif is_os("macosx") then
add_files("src/gui/runtime/*.mm", "src/gui/platform/tray/*.mm",
"src/gui/platform/window_drag_mac.mm")
"src/gui/platform/window_drag_mac.mm",
"src/gui/platform/metal_video_renderer.mm",
"src/gui/platform/video_renderer_factory_metal.mm")
elseif is_os("linux") then
add_files("src/gui/platform/tray/linux_tray.cpp")
add_links("GL")
add_files("src/gui/platform/tray/linux_tray.cpp",
"src/gui/platform/opengl_video_renderer.cpp",
"src/gui/platform/video_renderer_factory_opengl.cpp")
end
if is_os("windows") then