diff --git a/PRIVACY.md b/PRIVACY.md index 3bb6135b..66801b5b 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -1,18 +1,18 @@ # CrossDesk Privacy Policy -Last updated: October 10, 2026 +Last updated: October 15, 2026 [中文](#中文) | [English](#english) ## 中文 -更新日期:2026 年 10 月 10 日 +更新日期:2026 年 10 月 15 日 CrossDesk 重视您的个人信息与隐私保护。本政策说明我们在提供远程桌面服务时如何处理相关信息,以及您可以如何管理授权、保存的内容和数据请求。请在使用前阅读并了解本政策。 ### 一、适用范围 -本政策适用于 CrossDesk 桌面客户端、iOS 与 Android 客户端,以及由 CrossDesk 项目维护者(以下简称“我们”)提供的默认网络服务。 +本政策适用于 CrossDesk 桌面客户端、iOS、Android 与鸿蒙移动客户端,以及由 CrossDesk 项目维护者(以下简称“我们”)提供的默认网络服务。 如果您选择自行部署或使用其他运营者提供的服务器,该服务器所处理的信息、保存期限及安全管理由相应运营者负责。请在使用前了解其隐私政策。本政策中有关客户端本地数据处理的说明仍适用。 @@ -50,11 +50,13 @@ Android 中的设备身份及您选择保存的访问密码使用 Android Keysto iOS 通过系统钥匙串和文件保护机制保护相应本地数据,默认信令连接进行 TLS 证书校验。请妥善保管设备和连接凭据,避免向无关人员共享访问密码。 -Android 默认信令连接同样进行 TLS 证书校验。 +Android 与鸿蒙移动端默认信令连接同样进行 TLS 证书校验。 + +鸿蒙移动端使用系统 Asset Store 加密保存设备身份及您选择保存的访问密码,明确禁止凭据跨设备同步。设置和连接记录按服务器隔离。远程预览默认关闭,单独确认后从下一次连接起保存一张 640×360 预览;预览保存在应用私有缓存中,不上传,可随时清除,也可能被系统回收。关闭预览或撤回联网授权会清除预览。接收文件保存在应用私有目录,您可在会话中通过系统文件选择器导出。 ### 五、移动端权限与授权选择 -**隐私授权。** iOS 与 Android 首次使用,或从未提供此选择的旧版升级时,您需阅读本政策、主动勾选并点击同意后,客户端才会登记本机身份、连接信令服务和查询设备在线状态。拒绝后仍可查看设置、离线隐私政策和本地连接记录,远程连接功能暂不可用。未授权时,重新打开或从后台返回应用会再次提示。 +**隐私授权。** iOS、Android 与鸿蒙移动端 首次使用,或从未提供此选择的旧版升级时,您需阅读本政策、主动勾选并点击同意后,客户端才会登记本机身份、连接信令服务和查询设备在线状态。拒绝后仍可查看设置、离线隐私政策和本地连接记录,远程连接功能暂不可用。未授权时,重新打开或从后台返回应用会再次提示。 **撤回同意。** 您可以在“设置 → 隐私 → 隐私与授权”中查看隐私政策、授权状态或撤回授权。客户端将停止远程会话、信令连接和版本检查,关闭并清除本地画面预览。再次使用连接功能前需重新同意;重新同意不会自动开启画面预览保存。撤回不影响撤回前已经发生的信息处理,也不等同于删除服务端数据。 @@ -64,6 +66,8 @@ Android 客户端同样作为控制端接收远程画面与音频,不采集本 Android 远程会话优先使用系统画中画在切换应用后继续显示画面。画中画不可用时,通过前台服务和连接通知保持您已建立的远程连接;后台暂停本地画面渲染和音频播放,不申请悬浮窗权限。通知权限用于提供返回及断开入口,CPU 唤醒锁用于会话后台期间维持连接,返回前台或会话结束时释放。您可以在应用或连接通知中断开,移除应用任务也会结束会话及保活服务。 +鸿蒙手机和平板只作为控制端,不申请本机录屏、输入注入、摄像头或麦克风权限;文件选择和剪贴板操作由您主动触发。已建立会话在切入后台时优先使用系统画中画,提供静音控制;画中画不可用时申请数据传输后台任务并显示系统通知。若系统不允许或停止后台任务且画中画不可用,会话会断开。后台切换释放远端按键;空闲后台不维持信令连接。完成联网授权后,应用在前台检查鸿蒙本机版本,并在连接后检查远端版本;版本请求不携带设备身份或远端版本。 + 应用及通信组件使用本地文件元数据、应用偏好和计时接口实现文件传输、设置保存及网络计时,不使用这些信息制作设备指纹或进行广告追踪。 ### 六、查询与删除请求 @@ -86,13 +90,13 @@ Android 远程会话优先使用系统画中画在切换应用后继续显示画 ## English -Updated: October 2, 2026 +Updated: October 15, 2026 CrossDesk respects your privacy. This policy explains how information is processed when we provide remote desktop services and how you can manage consent, locally saved content and data requests. Please read this policy before using the software. ### 1. Scope -This policy applies to the CrossDesk desktop, iOS and Android clients and the default network services provided by the CrossDesk project maintainers ("we" or "us"). +This policy applies to the CrossDesk desktop, iOS, Android and HarmonyOS mobile clients and the default network services provided by the CrossDesk project maintainers ("we" or "us"). If you deploy your own server or choose a server provided by another operator, that operator is responsible for its data processing, retention and security practices. Please review its privacy policy before use. The descriptions of local client data processing in this policy still apply. @@ -130,11 +134,13 @@ On Android, device identities and access passwords you choose to save are encryp On iOS, the system Keychain and file protection mechanisms protect the relevant local data, and default signaling connections validate TLS certificates. Keep your devices and connection credentials secure and avoid sharing access passwords with unrelated people. -Default signaling connections on Android also validate TLS certificates. +Default signaling connections on Android and HarmonyOS mobile also validate TLS certificates. + +HarmonyOS mobile uses the system Asset Store to encrypt device identities and saved access passwords, with cross-device credential synchronization disabled. Settings and connection records are scoped to the server. Remote previews are off by default. After separate confirmation, one 640×360 preview is saved per connection starting with the next connection. Previews remain in private app cache, are never uploaded, can be cleared and may be reclaimed by the system. Disabling previews or withdrawing network consent clears them. Received files remain in private app storage and can be exported through the system document picker during the session. ### 5. Mobile permissions and consent choices -**Privacy authorization.** On first use of iOS or Android, or when upgrading from a version without this choice, you must read this policy, actively select the checkbox and agree before the client registers a local identity, connects to signaling or queries device presence. If you decline, settings, the offline policy and local connection history remain available, while remote connections are unavailable. Without consent, reopening the app or returning from the background presents the notice again. +**Privacy authorization.** On first use of iOS, Android or HarmonyOS mobile, or when upgrading from a version without this choice, you must read this policy, actively select the checkbox and agree before the client registers a local identity, connects to signaling or queries device presence. If you decline, settings, the offline policy and local connection history remain available, while remote connections are unavailable. Without consent, reopening the app or returning from the background presents the notice again. **Withdrawal.** You can review the privacy policy and your authorization status or withdraw consent in Settings → Privacy → Privacy and Authorization. The client stops remote sessions, signaling and version checks, disables preview saving and clears local previews. Consent is required again before reconnecting. Agreeing again does not automatically enable preview saving. Withdrawal does not undo processing that already occurred or delete server data. @@ -144,6 +150,8 @@ The Android client also receives remote screens and audio as a controller and do Android remote sessions prefer system picture-in-picture to continue displaying video when you switch apps. If picture-in-picture is unavailable, a foreground service and connection notification maintain the remote connection you have established. Local video rendering and audio playback pause in the background; no overlay permission is requested. Notification permission provides return and disconnect actions. A CPU wake lock helps maintain the connection while the session is backgrounded and is released on foreground return or session end. You can disconnect in the app or its connection notification; removing the app task also ends the session and its foreground service. +HarmonyOS phones and tablets act only as controllers and do not request local screen recording, input injection, camera or microphone permissions. File selection and clipboard actions are user initiated. An established session prefers system picture-in-picture with a mute control when backgrounded. If unavailable, the app requests a data-transfer background task with a system notification. The session ends if the system disallows or stops that task while picture-in-picture is unavailable. Backgrounding releases remote keys; idle background sessions do not retain signaling. After network consent, the foreground app checks the local HarmonyOS version and checks the remote version after connection. These requests contain neither device identity nor the remote version. + The app and its communication components use local file metadata, preferences and timing APIs for file transfer, settings and network timing, not for device fingerprinting or advertising tracking. ### 6. Access and deletion requests diff --git a/README.md b/README.md index 1dac846b..3b2f8152 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ CrossDesk 支持 Windows、macOS、Linux 之间的远程访问,也可从浏览 | Linux | Ubuntu 20.04+,amd64 / arm64,glibc 2.31 基线 | 普通版 / Xfce 虚拟桌面版 `.deb` | | iOS / iPadOS | iOS 16.0+,arm64 真机 | 原生客户端;使用 Xcode 构建并签名,CI 产物为未签名应用 | | Android | Android 8.0+,arm64 | 原生控制端;使用 Gradle 构建 APK,见 [开发说明](apps/android/README.md) | +| HarmonyOS(开发版) | Slint PC:API 26;ArkUI 移动端:API 20+;arm64,待真机验收 | [Slint PC 共用桌面实现](apps/harmony/desktop-port/README.md);[鸿蒙构建与移动端说明](apps/harmony/README.md),使用 DevEco 构建并签名 | | Web | 支持 WebRTC 的浏览器 | 访问 [web.crossdesk.cn](https://web.crossdesk.cn/) | Linux 每个架构提供两个可独立安装的包,**选择其中一个**: diff --git a/README_EN.md b/README_EN.md index 43609bf9..28aef343 100644 --- a/README_EN.md +++ b/README_EN.md @@ -80,6 +80,7 @@ Choose a package matching your OS and CPU architecture from [GitHub Releases](ht | Linux | Ubuntu 20.04+, amd64 / arm64, glibc 2.31 baseline | Standard / Xfce virtual desktop `.deb` | | iOS / iPadOS | iOS 16.0+, arm64 physical device | Native controller; build and sign with Xcode, or sign the unsigned CI app | | Android | Android 8.0+, arm64 | Native controller; build an APK with Gradle ([guide](apps/android/README.md)) | +| HarmonyOS (development) | Slint PC: API 26; ArkUI mobile: API 20+; arm64, device validation pending | [Shared Slint desktop port](apps/harmony/desktop-port/README.md); [HarmonyOS build and mobile guide](apps/harmony/README.md) (Chinese), build and sign with DevEco | | Web | A WebRTC-capable browser | Open [web.crossdesk.cn](https://web.crossdesk.cn/) | Linux provides two standalone packages per architecture. **Choose one**: diff --git a/apps/desktop/src/device_controller/device_controller.h b/apps/desktop/src/device_controller/device_controller.h index 7089cddf..bd8ef43b 100644 --- a/apps/desktop/src/device_controller/device_controller.h +++ b/apps/desktop/src/device_controller/device_controller.h @@ -45,6 +45,9 @@ class KeyboardCapturer : public DeviceController { // Backends with an asynchronous capture loop override this so the UI can // fall back to window events if that loop stops unexpectedly. virtual bool IsHookActive() const { return true; } + // Permission-denied backends can wait for an OS grant notification rather + // than repeatedly releasing held fallback keys to retry an impossible hook. + virtual bool ShouldRetryHook() const { return true; } virtual int SendKeyboardCommand(int key_code, bool is_down, uint32_t scan_code = 0, bool extended = false) = 0; diff --git a/apps/desktop/src/gui/application/gui_application.cpp b/apps/desktop/src/gui/application/gui_application.cpp index facc55f0..4654a116 100644 --- a/apps/desktop/src/gui/application/gui_application.cpp +++ b/apps/desktop/src/gui/application/gui_application.cpp @@ -53,7 +53,12 @@ #elif defined(__APPLE__) #include "platform/macos/gui/tray/mac_tray.h" #include "platform/window_drag.h" -#elif defined(__linux__) +#elif defined(__OHOS__) +#include "platform/harmony/harmony_autostart.h" +#include "platform/harmony/gui/tray/harmony_tray.h" +#include "platform/harmony/gui/window_state.h" +#include "tools/file_transfer.h" +#elif (defined(__linux__) && !defined(__OHOS__)) #include #include #include @@ -70,6 +75,7 @@ #include "signal_server_settings.h" #include "signal_server_address.h" #include "ui/announcement_text.h" +#include "ui/model_update.h" #include "ui/release_notes_text.h" #include "ui/ui_localization.h" #include "update_checker.h" @@ -111,7 +117,7 @@ std::string CreatePasswordChangeRequestId(uint64_t sequence) { std::to_string(sequence); } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) bool HasNonEmptyEnvironmentVariable(const char* name) { const char* value = std::getenv(name); return value != nullptr && value[0] != '\0'; @@ -639,14 +645,14 @@ struct WindowDragState { float global_to_physical_scale = 1; slint::PhysicalPosition window_start; slint::PhysicalPosition last_target; -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) std::unique_ptr x11_display{nullptr, XCloseDisplay}; slint::Timer pointer_timer; #endif bool ReadGlobalPointer(float* x, float* y, bool* left_pressed = nullptr) { -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) const char* driver = SDL_GetCurrentVideoDriver(); if (driver && std::strcmp(driver, "x11") == 0) { // Slint can dispatch motion before SDL pumps its XInput2 events, and @@ -683,7 +689,7 @@ bool CanUseGlobalPointerPosition() { if (driver == nullptr) { return false; } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) // Wayland deliberately does not expose global pointer coordinates or allow // clients to position top-level windows. return std::strcmp(driver, "wayland") != 0; @@ -821,7 +827,7 @@ void DragWindow(WindowHandle& component, int phase, float mouse_x, state.global_to_physical_scale = 1.0f; #endif } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (state.use_global_pointer && state.x11_display) { // Moving the window can leave consecutive pointer events at identical // local coordinates, so Slint need not emit another TouchArea::moved. @@ -841,7 +847,7 @@ void DragWindow(WindowHandle& component, int phase, float mouse_x, return; } if (phase == 0) { -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) state.pointer_timer.stop(); if (state.use_global_pointer && state.x11_display) { DragWindow(component, 2, mouse_x, mouse_y, state); @@ -1017,14 +1023,16 @@ struct GuiApplication::SlintUi { std::unique_ptr tray; #elif defined(__APPLE__) std::unique_ptr tray; -#elif defined(__linux__) +#elif defined(__OHOS__) + std::unique_ptr tray; +#elif (defined(__linux__) && !defined(__OHOS__)) std::unique_ptr tray; Display* x11_focus_display = nullptr; std::chrono::steady_clock::time_point last_x11_activation_poll{}; #endif ~SlintUi() { -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (x11_focus_display) { XCloseDisplay(x11_focus_display); } @@ -1059,7 +1067,7 @@ int GuiApplication::Run() { } InitializeSettings(); -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().BindSettings( config_center_.get(), [this] { std::lock_guard lock(password_change_mutex_); @@ -1121,7 +1129,7 @@ void GuiApplication::InitializeSettings() { } bool GuiApplication::InitializeSDL() { -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) ConfigureLinuxCudaGraphics(config_center_->IsHardwareVideoCodec()); // Standard Wayland top-level windows cannot choose an absolute screen // position. Use the session's XWayland compatibility server for the GUI so @@ -1146,7 +1154,7 @@ bool GuiApplication::InitializeSDL() { LOG_ERROR("SDL initialization failed: {}", SDL_GetError()); return false; } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) const char* active_video_driver = SDL_GetCurrentVideoDriver(); // X11 permits the manual move/position operations used by the frameless // main window. Keep native Wayland decorations as a compatibility fallback. @@ -1192,7 +1200,7 @@ void GuiApplication::InitializeUi() { LOG_INFO("Using requested Slint backend: {}", requested_slint_backend); } #endif -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) // Detect the desktop session before temporarily hiding Wayland variables // for the XWayland GUI backend below. const bool is_wayland_session = IsWaylandSession(); @@ -1214,7 +1222,7 @@ void GuiApplication::InitializeUi() { std::make_unique(*video_renderer_); #if _WIN32 ui_->main->set_custom_titlebar(true); -#elif defined(__linux__) +#elif (defined(__linux__) && !defined(__OHOS__)) ui_->main->set_custom_titlebar(use_x11_custom_titlebar_); ui_->main->set_wayland_titlebar(use_xwayland_gui_); // Set this only at startup so dismissing the notice lasts until exit. @@ -1342,7 +1350,11 @@ void GuiApplication::InitializeSystemTray() { ui_->tray = std::make_unique( std::move(show_window), std::move(hide_window), std::move(open_settings), std::move(exit_app), "CrossDesk", localization_language_index_); -#elif defined(__linux__) +#elif defined(__OHOS__) + ui_->tray = std::make_unique( + std::move(show_window), std::move(hide_window), std::move(open_settings), + std::move(exit_app), "CrossDesk", localization_language_index_); +#elif (defined(__linux__) && !defined(__OHOS__)) ui_->tray = std::make_unique( std::move(show_window), std::move(hide_window), std::move(open_settings), std::move(exit_app), "CrossDesk", @@ -1384,6 +1396,9 @@ void GuiApplication::ResetSettingsUi(bool gallery_window) { main->set_force_relay_enabled(config_center_->IsForceRelay()); main->set_self_hosted_enabled(enable_self_hosted_); main->set_autostart_enabled(enable_autostart_); +#if defined(__OHOS__) + main->set_autostart_system_managed(true); +#endif main->set_daemon_enabled(enable_daemon_); main->set_save_remote_previews(config_center_->IsSaveRemotePreviews()); main->set_privacy_on_connect_enabled( @@ -1393,12 +1408,20 @@ void GuiApplication::ResetSettingsUi(bool gallery_window) { main->set_capture_method_index( static_cast(config_center_->GetScreenCaptureMethod())); #endif -#if defined(_WIN32) || defined(__APPLE__) || defined(__linux__) +#if defined(_WIN32) || defined(__APPLE__) || (defined(__linux__) && !defined(__OHOS__)) main->set_privacy_setting_available(true); #else main->set_privacy_setting_available(false); #endif +#if defined(__OHOS__) + // The sandbox default is not the system desktop. Show the actual receive + // location until a public directory has been selected and authorized. + main->set_file_save_path(UiText(file_transfer_save_path_buf_[0] + ? std::string(file_transfer_save_path_buf_) + : FileReceiver().OutputDir().string())); +#else main->set_file_save_path(file_transfer_save_path_buf_); +#endif main->set_server_host(signal_server_ip_self_); main->set_server_port(signal_server_port_self_); main->set_self_host_settings_error(""); @@ -1633,6 +1656,9 @@ void GuiApplication::BindMainCallbacks(bool gallery_window) { main->on_cancel_settings([this, main] { ResetSettingsUi(main->get_gallery_mode()); }); +#if defined(__OHOS__) + main->on_refresh_autostart_status([] { RefreshHarmonyAutostartStatus(); }); +#endif main->on_save_self_hosted_settings([this, main] { // Stage the endpoint until the surrounding settings dialog is confirmed. return ValidateSelfHostedSettingsFromUi(main->get_gallery_mode()); @@ -1832,12 +1858,19 @@ void GuiApplication::BindStreamCallbacks() { }); stream->on_send_shortcut([this](slint::SharedString shortcut) { auto props = SelectedSession(); - if (!props || !props->peer_) { + if (!props || !props->peer_ || !props->control_mouse_ || + props->connection_status_.load() != ConnectionStatus::Connected) { return; } + const std::string name(shortcut); + if (name != "Ctrl+Alt+Del" && name != "Win+L") return; + { + std::lock_guard lock(props->remote_version_mutex_); + if (name == "Ctrl+Alt+Del" && props->remote_platform_ == HostPlatform::HarmonyOS) return; + } RemoteAction action{}; action.type = ControlType::service_command; - action.c.flag = std::string(shortcut) == "Ctrl+Alt+Del" + action.c.flag = name == "Ctrl+Alt+Del" ? ServiceCommandFlag::send_sas : ServiceCommandFlag::lock_workstation; const std::string message = action.to_json(); @@ -2099,7 +2132,7 @@ void GuiApplication::Tick() { signal_connection_indicator_.Reset(); CreateConnectionPeer(); } -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().Poll( signal_connected_.load(), [this](const std::string& password) { RequestPasswordChange(password); }, @@ -2130,7 +2163,7 @@ void GuiApplication::Tick() { } } clipboard_.ApplyPendingRemoteText(); -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (ui_->tray) { ui_->tray->ProcessEvents(); } @@ -2156,7 +2189,7 @@ void GuiApplication::Tick() { HandlePresenceProbeTimeout(); HandlePrivacy(); HandleWindowsServiceIntegration(); -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) SyncXWaylandWindowActivation(); #else SyncStreamKeyboardFocus(); @@ -2367,7 +2400,7 @@ void GuiApplication::ShareLocalCursorState() { bool GuiApplication::RequestPasswordChange(const std::string &password) { const auto notify = [](const std::string& message) { -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().NotifyKey(message.c_str()); #else (void)message; @@ -2486,7 +2519,7 @@ void GuiApplication::HandlePasswordChangeResult() { } if (!succeeded) { -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().NotifyKey( uncertain ? "console_password_uncertain" : "console_password_rejected"); #endif @@ -2517,7 +2550,7 @@ void GuiApplication::HandlePasswordChangeResult() { } LOG_INFO("Password changed successfully for [{}]", client_id_); -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().NotifyKey("console_password_confirmed"); #endif if (peer_) { @@ -2551,7 +2584,7 @@ void GuiApplication::HandleCredentialRecovery() { } if (promote_pending) { -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().NotifyKey("console_password_recovered"); #endif if (!settings_.PromotePendingPasswordChange()) { @@ -2560,7 +2593,7 @@ void GuiApplication::HandleCredentialRecovery() { LOG_INFO("Recovered password change with pending credential"); } } else if (clear_pending) { -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().NotifyKey("console_password_retained"); #endif if (!settings_.ClearPendingPasswordChange()) { @@ -2699,6 +2732,17 @@ void GuiApplication::SyncMainWindow() { const bool signal_online = signal_connected_.load(); const bool signal_reconnecting = signal_connection_indicator_.Update(signal_online); for (const auto& main : {ui_->main, ui_->wall}) { +#if defined(__OHOS__) + // Only the operating system owns this setting. Never bind it to the local + // saved checkbox or wait for an ArkTS request in this 16 ms GUI tick. + switch (GetHarmonyAutostartStatus()) { + case HarmonyAutostartStatus::checking: main->set_autostart_status(ui::AutostartStatus::Checking); break; + case HarmonyAutostartStatus::enabled: main->set_autostart_status(ui::AutostartStatus::Enabled); break; + case HarmonyAutostartStatus::disabled: main->set_autostart_status(ui::AutostartStatus::Disabled); break; + case HarmonyAutostartStatus::unsupported: main->set_autostart_status(ui::AutostartStatus::Unsupported); break; + case HarmonyAutostartStatus::failed: main->set_autostart_status(ui::AutostartStatus::Failed); break; + } +#endif main->set_signal_connected(signal_online); main->set_signal_reconnecting(signal_reconnecting); main->set_signal_tls_error(signal_status_ == @@ -2983,7 +3027,7 @@ void GuiApplication::SyncPlatformDialogs() { #endif } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) void GuiApplication::SyncXWaylandWindowActivation() { if (!ui_ || !use_xwayland_gui_) { return; @@ -3057,12 +3101,20 @@ void GuiApplication::SetStreamKeyboardFocus(bool focused) { } void GuiApplication::SyncStreamKeyboardFocus() { +#if defined(__OHOS__) + if (ui_) { + ui_->main->set_window_active(IsHarmonyWindowActive(ui_->main->window())); + ui_->wall->set_window_active(IsHarmonyWindowActive(ui_->wall->window())); + } +#endif if (ui_ && !ui_->gallery_control_id.empty()) { #if defined(__APPLE__) if (!IsGalleryWindowActive()) ReleaseControl(); #elif defined(_WIN32) if (GetForegroundWindow() != ui_->wall->window().win32_hwnd()) ReleaseControl(); +#elif defined(__OHOS__) + if (!IsHarmonyWindowActive(ui_->wall->window())) ReleaseControl(); #endif return; } @@ -3077,6 +3129,10 @@ void GuiApplication::SyncStreamKeyboardFocus() { GetForegroundWindow() == stream_hwnd); #elif defined(__APPLE__) SetStreamKeyboardFocus(IsStreamWindowActive()); +#elif defined(__OHOS__) + const bool active = IsHarmonyWindowActive((*ui_->stream)->window()); + (*ui_->stream)->set_window_active(active); + SetStreamKeyboardFocus(active); #endif } @@ -3093,7 +3149,7 @@ void GuiApplication::SyncStreamWindow() { if (!has_sessions) need_to_create_stream_window_ = false; if (need_to_create_stream_window_ && !ui_->stream) { ui_->stream.emplace(ui::StreamWindow::create()); -#if defined(__linux__) +#if (defined(__linux__) && !defined(__OHOS__)) (*ui_->stream)->set_custom_titlebar(use_xwayland_gui_); #endif RegisterFontAwesome((*ui_->stream)->window()); @@ -3257,6 +3313,9 @@ void GuiApplication::SyncStreamWindow() { } } (*ui_->stream)->set_remote_update_visible(remote_update_available); + // All desktop clients use the same menu for a Harmony host. Windows SAS is + // not an ordinary key chord and has no public Harmony implementation. + (*ui_->stream)->set_secure_attention_supported(remote_platform != HostPlatform::HarmonyOS); if (remote_update_available) { SetStreamKeyboardFocus(false); } @@ -3358,7 +3417,7 @@ void GuiApplication::SyncStreamWindow() { index, props->display_info_list_[index].name, localization_language_index_))); } - ui_->display_model->set_vector(std::move(displays)); + ui::UpdateModelRows(*ui_->display_model, std::move(displays)); (*ui_->stream)->set_selected_display(props->selected_display_); const auto net = props->net_traffic_stats_.Load(); @@ -3383,7 +3442,7 @@ void GuiApplication::SyncStreamWindow() { net.data_inbound_stats, net.data_outbound_stats); append_stats_row(localization::total[localization_language_index_], net.total_inbound_stats, net.total_outbound_stats); - ui_->stats_model->set_vector(std::move(stats_rows)); + ui::UpdateModelRows(*ui_->stats_model, std::move(stats_rows)); (*ui_->stream) ->set_stats_resolution(UiText(std::to_string(props->video_width_) + "x" + std::to_string(props->video_height_))); @@ -3444,7 +3503,7 @@ void GuiApplication::SyncStreamWindow() { item.size = UiText(FormatFileSize(info.file_size)); transfers.push_back(std::move(item)); } - ui_->transfer_model->set_vector(std::move(transfers)); + ui::UpdateModelRows(*ui_->transfer_model, std::move(transfers)); (*ui_->stream) ->set_file_transfer_visible( props->file_transfer_.file_transfer_window_visible_); @@ -3698,7 +3757,7 @@ void GuiApplication::SyncServerWindow() { // Keep the panel alive throughout privacy transitions so remote // controllers can still see and operate it without resetting its UI state. bool show_controller_window = has_connected_controller; -#if !defined(_WIN32) && !defined(__APPLE__) && !defined(__linux__) +#if !defined(_WIN32) && !defined(__APPLE__) && !(defined(__linux__) && !defined(__OHOS__)) // Preserve the existing panel behavior on other platforms. const auto privacy_status = privacy_.Snapshot(); const bool privacy_covering = privacy_status.overlay_active || @@ -3902,7 +3961,9 @@ bool GuiApplication::SaveSettingsFromUi(bool gallery_window) { main->get_hardware_codec_enabled(); enable_turn_ = main->get_turn_enabled(); enable_self_hosted_ = main->get_self_hosted_enabled(); +#if !defined(__OHOS__) enable_autostart_ = main->get_autostart_enabled(); +#endif enable_daemon_ = main->get_daemon_enabled(); localization_language_ = @@ -3917,7 +3978,9 @@ bool GuiApplication::SaveSettingsFromUi(bool gallery_window) { config_center_->SetHardwareVideoCodec(enable_hardware_video_codec_); config_center_->SetTurn(enable_turn_, main->get_force_relay_enabled()); main->set_force_relay_enabled(config_center_->IsForceRelay()); +#if !defined(__OHOS__) config_center_->SetAutostart(enable_autostart_); +#endif config_center_->SetDaemon(enable_daemon_); if (config_center_->SetSaveRemotePreviews(main->get_save_remote_previews()) != 0) { LOG_ERROR("Failed to save remote preview preference"); @@ -3945,7 +4008,7 @@ bool GuiApplication::SaveSettingsFromUi(bool gallery_window) { main->set_capture_method_index( static_cast(config_center_->GetScreenCaptureMethod())); #endif -#if defined(_WIN32) || defined(__APPLE__) || defined(__linux__) +#if defined(_WIN32) || defined(__APPLE__) || (defined(__linux__) && !defined(__OHOS__)) if (config_center_->SetPrivacyScreen(main->get_privacy_on_connect_enabled()) != 0) { main->set_privacy_on_connect_enabled( @@ -4000,7 +4063,7 @@ void GuiApplication::ApplySettingsFromConfig(bool reconnect) { enable_autostart_last_ = enable_autostart_; enable_daemon_last_ = enable_daemon_; file_transfer_save_path_last_ = path; -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().SetLanguage(localization_language_index_); #endif if (ui_) { @@ -4449,7 +4512,7 @@ void GuiApplication::ShowGalleryWindow() { #elif defined(_WIN32) ConfigureWindowsWindowIcons(window.win32_hwnd()); SetForegroundWindow(window.win32_hwnd()); -#else +#elif defined(__linux__) && !defined(__OHOS__) if (!ui_->x11_focus_display) ui_->x11_focus_display = XOpenDisplay(nullptr); X11ActivateGalleryWindow(ui_->x11_focus_display); #endif @@ -5758,7 +5821,7 @@ void GuiApplication::SyncGallery() { } void GuiApplication::Cleanup() { -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().BindSettings(nullptr, {}, {}); #endif if (!ui_) { diff --git a/apps/desktop/src/gui/application/gui_application.h b/apps/desktop/src/gui/application/gui_application.h index 29f1837a..74e597dc 100644 --- a/apps/desktop/src/gui/application/gui_application.h +++ b/apps/desktop/src/gui/application/gui_application.h @@ -91,7 +91,7 @@ private: void SyncServerWindow(); void SetServerPanelExpanded(bool expanded); void PositionServerWindows(); -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) void SyncXWaylandWindowActivation(); #endif void UpdateLocalization(); @@ -127,7 +127,7 @@ private: uint32_t cursor_state_sequence_ = 0; std::chrono::steady_clock::time_point next_video_frame_time_{}; SignalConnectionIndicator signal_connection_indicator_; -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) bool use_xwayland_gui_ = false; bool use_x11_custom_titlebar_ = false; #endif diff --git a/apps/desktop/src/gui/features/devices/session_device_manager.cpp b/apps/desktop/src/gui/features/devices/session_device_manager.cpp index ceb9497d..0f25ce18 100644 --- a/apps/desktop/src/gui/features/devices/session_device_manager.cpp +++ b/apps/desktop/src/gui/features/devices/session_device_manager.cpp @@ -30,7 +30,7 @@ namespace { constexpr uint64_t kCaptureResumeKeyFrameGapMs = 500; constexpr auto kFrameDeadlineTolerance = std::chrono::milliseconds(1); -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) constexpr auto kKeyboardCaptureRetryDelay = std::chrono::seconds(1); #endif @@ -419,7 +419,7 @@ int SessionDeviceManager::StartMouseController() { return -1; } -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (IsWaylandSession()) { if (!screen_capturer_) { return 1; @@ -467,7 +467,7 @@ int SessionDeviceManager::StopMouseController() { int SessionDeviceManager::StartKeyboardCapturer() { captured_keyboard_inputs_.Clear(); owner_.keyboard_capturer_uses_window_events_ = false; -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) next_keyboard_capture_retry_ = std::chrono::steady_clock::now() + kKeyboardCaptureRetryDelay; #endif @@ -479,7 +479,7 @@ int SessionDeviceManager::StartKeyboardCapturer() { } #endif -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (IsWaylandSession()) { owner_.keyboard_capturer_uses_window_events_ = true; LOG_INFO("Start keyboard capturer with Slint Wayland backend"); @@ -546,7 +546,7 @@ int SessionDeviceManager::InitializeAudioOutput() { }; if (!open_device()) { -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) LOG_WARN("Failed to open output stream with driver [{}]: {}", getenv("SDL_AUDIODRIVER") ? getenv("SDL_AUDIODRIVER") : "(default)", @@ -629,7 +629,7 @@ void SessionDeviceManager::UpdateInteractions() { } } #endif -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) const bool is_wayland_session = IsWaylandSession(); const bool stop_wayland_mouse_before_screen = is_wayland_session && !owner_.start_screen_capturer_ && @@ -664,24 +664,27 @@ void SessionDeviceManager::UpdateInteractions() { } if (owner_.screen_capturer_is_started_ && mouse_controller_ && SyncDisplayInfo()) { -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) const int active_display = screen_capturer_->GetCurrentMonitorIndex(); const auto displays = display_info_list(); if (active_display >= 0 && active_display < static_cast(displays.size())) { owner_.selected_display_ = active_display; +#ifdef _WIN32 if (displays[active_display].width <= 0 || displays[active_display].height <= 0) ReleaseRemoteMouseButtons(); +#endif } #endif } if (owner_.start_keyboard_capturer_ && owner_.focus_on_stream_window_ && !owner_.privacy_.Snapshot().input_blocked) { -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) if (owner_.keyboard_capturer_is_started_ && owner_.keyboard_capturer_uses_window_events_ && keyboard_capturer_ && + keyboard_capturer_->ShouldRetryHook() && std::chrono::steady_clock::now() >= next_keyboard_capture_retry_) { // A transient installation failure or stalled hook must not leave // system shortcuts in the window-event fallback until focus changes. @@ -699,7 +702,7 @@ void SessionDeviceManager::UpdateInteractions() { owner_.keyboard_.ForceReleasePressedKeys(); owner_.keyboard_capturer_uses_window_events_ = true; owner_.keyboard_capturer_is_started_ = true; -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) next_keyboard_capture_retry_ = std::chrono::steady_clock::now() + kKeyboardCaptureRetryDelay; #endif diff --git a/apps/desktop/src/gui/features/devices/session_device_manager.h b/apps/desktop/src/gui/features/devices/session_device_manager.h index da7caf76..9597a68b 100644 --- a/apps/desktop/src/gui/features/devices/session_device_manager.h +++ b/apps/desktop/src/gui/features/devices/session_device_manager.h @@ -111,8 +111,10 @@ private: std::vector display_info_list_; size_t registered_display_stream_count_ = 0; CapturedKeyboardQueue captured_keyboard_inputs_; -#ifdef _WIN32 +#if defined(_WIN32) || defined(__OHOS__) std::chrono::steady_clock::time_point next_keyboard_capture_retry_{}; +#endif +#ifdef _WIN32 std::mutex service_mouse_mutex_; unsigned service_mouse_buttons_ = 0; int service_mouse_x_ = 0, service_mouse_y_ = 0; diff --git a/apps/desktop/src/gui/rendering/slint_video_presenter.cpp b/apps/desktop/src/gui/rendering/slint_video_presenter.cpp index 4ae9168a..4feea74a 100644 --- a/apps/desktop/src/gui/rendering/slint_video_presenter.cpp +++ b/apps/desktop/src/gui/rendering/slint_video_presenter.cpp @@ -16,7 +16,7 @@ #if defined(__APPLE__) #include "platform/macos/gui/metal_video_renderer.h" -#else +#elif !defined(__OHOS__) #if defined(_WIN32) #include #endif @@ -28,10 +28,10 @@ namespace crossdesk { namespace { -#if !defined(__APPLE__) +#if !defined(__APPLE__) && !defined(__OHOS__) constexpr GLint kGlClampToEdge = 0x812F; constexpr float kStreamWindowCornerRadius = 12.0f; -#else +#elif defined(__APPLE__) constexpr int kMetalAttachmentAttemptLimit = 30; #endif @@ -78,7 +78,7 @@ struct SlintVideoPresenter::Impl { #if defined(__APPLE__) int attachment_attempts = 0; bool skip_attachment_once = false; -#else +#elif !defined(__OHOS__) std::mutex gl_mutex; std::vector gl_conversion_frame; std::vector gl_pending_frame; @@ -109,7 +109,7 @@ struct SlintVideoPresenter::Impl { return true; } -#if !defined(__APPLE__) +#if !defined(__APPLE__) && !defined(__OHOS__) void ConfigureOpenGlNotifier() { if (!stream) { return; @@ -238,7 +238,7 @@ void SlintVideoPresenter::PrepareWindow( stream->set_native_video_enabled(false); impl_->attachment_attempts = kMetalAttachmentAttemptLimit; impl_->skip_attachment_once = true; -#else +#elif !defined(__OHOS__) impl_->ConfigureOpenGlNotifier(); #endif } @@ -351,7 +351,7 @@ SlintVideoPresenter::PresentResult SlintVideoPresenter::Present( } return result; } -#else +#elif !defined(__OHOS__) auto* opengl_renderer = dynamic_cast(&impl_->renderer); if (opengl_renderer && opengl_renderer->IsReady()) { @@ -419,7 +419,7 @@ SlintVideoPresenter::PresentResult SlintVideoPresenter::Present( } } -#if !defined(__APPLE__) +#if !defined(__APPLE__) && !defined(__OHOS__) uint32_t texture_id = 0; { std::lock_guard lock(impl_->gl_mutex); diff --git a/apps/desktop/src/gui/runtime/connection_runtime.cpp b/apps/desktop/src/gui/runtime/connection_runtime.cpp index 798f7137..8402876a 100644 --- a/apps/desktop/src/gui/runtime/connection_runtime.cpp +++ b/apps/desktop/src/gui/runtime/connection_runtime.cpp @@ -229,7 +229,7 @@ void GuiRuntime::HandleServerControllerDisconnected( need_to_create_server_window_.store(false, std::memory_order_release); need_to_destroy_server_window_.store(true, std::memory_order_release); is_server_mode_ = false; -#if defined(__linux__) && !defined(__APPLE__) +#if (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) if (IsWaylandSession()) { // Keep Wayland capture session warm to avoid black screen on subsequent // reconnects. diff --git a/apps/desktop/src/gui/runtime/cursor_state_provider.cpp b/apps/desktop/src/gui/runtime/cursor_state_provider.cpp index ec48529f..f869be11 100644 --- a/apps/desktop/src/gui/runtime/cursor_state_provider.cpp +++ b/apps/desktop/src/gui/runtime/cursor_state_provider.cpp @@ -63,7 +63,7 @@ bool CursorStateProvider::Sample(const std::vector& displays, } // namespace crossdesk -#elif defined(__linux__) && !defined(__APPLE__) +#elif (defined(__linux__) && !defined(__OHOS__)) && !defined(__APPLE__) #include #include diff --git a/apps/desktop/src/gui/runtime/gui_runtime.cpp b/apps/desktop/src/gui/runtime/gui_runtime.cpp index 75a0d093..322e9de2 100644 --- a/apps/desktop/src/gui/runtime/gui_runtime.cpp +++ b/apps/desktop/src/gui/runtime/gui_runtime.cpp @@ -16,7 +16,7 @@ #include "localization.h" #include "platform/video_renderer.h" #include "rd_log.h" -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) #include "platform/linux/headless/headless_console.h" #endif @@ -143,7 +143,7 @@ int GuiRuntime::CreateConnectionPeer() { const char *active_identity = self_hosted ? self_hosted_user_id_ : client_id_with_password_; -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) // Publish the committed identity; an unconfirmed recovery credential must // not be advertised as the current connection password. HeadlessConsole::Instance().SetIdentity(active_identity); @@ -177,7 +177,7 @@ int GuiRuntime::CreateConnectionPeer() { sizeof(params_.log_path) - 1); params_.log_path[sizeof(params_.log_path) - 1] = '\0'; params_.hardware_acceleration = config_center_->IsHardwareVideoCodec(); -#if defined(_WIN32) || defined(__linux__) || defined(__APPLE__) +#if defined(_WIN32) || (defined(__linux__) && !defined(__OHOS__)) || defined(__APPLE__) // Windows and Linux retain pooled CPU NV12 frames or CUDA device frames; // macOS retains VideoToolbox CVPixelBuffers for direct Metal sampling. All // platforms fall back to a packed CPU copy when native upload is unavailable. diff --git a/apps/desktop/src/gui/runtime/peer_data_callbacks.cpp b/apps/desktop/src/gui/runtime/peer_data_callbacks.cpp index 39198419..fdeb8910 100644 --- a/apps/desktop/src/gui/runtime/peer_data_callbacks.cpp +++ b/apps/desktop/src/gui/runtime/peer_data_callbacks.cpp @@ -24,6 +24,9 @@ #include "platform.h" #include "rd_log.h" #include "runtime/gui_runtime.h" +#ifdef __OHOS__ +#include "platform/harmony/input/system_shortcuts.h" +#endif #if _WIN32 #include "desktop_transition_policy.h" @@ -308,6 +311,10 @@ void PeerEventHandler::OnReceiveDataBuffer( GetLastError()); } } +#elif defined(__OHOS__) + if (!SendHarmonySystemShortcut(remote_action.c.flag)) + LOG_WARN("Harmony system shortcut unsupported or unauthorized, command={}", + static_cast(remote_action.c.flag)); #endif return; } diff --git a/apps/desktop/src/gui/runtime/peer_event_handler.cpp b/apps/desktop/src/gui/runtime/peer_event_handler.cpp index 17ae7e44..ffd32ab0 100644 --- a/apps/desktop/src/gui/runtime/peer_event_handler.cpp +++ b/apps/desktop/src/gui/runtime/peer_event_handler.cpp @@ -22,7 +22,7 @@ #include "platform/video_renderer.h" #include "rd_log.h" #include "runtime/gui_runtime.h" -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) #include "platform/linux/headless/headless_console.h" #endif @@ -38,7 +38,9 @@ namespace { constexpr HostPlatform kClientPlatform = HostPlatform::Windows; #elif defined(__APPLE__) constexpr HostPlatform kClientPlatform = HostPlatform::MacOS; -#elif defined(__linux__) +#elif defined(__OHOS__) +constexpr HostPlatform kClientPlatform = HostPlatform::HarmonyOS; +#elif (defined(__linux__) && !defined(__OHOS__)) constexpr HostPlatform kClientPlatform = HostPlatform::Linux; #else constexpr HostPlatform kClientPlatform = HostPlatform::Unknown; @@ -558,7 +560,7 @@ void PeerEventHandler::OnNetStatusReport( LOG_INFO("Use client id [{}] and save id into cache file", id); runtime->settings_.Save(); } -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) HeadlessConsole::Instance().SetIdentity(id + "@" + password); #endif } diff --git a/apps/desktop/src/gui/ui/common.slint b/apps/desktop/src/gui/ui/common.slint index 27c09434..ba768510 100644 --- a/apps/desktop/src/gui/ui/common.slint +++ b/apps/desktop/src/gui/ui/common.slint @@ -182,6 +182,17 @@ export component CompactComboBox inherits Rectangle { property popup-progress: 0.0; callback selected(int); + // Session state can disable a setting while its popup is already open. + // Remove the stale options and stop a pending reveal on every backend. + changed enabled => { + if !root.enabled { + popup-open-timer.running = false; + root.popup-animation-enabled = false; + root.popup-progress = 0.0; + combo-popup.close(); + } + } + animate popup-progress { duration: 160ms; easing: ease-out; @@ -278,12 +289,15 @@ export component CompactComboBox inherits Rectangle { background: option-touch.has-hover ? #dbeafb : index == root.current-index ? #eef5fd : white; Text { x: 5px; width: parent.width - 10px; text: choice; color: #202124; font-size: root.subtle ? 12px : ImGuiFontStyle.body; overflow: elide; vertical-alignment: center; } option-touch := TouchArea { + enabled: root.enabled; clicked => { popup-open-timer.running = false; root.popup-animation-enabled = false; root.popup-progress = 0.0; - root.current-index = index; - root.selected(index); + if root.enabled && index >= 0 && index < root.model.length { + root.current-index = index; + root.selected(index); + } combo-popup.close(); } } @@ -439,6 +453,8 @@ export component SecondaryButton inherits Rectangle { export component ToggleSwitch inherits Rectangle { in-out property checked; in property enabled: true; + in property toggle-on-click: true; + callback clicked(); callback toggled(bool); width: 36px; @@ -459,8 +475,11 @@ export component ToggleSwitch inherits Rectangle { TouchArea { enabled: root.enabled; clicked => { - root.checked = !root.checked; - root.toggled(root.checked); + if root.toggle-on-click { + root.checked = !root.checked; + root.toggled(root.checked); + } + root.clicked(); } } } diff --git a/apps/desktop/src/gui/ui/crossdesk_ui.slint b/apps/desktop/src/gui/ui/crossdesk_ui.slint index 16b30d89..6133fbee 100644 --- a/apps/desktop/src/gui/ui/crossdesk_ui.slint +++ b/apps/desktop/src/gui/ui/crossdesk_ui.slint @@ -1,7 +1,7 @@ // Single AOT entry point. Generating one header for the entire UI avoids // duplicate imported-component definitions when C++ uses all three windows. export { AnnouncementItem, MessageStrings } from "message_panel.slint"; -export { MainWindow, RecentConnection, UpdateCheckStatus, UiStrings } from "main_window.slint"; +export { MainWindow, RecentConnection, UpdateCheckStatus, AutostartStatus, UiStrings } from "main_window.slint"; export { StreamWindow, StreamTab, FileTransferEntry, NetworkStatsRow, StreamStrings } from "stream_window.slint"; export { ServerWindow, ServerStatusBar, ServerStrings, ControllerEntry } from "server_window.slint"; diff --git a/apps/desktop/src/gui/ui/main_window.slint b/apps/desktop/src/gui/ui/main_window.slint index cdba87f2..8e001b03 100644 --- a/apps/desktop/src/gui/ui/main_window.slint +++ b/apps/desktop/src/gui/ui/main_window.slint @@ -32,6 +32,7 @@ export struct ReleaseNoteBlock { } export enum UpdateCheckStatus { idle, checking, up-to-date, available, failed } +export enum AutostartStatus { checking, enabled, disabled, unsupported, failed } enum CopyFeedback { idle, success, failed } export global UiStrings { @@ -401,6 +402,16 @@ export component MainWindow inherits Window { } in-out property self-hosted-enabled: false; in-out property autostart-enabled: false; + in property autostart-system-managed: false; + in property autostart-status: AutostartStatus.checking; + out property autostart-status-text: + root.autostart-status == AutostartStatus.checking ? GalleryStrings.tr("正在读取…", "Checking…", "Проверка…") : + root.autostart-status == AutostartStatus.enabled ? GalleryStrings.tr("已开启", "Enabled", "Включён") : + root.autostart-status == AutostartStatus.disabled ? GalleryStrings.tr("未开启", "Disabled", "Выключен") : + root.autostart-status == AutostartStatus.unsupported ? GalleryStrings.tr("设备不支持", "Unsupported", "Не поддерживается") : + GalleryStrings.tr("读取失败", "Read failed", "Ошибка чтения"); + in-out property autostart-guide-open: false; + changed autostart-guide-open => { if root.autostart-guide-open { focus-sink.focus(); } } in-out property daemon-enabled: false; in-out property save-remote-previews: true; // Show the preference on every platform; only adapted hosts can change it. @@ -427,6 +438,7 @@ export component MainWindow inherits Window { callback recent-delete(string); callback save-settings() -> bool; callback cancel-settings; + callback refresh-autostart-status; callback save-self-hosted-settings() -> bool; callback cancel-self-hosted-settings; callback browse-save-path; @@ -494,7 +506,10 @@ export component MainWindow inherits Window { root.gallery-frame-rate-draft = root.gallery-frame-rate-index; root.gallery-bandwidth-draft = root.gallery-bandwidth-index; root.gallery-action("settings-open", ""); + if root.autostart-system-managed { root.refresh-autostart-status(); } focus-sink.focus(); + } else { + root.autostart-guide-open = false; } } @@ -721,6 +736,10 @@ export component MainWindow inherits Window { focus-sink := FocusScope { x: 0px; y: 0px; width: 0px; height: 0px; key-pressed(event) => { + if root.autostart-guide-open { + if event.text == Key.Return || event.text == Key.Escape { root.autostart-guide-open = false; } + return accept; + } if root.wayland-notice-open { if event.text == Key.Return || event.text == Key.Escape { root.wayland-notice-open = false; @@ -939,7 +958,7 @@ export component MainWindow inherits Window { accessible-label: UiStrings.remote-id; vertical-alignment: center; selection-foreground-color: white; selection-background-color: WorkbenchStyle.accent; - edited => { root.connection-input-error = ""; remote-id-format-timer.restart(); } + edited => { root.connection-input-error = ""; remote-id-format-timer.running = true; } accepted => { if !root.connection-pending { root.connect-requested(root.remote-id-input); } } } } @@ -1550,7 +1569,12 @@ export component MainWindow inherits Window { } remote-id-format-timer := Timer { interval: 1ms; + running: false; triggered => { + // Use the running property for both transitions. restart() resumes + // the native repeating timer without restoring this property, so + // a later stop() cannot change it and the selection keeps resetting. + self.stop(); let formatted = root.format-remote-id(root.remote-id-input); root.remote-id-input = formatted; // IDs are ASCII-only, so character-count is also the UTF-8 offset @@ -1963,7 +1987,13 @@ export component MainWindow inherits Window { } SettingsGroup { title: GalleryStrings.tr("启动与后台", "Startup & background", "Запуск и фоновая работа"); rows: 2; - SettingsToggleRow { label: UiStrings.autostart; checked <=> root.autostart-enabled; enabled: !root.settings-session-active; } + if !root.autostart-system-managed: SettingsToggleRow { label: UiStrings.autostart; checked <=> root.autostart-enabled; enabled: !root.settings-session-active; } + if root.autostart-system-managed: SettingsToggleRow { + label: UiStrings.autostart; + checked: root.autostart-status == AutostartStatus.enabled; + toggle-on-click: false; + clicked => { root.autostart-guide-open = true; } + } SettingsToggleRow { label: UiStrings.daemon; checked <=> root.daemon-enabled; enabled: !root.settings-session-active; divider: false; } } } @@ -1998,6 +2028,59 @@ export component MainWindow inherits Window { } } + if root.autostart-guide-open && root.settings-open: Rectangle { + background: #15223835; z: 49; + TouchArea { clicked => { focus-sink.focus(); } } + DialogSurface { + x: (parent.width - self.width) / 2; y: (parent.height - self.height) / 2; + width: min(460px, parent.width - 32px); + height: min(380px, parent.height - 24px); + background: #f8fafc; border-color: #e1e7ef; + drop-shadow-color: #14294424; drop-shadow-blur: 18px; drop-shadow-offset-y: 6px; + TouchArea { clicked => { focus-sink.focus(); } } + Text { + x: 24px; y: 18px; width: parent.width - 48px; height: 26px; + text: GalleryStrings.tr("设置开机自启动", "Set up auto start", "Настройка автозапуска"); + font-size: 18px; font-weight: 600; color: WorkbenchStyle.text; + } + ScrollView { + x: 24px; y: 58px; width: parent.width - 48px; height: parent.height - 120px; + viewport-width: self.visible-width; + horizontal-scrollbar-policy: always-off; + viewport-height: guide-content.preferred-height; + guide-content := VerticalLayout { + width: parent.width; padding: 0px; spacing: 12px; + Text { text: root.autostart-status-text; color: WorkbenchStyle.accent; font-size: 13px; wrap: word-wrap; } + Text { + text: root.autostart-status == AutostartStatus.unsupported ? GalleryStrings.tr( + "当前设备不支持开机自启动。", "This device does not support auto start.", "Это устройство не поддерживает автозапуск.") : + root.autostart-status == AutostartStatus.failed ? GalleryStrings.tr( + "无法读取系统状态。请刷新重试,或在系统设置中核对。", "Could not read the system status. Retry or check it in Settings.", "Не удалось прочитать состояние. Повторите попытку или проверьте настройки системы.") : + GalleryStrings.tr("请在系统设置中添加或移除本应用,返回后会自动刷新状态。", "Add or remove this app in system Settings. The status refreshes when you return.", "Добавьте или удалите приложение в настройках системы. После возвращения статус обновится."); + color: WorkbenchStyle.text; font-size: 13px; wrap: word-wrap; + } + if root.autostart-status != AutostartStatus.unsupported: Text { + text: GalleryStrings.tr( + "设置 → 应用和元服务 → 应用启动管理\n\n开启:点击“添加应用”,选择 CrossDesk,然后点击“完成”。\n\n关闭:在自启动列表中找到本应用,点击“移除”。", + "Settings → Apps & services → App startup management\n\nEnable: choose Add app, select CrossDesk, then choose Done.\n\nDisable: find this app in the startup list and choose Remove.", + "Настройки → Приложения и сервисы → Управление запуском\n\nВключить: добавьте CrossDesk в список и подтвердите выбор.\n\nВыключить: удалите приложение из списка автозапуска."); + color: WorkbenchStyle.text; font-size: 13px; wrap: word-wrap; + } + } + } + SettingsButton { + x: 24px; y: parent.height - 48px; width: 112px; + text: GalleryStrings.tr("刷新状态", "Refresh status", "Обновить"); + enabled: root.autostart-status != AutostartStatus.checking; + clicked => { root.refresh-autostart-status(); } + } + SettingsButton { + x: parent.width - 116px; y: parent.height - 48px; width: 92px; primary: true; + text: UiStrings.ok; clicked => { root.autostart-guide-open = false; } + } + } + } + if root.self-host-settings-open: Rectangle { background: #15223835; z: 49; diff --git a/apps/desktop/src/gui/ui/model_update.h b/apps/desktop/src/gui/ui/model_update.h new file mode 100644 index 00000000..34247fe8 --- /dev/null +++ b/apps/desktop/src/gui/ui/model_update.h @@ -0,0 +1,34 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _MODEL_UPDATE_H_ +#define _MODEL_UPDATE_H_ + +#include + +#include +#include + +namespace crossdesk::ui { + +// A reset invalidates row-count bindings even if the count is unchanged. +// Repeating it from a timer can continually restart dependent size animations. +template +void UpdateModelRows(slint::VectorModel& model, std::vector rows) { + if (model.row_count() != rows.size()) { + model.set_vector(std::move(rows)); + return; + } + for (std::size_t i = 0; i < rows.size(); ++i) { + if (model.row_data(i) != rows[i]) { + model.set_row_data(i, rows[i]); + } + } +} + +} // namespace crossdesk::ui + +#endif diff --git a/apps/desktop/src/gui/ui/settings_controls.slint b/apps/desktop/src/gui/ui/settings_controls.slint index 1399ceaa..c5f7350a 100644 --- a/apps/desktop/src/gui/ui/settings_controls.slint +++ b/apps/desktop/src/gui/ui/settings_controls.slint @@ -130,9 +130,13 @@ export component SettingsRow inherits Rectangle { export component SettingsToggleRow inherits SettingsRow { in-out property checked; + in property toggle-on-click: true; + callback clicked(); label-reserve: 64px; ToggleSwitch { x: parent.width - self.width - 12px; y: (parent.height - self.height) / 2; checked <=> root.checked; enabled: root.enabled; + toggle-on-click: root.toggle-on-click; + clicked => { root.clicked(); } } } diff --git a/apps/desktop/src/gui/ui/stream_window.slint b/apps/desktop/src/gui/ui/stream_window.slint index 1c00938a..5fbaf019 100644 --- a/apps/desktop/src/gui/ui/stream_window.slint +++ b/apps/desktop/src/gui/ui/stream_window.slint @@ -238,6 +238,11 @@ export component StreamWindow inherits Window { callback reorder-tab(int, length, length); callback close-tab(string); callback switch-display(int); + in property secure-attention-supported: true; + private property <[string]> secure-shortcut-items: ["Ctrl+Alt+Del", "Win+L"]; + private property <[string]> ordinary-shortcut-items: ["Win+L"]; + private property <[string]> shortcut-items: root.secure-attention-supported ? root.secure-shortcut-items : root.ordinary-shortcut-items; + private property shortcut-menu-height: root.shortcut-items.length * 30px + 8px; callback send-shortcut(string); callback toggle-mouse-control; callback toggle-privacy; @@ -265,6 +270,10 @@ export component StreamWindow inherits Window { private property stats-header-height: 18px; private property stats-content-height: root.stats-header-height + root.stats-rows.length * root.stats-row-height + 65px; + private property file-panel-x: 18px; + private property file-panel-y: video.height - min(210px, video.height * 0.35); + private property file-panel-width: min(430px, video.width * 0.6); + private property file-panel-height: min(190px, video.height * 0.32); private property display-menu-open: false; private property display-menu-animation-enabled: false; private property display-menu-progress: 0.0; @@ -392,11 +401,14 @@ export component StreamWindow inherits Window { && pointer-x >= shortcut-menu-x && pointer-x <= shortcut-menu-x + 180px && pointer-y >= shortcut-menu-y - && pointer-y <= shortcut-menu-y + 68px; + && pointer-y <= shortcut-menu-y + root.shortcut-menu-height; let over-video-menu = root.video-menu-open && pointer-x >= root.video-menu-x && pointer-x <= root.video-menu-x + root.video-menu-width && pointer-y >= root.video-menu-y && pointer-y <= root.video-menu-y + root.video-menu-height; - return over-control || over-display-menu || over-shortcut-menu || over-video-menu; + let over-file-panel = root.file-transfer-visible + && pointer-x >= root.file-panel-x && pointer-x <= root.file-panel-x + root.file-panel-width + && pointer-y >= root.file-panel-y && pointer-y <= root.file-panel-y + root.file-panel-height; + return over-control || over-display-menu || over-shortcut-menu || over-video-menu || over-file-panel; } begin-control-drag(press-x, press-y) => { @@ -1246,11 +1258,11 @@ export component StreamWindow inherits Window { } if root.shortcut-menu-open: Rectangle { - x: control.x + shortcut-button.x; y: control.y + 40px; width: 180px; height: 68px; + x: control.x + shortcut-button.x; y: control.y + 40px; width: 180px; height: root.shortcut-menu-height; background: white; border-width: 1px; border-color: ImGuiLineStyle.border; border-radius: UiRadius.menu; z: 11; VerticalLayout { padding: 4px; spacing: 1px; - for shortcut in ["Ctrl+Alt+Del", "Win+L"]: Rectangle { + for shortcut in root.shortcut-items: Rectangle { height: 29px; background: shortcut-touch.has-hover ? #e8eef9 : transparent; Text { x: 8px; text: shortcut; color: #30343b; font-size: ImGuiFontStyle.body; vertical-alignment: center; } shortcut-touch := TouchArea { @@ -1365,10 +1377,21 @@ export component StreamWindow inherits Window { // Transfer progress overlay at bottom-left. if root.file-transfer-visible: Rectangle { - x: 18px; y: parent.height - min(210px, parent.height * 0.35); width: min(430px, parent.width * 0.6); height: min(190px, parent.height * 0.32); + x: root.file-panel-x; y: root.file-panel-y; + width: root.file-panel-width; height: root.file-panel-height; background: #fffffff0; border-width: 1px; border-color: ImGuiLineStyle.border; border-radius: UiRadius.surface; z: 7; + // Header/padding and drags leaving the panel belong to local UI. + TouchArea { + pointer-event(event) => { + if event.kind == PointerEventKind.down { root.suppress-next-remote-pointer-release = true; } + } + scroll-event(event) => { accept } + } Text { x: 12px; y: 8px; text: StreamStrings.file-transfer; color: #2d3138; font-weight: 600; } - IconButton { x: parent.width - 32px; y: 1px; icon: FontAwesomeIcons.xmark; icon-size: 13px; clicked => { root.close-file-transfer(); } } + IconButton { + x: parent.width - 32px; y: 1px; icon: FontAwesomeIcons.xmark; icon-size: 13px; + clicked => { root.suppress-next-remote-pointer-release = true; root.close-file-transfer(); } + } ScrollView { x: 10px; y: 32px; width: parent.width - 20px; height: parent.height - 42px; VerticalLayout { diff --git a/apps/desktop/src/log/rd_log.cpp b/apps/desktop/src/log/rd_log.cpp index 17e99a4c..306fa35e 100644 --- a/apps/desktop/src/log/rd_log.cpp +++ b/apps/desktop/src/log/rd_log.cpp @@ -6,11 +6,29 @@ #include "spdlog/sinks/rotating_file_sink.h" #include "spdlog/sinks/stdout_color_sinks.h" +#if defined(__OHOS__) +#include +#include "spdlog/sinks/base_sink.h" +#endif namespace crossdesk { namespace { +#if defined(__OHOS__) +class HarmonyLogSink final : public spdlog::sinks::base_sink { + void sink_it_(const spdlog::details::log_msg& message) override { + spdlog::memory_buf_t formatted; + formatter_->format(message, formatted); + const std::string text(formatted.data(), formatted.size()); + const auto level = message.level >= spdlog::level::err ? LOG_ERROR : + message.level >= spdlog::level::warn ? LOG_WARN : LOG_INFO; + OH_LOG_Print(LOG_APP, level, 0, "CrossDeskDesktop", "%{public}s", text.c_str()); + } + void flush_() override {} +}; +#endif + std::string g_log_dir = "logs"; std::string g_logger_name = LOGGER_NAME; std::once_flag g_logger_once_flag; @@ -56,7 +74,11 @@ std::shared_ptr get_logger() { g_log_dir + "/" + g_logger_name + ".log", 5 * 1024 * 1024, 3); std::vector sinks; +#if defined(__OHOS__) + sinks.push_back(std::make_shared()); +#else sinks.push_back(std::make_shared()); +#endif sinks.push_back(file_sink); auto logger = std::make_shared(g_logger_name, sinks.begin(), diff --git a/apps/desktop/src/platform/harmony/gui/tray/harmony_tray.h b/apps/desktop/src/platform/harmony/gui/tray/harmony_tray.h new file mode 100644 index 00000000..bed0b69f --- /dev/null +++ b/apps/desktop/src/platform/harmony/gui/tray/harmony_tray.h @@ -0,0 +1,30 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _HARMONY_TRAY_H_ +#define _HARMONY_TRAY_H_ + +#include +#include + +namespace crossdesk { +// Constructed and invoked on the shared Slint event thread. +class HarmonyTray { + public: + HarmonyTray(std::function show, std::function hide, + std::function settings, std::function exit, + const std::string& tooltip, int language); + ~HarmonyTray(); + bool MinimizeToTray(); + void RemoveTrayIcon(); + static void Dispatch(const std::string& action); + private: + std::function show_, hide_, settings_, exit_; + bool active_ = false; +}; +} + +#endif diff --git a/apps/desktop/src/platform/harmony/gui/window_state.h b/apps/desktop/src/platform/harmony/gui/window_state.h new file mode 100644 index 00000000..1145da9d --- /dev/null +++ b/apps/desktop/src/platform/harmony/gui/window_state.h @@ -0,0 +1,16 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _WINDOW_STATE_H_ +#define _WINDOW_STATE_H_ + +namespace slint { class Window; } +namespace crossdesk { +// GUI-thread query of the native window focus delivered by Harmony ArkUI. +bool IsHarmonyWindowActive(const slint::Window& window); +} + +#endif diff --git a/apps/desktop/src/platform/harmony/harmony_autostart.h b/apps/desktop/src/platform/harmony/harmony_autostart.h new file mode 100644 index 00000000..09b60766 --- /dev/null +++ b/apps/desktop/src/platform/harmony/harmony_autostart.h @@ -0,0 +1,19 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _HARMONY_AUTOSTART_H_ +#define _HARMONY_AUTOSTART_H_ + +namespace crossdesk { +enum class HarmonyAutostartStatus { checking, enabled, disabled, unsupported, failed }; + +// ArkTS publishes system state; the GUI reads it without waiting on a service. +HarmonyAutostartStatus GetHarmonyAutostartStatus(); +void SetHarmonyAutostartStatus(HarmonyAutostartStatus status); +void RefreshHarmonyAutostartStatus(); +} + +#endif diff --git a/apps/desktop/src/platform/harmony/input/system_shortcuts.h b/apps/desktop/src/platform/harmony/input/system_shortcuts.h new file mode 100644 index 00000000..90bf0677 --- /dev/null +++ b/apps/desktop/src/platform/harmony/input/system_shortcuts.h @@ -0,0 +1,15 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SYSTEM_SHORTCUTS_H_ +#define _SYSTEM_SHORTCUTS_H_ + +#include +namespace crossdesk { +bool SendHarmonySystemShortcut(ServiceCommandFlag command); +} + +#endif diff --git a/apps/desktop/src/privacy/privacy_backend.h b/apps/desktop/src/privacy/privacy_backend.h index 1973993a..2158ceae 100644 --- a/apps/desktop/src/privacy/privacy_backend.h +++ b/apps/desktop/src/privacy/privacy_backend.h @@ -57,7 +57,7 @@ std::unique_ptr CreateMacPrivacyBackend(); // Flush queued AppKit cleanup after joining the controller at app shutdown. void FlushMacPrivacyTasks(); #endif -#ifdef __linux__ +#if defined(__linux__) && !defined(__OHOS__) std::unique_ptr CreateLinuxPrivacyBackend(); #endif diff --git a/apps/desktop/src/privacy/privacy_controller.cpp b/apps/desktop/src/privacy/privacy_controller.cpp index f22cd341..8838f823 100644 --- a/apps/desktop/src/privacy/privacy_controller.cpp +++ b/apps/desktop/src/privacy/privacy_controller.cpp @@ -16,7 +16,7 @@ PrivacyController::PrivacyController(Factory factory) { if (!factory) factory = CreateWindowsPrivacyBackend; #elif defined(__APPLE__) if (!factory) factory = CreateMacPrivacyBackend; -#elif defined(__linux__) +#elif (defined(__linux__) && !defined(__OHOS__)) if (!factory) factory = CreateLinuxPrivacyBackend; #endif thread_ = std::thread([this, factory] { Run(factory); }); diff --git a/apps/desktop/src/tools/file_transfer.cpp b/apps/desktop/src/tools/file_transfer.cpp index 99df886c..437d48fa 100644 --- a/apps/desktop/src/tools/file_transfer.cpp +++ b/apps/desktop/src/tools/file_transfer.cpp @@ -2,6 +2,9 @@ #include #include "rd_log.h" +#if defined(__OHOS__) +#include "platform/path_backend.h" +#endif namespace crossdesk { @@ -18,6 +21,11 @@ FileReceiver::FileReceiver(const std::filesystem::path& output_dir) } std::filesystem::path FileReceiver::GetDefaultDesktopPath() { +#if defined(__OHOS__) + // Harmony has no process HOME/Desktop. Use writable application storage + // until the user selects and authorizes a public directory in settings. + const auto desktop_path = platform::GetCachePath("CrossDesk") / "Received Files"; +#else #ifdef _WIN32 const char* home_env = std::getenv("USERPROFILE"); #else @@ -29,6 +37,7 @@ std::filesystem::path FileReceiver::GetDefaultDesktopPath() { std::filesystem::path desktop_path = std::filesystem::path(home_env) / "Desktop"; +#endif std::error_code ec; std::filesystem::create_directories(desktop_path, ec); diff --git a/apps/desktop/src/version_checker/version_checker.cpp b/apps/desktop/src/version_checker/version_checker.cpp index 88926c41..179e3131 100644 --- a/apps/desktop/src/version_checker/version_checker.cpp +++ b/apps/desktop/src/version_checker/version_checker.cpp @@ -122,7 +122,11 @@ std::optional CheckUpdate() { const auto manifest = nlohmann::json::parse(res->body); auto info = ParseVersionInfo(manifest); if (info) { -#if defined(__linux__) +#if defined(__OHOS__) + // A Linux package cannot be installed by Harmony's package manager. + // Until the manifest has a Harmony artifact, use the release website. + info->download_url = "https://crossdesk.cn"; +#elif defined(__linux__) std::string architecture; #if defined(__x86_64__) architecture = "amd64"; diff --git a/apps/desktop/tests/slint_popup_render_test.cpp b/apps/desktop/tests/slint_popup_render_test.cpp new file mode 100644 index 00000000..31842d81 --- /dev/null +++ b/apps/desktop/tests/slint_popup_render_test.cpp @@ -0,0 +1,363 @@ +#include "crossdesk_ui.h" +#include "slint_render_test_platform.h" +#include "ui/model_update.h" + +#include +#include +#include +#include + +using namespace std::chrono_literals; + +namespace { +bool Check(bool value, const char* message) { + if (!value) std::cerr << message << '\n'; + return value; +} + +bool VerifyStreamPopups(RenderTestPlatform& platform) { + auto stream = crossdesk::ui::StreamWindow::create(); + auto& screen = *platform.adapter; + stream->window().set_size(slint::LogicalSize({1280, 720})); + stream->set_video_settings_enabled(true); + stream->show(); + const int top = int(stream->get_video_area_top()); + auto displays = std::make_shared>(); + auto transfers = std::make_shared>(); + stream->set_displays(displays); + stream->set_file_transfers(transfers); + bool file_visible = false; + unsigned ticks = 0; + slint::Timer sync; + sync.start(slint::TimerMode::Repeated, 16ms, [&] { + // Match the live session's model synchronization, including long clicks. + crossdesk::ui::UpdateModelRows(*displays, + std::vector{"Display 1", "Display 2", "Display 3"}); + crossdesk::ui::FileTransferEntry entry{}; + entry.name = "test.txt"; entry.progress = (++ticks % 100) / 100.f; + crossdesk::ui::UpdateModelRows(*transfers, std::vector(8, entry)); + stream->set_file_transfer_visible(file_visible); + }); + int remote_presses = 0, remote_releases = 0, remote_scrolls = 0; + stream->on_pointer_input([&](auto, auto kind, float, float) { + if (kind == slint::PointerEventKind::Down) ++remote_presses; + if (kind == slint::PointerEventKind::Up) ++remote_releases; + }); + stream->on_scroll_input([&](float, float, float, float) { ++remote_scrolls; }); + int selected_display = -1; + stream->on_switch_display([&](int index) { selected_display = index; }); + std::string shortcut; + stream->on_send_shortcut([&](slint::SharedString text) { shortcut = text; }); + int video_changes = 0; + stream->on_change_video_settings([&](int quality, int rate, int policy) { + stream->set_video_quality_index(quality); + stream->set_frame_rate_index(rate); + stream->set_adaptation_policy_index(policy); + ++video_changes; + }); + stream->on_select_file([&] { file_visible = true; }); + bool close_synchronously = false; + stream->on_close_file_transfer([&] { + // The real application hides this on its next synchronization tick. + file_visible = false; + if (close_synchronously) stream->set_file_transfer_visible(false); + }); + screen.Settle(); + for (int repeat = 0; repeat < 2; ++repeat) { + screen.Click(21, top + 19); + if (!Check(screen.Light(12, top + 128), "Display menu must fully open without pointer movement")) return false; + screen.Click(60, top + 86, 4); + if (!Check(selected_display == 1 && screen.Black(12, top + 128), + "Display selection must survive live refresh and close the menu")) return false; + screen.Click(53, top + 19); + if (!Check(screen.Light(44, top + 102), "Shortcut menu must open without pointer movement")) return false; + screen.Click(70, top + 88, 4); + if (!Check(shortcut == "Win+L" && screen.Black(44, top + 102), + "Shortcut selection must close the menu immediately")) return false; + screen.Click(213, top + 19); + if (!Check(stream->get_video_menu_open() && screen.Light(203, top + 210), + "Video settings menu must fully open without pointer movement")) return false; + screen.Click(240, top + 80, 4); + screen.Click(900, top + 300); + if (!Check(!stream->get_video_menu_open() && video_changes == repeat + 1, + "Video selection and outside dismissal must respond immediately")) return false; + } + if (!Check(remote_presses == 0 && remote_releases == 0 && remote_scrolls == 0, + "Menu gestures must not be forwarded to the peer")) return false; + screen.Click(181, top + 19); + const int panel_y = top + int(stream->get_video_area_height()) - 210; + if (!Check(file_visible && screen.Light(20, panel_y + 25), + "File progress must appear without pointer movement")) return false; + // Header/padding is outside the list's own input handler. + screen.Click(25, panel_y + 20, 4); + stream->window().dispatch_pointer_scroll_event( + slint::LogicalPosition({25, float(panel_y + 20)}), 0, -60); + screen.Settle(); + if (!Check(remote_presses == 0 && remote_releases == 0 && remote_scrolls == 0, + "File progress header and padding must consume input locally")) return false; + const slint::LogicalPosition panel_point({25, float(panel_y + 20)}); + const slint::LogicalPosition outside({900, float(top + 300)}); + stream->window().dispatch_pointer_press_event(panel_point, slint::PointerEventButton::Left); + screen.Settle(4); + stream->window().dispatch_pointer_move_event(outside); + stream->window().dispatch_pointer_release_event(outside, slint::PointerEventButton::Left); + screen.Settle(); + if (!Check(remote_presses == 0 && remote_releases == 0, + "Dragging out of the file panel must not release a button on the peer")) return false; + screen.Click(430, panel_y + 16, 4); + if (!Check(!file_visible && screen.Black(20, panel_y + 25), + "File progress close must clear its pixels without pointer movement")) return false; + if (!Check(remote_presses == 0 && remote_releases == 0, + "Closing file progress must not send an orphan pointer release")) return false; + close_synchronously = true; + screen.Click(181, top + 19); + screen.Click(430, panel_y + 16, 4); + if (!Check(!file_visible && screen.Black(20, panel_y + 25) && remote_releases == 0, + "Synchronous file panel dismissal must consume its closing release")) return false; + screen.Click(900, top + 300); + return Check(remote_presses == 1 && remote_releases == 1, + "Closing local menus must restore remote input"); +} + +bool VerifyWorkbenchPopups(RenderTestPlatform& platform) { + auto main = crossdesk::ui::MainWindow::create(); + auto& screen = *platform.adapter; + const float width = main->get_minimum_window_width(); + const float height = main->get_minimum_window_height(); + main->window().set_size(slint::LogicalSize({width, height})); + main->show(); + auto groups = std::make_shared>( + std::vector{"All", "Ungrouped", "Lab"}); + main->set_recent_groups(groups); + int selected_group = -1; + main->on_recent_group_selected([&](int index) { selected_group = index; }); + screen.Settle(); + const auto filter_closed = screen.Region(width - 170, 302, 140, 70); + screen.Click(width - 90, 276); + if (!Check(screen.Region(width - 170, 302, 140, 70) != filter_closed, + "Group filter must reveal its contents while the pointer stays on the trigger")) return false; + screen.Key(slint::platform::key_codes::Escape); + if (!Check(screen.Region(width - 170, 302, 140, 70) == filter_closed, + "Escape must dismiss the group filter")) return false; + screen.Click(width - 90, 276); + screen.Click(width - 90, 371, 4); + if (!Check(selected_group == 2 && main->get_recent_group_index() == 2, + "Group selection must survive a held click")) return false; + screen.Click(width - 38, 20); + screen.Click(width - 78, 56, 4); + if (!Check(main->get_settings_open(), "Application menu must open settings on click")) return false; + const float settings_y = (height - 446) / 2; + screen.Click(width / 2 + 87, settings_y + 76, 4); + if (!Check(main->get_settings_tab() == 1, "Settings tabs must switch without pointer movement")) return false; + screen.Click(width / 2 - 87, settings_y + 76, 4); + if (!Check(main->get_settings_tab() == 0, "Settings tabs must switch back immediately")) return false; + // The first row uses the same CompactComboBox as codec/capture settings. + const int combo_x = int(width / 2 + 100); + const int combo_y = int(settings_y + 150); + const int popup_y = int(settings_y + 164); + const auto combo_closed = screen.Region(combo_x - 35, popup_y + 2, 60, 60); + screen.Click(combo_x, combo_y); + if (!Check(screen.Region(combo_x - 35, popup_y + 2, 60, 60) != combo_closed, + "Settings dropdown must animate open without pointer movement")) return false; + screen.Key(slint::platform::key_codes::Escape); + if (!Check(screen.Region(combo_x - 35, popup_y + 2, 60, 60) == combo_closed, + "Escape must dismiss the settings dropdown")) return false; + screen.Click(combo_x, combo_y); + screen.Click(combo_x, popup_y + 34, 4); + if (!Check(main->get_language_index() == 1, "Settings dropdown must apply a selected row")) return false; + // A session starting while a codec menu is open disables the setting. + // Existing popup rows must not remain actionable after that transition. + const int codec_y = int(settings_y + 224); + screen.Click(combo_x, codec_y); + main->set_settings_session_active(true); + screen.Settle(); + screen.Click(combo_x, codec_y + 48, 4); + if (!Check(main->get_codec_index() == 0, + "A disabled codec dropdown must close and reject its stale options")) return false; + main->set_settings_session_active(false); + screen.Click(width / 2 + 32, settings_y + 412); + if (!Check(!main->get_settings_open(), "Settings Cancel must close the dialog immediately")) return false; + screen.Click(width - 38, 20); + screen.Click(width - 78, 86, 4); + if (!Check(main->get_messages_open(), "Messages must open directly from the application menu")) return false; + screen.Key(slint::platform::key_codes::Escape); + if (!Check(!main->get_messages_open(), "Escape must dismiss Messages")) return false; + screen.Click(width - 38, 20); + screen.Click(width - 78, 132, 4); + if (!Check(main->get_about_open(), "About must open directly from the application menu")) return false; + screen.Key(slint::platform::key_codes::Escape); + return Check(!main->get_about_open(), "Escape must dismiss About"); +} + +bool VerifyAutostartSettings(RenderTestPlatform& platform) { + auto main = crossdesk::ui::MainWindow::create(); + auto& screen = *platform.adapter; + using Status = crossdesk::ui::AutostartStatus; + const float width = main->get_minimum_window_width(); + const float height = main->get_minimum_window_height(); + main->window().set_size(slint::LogicalSize({width, height})); + main->set_autostart_system_managed(true); + main->set_autostart_enabled(true); // Stale app preference must not label system state. + main->set_autostart_status(Status::Disabled); + int refreshes = 0; + main->on_refresh_autostart_status([&] { ++refreshes; }); + main->show(); main->set_settings_open(true); screen.Settle(); + if (!Check(refreshes == 1, "Opening managed autostart settings must query the system")) return false; + const float top = (height - 446) / 2; + main->window().dispatch_pointer_scroll_event(slint::LogicalPosition({width / 2, top + 260}), 0, -1200); + screen.Settle(); + const float toggle_x = width / 2 + 144, toggle_y = top + 322; + const auto toggle_pixels = [&] { + return screen.Region(int(toggle_x - 18), int(toggle_y - 10), 36, 20); + }; + const auto save_snapshot = [&](const std::string& suffix) { + if (const char* snapshot = std::getenv("CROSSDESK_AUTOSTART_UI_SNAPSHOT")) { + std::ofstream out(std::string(snapshot) + suffix, std::ios::binary); + out << "P6\n" << screen.extent.width << ' ' << screen.extent.height << "\n255\n"; + for (const auto pixel : screen.pixels) { out.put(pixel.r); out.put(pixel.g); out.put(pixel.b); } + } + }; + const auto disabled = toggle_pixels(); + save_snapshot(".settings.ppm"); + main->set_autostart_status(Status::Enabled); screen.Settle(); + const auto enabled = toggle_pixels(); + if (!Check(enabled != disabled, + "The autostart switch must reflect system changes while settings stay open")) return false; + main->set_autostart_status(Status::Disabled); screen.Settle(); + screen.Click(toggle_x, toggle_y); + if (!Check(main->get_autostart_guide_open() && main->get_autostart_status() == Status::Disabled && + main->get_autostart_enabled(), + "Clicking the disabled system switch opens guidance without changing state or preference")) return false; + screen.Key(slint::platform::key_codes::Escape); + if (!Check(toggle_pixels() == disabled, "The system switch must not flip when guidance closes")) return false; + screen.Click(toggle_x, toggle_y); + // Both actions stay below the scrollable instructions at minimum size. + const auto button_y = [&]() { + const int x = int(width / 2 + 120), y = int(height / 2 + 158); + const auto p = screen.pixels.at(y * screen.extent.width + x); + return p.b > 180 && p.r < 90 && p.g > 80 && p.g < 160 ? y : -1; + }; + int refresh_y = button_y(); + if (!Check(refresh_y > 0, "Autostart guide buttons must fit inside the minimum window")) return false; + save_snapshot(""); + screen.Click(width / 2 - 145, refresh_y); + if (!Check(refreshes == 2, "Autostart guide Refresh must query the actual system")) return false; + main->set_autostart_status(Status::Checking); screen.Settle(); + screen.Click(width / 2 - 145, button_y()); + if (!Check(refreshes == 2, "Refresh is disabled while the system query is pending")) return false; + for (const auto status : {Status::Failed, Status::Unsupported, Status::Enabled}) { + main->set_autostart_status(status); screen.Settle(); + if (!Check(button_y() > 0, "Error and unsupported states must remain readable and dismissible")) return false; + } + screen.Key(slint::platform::key_codes::Escape); + if (!Check(!main->get_autostart_guide_open() && main->get_settings_open(), + "Escape closes the guide without cancelling other settings")) return false; + if (!Check(toggle_pixels() == enabled, "The switch must still follow system state after opening guidance")) return false; + screen.Click(toggle_x, toggle_y); + if (!Check(main->get_autostart_guide_open() && main->get_autostart_status() == Status::Enabled, + "Clicking the enabled system switch must also open guidance without changing state")) return false; + screen.Click(width / 2 + 160, button_y()); + if (!Check(!main->get_autostart_guide_open(), "The guide confirmation must close immediately")) return false; + if (!Check(toggle_pixels() == enabled, "Dismissing guidance must keep the system switch enabled")) return false; + main->set_autostart_guide_open(true); main->set_settings_open(false); screen.Settle(); + if (!Check(!main->get_autostart_guide_open(), "Closing settings must also dismiss its guide")) return false; + main->set_autostart_system_managed(false); + main->set_settings_open(true); screen.Settle(); + main->window().dispatch_pointer_scroll_event(slint::LogicalPosition({width / 2, top + 260}), 0, -1200); + screen.Settle(); + screen.Click(toggle_x, toggle_y); + return Check(refreshes == 2 && !main->get_autostart_enabled() && !main->get_autostart_guide_open(), + "Other platforms toggle their autostart preference directly without querying Harmony or opening guidance"); +} + +bool VerifyGalleryPopups(RenderTestPlatform& platform) { + auto wall = crossdesk::ui::MainWindow::create(); + auto& screen = *platform.adapter; + wall->set_gallery_mode(true); + wall->window().set_size(slint::LogicalSize({900, 600})); + wall->set_gallery_count(8); + wall->set_gallery_connected(8); + wall->set_gallery_groups(std::make_shared>( + std::vector{"All", "Lab"})); + auto network = std::make_shared>(); + wall->set_gallery_network_devices(network); + wall->show(); + unsigned ticks = 0; + slint::Timer sync; + sync.start(slint::TimerMode::Repeated, 16ms, [&] { + crossdesk::ui::GalleryNetworkDevice row{}; + row.id = "123456789"; row.name = "Test device"; + row.inbound = slint::SharedString(std::to_string(++ticks) + " kbps"); + crossdesk::ui::UpdateModelRows(*network, std::vector(8, row)); + }); + screen.Settle(); + const auto baseline = screen.Region(75, 100, 740, 400); + for (int repeat = 0; repeat < 2; ++repeat) { + screen.Click(160, 578); + if (!Check(wall->get_gallery_network_open() && screen.Region(75, 100, 740, 400) != baseline, + "DeskHub network dialog must paint immediately during live refresh")) return false; + if (repeat == 0) screen.Click(780, 102, 4); + else screen.Key(slint::platform::key_codes::Escape); + if (!Check(!wall->get_gallery_network_open() && screen.Region(75, 100, 740, 400) == baseline, + "DeskHub network dialog must close completely without pointer movement")) return false; + } + // Open the add menu, leave the pointer on it, then select its first row. + const auto menu_closed = screen.Region(490, 42, 220, 60); + screen.Click(680, 24); + if (!Check(screen.Region(490, 42, 220, 60) != menu_closed, + "DeskHub add menu must appear without pointer movement")) return false; + screen.Click(600, 55, 4); + if (!Check(wall->get_gallery_adding(), "Add menu must open the device dialog")) return false; + screen.Click(458, 421, 4); + return Check(!wall->get_gallery_adding(), "Add dialog Cancel must respond immediately"); +} + +bool VerifyServerPanel(RenderTestPlatform& platform) { + auto panel = crossdesk::ui::ServerWindow::create(); + auto& screen = *platform.adapter; + panel->set_controllers(std::make_shared>( + std::vector{{"1", "First"}, {"2", "Second"}})); + panel->window().set_size(slint::LogicalSize({225, panel->get_content_height()})); + panel->show(); + int selected = -1, disconnected = 0, picked = 0; + panel->on_controller_selected([&](int index) { selected = index; }); + panel->on_disconnect_all([&] { ++disconnected; }); + panel->on_select_file([&] { ++picked; panel->set_file_transfer_visible(true); }); + slint::Timer sync; + sync.start(slint::TimerMode::Repeated, 16ms, [&] { + const auto height = unsigned(panel->get_content_height()); + if (screen.extent.height != height) + panel->window().set_size(slint::LogicalSize({225, float(height)})); + panel->set_file_progress(0.5); + }); + screen.Settle(); + screen.Click(50, 84, 4); + if (!Check(selected == 1, "Controlled-side controller selection must respond to a held click")) return false; + const auto normal_height = screen.extent.height; + screen.Click(180, 123, 4); + if (!Check(panel->get_confirming_disconnect() && screen.extent.height == normal_height + 36, + "Disconnect confirmation must expand immediately")) return false; + screen.Click(110, 123, 4); + if (!Check(!panel->get_confirming_disconnect() && disconnected == 0 && screen.extent.height == normal_height, + "Cancel must collapse the controlled-side confirmation")) return false; + screen.Click(180, 123, 4); + screen.Click(180, 123, 4); + if (!Check(disconnected == 1 && !panel->get_confirming_disconnect(), + "Disconnect all must require and accept confirmation")) return false; + screen.Click(110, 123, 4); + return Check(picked == 1 && screen.extent.height == normal_height + 58, + "Controlled-side file progress must resize the panel immediately"); +} +} // namespace + +int main() { + auto platform = std::make_unique(); + auto* backend = platform.get(); + slint::platform::set_platform(std::move(platform)); + if (!VerifyStreamPopups(*backend)) return 1; + if (!VerifyWorkbenchPopups(*backend)) return 2; + if (!VerifyGalleryPopups(*backend)) return 3; + if (!VerifyServerPanel(*backend)) return 4; + if (!VerifyAutostartSettings(*backend)) return 5; + std::cout << "Slint popup rendering and local input passed.\n"; +} diff --git a/apps/desktop/tests/slint_render_test_platform.h b/apps/desktop/tests/slint_render_test_platform.h new file mode 100644 index 00000000..0e874302 --- /dev/null +++ b/apps/desktop/tests/slint_render_test_platform.h @@ -0,0 +1,91 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SLINT_RENDER_TEST_PLATFORM_H_ +#define _SLINT_RENDER_TEST_PLATFORM_H_ + +#include + +#include +#include +#include +#include +#include + +// Retain pixels and honor redraw notifications, as the Harmony adapter does. +// take_snapshot() forces a repaint and can conceal a stalled popup/animation. +class RenderTestWindow : public slint::platform::WindowAdapter { + public: + slint::platform::SoftwareRenderer software{ + slint::platform::SoftwareRenderer::RepaintBufferType::ReusedBuffer}; + slint::PhysicalSize extent{{1280, 720}}; + std::vector pixels = std::vector(1280 * 720); + bool dirty = true; + slint::PhysicalSize size() override { return extent; } + slint::platform::AbstractRenderer& renderer() override { return software; } + void request_redraw() override { dirty = true; } + void set_size(slint::PhysicalSize value) override { + extent = value; + pixels.resize(extent.width * extent.height); + window().dispatch_resize_event(slint::LogicalSize( + {float(extent.width), float(extent.height)})); + dirty = true; + } + void Render() { + if (!dirty && !window().has_active_animations()) return; + dirty = false; + software.render(pixels, extent.width); + } + void Settle(int frames = 15) { + for (int i = 0; i < frames; ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(16)); + slint::platform::update_timers_and_animations(); + Render(); + } + } + void Click(float x, float y, int hold_frames = 0) { + const slint::LogicalPosition point({x, y}); + window().dispatch_pointer_move_event(point); + window().dispatch_pointer_press_event(point, slint::PointerEventButton::Left); + Settle(hold_frames); + window().dispatch_pointer_release_event(point, slint::PointerEventButton::Left); + Settle(); // Do not move the pointer to make a delayed repaint pass. + } + bool Light(int x, int y) const { + const auto p = pixels.at(y * extent.width + x); + return p.r >= 230 && p.g >= 230 && p.b >= 230; + } + bool Black(int x, int y) const { + const auto p = pixels.at(y * extent.width + x); + return p.r == 0 && p.g == 0 && p.b == 0; + } + std::uint64_t Region(int x, int y, int width, int height) const { + std::uint64_t hash = 14695981039346656037ull; + for (int row = y; row < y + height; ++row) + for (int col = x; col < x + width; ++col) { + const auto p = pixels.at(row * extent.width + col); + for (auto channel : {p.r, p.g, p.b}) hash = (hash ^ channel) * 1099511628211ull; + } + return hash; + } + void Key(const slint::SharedString& key) { + window().dispatch_key_press_event(key); + window().dispatch_key_release_event(key); + Settle(); + } +}; + +class RenderTestPlatform : public slint::platform::Platform { + public: + RenderTestWindow* adapter = nullptr; + std::unique_ptr create_window_adapter() override { + auto result = std::make_unique(); + adapter = result.get(); + return result; + } +}; + +#endif diff --git a/apps/desktop/tests/slint_stream_render_test.cpp b/apps/desktop/tests/slint_stream_render_test.cpp new file mode 100644 index 00000000..f82494b5 --- /dev/null +++ b/apps/desktop/tests/slint_stream_render_test.cpp @@ -0,0 +1,112 @@ +#include "crossdesk_ui.h" +#include "ui/model_update.h" + +#include + +#include +#include +#include +#include +#include +#include + +namespace { +// Match the retained software buffer used by the Harmony desktop adapter. +// A full window snapshot would conceal missing damage/redraw notifications. +class TestWindow : public slint::platform::WindowAdapter { + public: + slint::platform::SoftwareRenderer software{ + slint::platform::SoftwareRenderer::RepaintBufferType::ReusedBuffer}; + std::vector pixels = std::vector(1280 * 720); + bool dirty = true; + slint::PhysicalSize size() override { return slint::PhysicalSize({1280, 720}); } + slint::platform::AbstractRenderer& renderer() override { return software; } + void request_redraw() override { dirty = true; } + void Render() { + if (!dirty && !window().has_active_animations()) return; + dirty = false; + software.render(pixels, 1280); + } +}; + +class TestPlatform : public slint::platform::Platform { + public: + TestWindow* adapter = nullptr; + std::unique_ptr create_window_adapter() override { + auto result = std::make_unique(); + adapter = result.get(); + return result; + } +}; +} // namespace + +int main() { + auto platform = std::make_unique(); + auto* backend = platform.get(); + slint::platform::set_platform(std::move(platform)); + auto stream = crossdesk::ui::StreamWindow::create(); + auto& adapter = *backend->adapter; + stream->window().dispatch_resize_event(slint::LogicalSize({1280, 720})); + stream->show(); + auto rows = std::make_shared>(); + stream->set_stats_rows(rows); + std::vector stats(4); + unsigned samples = 0; + bool stats_visible = false; + stream->on_toggle_stats([&] { stats_visible = !stats_visible; }); + // Synchronize inside a Slint timer, exactly as GuiApplication does. Updating + // before update_timers_and_animations would use the previous animation tick. + slint::Timer sync; + sync.start(slint::TimerMode::Repeated, std::chrono::milliseconds(16), [&] { + stream->set_stats_visible(stats_visible); + // Keep values changing while the panel animates, as during a live session. + stats[0].inbound = slint::SharedString(std::to_string(++samples) + " kbps"); + crossdesk::ui::UpdateModelRows(*rows, stats); + }); + const auto tick = [&] { + slint::platform::update_timers_and_animations(); + adapter.Render(); + }; + const auto settle = [&](int frames = 12) { + for (int i = 0; i < frames; ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(16)); + tick(); + } + }; + settle(); + const float top = stream->get_video_area_top(); + for (bool dock_right : {false, true}) { + if (dock_right) { + stream->invoke_begin_control_drag(5, 20); + stream->invoke_move_control_drag(1275, 20); + stream->invoke_end_control_drag(); + settle(); + } + const int control_x = dock_right ? 916 : 0; + const slint::LogicalPosition button({float(control_x + (dock_right ? 278 : 245)), top + 19}); + stream->window().dispatch_pointer_move_event(button); + settle(35); // Include the tooltip that appears before the click. + for (bool expanded : {true, false, true, false}) { + stream->window().dispatch_pointer_press_event(button, slint::PointerEventButton::Left); + stream->window().dispatch_pointer_release_event(button, slint::PointerEventButton::Left); + settle(); // No pointer movement: animation must finish on its own. + for (int y : {65, 195}) { + const auto pixel = adapter.pixels[(int(top) + y) * 1280 + control_x + 4]; + const bool painted = pixel.r >= 240 && pixel.g >= 240 && pixel.b >= 240; + const bool cleared = pixel.r == 0 && pixel.g == 0 && pixel.b == 0; + if (stats_visible != expanded || (expanded ? !painted : !cleared)) { + std::cerr << "Stats must fully " << (expanded ? "expand" : "collapse") + << " without pointer movement; dock_right=" << dock_right + << ", y=" << y << ", pixel=" << int(pixel.r) << ',' + << int(pixel.g) << ',' << int(pixel.b) << '\n'; + return 1; + } + if (stream->invoke_is_local_control_area(control_x + 4, y) != expanded) { + std::cerr << "Stats input area must match the rendered panel\n"; + return 1; + } + } + } + } + std::cout << "Stream stats retained-buffer rendering passed.\n"; +} diff --git a/apps/desktop/tests/slint_ui_smoke_test.cpp b/apps/desktop/tests/slint_ui_smoke_test.cpp index 978e3246..643abc72 100644 --- a/apps/desktop/tests/slint_ui_smoke_test.cpp +++ b/apps/desktop/tests/slint_ui_smoke_test.cpp @@ -4,6 +4,8 @@ #include "ui/release_notes_text.h" #include "ui/ui_localization.h" +#include + #include #include #include @@ -14,6 +16,7 @@ #include #include #include +#include #include namespace { @@ -845,9 +848,71 @@ bool VerifyWorkbenchActions() { return true; } +bool VerifySystemShortcutMenu() { + auto stream = crossdesk::ui::StreamWindow::create(); + stream->window().set_size(slint::LogicalSize(slint::Size{1280, 720})); + std::vector commands; + stream->on_send_shortcut([&](slint::SharedString command) { commands.emplace_back(command); }); + const auto click = [&](float y) { + const slint::LogicalPosition point({55, stream->get_video_area_top() + y}); + stream->window().dispatch_pointer_move_event(point); + stream->window().dispatch_pointer_press_event(point, slint::PointerEventButton::Left); + stream->window().dispatch_pointer_release_event(point, slint::PointerEventButton::Left); + }; + click(20); + if (!stream->invoke_is_local_control_area(55, 90)) return false; + click(58); + stream->set_secure_attention_supported(false); + click(20); + if (stream->invoke_is_local_control_area(55, 90)) return false; + click(58); + stream->set_secure_attention_supported(true); + click(20); click(88); + return commands == std::vector{"Ctrl+Alt+Del", "Win+L", "Win+L"}; +} + +bool VerifyRemoteIdEditing() { + auto window = crossdesk::ui::MainWindow::create(); + window->window().set_size(slint::LogicalSize(slint::Size{ + window->get_minimum_window_width(), window->get_minimum_window_height()})); + window->on_format_remote_id([](slint::SharedString text) { return text; }); + window->invoke_reset_remote_id(); + const auto key = [&](const slint::SharedString& text) { + window->window().dispatch_key_press_event(text); + window->window().dispatch_key_release_event(text); + }; + const auto settle = [] { + for (int i = 0; i < 10; ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + slint::platform::update_timers_and_animations(); + } + }; + settle(); // Let the initial timer state settle before the first user edit. + for (char digit : std::string("1234")) { + key(slint::SharedString(std::string(1, digit))); + settle(); // Separate edits must not leave the formatter repeating. + } + if (std::string(window->get_remote_id_input()) != "1234") return false; + window->window().dispatch_key_press_event(slint::platform::key_codes::Control); + key("a"); + window->window().dispatch_key_release_event(slint::platform::key_codes::Control); + settle(); // Formatting must not repeatedly collapse the selection. + key("5"); + settle(); + return std::string(window->get_remote_id_input()) == "5"; +} + } // namespace int main() { + if (!VerifySystemShortcutMenu()) { + std::cerr << "Shortcut menu must omit unsupported SAS and resize its input area\n"; + return 13; + } + if (!VerifyRemoteIdEditing()) { + std::cerr << "Remote ID editor must preserve Ctrl+A selection after formatting\n"; + return 12; + } const CaptureOptions capture_options = LoadCaptureOptions(); if (!VerifyMainWindowGeometry()) { std::cerr << "Main window dimensions must stay stable through startup\n"; diff --git a/apps/desktop/xmake/targets.lua b/apps/desktop/xmake/targets.lua index 3ff3d883..e63d17d7 100644 --- a/apps/desktop/xmake/targets.lua +++ b/apps/desktop/xmake/targets.lua @@ -230,6 +230,32 @@ function setup_targets() add_files("apps/desktop/tests/slint_ui_smoke_test.cpp") add_rules("crossdesk.slint_runtime") + target("slint_stream_render_test") + set_kind("binary") + set_languages("c++20") + set_default(false) + if is_os("windows") then + add_cxxflags("/bigobj") + end + add_packages("slint") + add_deps("crossdesk_ui") + add_includedirs("apps/desktop/src/gui") + add_files("apps/desktop/tests/slint_stream_render_test.cpp") + add_rules("crossdesk.slint_runtime") + + target("slint_popup_render_test") + set_kind("binary") + set_languages("c++20") + set_default(false) + if is_os("windows") then + add_cxxflags("/bigobj") + end + add_packages("slint") + add_deps("crossdesk_ui") + add_includedirs("apps/desktop/src/gui") + add_files("apps/desktop/tests/slint_popup_render_test.cpp") + add_rules("crossdesk.slint_runtime") + target("version_checker_test") set_kind("binary") set_default(false) diff --git a/apps/harmony/.clang-tidy b/apps/harmony/.clang-tidy new file mode 100644 index 00000000..69a91485 --- /dev/null +++ b/apps/harmony/.clang-tidy @@ -0,0 +1 @@ +Checks: 'misc-missing-switch-cases,misc-napi-module-name,misc-replace-if-else-with-ternary-operator,misc-unused-local-variable,misc-unused-parameters,modernize-use-auto,readability-system-capabilities,misc-napi-macro-module-name,capi-version-validation' diff --git a/apps/harmony/.clangd b/apps/harmony/.clangd new file mode 100644 index 00000000..6f7c7f85 --- /dev/null +++ b/apps/harmony/.clangd @@ -0,0 +1,16 @@ +CompileFlags: + Add: [-Wunreachable-code-aggressive] + Remove: [-fsanitize=hwaddress, -fsanitize=thread] +Diagnostics: + ClangTidy: + Add: [misc-missing-switch-cases,misc-napi-module-name,misc-replace-if-else-with-ternary-operator,misc-unused-local-variable,misc-unused-parameters,modernize-use-auto,readability-system-capabilities,misc-napi-macro-module-name,capi-version-validation] + Remove: [] + CheckOptions: + misc-unused-parameters.StrictMode: true + misc-unused-parameters.IgnoreVirtual: true + modernize-use-auto.MinTypeNameLength: 0 + + UnusedIncludes: Strict + UnusedFunctions: Check +Hover: + ShowNewHover: Yes \ No newline at end of file diff --git a/apps/harmony/.gitignore b/apps/harmony/.gitignore new file mode 100644 index 00000000..83ca3c67 --- /dev/null +++ b/apps/harmony/.gitignore @@ -0,0 +1,25 @@ +.hvigor/ +.native/ +.idea/ +**/__pycache__/ +*.py[cod] +oh_modules/ +**/oh_modules/ +**/build/ +**/.cxx/ +native/.xmake/ +entry/libs/ +local.properties +*.p12 +*.p7b +*.cer +*.csr + +# Generated mobile legal resources; regenerated from checked-in sources and the SDK. +entry/src/main/resources/rawfile/about/ +entry/src/main/resources/rawfile/LICENSE +entry/src/main/resources/rawfile/PRIVACY.md +entry/src/main/resources/rawfile/OPEN_SOURCE_NOTICE.txt +entry/src/main/resources/rawfile/THIRD_PARTY_NOTICES.txt +entry/src/main/resources/rawfile/ThirdPartyLicenses.json +entry/src/main/resources/rawfile/SourceMetadata.json diff --git a/apps/harmony/AppScope/app.json5 b/apps/harmony/AppScope/app.json5 new file mode 100644 index 00000000..367349bf --- /dev/null +++ b/apps/harmony/AppScope/app.json5 @@ -0,0 +1,10 @@ +{ + "app": { + "bundleName": "com.chunkdash.crossdesk", + "vendor": "CrossDesk", + "versionCode": 10000, + "versionName": "0.1.0", + "icon": "$media:app_icon", + "label": "$string:app_name" + } +} diff --git a/apps/harmony/AppScope/resources/base/element/string.json b/apps/harmony/AppScope/resources/base/element/string.json new file mode 100644 index 00000000..1e6ae33b --- /dev/null +++ b/apps/harmony/AppScope/resources/base/element/string.json @@ -0,0 +1,8 @@ +{ + "string": [ + { + "name": "app_name", + "value": "CrossDesk" + } + ] +} diff --git a/apps/harmony/AppScope/resources/base/media/app_icon.png b/apps/harmony/AppScope/resources/base/media/app_icon.png new file mode 100644 index 00000000..e7c2e93b Binary files /dev/null and b/apps/harmony/AppScope/resources/base/media/app_icon.png differ diff --git a/apps/harmony/MOBILE_PARITY.md b/apps/harmony/MOBILE_PARITY.md new file mode 100644 index 00000000..f97809db --- /dev/null +++ b/apps/harmony/MOBILE_PARITY.md @@ -0,0 +1,81 @@ +# 鸿蒙移动控制端与 Android 功能对照 + +手机、平板使用 `pages/Mobile`,角色固定为控制端。对照基线为本仓库 `apps/android`,不包含 PC 被控能力。移动专用 HAP 只声明 `INTERNET` 和 `KEEP_BACKGROUND_RUNNING`;不申请录屏、输入注入、相机、麦克风或整个文件库权限。 + +## 构建与安装 + +```bash +python3 apps/harmony/scripts/build.py --mobile +python3 apps/harmony/tests/run.py --mobile +``` + +需要 Python 3.10+、Xmake 3.1.1+、DevEco SDK。兼容 API 20、目标 API 26、ARM64。产物: + +``` +apps/harmony/.native/mobile-project/entry/build/default/outputs/default/entry-default-unsigned.hap +``` + +手机真机调试时,在 DevEco 打开生成的 `.native/mobile-project` 并配置本机签名,再安装。不要把签名材料提交到仓库。重新执行构建脚本会重新生成隔离工程,移除签名配置。PC 的 ACL 和签名配置不用于移动包。 + +移动构建只复制 `ets/mobile/`、`pages/Mobile.ets`、N-API 桥接和实际使用的资源,只有 `pages/Mobile` 一个页面入口。`crossdesk_harmony_mobile` 将控制会话与共享音视频播放编译为独立原生库,输出到 `.native/mobile-libs/arm64-v8a/`;不编译 PC 会话、屏幕采集或输入注入,也不导出 PC 桥接 API。只重编原生库时使用 `build.py --mobile --native-only`,随后可使用 `--mobile --hap-only` 打包。PC 直接链接 MiniRTC 与平台服务,不再生成旧 ArkUI PC 核心;两端产物不会互相覆盖。 + +生成工程每次重建源码与库目录,以移除旧版本残留;请修改 `entry/src/main/` 下的源码,再运行构建脚本。桌面字体不进入移动包,开源许可证只保留界面可逐项阅读并校验 SHA-256 的完整文档,移除了重复汇总副本。 + +## 功能对照 + +| Android 功能 | 鸿蒙实现 | +| --- | --- | +| 前台自动登录、空闲后台停止、服务器隔离 | 独立 identity peer;只有主动远控才创建 controller peer;拒绝任何入站被控连接 | +| 首次隐私确认、撤回与离线阅读 | 同意前不发起身份、公告、在线状态或版本网络请求;撤回结束会话并清除预览 | +| 加密身份、可选记住访问密码 | 系统 Asset Store,设备首次解锁后可读,禁止跨设备同步;普通 Preferences 不存密码 | +| 最近连接、删除、在线状态 | 最近 20 台、在线优先、真实信令订阅、30 秒刷新和 60 秒过期;删除同步清理密码与预览 | +| H.264 / AV1 | 系统硬件解码;无能力、配置/运行失败时保持协商格式回退 OpenH264 / dav1d;NV12 GLES 显示 | +| 远端音频与静音 | 48 kHz 单声道 OHAudio,控制菜单与画中画静音同步 | +| 相对/绝对鼠标、触控 | 单击、绝对拖动、长按右键、1–10 倍焦点缩放、放大后双指平移、1 倍时双指滚轮 | +| 外接键鼠 | 实体按键映射、鼠标移动/按键/滚轮;失焦、后台、断开及切屏释放输入 | +| 悬浮控制 | 可拖动品牌悬浮球、三列菜单(低高度横屏四列)、可拖动 ASCII/电脑键盘、完整快捷键组合、140×48 虚拟鼠标 | +| 视频设置 | 低/中/高、30/60 fps、帧率/画质/平衡;能力协商、请求序号、旧 ACK 丢弃、拒绝回滚、5 秒超时 | +| 网络诊断 | 视频/音频/数据/总流量、双向速率、丢包率、实际提交帧率/尺寸、画面延迟、平滑 RTT、P2P/TURN、SRTP;3 秒失效 | +| 多显示器、Ctrl+Alt+Del | 按远端显示器列表切换;SAS 仅在远端 Windows 服务报告可用时发送 | +| 剪贴板 | 主动发送(128 KB 上限)、远端粘贴、手动复制收到的文本;退出清空内存文本 | +| 文件双向传输 | 系统文件选择/导出、进度、可靠分块/ACK、乱序覆盖、失败/超时清理;直接共用 Android 文件引擎 | +| 系统画中画与后台保活 | VIDEO_LIVE PiP 与静音控制,后台数据传输任务及系统通知兜底;选择器期间抑制 PiP,系统拒绝后台时结束会话 | +| 公告 | 摘要每页 200、正文每页 20、已读/删除与修订、分页、刷新、12 秒超时、身份及服务器隔离、迟到应答过滤 | +| 版本提示 | 每会话检查远端版本;本机更新使用 `harmony-arm64`,无该发布项时明确显示无鸿蒙版本信息 | +| 远程预览 | 默认关闭、单独确认、下一次连接起每会话一张 640×360、私有缓存、不上传;关闭/撤回清除,异步任务不能恢复已撤回内容 | +| 关于和开源合规 | 实际包版本、离线许可证和组件清单、SHA-256 文档校验、源码版本与构建说明 | +| 手机/平板布局 | 900 vp 双栏断点、最近连接 1–4 列、主体最大 1200/720 vp、手机首页竖屏、会话/平板自动旋转、200 ms 页面/选项过渡 | + +平台 API 有差异:鸿蒙使用 Asset Store、OHAudio、ArkUI PiP 和后台任务,Android 使用 Keystore、AAudio、Android PiP 和前台服务。系统保留快捷键、后台策略、硬件解码能力受具体设备约束。 + +## 移动界面 + +移动端滚动容器显式隐藏叠加式滚动条,包括首页内外层、设置与公告、长文档、弹窗、远控菜单、虚拟键盘及剪贴板文本框;保留原有触摸和滚轮滚动,避免滑块覆盖文字、按钮或按键。新增滚动区域应沿用这一设置。 + +首页、设置、弹窗和远控菜单以 Android `MobileUi` 为视觉基准:统一灰白底色、蓝色主操作、文字层级、线条图标、圆角与阴影。首页提供连接卡片、最近设备卡片及居中空态,宽屏采用固定宽度连接栏与自适应设备列表;设置使用分组卡片、分段选择器和内联服务器配置,隐私、关于、公告及许可页面沿用相同导航与列表样式。 + +访问密码在手机上以底部面板显示,在平板上居中显示;输入法弹出时缩小可用区域,保留密码、保存开关和连接按钮,较小窗口可滚动。隐私政策面板保持标题与授权操作可见,正文独立滚动。删除、撤回、清除和断开操作使用统一确认框。远控菜单提供图标网格、画面设置、对齐的网络统计、品牌悬浮球以及可拖动键盘和鼠标;菜单和键盘按可用窗口尺寸限制位置。 + +此次 UI 检查在 API 26 手机和平板模拟器中使用独立包名、无联网权限的离线状态预览,检查了首页、设置、确认取消、密码显隐与输入法避让、弹窗及远控面板。预览中的设备、传输状态和屏幕尺寸为测试数据,不作为网络互通验收依据。图标与鼠标指针按显示密度绘制,避免 SVG 路径的像素坐标在高密度屏幕上缩小。 + +## 自动化验证 + +`tests/run.py --mobile` 运行生产 ArkTS 模型、控制器和平台生命周期代码的测试,包含隐私前置、身份/密码调用、重复/跨会话回调、重连后台任务串行化、PiP 去重及静音、文件选择器保护、双指/虚拟鼠标独立按键、画质应答乱序与序号回绕、统计失效、在线状态与公告完整分页。 + +同时运行 Android 原生协议及视频管线回归测试、共享文件传输测试、鸿蒙输入/采集适配与公共 wire 测试。共享代码位于 `libs/controller/include`;Android 旧头文件保留兼容导出,避免两平台协议漂移。 + +`tests/verify_hap.py --mobile ` 校验 phone/tablet 设备类型、最小权限、后台模式、编译后的 ArkTS、ARM64 原生库依赖及许可证资料。构建脚本自动执行该检查。 + +## 设备验收 + +### 确认实际解码器 + +解码诊断通过 HiLog 标签 `CrossDeskDecoder` 输出,不受 MiniRTC 通用日志关闭的影响。在 DevEco 的 HiLog 中按此标签过滤,或运行 `hdc shell hilog -x -T CrossDeskDecoder`。`Decode request` 记录当前格式的硬解能力,`Decoder selected` 记录选中的实现和软解回退原因;应以 `First decoded frame` 的 `mode` 和 `name` 确认实际成功工作的解码器,而不能只看硬件加速配置或初始化结果。会话结束的 `Decode summary` 分别统计硬解和软解成功帧数,也能识别中途回退。日志仅记录格式、解码器、尺寸、错误阶段及帧数,不输出连接凭据或视频内容。 + +2026-10-15 在 API 26 ARM64 Mate 90 Pro 模拟器实测:H.264 3840×2160 远控会话使用 OpenH264,成功软解 2722 帧、硬解 0 帧;AV1 640×360 合成视频使用 Dav1dAv1,成功软解 1203 帧、硬解 0 帧。两者均为 `hardware_available=0`,回退原因 `no hardware capability`。这只证明当前模拟器的解码路径,不代表真机硬解能力。 + +### 已覆盖的功能 + +已在 API 26 ARM64 手机(Mate 90 Pro)和平板(MatePad Pro 13)模拟器安装运行,验证首次同意、登录、公告同步、设置及双栏布局。在手机上使用同版本 MiniRTC 合成发送端验证 H.264 / AV1 640×360 画面、48 kHz PCM 接收、鼠标与虚拟键盘回传、画质请求/应答、双屏切换、剪贴板、文件接收及系统导出/选择后反向发送的字节一致性、旋转和后台画中画恢复。发现并修复了公开 ID 覆盖完整登录凭据、连接成功回调前的主机信息丢失、返回前台后画中画遮挡文件选择器等问题,并补充回归测试。主机侧测试不能替代真机上的触摸、多指、外接键鼠、音频路由、硬件编解码性能以及长期后台/画中画测试;这些仍应在发布前用 API 20 与当前系统真机验收。 + +可使用 `tests/mobile_synthetic_host.cpp` 作为互通测试发送端:它只生成测试色条、静音 PCM、虚拟显示器和小文件,不采集本机屏幕或输入。运行参数为 ` [av1]`,测试身份写入工作目录 `fixture-login`,运行状态写入 `fixture-status`,15 分钟自动结束。使用独立私有测试目录,测试后删除身份、日志和接收文件。鸿蒙 fixture 可用 `tests/build_mobile_fixture.py` 编译;受限模拟器 shell 可能不能加载系统媒体库,可在开发机链接 MiniRTC 运行同一源文件。 diff --git a/apps/harmony/README.md b/apps/harmony/README.md new file mode 100644 index 00000000..3c68703b --- /dev/null +++ b/apps/harmony/README.md @@ -0,0 +1,79 @@ +# CrossDesk HarmonyOS + +鸿蒙 PC 只保留共享 Slint 桌面客户端;手机和平板使用独立 ArkUI 控制端。旧 ArkUI PC 界面、独立 PC 会话状态机和早期 Slint 验证程序已移除。 + +| 设备 | 实现 | 要求与说明 | +| --- | --- | --- | +| PC(`2in1`) | 公共桌面 UI、业务逻辑和鸿蒙平台适配 | API 26、arm64;[桌面构建、权限与验收](desktop-port/README.md) | +| 手机 / 平板 | ArkUI 移动控制端 | 兼容 API 20、目标 API 26、arm64;[功能对照与验收](MOBILE_PARITY.md) | + +两者仍为开发版。已完成的模拟器验证及未完成的真机验收见各端说明。 + +## 构建 + +在仓库根目录执行,当前构建脚本面向 macOS: + +```bash +# 默认构建 PC:MiniRTC、Slint runtime、公共桌面应用和 HAP +python3 apps/harmony/scripts/build.py + +# PC 增量构建:复用已生成的 runtime;补丁变化仍会触发重建 +python3 apps/harmony/scripts/build.py --reuse-runtime + +# 手机 / 平板:独立控制端,不包含 PC 采集和输入注入代码 +python3 apps/harmony/scripts/build.py --mobile + +# 仅准备 PC 的 MiniRTC 依赖;不再构建旧 ArkUI PC 核心 +python3 apps/harmony/scripts/build.py --native-only + +# 使用已有原生依赖重新打包 +python3 apps/harmony/scripts/build.py --hap-only --reuse-runtime +python3 apps/harmony/scripts/build.py --mobile --hap-only +``` + +`--release` 选择 release HAP;原生依赖默认使用 release。PC 的安装、设备选择、版本号、权限和签名选项见 `python3 apps/harmony/scripts/build_desktop.py --help`,也可直接传给默认的 `build.py`。移动端仅构建控制端核心时使用 `--mobile --native-only`。 + +需要 DevEco Studio(SDK、Native SDK、Node、ohpm、hvigor、Java)、Xmake 3.1.1+ 和完整的 MiniRTC 子模块。PC 另需 Python 3.12+、Rust 1.92+ 和主机版 Slint compiler 1.17.1;移动端需 Python 3.10+。如果系统 Python 版本过旧,可使用 `/opt/homebrew/bin/python3`。 + +| 环境变量 | 用途 / 默认值 | +| --- | --- | +| `DEVECO_STUDIO_HOME` | `/Applications/DevEco-Studio.app/Contents` | +| `DEVECO_SDK_HOME` | `$DEVECO_STUDIO_HOME/sdk`,包含 `default/openharmony/native` | +| `XMAKE_BIN` | PATH 中的 `xmake`;本机也可指定 `$HOME/.local/bin/xmake` | +| `SLINT_COMPILER` | 主机 Slint compiler;默认查找 Xmake 的 1.17.1 缓存 | + +脚本使用独立的 Xmake 配置与包缓存,不切换仓库根目录的桌面配置。Slint、httplib、Xmake 包仓库和 Python 构建工具版本固定。PC 与移动端构建共用锁,避免同时改写原生依赖。 + +## 工程与产物 + +| 客户端 | 生成的 DevEco 工程 | +| --- | --- | +| PC | `apps/harmony/.native/slint-poc/desktop-project` | +| 手机 / 平板 | `apps/harmony/.native/mobile-project` | + +HAP 均位于对应工程的 `entry/build/default/outputs/default/entry-default-unsigned.hap`。PC 的 `slint-poc` 缓存名保留以复用现有 Rust 构建缓存,源码中已没有验证程序。PC 与手机 / 平板共用 `AppScope/app.json5` 和应用名称资源,包名统一为 `com.chunkdash.crossdesk`,名称统一为 **CrossDesk**。旧 PC 验证包 `cn.crossdesk.slintdesktop` 与新包属于不同应用,已有数据不会自动迁移。 + +使用 DevEco 打开生成工程进行调试和签名。请修改本目录源码后重新执行脚本;生成工程不是源码入口。移动端构建会重新生成源码与库目录;PC 首次从旧构建迁移时会清除旧桥接目标的产物。两端打包后均检查实际 HAP 的设备类型、权限、ArkTS、ARM64 库、SONAME 和依赖,拒绝混入已移除入口的库。 + +签名材料与本机配置放在仓库外或已忽略的 `.native` 中;PC 使用显式 `--signing-config`,不会继承移动工程的签名。MiniRTC 是独立 Git 子模块,交付时需要保留其中的鸿蒙适配改动。 + +## 代码与验证 + +- `desktop-port/`:PC 平台服务、窗口、输入法、Slint 适配及 runtime 补丁;业务逻辑和 UI 直接编译 `apps/desktop`。 +- `entry/src/main/`:移动 ArkUI 页面与移动 N-API 桥接。 +- `native/mobile_session.*`、`media.*`:移动控制会话与 NV12 视频显示。 +- `native/audio.*`:两端共用的 OHAudio 播放服务。 +- `native/capture*`、`input.*`:PC 录屏与被控输入;`key_mapping.h` 也由移动端使用。 +- `libs/wire`:仓库公共协议。 +- `libs/controller/include/file_transfer_core.h`:鸿蒙 PC、其他桌面平台和移动端共用的文件传输核心;`mobile_file_transfer_session.h` 仅保留移动端文件生命周期适配。 + +文件传输使用同一份分块、确认窗口、乱序重组、超时和取消逻辑,保持现有 wire 格式。PC 继续使用公共桌面的多文件队列、进度列表、目录设置及同名文件追加时间戳规则,不删除发送原文件;移动端继续使用文件选择器生成的临时副本,并在完成或失败后清理副本,接收文件仍保存在独立子目录。桌面接收状态按连接和对端隔离,反馈只接受对应对端的 ACK,断开时清理未完成的接收文件。 + +```bash +python3 apps/harmony/tests/run.py --desktop --mobile +python3 apps/harmony/tests/verify_hap.py +python3 apps/harmony/tests/verify_hap.py --mobile \ + apps/harmony/.native/mobile-project/entry/build/default/outputs/default/entry-default-unsigned.hap +``` + +主机侧测试覆盖协议、输入、采集、显示器、键盘钩子、窗口、输入法、自启动状态及移动生命周期。它们使用 SDK 声明与模拟系统接口;真机录屏、输入授权、多屏、音频、后台与长期会话仍需设备验收。 diff --git a/apps/harmony/build-profile.json5 b/apps/harmony/build-profile.json5 new file mode 100644 index 00000000..5f42883d --- /dev/null +++ b/apps/harmony/build-profile.json5 @@ -0,0 +1,41 @@ +{ + "app": { + "signingConfigs": [], + "products": [ + { + "name": "default", + "targetSdkVersion": "26.0.0", + "compatibleSdkVersion": "6.0.0(20)", + "runtimeOS": "HarmonyOS", + "buildOption": { + "strictMode": { + "caseSensitiveCheck": true, + "useNormalizedOHMUrl": true + } + } + } + ], + "buildModeSet": [ + { + "name": "debug" + }, + { + "name": "release" + } + ] + }, + "modules": [ + { + "name": "entry", + "srcPath": "./entry", + "targets": [ + { + "name": "default", + "applyToProducts": [ + "default" + ] + } + ] + } + ] +} diff --git a/apps/harmony/desktop-port/AutostartHost.ets b/apps/harmony/desktop-port/AutostartHost.ets new file mode 100644 index 00000000..2b8cc8b0 --- /dev/null +++ b/apps/harmony/desktop-port/AutostartHost.ets @@ -0,0 +1,66 @@ +import { autoStartupManager } from '@kit.AbilityKit'; + +export interface AutostartEvent { + op: string; + pending?: boolean; + ok?: boolean; + supported?: boolean; + active?: boolean; +} + +// Public APIs can only query the user's system setting. Never derive this +// state from an application preference, or treat a failed query as disabled. +export class AutostartHost { + private static running: boolean = false; + private static sequence: number = 0; + private static timer: number = -1; + private static notify: (event: AutostartEvent) => void = () => {}; + + static start(notify: (event: AutostartEvent) => void): void { + AutostartHost.stop(); + AutostartHost.notify = notify; + AutostartHost.running = true; + AutostartHost.refresh(); + } + + static stop(): void { + AutostartHost.running = false; + ++AutostartHost.sequence; + clearTimeout(AutostartHost.timer); + AutostartHost.timer = -1; + } + + static refresh(): void { + if (!AutostartHost.running) return; + const sequence = ++AutostartHost.sequence; + clearTimeout(AutostartHost.timer); + AutostartHost.notify({ op: 'autostartState', pending: true }); + // A stalled system service must not leave the settings page loading forever. + AutostartHost.timer = setTimeout(() => { + AutostartHost.finish(sequence, { op: 'autostartState', ok: false }); + }, 5000); + try { + if (!autoStartupManager.isAutoStartupSupported()) { + AutostartHost.finish(sequence, { op: 'autostartState', ok: true, supported: false }); + return; + } + autoStartupManager.getAutoStartupStatusForSelf().then((active: boolean) => { + AutostartHost.finish(sequence, { op: 'autostartState', ok: true, supported: true, active }); + }).catch((error: Error) => { + console.warn(`Slint autostart query: ${error.message}`); + AutostartHost.finish(sequence, { op: 'autostartState', ok: false }); + }); + } catch (error) { + console.warn(`Slint autostart capability: ${error.message}`); + AutostartHost.finish(sequence, { op: 'autostartState', ok: false }); + } + } + + private static finish(sequence: number, event: AutostartEvent): void { + if (!AutostartHost.running || sequence !== AutostartHost.sequence) return; + ++AutostartHost.sequence; // Also reject a result arriving after its timeout. + clearTimeout(AutostartHost.timer); + AutostartHost.timer = -1; + AutostartHost.notify(event); + } +} diff --git a/apps/harmony/desktop-port/CMakeLists.txt b/apps/harmony/desktop-port/CMakeLists.txt new file mode 100644 index 00000000..2356b426 --- /dev/null +++ b/apps/harmony/desktop-port/CMakeLists.txt @@ -0,0 +1,50 @@ +# Compile the desktop application itself. Harmony owns only platform services; +# callbacks, settings, history, DeskHub and session state stay in apps/desktop. +set(DESKTOP "${CROSSDESK_ROOT}/apps/desktop/src") +set(HARMONY "${CROSSDESK_ROOT}/apps/harmony") +file(GLOB RUNTIME CONFIGURE_DEPENDS "${DESKTOP}/gui/runtime/*.cpp") +file(GLOB_RECURSE FEATURES CONFIGURE_DEPENDS "${DESKTOP}/gui/features/*.cpp") +file(GLOB COMMON CONFIGURE_DEPENDS "${DESKTOP}/common/*.cpp" "${DESKTOP}/thumbnail/*.cpp" + "${DESKTOP}/version_checker/*.cpp" "${CROSSDESK_ROOT}/libs/wire/src/*.cpp") +target_sources(slint_desktop PRIVATE ${RUNTIME} ${FEATURES} ${COMMON} + "${DESKTOP}/gui/application/gui_application.cpp" + "${DESKTOP}/gui/rendering/slint_video_presenter.cpp" + "${DESKTOP}/platform/common/gui/service_status_runtime.cpp" + "${DESKTOP}/privacy/privacy_controller.cpp" + "${DESKTOP}/speaker_capturer/speaker_capture_controller.cpp" + "${DESKTOP}/config_center/config_center.cpp" "${DESKTOP}/autostart/autostart.cpp" + "${DESKTOP}/path_manager/path_manager.cpp" "${DESKTOP}/log/rd_log.cpp" + "${DESKTOP}/tools/file_transfer.cpp" + "${CMAKE_CURRENT_LIST_DIR}/tray.cpp" "${CMAKE_CURRENT_LIST_DIR}/services.cpp" "${CMAKE_CURRENT_LIST_DIR}/devices.cpp" + "${CMAKE_CURRENT_LIST_DIR}/keyboard_hook.cpp" + "${HARMONY}/native/input.cpp" "${HARMONY}/native/audio.cpp" "${HARMONY}/native/capture.cpp" + "${HARMONY}/native/capture_surface.cpp") +target_include_directories(slint_desktop PRIVATE "${CMAKE_CURRENT_LIST_DIR}/include" + "${CMAKE_CURRENT_LIST_DIR}" "${HARMONY}/native" "${DESKTOP}" "${DESKTOP}/gui" + "${DESKTOP}/common" "${DESKTOP}/config_center" "${DESKTOP}/autostart" + "${DESKTOP}/path_manager" "${DESKTOP}/log" "${DESKTOP}/tools" + "${DESKTOP}/thumbnail" "${DESKTOP}/version_checker" "${DESKTOP}/privacy" + "${DESKTOP}/device_controller" "${DESKTOP}/speaker_capturer" "${DESKTOP}/screen_capturer" + "${DESKTOP}/platform/common/input" "${CROSSDESK_ROOT}/libs/wire/include" + "${CROSSDESK_ROOT}/libs/controller/include" + "${CROSSDESK_ROOT}/deps/submodules/minirtc/src/api" + "${CROSSDESK_ROOT}/deps/submodules/minirtc/src/common" + "${CROSSDESK_ROOT}/deps/submodules/minirtc/src/ws" + "${CROSSDESK_ROOT}/deps/submodules/minirtc/src/pc" + "${CROSSDESK_ROOT}/deps/submodules/minirtc/src/media/resolution_adapter") +file(GLOB PACKAGE_INCLUDES "${HARMONY}/.native/packages/*/*/*/*/include") +target_include_directories(slint_desktop SYSTEM PRIVATE ${PACKAGE_INCLUDES}) +file(GLOB QUEUE_INCLUDE "${HARMONY}/.native/packages/c/concurrentqueue/*/*/include/concurrentqueue/moodycamel") +target_include_directories(slint_desktop SYSTEM PRIVATE ${QUEUE_INCLUDE} "${SLINT_DESKTOP_DEPS}") +target_compile_definitions(slint_desktop PRIVATE FMT_CONSTEVAL= CPPHTTPLIB_OPENSSL_SUPPORT + CROSSDESK_VERSION="${CROSSDESK_VERSION}" CROSSDESK_HARMONY_DESKTOP=1) +# Use exactly the native dependency versions built by scripts/build.py. +set(NATIVE_LIB "${HARMONY}/.native/build/harmony/arm64-v8a/release/libminirtc.a") +if(NOT EXISTS "${NATIVE_LIB}") + message(FATAL_ERROR "Build apps/harmony/scripts/build.py --native-only first") +endif() +file(GLOB PACKAGE_LIBS "${HARMONY}/.native/packages/*/*/*/*/lib/*.a") +target_link_libraries(slint_desktop PRIVATE "-Wl,--start-group" "${NATIVE_LIB}" + ${PACKAGE_LIBS} "-Wl,--end-group" native_buffer native_image ohaudio native_avscreen_capture + native_media_core native_media_codecbase native_media_vdec ohinput pasteboard udmf) +target_link_options(slint_desktop PRIVATE -Wl,--no-undefined) diff --git a/apps/harmony/desktop-port/DesktopAccess.ets b/apps/harmony/desktop-port/DesktopAccess.ets new file mode 100644 index 00000000..c87aa01d --- /dev/null +++ b/apps/harmony/desktop-port/DesktopAccess.ets @@ -0,0 +1,133 @@ +import { abilityAccessCtrl, common, Permissions } from '@kit.AbilityKit'; +import { UIContext } from '@kit.ArkUI'; +import { DesktopEvent } from './DesktopHost'; + +// OS authorization only; session policy stays in the shared desktop runtime. +export class DesktopAccess { + private static context: common.UIAbilityContext; + private static enabled: boolean = false; + private static hostEnabled: boolean = false; + private static hostRequested: boolean = false; + private static running: boolean = false; + private static generation: number = 0; + private static requested: Set = new Set(); + private static requests: Promise = Promise.resolve(); + private static notify: (event: DesktopEvent) => void = () => {}; + private static manager: abilityAccessCtrl.AtManager = abilityAccessCtrl.createAtManager(); + private static permissions: Permissions[] = []; + private static changed: (info: abilityAccessCtrl.PermissionStateChangeInfo) => void = () => DesktopAccess.refresh(); + + static start(context: common.UIAbilityContext, enabled: boolean, notify: (event: DesktopEvent) => void, + hostEnabled: boolean = false): void { + if (DesktopAccess.running) DesktopAccess.stop(); + DesktopAccess.context = context; DesktopAccess.enabled = enabled; + DesktopAccess.hostEnabled = hostEnabled; + DesktopAccess.hostRequested = false; + DesktopAccess.running = true; DesktopAccess.notify = notify; + ++DesktopAccess.generation; DesktopAccess.requested.clear(); + DesktopAccess.permissions = []; + if (enabled) DesktopAccess.permissions.push('ohos.permission.HOOK_KEY_EVENT', 'ohos.permission.READ_PASTEBOARD'); + if (hostEnabled) DesktopAccess.permissions.push('ohos.permission.CUSTOM_SCREEN_RECORDING', 'ohos.permission.CONTROL_DEVICE'); + if (DesktopAccess.permissions.length > 0) { + try { DesktopAccess.manager.on('selfPermissionStateChange', DesktopAccess.permissions, DesktopAccess.changed); } + catch (error) { console.warn(`Slint permission observation: ${error.message}`); } + } + DesktopAccess.refresh(); + } + + static stop(): void { + if (!DesktopAccess.running) return; + DesktopAccess.notify({ op: 'hostPermissions', controlDevice: false, screenRecording: false }); + DesktopAccess.running = false; + ++DesktopAccess.generation; + if (DesktopAccess.permissions.length > 0) { + try { DesktopAccess.manager.off('selfPermissionStateChange', DesktopAccess.permissions, DesktopAccess.changed); } + catch (error) { console.warn(`Slint permission cleanup: ${error.message}`); } + } + } + + static granted(permission: Permissions): boolean { + try { + return DesktopAccess.manager.checkAccessTokenSync(DesktopAccess.context.applicationInfo.accessTokenId, permission) === + abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED; + } catch { return false; } + } + + static refresh(): void { + if (!DesktopAccess.running) return; + DesktopAccess.notify({ op: 'keyboardPermission', ok: DesktopAccess.granted('ohos.permission.HOOK_KEY_EVENT') }); + DesktopAccess.notify({ op: 'clipboardChanged' }); + DesktopAccess.notify({ op: 'hostPermissions', + controlDevice: DesktopAccess.hostEnabled && DesktopAccess.granted('ohos.permission.CONTROL_DEVICE'), + screenRecording: DesktopAccess.hostEnabled && DesktopAccess.granted('ohos.permission.CUSTOM_SCREEN_RECORDING') }); + } + + static requestKeyboard(): void { + if (DesktopAccess.enabled) DesktopAccess.request('ohos.permission.HOOK_KEY_EVENT', true); + } + + static requestClipboard(): void { + if (DesktopAccess.enabled) DesktopAccess.request('ohos.permission.READ_PASTEBOARD', false); + } + + static requestHost(ui: UIContext): void { + if (!DesktopAccess.running || !DesktopAccess.hostEnabled || DesktopAccess.hostRequested) return; + DesktopAccess.hostRequested = true; + const generation = DesktopAccess.generation; + DesktopAccess.requests = DesktopAccess.requests.then(async () => { + if (!DesktopAccess.running || generation !== DesktopAccess.generation) return; + const switches = DesktopAccess.missingHostSwitches(); + if (switches.length === 0) return; + const prompt = ui.getPromptAction(); + const result = await prompt.showDialog({ + title: '设置远程访问权限', + message: '请在接下来打开的应用设置页开启:\n\n' + switches.join('\n') + + '\n\n仅用于已通过连接密码验证的远程会话。你可以随时在该页面关闭权限。', + buttons: [{ text: '暂不开启', color: '#666666' }, { text: '前往设置', color: '#007DFF' }] + }); + if (result.index !== 1 || !DesktopAccess.running || generation !== DesktopAccess.generation || + DesktopAccess.missingHostSwitches().length === 0) return; + try { + // The system manual-permission dialog describes a different settings + // route on PC. Open this app's actual switches together instead. + await DesktopAccess.context.startAbility({ + bundleName: 'com.huawei.hmos.settings', + abilityName: 'com.huawei.hmos.settings.MainAbility', + uri: 'application_info_entry', + parameters: { pushParams: DesktopAccess.context.applicationInfo.name } + }); + } catch (error) { + if (!DesktopAccess.running || generation !== DesktopAccess.generation) return; + console.warn(`Slint host permission settings: ${error.message}`); + await prompt.showDialog({ + title: '未能打开应用设置', + message: '请在系统设置的应用列表中找到本应用,再开启:\n\n' + + DesktopAccess.missingHostSwitches().join('\n'), + buttons: [{ text: '知道了', color: '#007DFF' }] + }); + } + }).catch((error: Error) => console.warn(`Slint host permission guide: ${error.message}`)) + .then(() => { if (generation === DesktopAccess.generation) DesktopAccess.refresh(); }); + } + + private static missingHostSwitches(): string[] { + const switches: string[] = []; + if (!DesktopAccess.granted('ohos.permission.CUSTOM_SCREEN_RECORDING')) + switches.push('“屏幕共享”:允许远程设备查看本机屏幕。'); + if (!DesktopAccess.granted('ohos.permission.CONTROL_DEVICE')) + switches.push('“辅助操作”:允许远程设备使用本机键盘和鼠标。'); + return switches; + } + + private static request(permission: Permissions, manual: boolean): void { + if (!DesktopAccess.running || DesktopAccess.requested.has(permission) || DesktopAccess.granted(permission)) return; + DesktopAccess.requested.add(permission); + const generation = DesktopAccess.generation; + DesktopAccess.requests = DesktopAccess.requests.then(async () => { + if (!DesktopAccess.running || generation !== DesktopAccess.generation || DesktopAccess.granted(permission)) return; + if (manual) await DesktopAccess.manager.openPermissionOnSetting(DesktopAccess.context, permission); + else await DesktopAccess.manager.requestPermissionsFromUser(DesktopAccess.context, [permission]); + }).catch((error: Error) => console.warn(`Slint permission ${permission}: ${error.message}`)) + .then(() => { if (generation === DesktopAccess.generation) DesktopAccess.refresh(); }); + } +} diff --git a/apps/harmony/desktop-port/DesktopHost.ets b/apps/harmony/desktop-port/DesktopHost.ets new file mode 100644 index 00000000..81d5e24a --- /dev/null +++ b/apps/harmony/desktop-port/DesktopHost.ets @@ -0,0 +1,448 @@ +import native from 'libslint_desktop.so'; +import { TextInputHost } from './TextInputHost'; +import { TrayHost } from './TrayHost'; +import { FileHost } from './FileHost'; +import { DisplayHost, DesktopDisplay } from './DisplayHost'; +import { DesktopAccess } from './DesktopAccess'; +import { AutostartHost, AutostartEvent } from './AutostartHost'; +import { desktopPermissions, hostPermissions } from './DesktopCapabilities'; +import { pointer } from '@kit.InputKit'; +import { common, abilityAccessCtrl } from '@kit.AbilityKit'; +import { display, window } from '@kit.ArkUI'; +import { pasteboard } from '@kit.BasicServicesKit'; +import { certificateManager } from '@kit.DeviceCertificateKit'; + +export interface DesktopEvent { + op: string; + displays?: DesktopDisplay[]; + decor?: number; + noFrame?: boolean; + topmost?: boolean; + cursor?: number; + anchor?: number; + inputType?: number; + commit?: boolean; + id?: number; + request?: number; + ok?: boolean; + controlDevice?: boolean; + screenRecording?: boolean; + text?: string; + message?: string; + url?: string; + path?: string; + title?: string; + show?: string; + settings?: string; + exit?: string; + visible?: boolean; + width?: number; + height?: number; + minWidth?: number; + minHeight?: number; + maxWidth?: number; + maxHeight?: number; + x?: number; + y?: number; + dx?: number; + dy?: number; + fullscreen?: boolean; + maximized?: boolean; + minimized?: boolean; + surface?: string; + density?: number; + active?: boolean; + kind?: number; + button?: number; + code?: number; + down?: boolean; + control?: boolean; + shift?: boolean; + unicode?: number; + capsLock?: boolean; + numLock?: boolean; + imeSession?: number; + stateRevision?: number; +} + +// Window/OS service host only. The desktop C++ application owns every product +// action and keeps running when another window gains focus or is minimized. +export class DesktopHost { + private static stage: window.WindowStage; + private static context: common.UIAbilityContext; + private static windows: Map = new Map(); + private static queues: Map> = new Map(); + private static clipboardQueue: Promise = Promise.resolve(); + private static clipboardRevision: number = 0; + private static clipboardCacheRevision: number = -1; + private static clipboardCache: string = ''; + private static clipboardObserved: boolean = false; + private static clipboardChanged: () => void = () => { + if (!DesktopHost.started || DesktopHost.terminating) return; + ++DesktopHost.clipboardRevision; + DesktopHost.clipboardCache = ''; + DesktopHost.send({ op: 'clipboardChanged' }); + }; + private static properties: Map = new Map(); + private static timer: number = -1; + private static cursorSupported: boolean = true; + private static cursorHidden: boolean = false; + private static trayRemoval: Promise = Promise.resolve(); + private static started: boolean = false; + private static terminating: boolean = false; + private static activeWindow: number = 1; + private static mainShown: boolean = false; + private static stateRevisions: Map = new Map(); + + static send(event: DesktopEvent): void { native.send(JSON.stringify(event)); } + + static requestExit(): boolean { + if (!DesktopHost.started || DesktopHost.terminating) return false; + DesktopHost.send({ op: 'tray', text: 'exit' }); + return true; + } + + static restore(): void { + const target = DesktopHost.windows.get(1); + if (target === undefined) return; + DesktopHost.send({ op: 'restore', id: 1 }); + } + + static foreground(): void { + if (!DesktopHost.started || DesktopHost.terminating) return; + AutostartHost.refresh(); + } + + static start(stage: window.WindowStage, context: common.UIAbilityContext): void { + DesktopHost.stage = stage; DesktopHost.context = context; + const screen = display.getDefaultDisplaySync(); + const storage = new LocalStorage(); + storage.setOrCreate('slintWindowId', 1); + DesktopHost.windows.set(1, stage.getMainWindowSync()); + stage.loadContent('pages/Index', storage).then(async () => { + await FileHost.prepare(context); + const screens = await DisplayHost.snapshot(); + DesktopHost.observe(1, stage.getMainWindowSync()); + // Publish actual grants before native auto-connect can start capture. + DesktopAccess.start(context, desktopPermissions, (event: DesktopEvent) => { + if (event.op === 'clipboardChanged') DesktopHost.clipboardChanged(); + else DesktopHost.send(event); + }, hostPermissions); + native.start(JSON.stringify({ filesDir: context.filesDir, cacheDir: context.cacheDir, + width: screen.width, height: screen.height, density: screen.densityPixels, + displayId: screen.id, displays: screens, name: 'Harmony PC', + certificates: certificateManager.getCertificateStorePath({ certType: certificateManager.CertType.CA_CERT_SYSTEM }) })); + DesktopHost.started = true; + AutostartHost.start((event: AutostartEvent) => native.send(JSON.stringify(event))); + try { + pasteboard.getSystemPasteboard().on('update', DesktopHost.clipboardChanged); + DesktopHost.clipboardObserved = true; + } catch (error) { console.warn(`Slint clipboard observation: ${error.message}`); } + DisplayHost.observe((screens: DesktopDisplay[]) => { + DesktopHost.send({ op: 'displays', displays: screens }); + DesktopHost.windows.forEach((target: window.Window, id: number) => DesktopHost.metrics(id, target)); + }); + DesktopHost.timer = setInterval(() => DesktopHost.poll(), 8); + }).catch((error: Error) => console.error(`Slint startup: ${error.message}`)); + } + + private static observe(id: number, target: window.Window): void { + const current = (): boolean => DesktopHost.windows.get(id) === target && !DesktopHost.terminating; + const watch = (subscribe: () => void): void => { + try { subscribe(); } + catch (error) { console.warn(`Slint window observation: ${error.message}`); } + }; + watch(() => target.on('windowEvent', (event: window.WindowEventType) => { + if (!current()) return; + if (event === window.WindowEventType.WINDOW_ACTIVE) { + DesktopHost.activeWindow = id; + DesktopAccess.refresh(); + DesktopHost.foreground(); + } + if (event === window.WindowEventType.WINDOW_ACTIVE || event === window.WindowEventType.WINDOW_INACTIVE) + DesktopHost.send({ op: 'active', id, active: event === window.WindowEventType.WINDOW_ACTIVE }); + })); + watch(() => target.on('windowRectChange', () => { if (current()) DesktopHost.metrics(id, target); })); + watch(() => target.on('displayIdChange', () => { if (current()) DesktopHost.metrics(id, target); })); + DesktopHost.metrics(id, target); + watch(() => target.on('windowStatusDidChange', (status: window.WindowStatusType) => { + if (!current()) return; + const revision = (DesktopHost.stateRevisions.get(id) ?? 0) + 1; + DesktopHost.stateRevisions.set(id, revision); + // Minimization does not change the underlying fullscreen/maximized mode. + const state: DesktopEvent = status === window.WindowStatusType.MINIMIZE ? + { op: 'state', id, stateRevision: revision, minimized: true } : + { op: 'state', id, stateRevision: revision, fullscreen: status === window.WindowStatusType.FULL_SCREEN, + maximized: status === window.WindowStatusType.MAXIMIZE, minimized: false }; + const saved = DesktopHost.properties.get(id) ?? { op: 'properties', id }; + saved.fullscreen = state.fullscreen ?? saved.fullscreen; + saved.maximized = state.maximized ?? saved.maximized; + saved.minimized = state.minimized; + saved.stateRevision = revision; + DesktopHost.properties.set(id, saved); + DesktopHost.send(state); + })); + watch(() => target.on('windowWillClose', async () => { + if (!current()) return false; + DesktopHost.send({ op: 'close', id }); + return true; // The shared desktop callback decides hide/disconnect/exit. + })); + } + + private static forget(id: number, target: window.Window): void { + DesktopHost.windows.delete(id); + TextInputHost.release(id); + // Some SDKs do not expose all listeners. Cleanup must still reach destroy. + try { target.off('windowEvent'); } catch (_) {} + try { target.off('windowRectChange'); } catch (_) {} + try { target.off('displayIdChange'); } catch (_) {} + try { target.off('windowStatusDidChange'); } catch (_) {} + try { target.off('windowWillClose'); } catch (_) {} + } + + private static metrics(id: number, target: window.Window): void { + try { + const point = DisplayHost.global(target); + DesktopHost.send({ op: 'metrics', id, x: point.x, y: point.y, + density: DisplayHost.screen(target).densityPixels, decor: DesktopHost.decorHeight(target) }); + } catch (error) { console.warn(`Slint window metrics: ${error.message}`); } + } + + private static poll(): void { + const events: DesktopEvent[] = JSON.parse(native.poll()) as DesktopEvent[]; + events.forEach((event: DesktopEvent) => { + if (event.request !== undefined) { + const service = (): Promise => DesktopHost.service(event).catch((error: Error) => + DesktopHost.send({ op: 'reply', request: event.request, ok: false, message: error.message })); + // Slint edits and the shared clipboard monitor run on different native + // threads. The OS permits only one copy/paste operation at a time. + if (event.op.startsWith('clipboard')) DesktopHost.clipboardQueue = DesktopHost.clipboardQueue.then(service); + else service(); + } else if (event.id !== undefined) { + const id = event.id; + const prior = DesktopHost.queues.get(id) ?? Promise.resolve(); + const next = prior.then(() => DesktopHost.windowCommand(event)).catch((error: Error) => console.error(`Slint window ${id}: ${error.message}`)); + DesktopHost.queues.set(id, next); + next.then(() => { if (DesktopHost.queues.get(id) === next) DesktopHost.queues.delete(id); }); + } else if (event.op === 'exit') { + DesktopHost.terminating = true; + DesktopAccess.stop(); + AutostartHost.stop(); + if (DesktopHost.clipboardObserved) { + try { pasteboard.getSystemPasteboard().off('update', DesktopHost.clipboardChanged); } + catch (error) { console.warn(`Slint clipboard cleanup: ${error.message}`); } + DesktopHost.clipboardObserved = false; + } + clearInterval(DesktopHost.timer); + DesktopHost.trayRemoval.finally(() => DesktopHost.context.terminateSelf()); + } else if (event.op === 'keyboardPermission') { + DesktopAccess.requestKeyboard(); + } else if (event.op === 'autostartRefresh') { + AutostartHost.refresh(); + } else if (event.op === 'trayRemove') { + DesktopHost.trayRemoval = TrayHost.remove(DesktopHost.context) + .catch((error: Error) => console.warn(`Slint tray removal: ${error.message}`)); + } else if (event.op === 'error') { + console.error(`Slint desktop: ${event.message}`); + } + }); + } + + private static async windowCommand(event: DesktopEvent): Promise { + const id = event.id ?? 0; + if (event.op === 'properties') { + // Ignore mode commands based on a state older than the latest native + // transition. Keep independent title/size/stacking updates applicable. + if ((event.stateRevision ?? 0) < (DesktopHost.stateRevisions.get(id) ?? 0)) { + event.fullscreen = undefined; event.maximized = undefined; event.minimized = undefined; + } + const prior = DesktopHost.properties.get(id); + DesktopHost.properties.set(id, { op: 'properties', id, title: event.title, noFrame: event.noFrame, topmost: event.topmost, + minWidth: event.minWidth, minHeight: event.minHeight, maxWidth: event.maxWidth, maxHeight: event.maxHeight, + fullscreen: event.fullscreen ?? prior?.fullscreen, + maximized: event.maximized ?? prior?.maximized, minimized: event.minimized ?? prior?.minimized, + stateRevision: DesktopHost.stateRevisions.get(id) ?? 0 }); + } + if (event.op === 'destroy') { + DesktopHost.properties.delete(id); + DesktopHost.stateRevisions.delete(id); + } + let target = DesktopHost.windows.get(id); + if (event.op === 'visible' && event.visible && target === undefined) { + target = await DesktopHost.stage.getMainWindowSync().createSubWindowWithOptions(`crossdesk-slint-${id}`, + { title: DesktopHost.properties.get(id)?.title ?? 'CrossDesk', decorEnabled: !DesktopHost.properties.get(id)?.noFrame, isModal: false, + zLevelAboveParentLoosened: !DesktopHost.properties.get(id)?.topmost, maximizeSupported: true }); + DesktopHost.windows.set(id, target); + try { + const storage = new LocalStorage(); + storage.setOrCreate('slintWindowId', id); + await target.loadContent('pages/Index', storage); + await DesktopHost.resizeContent(target, event.width ?? 1216, event.height ?? 903); + if (event.x !== undefined && event.y !== undefined) await DisplayHost.move(target, event.x, event.y); + const saved = DesktopHost.properties.get(id); + if (saved !== undefined) await DesktopHost.applyProperties(target, saved); + // Publish geometry only after restoring it; the default placement must + // not overwrite the shared desktop's saved position during recreation. + DesktopHost.observe(id, target); + } catch (error) { + DesktopHost.forget(id, target); + try { await target.destroyWindow(); } catch (_) {} + throw new Error(error.message); + } + } + if (target === undefined) return; + if (event.op === 'ime') { TextInputHost.request(event, target, (reply: DesktopEvent) => DesktopHost.send(reply)); return; } + if (event.op === 'cursor') { + if (!DesktopHost.cursorSupported) return; + // Slint 1.17.1 MouseCursor order is pinned by os-services.patch. + const styles: pointer.PointerStyle[] = [pointer.PointerStyle.DEFAULT, pointer.PointerStyle.DEFAULT, + pointer.PointerStyle.HELP, pointer.PointerStyle.HAND_POINTING, pointer.PointerStyle.RUNNING, + pointer.PointerStyle.LOADING, pointer.PointerStyle.CROSS, pointer.PointerStyle.TEXT_CURSOR, + pointer.PointerStyle.CURSOR_COPY, pointer.PointerStyle.CURSOR_COPY, pointer.PointerStyle.MOVE, + pointer.PointerStyle.CURSOR_FORBID, pointer.PointerStyle.CURSOR_FORBID, pointer.PointerStyle.HAND_OPEN, + pointer.PointerStyle.HAND_GRABBING, pointer.PointerStyle.WEST_EAST, pointer.PointerStyle.NORTH_SOUTH, + pointer.PointerStyle.NORTH, pointer.PointerStyle.EAST, pointer.PointerStyle.SOUTH, pointer.PointerStyle.WEST, + pointer.PointerStyle.NORTH_EAST, pointer.PointerStyle.NORTH_WEST, pointer.PointerStyle.SOUTH_EAST, + pointer.PointerStyle.SOUTH_WEST, pointer.PointerStyle.WEST_EAST, pointer.PointerStyle.NORTH_SOUTH, + pointer.PointerStyle.NORTH_EAST_SOUTH_WEST, pointer.PointerStyle.NORTH_WEST_SOUTH_EAST]; + try { + await pointer.setPointerStyle(target.getWindowProperties().id, styles[event.kind ?? 0] ?? pointer.PointerStyle.DEFAULT); + if (DesktopHost.cursorHidden !== (event.kind === 1)) { + await pointer.setPointerVisible(event.kind !== 1); + DesktopHost.cursorHidden = event.kind === 1; + } + } catch (error) { + DesktopHost.cursorSupported = false; + console.warn(`Slint cursor service unavailable: ${error.message}`); + } + return; + } + if (event.op === 'visible') { + if (event.visible) { + if (id === 1 && !DesktopHost.mainShown) { + const status = target.getWindowStatus(); + if (status !== window.WindowStatusType.MAXIMIZE && status !== window.WindowStatusType.FULL_SCREEN) + await DesktopHost.resizeContent(target, event.width ?? 1216, event.height ?? 903); + DesktopHost.mainShown = true; + } + await DesktopHost.show(id, target); + if (id === 1) DesktopAccess.requestHost(target.getUIContext()); + } else if (id === 1) await target.minimize(); + else { DesktopHost.forget(id, target); await target.destroyWindow(); } + } else if (event.op === 'resize') { + await DesktopHost.resizeContent(target, event.width ?? 1216, event.height ?? 903); + } else if (event.op === 'move') { + await DisplayHost.move(target, event.x ?? 0, event.y ?? 0); + } else if (event.op === 'properties') await DesktopHost.applyProperties(target, event); + else if (event.op === 'destroy' && id !== 1) { + DesktopHost.forget(id, target); DesktopHost.properties.delete(id); + await target.destroyWindow(); + } + } + + private static decorHeight(target: window.Window): number { + try { return target.getWindowDecorVisible() ? Math.round(target.getWindowDecorHeight() * DisplayHost.screen(target).densityPixels) : 0; } + catch { return 0; } + } + + private static async resizeContent(target: window.Window, width: number, height: number): Promise { + await target.resize(Math.max(1, Math.round(width)), Math.max(1, Math.round(height) + DesktopHost.decorHeight(target))); + } + + private static async show(id: number, target: window.Window): Promise { + // restore() restores a minimized main window on 2in1; showWindow() only + // restores child windows. Avoid resizing an already-visible/maximized one. + if (id === 1 && target.getWindowStatus() === window.WindowStatusType.MINIMIZE) await target.restore(); + else await target.showWindow(); + } + + private static async applyProperties(target: window.Window, event: DesktopEvent): Promise { + if (event.noFrame !== undefined && target.getWindowDecorVisible() === event.noFrame) + target.setWindowDecorVisible(!event.noFrame); + // A decor-free subwindow has no native title component. Harmony rejects + // setWindowTitle for it, which must not prevent the Slint status bar from + // receiving its size/stacking properties or becoming visible. + if (event.title !== undefined && target.getWindowDecorVisible()) { + try { await target.setWindowTitle(event.title); } + catch (error) { console.warn(`Slint window title: ${error.message}`); } + } + if (event.id !== 1 && event.topmost !== undefined) { + try { await target.setSubWindowZLevel(event.topmost ? 10000 : 0); } + catch (error) { console.warn(`Slint window stacking: ${error.message}`); } + } + const decor = DesktopHost.decorHeight(target); + const limits: window.WindowLimits = { minWidth: Math.max(1, Math.round(event.minWidth ?? 1)), + minHeight: Math.max(1, Math.round(event.minHeight ?? 1) + decor) }; + if (event.maxWidth !== undefined) limits.maxWidth = Math.max(limits.minWidth ?? 1, Math.round(event.maxWidth)); + if (event.maxHeight !== undefined) limits.maxHeight = Math.max(limits.minHeight ?? 1, Math.round(event.maxHeight) + decor); + await target.setWindowLimits(limits); + if (event.fullscreen === undefined && event.maximized === undefined && event.minimized === undefined) return; + if ((event.stateRevision ?? 0) < (DesktopHost.stateRevisions.get(event.id ?? 0) ?? 0)) return; + const saved = DesktopHost.properties.get(event.id ?? 0); + const minimized = event.minimized ?? saved?.minimized ?? false; + const fullscreen = event.fullscreen ?? saved?.fullscreen ?? false; + const maximized = event.maximized ?? saved?.maximized ?? false; + let status = target.getWindowStatus(); + if (minimized) { + if (status !== window.WindowStatusType.MINIMIZE) await target.minimize(); + return; + } + if (status === window.WindowStatusType.MINIMIZE) { + await DesktopHost.show(event.id ?? 0, target); + status = target.getWindowStatus(); + } + if (fullscreen) { + if (status !== window.WindowStatusType.FULL_SCREEN) await target.maximize(window.MaximizePresentation.ENTER_IMMERSIVE); + } else if (maximized) { + if (status !== window.WindowStatusType.MAXIMIZE) await target.maximize(window.MaximizePresentation.EXIT_IMMERSIVE); + } else if (status === window.WindowStatusType.MAXIMIZE || status === window.WindowStatusType.FULL_SCREEN) await target.recover(); + } + + private static async service(event: DesktopEvent): Promise { + const response: DesktopEvent = { op: 'reply', request: event.request, ok: true }; + const board = pasteboard.getSystemPasteboard(); + // A Ctrl+V gesture grants one read. Background synchronization must never + // consume that grant before the editor does. It requires READ_PASTEBOARD; + // the explicit paste uses the system's normal gesture authorization. + if ((event.op === 'clipboardRead' || event.op === 'clipboardHas') && + abilityAccessCtrl.createAtManager().checkAccessTokenSync(DesktopHost.context.applicationInfo.accessTokenId, + 'ohos.permission.READ_PASTEBOARD') !== abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED) { + response.ok = false; + DesktopHost.clipboardCache = ''; + DesktopHost.clipboardCacheRevision = -1; + DesktopAccess.requestClipboard(); + response.message = 'Clipboard synchronization requires READ_PASTEBOARD permission'; + } else if (event.op === 'clipboardRead' || event.op === 'clipboardPaste') { + const revision = DesktopHost.clipboardRevision; + const explicitPaste = event.op === 'clipboardPaste'; + if (explicitPaste || !DesktopHost.clipboardObserved || DesktopHost.clipboardCacheRevision !== revision) { + const text = board.hasDataType(pasteboard.MIMETYPE_TEXT_PLAIN) ? (await board.getData()).getPrimaryText() : ''; + if (!explicitPaste && abilityAccessCtrl.createAtManager().checkAccessTokenSync( + DesktopHost.context.applicationInfo.accessTokenId, 'ohos.permission.READ_PASTEBOARD') !== + abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED) { + DesktopHost.clipboardCache = ''; DesktopHost.clipboardCacheRevision = -1; + DesktopHost.send({ op: 'reply', request: event.request, ok: false, message: 'Clipboard permission revoked' }); + return; + } + response.text = text; + // A change during getData must be read again. Explicit paste never + // populates the background cache from a one-use gesture grant. + if (!explicitPaste && revision === DesktopHost.clipboardRevision) { + DesktopHost.clipboardCache = text; DesktopHost.clipboardCacheRevision = revision; + } + } else response.text = DesktopHost.clipboardCache; + } else if (event.op === 'clipboardHas') response.ok = board.hasDataType(pasteboard.MIMETYPE_TEXT_PLAIN); + else if (event.op === 'clipboardWrite') { + await board.setData(pasteboard.createData(pasteboard.MIMETYPE_TEXT_PLAIN, event.text ?? '')); + DesktopHost.clipboardChanged(); + } + else if (event.op === 'openUrl') await DesktopHost.context.openLink(event.url ?? ''); + else if (event.op === 'trayCreate') response.ok = await TrayHost.create(DesktopHost.context, event, + (reply: DesktopEvent) => DesktopHost.send(reply)); + else if (event.op === 'selectFile' || event.op === 'selectFolder') { + const owner = DesktopHost.windows.get(DesktopHost.activeWindow) ?? DesktopHost.stage.getMainWindowSync(); + response.path = await FileHost.select(DesktopHost.context, event.op === 'selectFolder', owner); + } else response.ok = false; + DesktopHost.send(response); + } +} diff --git a/apps/harmony/desktop-port/DisplayHost.ets b/apps/harmony/desktop-port/DisplayHost.ets new file mode 100644 index 00000000..5df4b55e --- /dev/null +++ b/apps/harmony/desktop-port/DisplayHost.ets @@ -0,0 +1,83 @@ +import { display, window } from '@kit.ArkUI'; + +export interface DesktopDisplay { + id: number; + name: string; + primary: boolean; + x: number; + y: number; + width: number; + height: number; + density: number; + usableX: number; + usableY: number; + usableWidth: number; + usableHeight: number; +} + +export class DisplayHost { + private static revision: number = 0; + private static observed: Map = new Map(); + + static screen(target: window.Window): display.Display { + return display.getDisplayByIdSync(target.getWindowProperties().displayId ?? display.getDefaultDisplaySync().id); + } + + static global(target: window.Window): display.Position { + const properties = target.getWindowProperties(); + return display.convertRelativeToGlobalCoordinate({ + displayId: properties.displayId ?? display.getDefaultDisplaySync().id, + position: { x: properties.windowRect.left, y: properties.windowRect.top } + }); + } + + static async move(target: window.Window, x: number, y: number): Promise { + const point = display.convertGlobalToRelativeCoordinate({ x: Math.round(x), y: Math.round(y) }); + await target.moveWindowToAsync(point.position.x, point.position.y, { displayId: point.displayId }); + } + + static async snapshot(): Promise { + const primary = display.getDefaultDisplaySync().id; + const screens = await display.getAllDisplays(); + const result: DesktopDisplay[] = []; + for (const screen of screens) { + const origin = display.convertRelativeToGlobalCoordinate({ displayId: screen.id, position: { x: 0, y: 0 } }); + let area: display.Rect = { left: 0, top: 0, width: screen.width, height: screen.height }; + try { area = await screen.getAvailableArea(); } + catch (error) { console.warn(`Slint available display area: ${error.message}`); } + result.push({ id: screen.id, name: screen.name, primary: screen.id === primary, + x: origin.x, y: origin.y, width: screen.width, height: screen.height, density: screen.densityPixels, + usableX: origin.x + area.left, usableY: origin.y + area.top, + usableWidth: area.width, usableHeight: area.height }); + } + return result; + } + + static observe(changed: (screens: DesktopDisplay[]) => void): void { + const refresh = async (): Promise => { + const revision = ++DisplayHost.revision; + try { + const screens = await DisplayHost.snapshot(); + if (revision !== DisplayHost.revision) return; + DisplayHost.observed.forEach((screen: display.Display, id: number) => { + if (!screens.some((item: DesktopDisplay) => item.id === id)) { + try { screen.off('availableAreaChange'); } catch (_) {} + DisplayHost.observed.delete(id); + } + }); + screens.forEach((item: DesktopDisplay) => { + if (DisplayHost.observed.has(item.id)) return; + const screen = display.getDisplayByIdSync(item.id); + try { screen.on('availableAreaChange', () => { refresh(); }); } + catch (error) { console.warn(`Slint display area events: ${error.message}`); } + DisplayHost.observed.set(item.id, screen); + }); + changed(screens); + } catch (error) { console.warn(`Slint display refresh: ${error.message}`); } + }; + display.on('add', () => { refresh(); }); + display.on('remove', () => { refresh(); }); + display.on('change', () => { refresh(); }); + refresh(); + } +} diff --git a/apps/harmony/desktop-port/EntryAbility.ets b/apps/harmony/desktop-port/EntryAbility.ets new file mode 100644 index 00000000..15bf2670 --- /dev/null +++ b/apps/harmony/desktop-port/EntryAbility.ets @@ -0,0 +1,12 @@ +import { UIAbility } from '@kit.AbilityKit'; +import { window } from '@kit.ArkUI'; +import { DesktopHost } from '../pages/DesktopHost'; + +export default class EntryAbility extends UIAbility { + onPrepareToTerminate(): boolean { return DesktopHost.requestExit(); } + onNewWant(): void { DesktopHost.restore(); } + onForeground(): void { DesktopHost.foreground(); } + onWindowStageCreate(stage: window.WindowStage): void { + DesktopHost.start(stage, this.context); + } +} diff --git a/apps/harmony/desktop-port/FileHost.ets b/apps/harmony/desktop-port/FileHost.ets new file mode 100644 index 00000000..3e45757c --- /dev/null +++ b/apps/harmony/desktop-port/FileHost.ets @@ -0,0 +1,51 @@ +import { common } from '@kit.AbilityKit'; +import { picker, fileIo, fileUri, fileShare } from '@kit.CoreFileKit'; +import { preferences } from '@kit.ArkData'; +import { window } from '@kit.ArkUI'; + +// Only URI grants and native file access live here. Transfer/progress/cancel +// remain in the shared desktop FileTransferController and FileReceiver. +export class FileHost { + private static grants: preferences.Preferences; + private static sequence: number = 0; + static async prepare(context: common.UIAbilityContext): Promise { + FileHost.grants = await preferences.getPreferences(context, { name: 'slint-file-grants' }); + const folders = await FileHost.grants.get('folders', []) as string[]; + for (const uri of folders) { + try { await fileShare.activatePermission([FileHost.policy(uri)]); } + catch (error) { console.warn(`Slint folder permission is no longer available: ${error.code}`); } + } + // These are snapshots from previous processes, never live transfers. + const staging = `${context.cacheDir}/slint-outgoing`; + if (fileIo.accessSync(staging)) await fileIo.rmdir(staging); + await fileIo.mkdir(staging); + } + private static policy(uri: string): fileShare.PolicyInfo { + return { uri, operationMode: fileShare.OperationMode.READ_MODE | fileShare.OperationMode.WRITE_MODE }; + } + static async select(context: common.UIAbilityContext, folder: boolean, owner: window.Window): Promise { + const selected = await new picker.DocumentViewPicker(context, owner).select({ maxSelectNumber: 1, + selectMode: folder ? picker.DocumentSelectMode.FOLDER : picker.DocumentSelectMode.FILE }); + if (selected.length === 0) return ''; + const uri = new fileUri.FileUri(selected[0]); + if (folder) { + const policy = FileHost.policy(selected[0]); + await fileShare.persistPermission([policy]); + await fileShare.activatePermission([policy]); + const folders = await FileHost.grants.get('folders', []) as string[]; + if (!folders.includes(selected[0])) folders.push(selected[0]); + await FileHost.grants.put('folders', folders); + await FileHost.grants.flush(); + return uri.path; + } + if (!uri.name || uri.name === '.' || uri.name === '..' || uri.name.includes('/')) + throw new Error('Invalid selected filename'); + const directory = `${context.cacheDir}/slint-outgoing/${Date.now()}-${++FileHost.sequence}`; + await fileIo.mkdir(directory); + const path = `${directory}/${uri.name}`; + const source = await fileIo.open(selected[0], fileIo.OpenMode.READ_ONLY); + try { await fileIo.copyFile(source.fd, path); } + finally { await fileIo.close(source); } + return path; + } +} diff --git a/apps/harmony/desktop-port/Index.ets b/apps/harmony/desktop-port/Index.ets new file mode 100644 index 00000000..157913c6 --- /dev/null +++ b/apps/harmony/desktop-port/Index.ets @@ -0,0 +1,76 @@ +import { DesktopEvent, DesktopHost } from './DesktopHost'; +import { KeyboardHost } from './KeyboardHost'; + + +// This host owns only the native surface and event delivery. All visible +// application controls are compiled from apps/desktop's existing Slint files. +@Entry({ useSharedStorage: true }) +@Component +struct Index { + private surface: XComponentController = new XComponentController(); + private keyboard: KeyboardHost = new KeyboardHost(); + private surfaceId: string = ''; + + private send(event: DesktopEvent): void { + event.surface = this.surfaceId; + DesktopHost.send(event); + } + @LocalStorageProp('slintWindowId') private windowId: number = 1; + + aboutToDisappear(): void { + this.keyboard.reset(); + if (this.surfaceId.length === 0) return; + this.send({ op: 'detach', id: this.windowId }); + this.surfaceId = ''; + } + + build() { + Stack() { + XComponent({ id: 'slint-desktop', type: XComponentType.SURFACE, controller: this.surface }) + .width('100%').height('100%') + .onSizeChange((_old: SizeOptions, size: SizeOptions) => { + this.surface.setXComponentSurfaceSize({ surfaceWidth: Math.round(vp2px(Number(size.width))), + surfaceHeight: Math.round(vp2px(Number(size.height))) }); + this.send({ op: 'redraw', id: this.windowId }); + }) + .onLoad(() => { + this.surfaceId = this.surface.getXComponentSurfaceId(); + this.send({ op: 'attach', id: this.windowId, surface: this.surface.getXComponentSurfaceId(), + density: this.getUIContext().vp2px(1) }); + }) + .onDestroy(() => this.aboutToDisappear()) + // A focusable leaf is required: an empty Column is a focus scope and + // receives touch events but cannot retain hardware keyboard focus. + Text('').id('slintInput').width('100%').height('100%').focusable(true).defaultFocus(true).focusOnTouch(true) + .onBlur(() => this.keyboard.reset()) + .onTouch((event: TouchEvent) => { + if (event.type === TouchType.Down) this.getUIContext().getFocusController().requestFocus('slintInput'); + if (event.source === SourceType.Mouse) return; + if (event.changedTouches.length === 0) return; + const p = event.changedTouches[0]; + const kind = event.type === TouchType.Down ? 0 : event.type === TouchType.Up ? 1 : + event.type === TouchType.Cancel ? 3 : 2; + this.send({ op: 'pointer', id: this.windowId, kind, x: p.x, y: p.y, button: 0 }); + }) + .onMouse((event: MouseEvent) => { + if (event.action === MouseAction.Press) this.getUIContext().getFocusController().requestFocus('slintInput'); + const kind = event.action === MouseAction.Press ? 0 : event.action === MouseAction.Release ? 1 : 2; + this.send({ op: 'pointer', id: this.windowId, kind, x: event.x, y: event.y, + button: event.button === MouseButton.Right ? 1 : event.button === MouseButton.Middle ? 2 : 0 }); + event.stopPropagation(); + }) + .onHover((hovered: boolean) => { + if (!hovered) this.send({ op: 'pointerExit', id: this.windowId }); + }) + .onKeyPreIme((event: KeyEvent) => this.keyboard.preIme(event, this.windowId, (reply: DesktopEvent) => this.send(reply))) + .onKeyEvent((event: KeyEvent) => { + this.send(KeyboardHost.event(event, this.windowId)); + event.stopPropagation(); + }) + .onAxisEvent((event: AxisEvent) => { + this.send({ op: 'scroll', id: this.windowId, x: event.x, y: event.y, + dx: -event.getHorizontalAxisValue(), dy: -event.getVerticalAxisValue() }); + }) + }.width('100%').height('100%').backgroundColor(Color.White) + } +} diff --git a/apps/harmony/desktop-port/KeyboardHost.ets b/apps/harmony/desktop-port/KeyboardHost.ets new file mode 100644 index 00000000..b2d94b87 --- /dev/null +++ b/apps/harmony/desktop-port/KeyboardHost.ets @@ -0,0 +1,40 @@ +import type { DesktopEvent } from './DesktopHost'; + +export class KeyboardHost { + private captured: Set = new Set(); + private modifiers: Set = new Set(); + reset(): void { this.captured.clear(); this.modifiers.clear(); } + + static event(event: KeyEvent, id: number): DesktopEvent { + return { op: 'key', id, code: event.keyCode, down: event.type === KeyType.Down, + unicode: event.unicode, capsLock: event.isCapsLockOn, numLock: event.isNumLockOn }; + } + + preIme(event: KeyEvent, id: number, send: (event: DesktopEvent) => void): boolean { + const code = event.keyCode; + const modifier = code === 2072 || code === 2073 || code === 2045 || code === 2046 || + code === 2047 || code === 2048 || code === 2076 || code === 2077; + if (modifier) { + if (event.type === KeyType.Down) this.modifiers.add(code); + else this.modifiers.delete(code); + } + // ArkUI's query accepts only Ctrl/Alt/Shift, not Meta. Track Meta through + // its actual key transitions instead of issuing an invalid SDK query. + const control = (event.getModifierKeyState?.(['Ctrl']) ?? false) || + this.modifiers.has(2072) || this.modifiers.has(2073); + const shortcutModifier = code === 2072 || code === 2073 || code === 2076 || code === 2077; + const shortcut = shortcutModifier || (!modifier && code !== 2050 && + (control || event.metaKey === 1 || this.modifiers.has(2076) || this.modifiers.has(2077))); + // IMEs may turn Ctrl+A into insertText("a") instead of an edit command. + // Slint must receive shortcuts as keys before the IME consumes them. Keep + // the matching release even when the modifier is released first. Space + // remains available for the input method's own toggle. Its native event + // still carries the OS modifier state. Forward Ctrl/Meta transitions too: + // some IMEs otherwise consume the down event but pass the key-up along. + if (!shortcut && !this.captured.has(code)) return false; + if (event.type === KeyType.Down) this.captured.add(code); + else this.captured.delete(code); + send(KeyboardHost.event(event, id)); + return true; + } +} diff --git a/apps/harmony/desktop-port/README.md b/apps/harmony/desktop-port/README.md new file mode 100644 index 00000000..8ac7bffb --- /dev/null +++ b/apps/harmony/desktop-port/README.md @@ -0,0 +1,251 @@ +# HarmonyOS Slint 桌面移植 + +目标是与 Windows、macOS、Linux 使用同一套界面和产品行为。此目录直接编译 `apps/desktop/src/gui/application/gui_application.cpp`、`gui/runtime`、`gui/features` 和原始 `crossdesk_ui.slint`。鸿蒙 PC 只维护这一套入口,旧 ArkUI PC 会话和 Slint 验证程序已移除。 + +**当前是可运行的移植开发版,尚未达到全部 PC 行为一致的验收标准。** PC 与移动端共用应用信息,包名统一为 `com.chunkdash.crossdesk`,桌面名称统一为 **CrossDesk**。移动端使用独立 ArkUI 工程;旧 PC 验证包 `cn.crossdesk.slintdesktop` 的数据不会自动迁移到统一包名的新应用。 + +## 一致性约束 + +- 界面、布局、菜单、密码确认、设置保存/取消、历史记录、分组、桌面墙、会话标签、文件传输状态机、剪贴板同步和网络连接全部来自桌面公共代码。业务改动应在公共代码修复,不能在 ArkTS 再写一套行为。 +- ArkTS 只负责系统窗口、XComponent、键鼠事件、输入法、剪贴板、文件选择器、目录授权、系统托盘和打开链接。 +- C++ 平台层实现窗口/渲染、音频、屏幕采集及输入注入。私有 `include/SDL3` 只适配公共代码实际使用的 SDL 服务,不是完整的 SDL 移植,不能用于其他目标。 +- 屏幕、窗口和输入法坐标使用系统的实际显示器、工作区和各窗口缩放比例。Slint 的无边框属性直接映射到原生窗口,置顶属性目前只保证应用内部子窗口层级。 +- 所有 Slint 对象、事件、定时器和渲染在同一 GUI 线程运行。系统服务通过 ArkTS 线程处理;退出会取消等待,切换窗口保持已有会话。 +- Harmony 也定义 `__linux__`。共享代码仅在使用 X11/Wayland、Linux 原生窗口等位置排除 `__OHOS__`;POSIX 和 TLS 代码继续复用。 + +## 本机构建和运行 + +当前桌面适配使用 API 26 的[独立子窗口](https://developer.huawei.com/consumer/cn/doc/HarmonyOS-Guides/subwindow-guide)行为,要求 HarmonyOS API 26;移动端仍兼容 API 20。当前产物为 arm64,已在本机 2in1 模拟器验证。 + +```bash +# 构建 MiniRTC、Slint runtime、公共桌面程序和 HAP,安装到正在运行的模拟器 +python3 apps/harmony/scripts/build_slint_desktop.py --desktop --install + +# 后续增量构建;runtime 补丁变更仍会自动重建 +python3 apps/harmony/scripts/build_slint_desktop.py --desktop --reuse-runtime --install + +# 原生采集、输入 / wire 协议 / Slint 平台适配回归 +python3 apps/harmony/tests/run.py --desktop +``` + +`--device` 可指定 HDC 设备。`--version` 设置与其他桌面构建相同的版本号,默认 `0.0.0`,因此开发包会出现现有的新版提示。未提供鸿蒙发布文件时,更新入口打开官网,不能下载 Linux 安装包。 + +需要本机 Slint compiler 1.17.1、Rust OHOS 标准库和 DevEco SDK。工具链定位及首次下载要求见 [鸿蒙构建说明](../README.md),签名选项见下文。Slint 和 httplib 源码均固定版本及 SHA-256。生成的工程位于 `.native/slint-poc/desktop-project`,可用 DevEco Studio 打开。HAP 位于其 `entry/build/default/outputs/default/entry-default-unsigned.hap`;真机安装仍需自己的调试签名。 + +## 本次验证(2026-10-15) + +工程精简后重新通过 `tests/run.py --desktop --mobile`、默认 PC HAP、全部可选权限声明的 PC HAP 以及移动端完整构建。包检查确认 PC 仅支持 `2in1`、移动端仅支持 `phone/tablet`,两端均只包含各自所需的三个原生库;PC 不含旧验证库或移动视频渲染器。以下设备交互记录来自此前验证,本次精简未重复真机或模拟器端到端会话。 + +| 范围 | 证据 / 状态 | +| --- | --- | +| 共享桌面程序 | 完整 C++ 源码交叉编译、无未解析符号;HAP 的 arm64 ELF、SONAME 和动态依赖检查通过 | +| 启动和登录 | 本机 2in1 模拟器启动;系统 CA 校验下连接信令服务器;重启恢复设备身份和近期连接 | +| 真正远程会话 | 与测试 MiniRTC 被控端建立连接;共享密码对话框、会话窗口、工具栏和动态 NV12 测试画面正常;远端实际收到 control / keyboard / mouse 数据流,关闭会话后收到断开事件 | +| 中文输入 | 连接别名文本框全选替换;鸿蒙系统拼音候选 `nihao` 选择“你好”并提交到原 Slint 文本框 | +| 键盘 | 补齐扫描码、扩展位、左右修饰键及 F13–F24;复用公共按键状态对账,按下时先修饰键、释放时先普通键。左右 Ctrl、双 Shift、乱序快照、重复按键、释放失败重试和失焦清理的主机测试通过 | +| 被控输入实测 | Mac MiniRTC 测试主控经网络发送真实输入:拖动鸿蒙窗口、Ctrl+A 替换文本、右 Shift+方向键选择后删除、右 Ctrl、双 Ctrl 先放开左侧仍保持组合键均通过;按住右 Ctrl 时断开,之后本地输入正常,无修饰键滞留 | +| 剪贴板快捷键 | 最终 HAP 中经远程 Ctrl+A/C、End、Ctrl+V、Ctrl+A/X、Ctrl+V,`987` 完整复制为 `987 987`,剪切后再次粘贴恢复;补充后台同步不能抢占用户粘贴授权的回归测试 | +| 窗口 | 桌面墙独立窗口、全屏进入/退出恢复;主窗口关闭后从应用入口重新打开,保持原进程与登录状态 | +| 窗口重建 | 模拟器桌面墙移动后关闭、重开保留位置和尺寸;修复重建 XComponent 后因局部重绘缓冲被清空而出现黑块的问题 | +| 显示器适配 | 接入显示器增删、分辨率及工作区变化;负坐标、非连续屏幕 ID、主屏 ID 为 0、可用工作区和最近屏幕选择的主机测试通过。外接多屏与混合缩放仍需真机验收 | +| 被控端适配测试 | 直接编译生产 `devices.cpp`,以模拟系统接口验证 Display1/Display2 切换、失败回退、拔屏后编号稳定及恢复、采集帧率、私密场景暂停和授权失效时拒绝输入 | +| 原生采集回归 | 直接编译生产 `capture.cpp`,模拟异步授权;验证启动前不发送数据、GPU 路径优先及失败回退、CPU Surface 合并通知后排空队列、同步 fence 等待/超时/释放、私密场景和暂停/恢复、音频范围检查、重复停止及初始化失败清理;麦克风静音不影响视频和输入 | +| 模拟器被控实测 | 改用系统 Surface 输出及 GLES 纹理读取后,Mac 测试主控实际解码出 3120×2080 桌面;检查了壁纸、Slint 窗口、文字、颜色和方向。移动窗口时画面持续更新。同一鸿蒙进程断开、重新授权、重连再次收到正常画面;断开后被控栏消失,桌面墙仍能创建和绘制;授权弹窗选择“返回”收到取消状态且解码帧数为 0,授权等待中断开也能正常清理 | +| 鼠标状态 | 多键同时按下、拖出边界、取消和重复失焦的释放测试通过;切屏释放旧屏按键/鼠标,之后映射到目标屏像素范围 | +| 桌面墙 | 共享空状态、从近期连接导入设备、密码确认、单设备连接和动态视频预览通过;批量多设备会话尚未完整验收 | +| 系统托盘 | 接入 PC 原生托盘;关闭主窗口后保留原进程,左键恢复、菜单打开共享设置及退出通过;退出执行公共连接清理并移除图标 | +| 音频 | 合成 48 kHz 单声道测试音经 MiniRTC 解码后进入 OH_AudioRenderer;连接后自动播放、静音停止、取消静音恢复;尚未验收真机扬声器听感 | +| 文件 | 系统选择器发送 1 MiB 二进制文件,远端逐字节校验一致;接收 UTF-8 文件到默认应用目录和用户选择的公共目录;进程重启后无需重新选择目录即可接收 | +| 对话框及焦点 | 修复空初始目录导致的闪退;文件选择器绑定发起操作的窗口,取消后返回原会话;选择器尚未关闭时从托盘退出,也能取消等待并完成会话清理 | +| macOS 回归 | 公共 Slint UI 冒烟程序完整编译、链接和运行通过;新增间隔输入后 Ctrl+A 替换的回归,修复前失败、修复后通过。未完成整个 macOS 客户端链接回归 | + +密码框沿用共享桌面的明文文本控件。输入英文密码时,请切换系统输入法到英文或先提交组合文本;自动化回归也必须控制输入法状态,不能把尚未提交的拼音候选当成已输入的密码。 + +文件测试使用独立的合成 MiniRTC 测试端及测试文件,不采集本机桌面。默认目录接收和公共目录接收均检查了 UTF-8 内容,发送测试文件大小为 1,048,576 字节,SHA-256 为 `fbbab289f7f94b25736c58be46a994c441fd02552cc6022352e3d86d2fab7c83`。测试记录和临时文件已清理,原有连接记录保留。 + +被控端联调仅共享鸿蒙模拟器桌面,临时保存主控解码帧以检查实际像素,测试结束清理;没有采集工作站桌面。原始数据回调在本机模拟器只产生音频,CPU 映射的 Surface 视频缓冲也几乎全黑;`StartScreenCaptureWithSurface` 配合 NativeImage 外部纹理、V2 变换矩阵、GLES 读取和 NV12 转换后,主控可显示实际桌面。保留 CPU Surface 和原始回调作为初始化失败时的后备路径。采集帧率设置遵循 [AVScreenCapture 接口约束](https://github.com/openharmony/docs/blob/master/en/application-dev/reference/apis-media-kit/capi-native-avscreen-capture-h.md),在异步启动成功通知之后执行。 + +采集纹理及 EGL context 由专用线程创建、使用和释放,生产者窗口由 NativeImage 持有,避免重复释放。停止采集只销毁自身资源,不终止与 Slint/ArkUI 共用的 EGL display。界面、会话、编码发送及连接清理继续使用公共桌面逻辑。远程拖动原先因 MOVE 事件携带 `BUTTON_NONE` 被窗口管理器拒绝,改为携带实际按下的按钮后,模拟器窗口实际移动并恢复原位。系统全局指针位置查询仍返回能力错误,与已验证的输入注入分别处理。 + +鸿蒙输入法可能消费硬件 Ctrl 组合键。`KeyboardHost` 在 `onKeyPreIme` 中把 Ctrl/Meta 及组合键交给原 Slint 按键分发;普通文本与 Ctrl+Space 保留系统输入法路径。输入法自身的全选、剪切、复制、粘贴、移动和选区回调也交给 Slint TextInput。公共 ID 格式化定时器改为通过 `running` 属性启停,避免原生重复定时器持续清除选区,修复同时适用于其他桌面平台。 + +系统 Ctrl+V 提供的剪贴板读取授权只能使用一次。显式粘贴使用独立的 `clipboardPaste` 请求,后台同步先检查 `READ_PASTEBOARD` 权限,避免公共剪贴板轮询抢先消费粘贴授权。剪贴板服务按顺序执行,读取失败不会伪装成成功的空字符串。已接入 `SystemPasteboard.update` 变更通知,隐藏/失焦窗口也向公共桌面投递 `SDL_EVENT_CLIPBOARD_UPDATE`,同步去重、大小限制及会话过滤继续复用 `ClipboardController`。后台读取使用按变更版本失效的缓存,读取期间再次变更会重读,撤权会清空缓存并拒绝结果。显式粘贴不进入后台缓存。当前普通调试包没有 `READ_PASTEBOARD` 的签名 ACL,因此跨应用后台同步的授权实机验收尚未完成。权限限制参见[系统剪贴板说明](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-v5/use_pasteboard_to_copy_and_paste-V5)。 + +## 全局输入和后台剪贴板权限 + +`KeyboardHook` 使用 API 21 的 `OH_Input_AddKeyEventHook`,而不是局部窗口监听或热键订阅。共享 `SessionDeviceManager` 决定捕获生命周期,原生回调仅进入公共按键队列;窗口失焦、撤权和取消时释放捕获状态,未知按键/未捕获的释放事件交回系统。卸载失败后回调只放行,不再访问已销毁的会话。未授权时继续使用原 Slint 窗口事件;收到授权变化后自动重试,不因周期性失败重试打断用户正按住的回退组合键。 + +本机 API 26 SDK 的 `PermissionDefinitions.json` 将 `HOOK_KEY_EVENT` 定义为 `system_basic / manual_settings`(tablet、2in1),将 `READ_PASTEBOARD` 定义为 `system_basic / user_grant`,二者均需签名 profile 中获批的 ACL。代码声明不能替代签名授权。`DesktopAccess` 串行处理系统授权界面,每次启动只引导一次;用户取消后不循环弹窗,并监听自身权限变化。键盘使用 `openPermissionOnSetting`,剪贴板使用 `requestPermissionsFromUser`。普通开发包不声明这两项权限,也不打开无法使用的授权页。 + +获得获批 profile 后,导出一项 DevEco `signingConfigs` 配置到仓库外的私有 JSON 文件,`material.certpath`、`material.storeFile`、`material.profile` 使用绝对路径: + +```sh +python3 apps/harmony/scripts/build_slint_desktop.py --desktop --reuse-runtime \ + --desktop-permissions --signing-config /absolute/private/signing.json --install +``` + +`--desktop-permissions` 生成权限声明、用途说明和授权流程开关;`--signing-config` 生成签名配置并选择 `entry-default-signed.hap` 安装。生成的配置仅位于忽略的 `.native` 缓存,文件权限为 `0600`。无签名参数时仍输出普通 `entry-default-unsigned.hap`。可单独用 `--desktop-permissions` 做带权限声明的编译检查,不能据此认为已获得权限或可安装到真机。 + +远程 `lock_workstation` 命令已接到经过采集/输入授权检查的 Meta+L 路径,先释放残留键鼠,发送后配对释放,失败保留待清理状态。Windows `send_sas` 没有等价的鸿蒙公开接口,明确返回不支持;所有 PC 客户端共用的 Slint 快捷键菜单在目标为鸿蒙时不显示 Ctrl+Alt+Del,仍保留 Win+L。普通 Ctrl/Alt/Meta 组合键由原生钩子按原顺序转发,系统最终允许拦截哪些保留键仍需授权后的真机验证。 + +本轮验证:普通 HAP 与带权限声明的 HAP 构建通过;普通 HAP 安装本机 2in1 模拟器后,全局钩子实际返回 `201`,无授权弹窗循环,仍收到共享窗口回退发送的 `keyboard` 数据流。真实 Ctrl+A/C、End、Ctrl+V 把 `246` 复制为 `246 246`。生产 C++/ArkTS 的模拟 SDK 回归覆盖 Alt+Tab、Meta+L、重复键、小键盘 Enter、取消、失焦、撤权、重复注册、卸载失败、后台通知、缓存失效、读取中变更/撤权,以及权限拒绝后不重复引导。macOS 公共 Slint 冒烟测试新增快捷键菜单的真实点击回归,验证移除 SAS 后首项为锁屏、弹层命中范围随菜单缩小、切回 Windows 菜单仍正常。成功授权的钩子和后台跨应用读取没有在无 ACL 包中冒充验证通过。 + +## 被控端持久授权(API 26) + +已接入一次系统设置授权、后续连接复用权限的路径。它解决的是应用运行中重复连接的录屏和键鼠确认;重启后自启动、系统登录前控制、锁屏及休眠恢复仍需分别验收。已使用包含这两项 ACL 的调试 Profile 完成签名和模拟器安装,尚未完成本机首次设置授权及真机免确认重连验证。 + +| 权限 | 用途 | SDK 要求 | +| --- | --- | --- | +| `ohos.permission.CUSTOM_SCREEN_RECORDING` | 录屏持久授权,指定显示器采集时无需重复确认 | API 22 起,`system_basic / manual_settings` | +| `ohos.permission.CONTROL_DEVICE` | 直接注入键鼠,无需每次请求临时输入授权 | API 26 起,`system_basic / manual_settings`,PC/2in1 | + +两项权限都需要签名 Profile 的 ACL 和用户在系统设置中的授权。`CUSTOM_SCREEN_CAPTURE` 是截图权限,不能替代上述录屏权限。`HOOK_KEY_EVENT` 是控制端捕获快捷键的权限,也不能替代被控端的 `CONTROL_DEVICE`。 + +使用独立的 `--host-permissions` 选项生成声明、用途说明及设置引导,不要求同时申请全局键盘钩子或剪贴板权限: + +```sh +# 生成 DevEco 工程并编译未签名包,供申请权限和检查构建使用。 +python3 apps/harmony/scripts/build_slint_desktop.py --desktop --reuse-runtime --host-permissions +``` + +### 申请与签名 + +1. 在 DevEco Studio 打开 `apps/harmony/.native/slint-poc/desktop-project`。PC 与移动端的 bundle name 均为 `com.chunkdash.crossdesk`;AGC 应用、签名 Profile 和包名必须一致。构建从共享的 `AppScope/app.json5` 读取包名,不支持单独为 PC 覆盖包名。 +2. 在 **File > Project Structure > Project > Signing Configs** 登录自己的华为开发者账号,关联 AGC 注册应用。在支持该流程的 DevEco Studio 中,通过 **Enable ACL Permissions** 为 `entry / EntryAbility` 申请上述两项权限。调用时机选择 `always`,因为最小化到后台后仍需接受经过认证的连接。若权限列表中没有 `CONTROL_DEVICE`,核对 IDE/SDK 的 API 26 支持,并通过 AGC 项目的“ACL权限”页签确认该权限的开放范围;网页端需先取得 ACL,再将其加入新生成的 Profile。 +3. 分别填写用途和申请理由,并提交实际功能演示材料。可使用以下说明作为草稿,按真实发布场景修改: + + - `CUSTOM_SCREEN_RECORDING`:CrossDesk 为鸿蒙 PC 提供远程桌面被控功能。设备所有者在本机设置中主动授权后,应用仅在远程连接通过密码验证且会话有效时采集指定显示器并传输画面,断开后停止采集。申请持久权限以支持所有者离开设备后的远程访问,避免每次连接都要求现场确认。 + - `CONTROL_DEVICE`:在已认证且屏幕采集有效的远程会话中,将远端操作转换为本机键鼠输入。应用停止共享、系统进入私密场景或授权失效时不接受远程输入;用户可在系统设置中撤销授权。 + +4. 检查生成 Profile 的 ACL 确实包含两项权限。调试阶段官方自动签名流程可能提供临时 Profile,其有效期和权限以实际签发结果为准;发布阶段须完成正式权限及发布 Profile 申请。登录、提交申请及用户设置授权需要由账号/设备所有者完成。 +5. 将工程 `build-profile.json5` 的一项 `app.signingConfigs` 对象保存为仓库外的私有 JSON 文件(不要包含外层数组)。保持原有签名字段,`material.certpath`、`material.storeFile`、`material.profile` 均使用绝对路径。构建脚本会重新生成工程配置,因此在再次运行脚本前先导出该对象。 + +```sh +python3 apps/harmony/scripts/build_slint_desktop.py --desktop --reuse-runtime \ + --host-permissions --signing-config /absolute/private/signing.json \ + --device YOUR_DEVICE_ID --install +``` + +Profile 必须对应 `com.chunkdash.crossdesk`,无需额外传入 `--bundle-name`;该选项仅为兼容已有命令保留,且只接受共享包名。目标设备已有同包名应用时将尝试更新,需满足签名和版本要求。脚本只使用显式传入的签名配置,不会自动继承源工程的签名;DevEco 写入源工程的 ACL 声明也按本次构建开关重新生成,避免重复声明或普通构建意外启用权限。 + +如果也需要控制端快捷键和后台剪贴板同步,可同时加 `--desktop-permissions`,但 Profile 还需包含对应的两项 ACL。未签名包的构建成功不代表有权限或能安装到真机。 + +申请入口和流程依据:[华为受限权限申请](https://developer.huawei.com/consumer/cn/doc/doccenter-capabilities/declare-permissions-in-acl)、[DevEco 自动签名及 ACL 配置](https://developer.huawei.com/consumer/cn/doc/doccenter-deveco-studio/ide-signing-auto)。 + +### 运行行为与验收 + +首次显示主窗口时,启用 `--host-permissions` 的构建合并提示缺少的“屏幕共享”和“辅助操作”开关;点击“前往设置”后打开本应用的系统设置详情页。两项均已授予时不弹窗,只缺一项时只解释该项。“暂不开启”或关闭引导后本次运行不重复打开,可自行进入系统设置开启或重启应用再次引导。普通构建不声明这两项权限、不打开对应设置。 + +PC 系统 `openPermissionOnSetting` 弹窗中的“隐私和安全 > 辅助操作”路径与实际应用设置页不一致,且该系统文案没有应用侧修改接口。因此被控权限改用窗口所属 `UIContext` 的一次引导,再按[华为官方应用设置跳转示例](https://developer.huawei.com/consumer/cn/doc/doccenter-dev-faq/faqs-access-control-16)打开 `application_info_entry`,`pushParams` 取运行中应用的实际包名,兼容默认验证包和已签名包。跳转失败时显示手动查找本应用及开关的说明,不退回错误路径的系统提示。仅打开设置不视为获得权限,仍读取系统授权状态。 + +启动前和设置变化后,ArkTS 都读取系统实际权限并同步给 C++。输入允许条件为“采集有效,且持有 `CONTROL_DEVICE` 或已获得临时输入授权”。不能仅查询 `OH_Input_QueryAuthorizedStatus`,因为 API 26 明确说明该接口只反映弹窗授权,不反映 `CONTROL_DEVICE`。参见 [Input Kit 接口说明](https://github.com/openharmony/docs/blob/master/zh-cn/application-dev/reference/apis-input-kit/capi-oh-input-manager-h.md#oh_input_requestinjection)。 + +持有 `CUSTOM_SCREEN_RECORDING` 时,指定屏幕采集设置不弹出内容选择器,并由系统检查权限是否允许免确认启动;未持有权限时保留原有授权流程。此行为依据本机 SDK 和 [OpenHarmony 录屏服务的 `IsSkipPrivacyWindow`](https://github.com/openharmony/multimedia_player_framework/blob/master/services/services/screen_capture/server/screen_capture_server.cpp),需要目标 HarmonyOS 固件验证。无论哪条路径,都等待 `STARTED` 后才发送画面。断开仍停止录屏、释放键鼠和撤销临时输入授权,不通过持续录屏维持权限。 + +主机回归覆盖持久权限下临时授权状态仍为 `UNAUTHORIZED` 的输入、断开重连、权限撤销、私密场景、设置串行化、取消后不重复引导、退出后不再打开待处理设置,以及录屏策略失败回退。真机验收需使用带相应 ACL 的签名包,依次验证首次授权、两次重连、退出重开、分别撤销两项权限及恢复授权;锁屏/唤醒和重启后场景单独记录,不能用模拟 SDK 测试代替。 + +2026-10-15 验证:`python3 apps/harmony/tests/run.py --desktop` 通过;普通构建与 `--host-permissions` 构建均通过 ArkTS、原生链接和 HAP 检查。构建脚本检查打包后的权限声明与所选开关一致。 + +2026-10-15 签名验证:使用本机 DevEco 生成的 `com.chunkdash.crossdesk` 调试 Profile,确认其 ACL 包含上述两项权限、已登记当前模拟器。签名 HAP 通过官方 `hap-sign-tool verify-app`(含代码和权限签名校验),在 API 26 的 2in1 模拟器安装并启动成功。系统读取到两项权限,但首次设置授权尚未开启,未据此宣称免确认重连已通过。交付包保存在 `.native/artifacts/acl-host/crossdesk-acl-signed.hap`。同时验证普通构建不会继承源工程签名或可选 ACL 声明。 + +2026-10-15 引导修正:回归覆盖两个开关合并、仅缺一项、已授权免提示、取消不跳转、不重复引导、实际包名路由、退出后的过期回调、跳转失败及设置打开不等于授权。完整 `--desktop` 主机回归和带 ACL 的签名构建通过;实际开关授权和远程重连仍需设备端验收。 + +## 无 ACL 的窗口与输入行为(2026-10-15) + +继续使用同一套公共 Slint 界面、桌面墙、会话及编辑逻辑;本轮修复集中在鸿蒙窗口和输入事件适配层,不增加平台专用的产品交互。 + +- 原生标题栏的最大化/还原状态作为一次完整确认传回 Slint。Slint 的状态设置器会同步回调平台;`WindowMode` 阻止确认过程中产生的中间状态再次控制系统窗口,修复点击还原后立即重新最大化的循环。属性事件只携带实际变化的模式,状态版本过滤延迟命令,包括等待标题/尺寸服务期间发生的原生状态变化。 +- 最小化保留原有全屏/最大化模式。主窗口恢复使用 `restore()`,子窗口使用 `showWindow()`;重复显示不重设主窗口的还原尺寸。由全屏返回普通最大化明确使用 `EXIT_IMMERSIVE`。窗口 API 行为参见[官方窗口参考](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/js-apis-window)。 +- 子窗口隐藏/重建时注销监听,按原生窗口实例过滤旧的关闭、失焦、尺寸和状态回调。XComponent 事件携带表面标识,旧表面的延迟销毁和输入不能作用于新表面。创建失败清理半成品,窗口命令队列完成后释放;标题/层级服务失败不再阻断尺寸和模式处理。 +- 输入法增加独立会话编号。相同窗口内切换输入框时,旧回调、旧更新和正在等待的 attach 不能写入新编辑器;原生分发再次检查会话编号。窗口销毁会立即使输入法回调失效,异步完成后释放连接。 +- 硬件事件传递系统 CapsLock、NumLock 快照和 SDK 提供的可打印 ASCII Unicode 值;小键盘在 NumLock 关闭时映射导航键,右 Alt 单独映射为 Slint AltGr。重复键和释放继续使用首次按下的符号。此项没有声称覆盖 SDK 未提供的非 ASCII 布局或全部输入法行为。 + +验证:新增生产 ArkTS 窗口适配回归,覆盖恢复/全屏、过期命令、重建、晚到回调、可选服务失败和创建失败清理;C++ 回归覆盖 Slint 同步状态回写,输入法回归覆盖同窗口换字段和 attach 期间销毁;扩展锁键、布局符号、右 Alt 和小键盘测试。`python3 apps/harmony/tests/run.py --desktop` 及普通无 ACL HAP 构建通过。2in1 模拟器实测主窗口最大化后还原到原尺寸、最小化后重新激活、桌面墙进入/退出全屏及关闭后重建;重建后的表面继续绘制,没有再出现状态往返。最终安装包验证普通输入及 Ctrl+A/C、End、Ctrl+V,将 `246` 复制为 `246 246`,随后清空测试输入并恢复原窗口位置。 + +## 网络状态面板展开回归(2026-10-15) + +已复现点击网络状态图标后,鼠标留在按钮上时面板不展开、移开后才部分展开的问题。公共 `GuiApplication` 每 16 ms 对统计模型调用 `set_vector`,即使行数不变也会重置行数依赖,反复重启控制栏的高度动画。改为只在行数变化时重置模型,其余时间仅更新变化的数据行,保留各 PC 共用的 Slint 布局和 60 ms 动画。 + +新增 `slint_stream_render_test` 使用真实 Slint 定时器与保留缓冲区的软件渲染,持续刷新统计值,在左右停靠布局下连续点击展开/收起且不移动鼠标,同时检查完整面板像素和输入范围。修复前可复现失败,修复后与 `slint_ui_smoke_test` 均通过。普通无 ACL HAP 已构建、安装,在本机 2in1 模拟器连接合成视频源后确认点击展开和收起均不再等待鼠标移开。 + +```bash +xmake build slint_stream_render_test +xmake run slint_stream_render_test +``` + +## 其他按钮与弹窗回归(2026-10-15) + +继续排查公共桌面的模型更新和弹窗输入路径,修复以下问题,所有 PC 仍复用同一套 Slint 组件和回调: + +- 显示器和文件传输模型也改为行数变化时才整体重置,其余时间只更新变化的行,避免每帧刷新重建菜单及列表。 +- 文件传输浮层的标题、边缘和空白区域原先会把点击、滚轮传到远端。浮层现在完整占用本地输入区域,拖出边界以及关闭瞬间都不会向远端漏发鼠标释放事件。 +- 编码下拉框打开后,连接状态切换为会话中会禁用设置,但原菜单仍可修改编码。公共 `CompactComboBox` 现在在禁用时关闭菜单、取消展开定时器,选项回调也检查当前有效性。 + +新增 `slint_popup_render_test`,直接运行共享 Slint 窗口,使用保留像素缓冲区、真实鼠标/键盘事件和 16 ms 模型更新定时器。覆盖显示器、快捷键、视频设置、文件传输、分组筛选、设置下拉框和标签、消息、关于、桌面墙网络状态及添加弹窗、被控端控制者选择及断开确认。检查鼠标留在触发位置时的绘制、按住期间的数据刷新、Esc/取消关闭,以及本地输入不泄漏到远端。文件浮层与禁用下拉框的用例均在修复前失败、修复后通过。 + +验证:`slint_popup_render_test`、`slint_stream_render_test`、`slint_ui_smoke_test` 在本机 macOS 通过;普通无 ACL HAP 构建、安装成功。2in1 模拟器实测主菜单、设置下拉框、设置标签、桌面墙网络状态、添加菜单和添加弹窗,点击后鼠标保持原位也可立即显示,取消/关闭后正常恢复。模拟器本轮未连接真实设备;会话内持续刷新及被控面板状态转换由共享 Slint 回归覆盖,不代表所有平台的全部端到端验收已完成。 + +```bash +xmake build slint_popup_render_test +xmake run slint_popup_render_test +``` + +## 被控端鼠标同步(2026-10-15) + +鸿蒙被控端原先忽略公共 `Start(show_cursor)` 参数,始终请求将鼠标录入视频,并仅发送 `embedded` 状态。现在原生 PC 会话关闭录屏中的鼠标,通过 `OH_Input_GetCursorInfo` 同步箭头、文本、链接、拖拽、忙碌和窗口缩放等语义形状及可见状态,复用 macOS/Windows 控制端已有的 `cursor_state` 协议和显示逻辑。需要视频内鼠标的会话继续使用 `show_cursor=true`。录屏只有确认移除鼠标后才报告独立绘制;移除失败会尝试视频内鼠标,均失败则报告采样不可用。切屏、暂停恢复及私密场景恢复保留该策略。 + +位置查询优先使用 `OH_Input_GetPointerLocation`;后台查询不可用时,使用录屏期间 `OH_Input_AddMouseEventMonitor` 收到的真实鼠标事件。两种来源均将屏内像素坐标和物理显示器 ID 转换为已协商的显示器编号和归一化位置,兼容负坐标排列及重叠屏幕。监听仅在有效的独立鼠标录屏会话期间注册,暂停、私密场景、切屏和停止时清空位置并移除监听;移除失败的回调保持停用。没有有效查询或监听事件时,不伪造位置,仍保留已获取的形状/可见状态。 + +后台位置同步使用 **`ohos.permission.INPUT_MONITORING`**,其授权方式是 `system_basic / system_grant`,需要签名 Profile 中对应 ACL。该权限出现在[华为自动签名支持的 ACL 清单](https://developer.huawei.com/consumer/cn/doc/doccenter-deveco-studio/ide-signing-auto)中;仍需满足应用场景及审批要求。获得该 ACL 后,构建时增加 `--cursor-position-permission` 生成并校验声明。它独立于录屏和输入注入授权,普通构建及已有两项被控权限的构建不会自动加入该 ACL。接口依据:[OpenHarmony Input Kit 的鼠标监听接口](https://github.com/openharmony/docs/blob/master/zh-cn/application-dev/reference/apis-input-kit/capi-oh-input-manager-h.md#oh_input_addmouseeventmonitor)。 + +更正:本机 API 26 SDK 确实定义 `INPUT_DEVICE_CONFIGURATOR`,并注明它可允许后台直接查询位置,但 SDK 定义和证书授权标记不能证明该权限已在 HarmonyOS 开发者 ACL 申请页面开放。此前据此建议直接申请不充分;当前构建不再声明或依赖它。`INPUT_MONITORING` 用于事件监听,不能给 `OH_Input_GetPointerLocation` 直接查询授权。未获得监听权限时,后台位置仍不能保证反馈;鼠标形状和可见状态同步不依赖该权限。 + +回归测试调用生产采集适配和鼠标采样代码,覆盖形状映射、隐藏/无设备、后台查询拒绝后的监听位置、监听权限拒绝、暂停/切屏清理、迟到回调、移除失败与重试、系统已移除监听、采样失败、屏幕 ID 映射、无效坐标、协议往返、公共显示策略及录屏模式回退。`python3 apps/harmony/tests/run.py --desktop` 通过,带监听权限声明的 API 26 AArch64 桌面 HAP 构建通过,并检查打包后只声明 `INPUT_MONITORING`、未声明 `INPUT_DEVICE_CONFIGURATOR`。尚需获批签名权限的鸿蒙 PC 真机与 macOS/Windows 控制端进行端到端验收,模拟 SDK 测试不替代实际鼠标服务和签名权限验证。 + +## 开机自启动设置(2026-10-15) + +共享设置页的「启动与后台」保留原有自启动开关样式。鸿蒙上开关按系统查询结果显示,点击开关弹出设置引导,不直接翻转状态;引导中显示详细状态,并提供「刷新状态」。打开设置、应用回到前台或原生窗口重新激活时会重新查询;查询失败、设备不支持和未开启在引导中分别显示。查询超过 5 秒可重试,旧查询、超时后的迟到结果及退出后的回调不会覆盖新状态。其他 PC 平台继续使用原有自启动开关行为。 + +鸿蒙 PC 用户需要在 **系统设置 → 应用和元服务 → 应用启动管理 → 添加应用** 中选择 **CrossDesk**,点击「完成」;关闭时在该列表移除。应用内提供完整操作说明,不使用未经验证的设置深链接,也不会把保存本地配置当作系统已经开启。步骤依据[华为电脑自启动说明](https://consumer.huawei.com/cn/support/content/zh-cn16048401/);本机 API 26 SDK 的公开接口仅提供 `isAutoStartupSupported()` 和 `getAutoStartupStatusForSelf()`,不需要增加自启动 ACL。系统接口 `setApplicationAutoStartup()` 不适用于普通应用。 + +程序启动后继续由公共 `GuiApplication::InitializeModules()` 和 `GuiRuntime::CreateConnectionPeer()` 恢复设备身份并连接服务器,托盘也复用原有启动流程。自启动设置只决定系统是否启动应用,不等于获得录屏、输入注入权限或登录前控制能力。 + +验证:`python3 apps/harmony/tests/run.py --desktop` 通过,新增生产 ArkTS 回归覆盖状态变化、查询失败、超时、过期结果、进程退出,以及启动/前台/窗口激活和原生刷新命令。`slint_popup_render_test` 和 `slint_ui_smoke_test` 在 macOS 编译运行通过,覆盖开关开启/关闭时点击弹出引导且不改变状态、刷新按钮、异步状态变化、最小窗口布局、Esc/确认关闭及其他平台开关仍可直接切换。普通无 ACL HAP 构建及包检查通过,已安装到 API 26 的 2in1 模拟器,启动后自动连接服务器,自启动开关显示关闭,点击后弹出引导并显示「未开启」,手动刷新及从系统设置返回均正常。未修改模拟器的系统自启动名单;开启/关闭的状态转换由模拟 SDK 回归覆盖。真机系统自启动、重启/切换用户后自动上线仍需设备验收。 + +## 未完成的验收项 + +以下项目不能因为公共逻辑已编译就标记为支持完整: + +- 真机被控端:录屏和输入注入的授权、拒绝、撤销、私密场景恢复;系统声音采集;已实现启动时连接的多显示器枚举、切换和输入映射,但模拟器只有一个屏幕。输入注入仅在采集有效且系统授权时执行。 +- 真机主控音频听感、长时间连接、多会话及桌面墙批量操作;目前远程画面通过合成测试源验证。 +- 文件传输中途取消、大文件与磁盘满、用户撤销目录授权、后台多任务传输;基础发送、接收及重启授权已经通过。 +- 系统自启动后的真机重启/登录验收、系统服务和隐私遮罩;自启动状态、设置引导及返回刷新已接入,开启/关闭由用户在系统设置管理。无人值守的持久录屏/输入授权路径仍需带 ACL 的真机验收,详见上文。原生托盘已接入,仅在系统报告支持时启用。 +- 多屏被控:MiniRTC 在初始化时协商 DisplayN;已协商屏幕拔出后保留编号、恢复到仍可用的屏幕,重新插入可恢复。运行中新增屏幕需要重新初始化连接端(当前重启应用),不能直接向已有会话发送未协商的流。切屏通过重新启动系统采集实现,系统可能再次要求授权,尚不能视为与其他 PC 无感切换完全一致。 +- 全局键盘钩子的授权实机验收、系统保留键在不同 PC 固件上的路由、Meta+L 实际锁屏效果及跨应用后台剪贴板同步。无 ACL 开发包只能验证拒绝/回退路径;模拟 SDK 的成功测试不能替代系统授权后的端到端验收。 +- 窗口和输入:外接多屏、混合缩放、窗口位置/尺寸的长周期恢复、所有输入法组合范围、非 US 键盘布局、系统保留快捷键及跨应用置顶浮层。模拟器不具备的鼠标服务会报告能力不足。 +- GPU 视频直接导入、性能与耗电。当前视频走公共 Slint CPU 图像后备路径,再用 GLES3 呈现;手机和平板使用独立移动端。 + +## 代码定位 + +- `CMakeLists.txt`:公共桌面源码和原生依赖集合,只构建 `slint_desktop` 桥接。 +- `font-discovery.patch`:关闭 OHOS 运行库不适用的 Fontconfig 自动发现;字体仍由后端显式加载。 +- `../native/audio.*`:PC 与移动端共享的 OHAudio 播放;PC 不编译移动视频渲染器。 +- `DisplayHost.ets` / `displays.h`:系统显示器与工作区、全局/屏内坐标转换、窗口所属屏幕及热插拔事件。 +- `window-properties.patch`:把原 Slint 窗口的无边框和置顶属性交给鸿蒙窗口层;补丁只影响 OHOS。 +- `pointer.h`:逐窗口跟踪鼠标按下状态,失焦/取消时完整释放。 +- `desktop.cpp` / `surface.inc`:Slint GUI 线程、窗口适配、EGL/GLES3 呈现。 +- `DesktopHost.ets` / `Index.ets`:鸿蒙原生窗口、表面及事件分发。 +- `TextInputHost.ets` / `os-services.patch`:Slint 1.17.1 C++ 后端缺少的输入法/鼠标形状桥接,补丁只改变 OHOS 目标。 +- `keyboard_hook.cpp` / `DesktopAccess.ets`:系统键盘钩子、权限变化及公共捕获回退。 +- `KeyboardHost.ets`:输入法之前的硬件快捷键路由及配对释放,调用原 Slint 按键处理。 +- `services.cpp` / `devices.cpp`:文件、剪贴板、音频、采集和输入等平台服务。 +- `../native/capture.cpp`:原生采集生命周期、异步授权状态、音视频缓冲处理。 +- `../native/capture_surface.cpp`:系统录屏 Surface 的 NativeImage/GLES 读取、方向变换及线程资源管理。 +- `../native/input.cpp` / `key_mapping.h`:被控键鼠注入、扩展扫描码映射、按键快照对账及授权/断开后的释放。 +- `FileHost.ets`:系统文件选择器、URI 读取快照、目录持久授权及进程启动时激活。未选公共目录时接收文件保存在应用 `files/desktop/Received Files`;可在共享设置中选择公开目录。 +- `TrayHost.ets` / `tray.cpp`:PC 系统托盘事件接回公共桌面的显示、设置和退出回调。 +- `AutostartHost.ets` / `platform/harmony/harmony_autostart.h`(桌面源码目录):异步查询系统自启动状态,提供共享设置页的只读状态及刷新桥接。 +- `platform/harmony/gui/window_state.h`(桌面源码目录):原生窗口激活状态用于公共音频、剪贴板和输入释放逻辑。 +- `keyboard.h`:鸿蒙按键到 Slint 按键的状态化映射。 + +日志:`hdc shell hilog -T CrossDeskDesktop`(共享桌面应用)和 `hdc shell hilog -T SlintDesktop`(表面/采集)。登录凭据不应复制到问题报告;公共桌面版本的本机密码默认可见,录屏/截图前请隐藏。 diff --git a/apps/harmony/desktop-port/TextInputHost.ets b/apps/harmony/desktop-port/TextInputHost.ets new file mode 100644 index 00000000..80f0d69d --- /dev/null +++ b/apps/harmony/desktop-port/TextInputHost.ets @@ -0,0 +1,129 @@ +import { inputMethod } from '@kit.IMEKit'; +import { window } from '@kit.ArkUI'; +import { DisplayHost } from './DisplayHost'; +import type { DesktopEvent } from './DesktopHost'; + +// The OS owns composition/candidates; Slint owns the text, selection and edits. +export class TextInputHost { + private static editor: inputMethod.InputMethodController = inputMethod.getController(); + private static id: number = 0; + private static preview: string = ''; + private static queue: Promise = Promise.resolve(); + private static revision: number = 0; + private static requestedId: number = 0; + private static requestedSession: number = 0; + + static request(event: DesktopEvent, target: window.Window, send: (event: DesktopEvent) => void): void { + const id = event.id ?? 0; + if (event.kind === 1) { + TextInputHost.requestedId = id; + TextInputHost.requestedSession = event.imeSession ?? 0; + ++TextInputHost.revision; + } else if (event.kind === 0) { + if (TextInputHost.requestedId !== id) return; + TextInputHost.requestedId = 0; + ++TextInputHost.revision; + } else if (TextInputHost.requestedId !== id || TextInputHost.requestedSession !== (event.imeSession ?? 0)) return; + const revision = TextInputHost.revision; + TextInputHost.queue = TextInputHost.queue.then(() => TextInputHost.apply(event, target, send, revision)) + .catch((error: Error) => console.error(`Slint input method: ${error.message}`)); + } + + static release(id: number): void { + if (TextInputHost.requestedId === id) { + TextInputHost.requestedId = 0; + ++TextInputHost.revision; + } + TextInputHost.queue = TextInputHost.queue.then(async () => { + if (TextInputHost.id !== id || TextInputHost.requestedId === id) return; + TextInputHost.id = 0; TextInputHost.preview = ''; + await TextInputHost.editor.detach(); + }).catch((error: Error) => console.warn(`Slint input cleanup: ${error.message}`)); + } + + private static async apply(event: DesktopEvent, target: window.Window, send: (event: DesktopEvent) => void, + revision: number): Promise { + if (revision !== TextInputHost.revision) return; + const id = event.id ?? 0; + const editor = TextInputHost.editor; + if (event.kind === 0) { + if (TextInputHost.id === id) { TextInputHost.id = 0; TextInputHost.preview = ''; await editor.detach(); } + return; + } + const rect = target.getWindowProperties().windowRect; + const scale = DisplayHost.screen(target).densityPixels; + const cursor: inputMethod.CursorInfo = { left: rect.left + (event.x ?? 0) * scale, + top: rect.top + ((event.y ?? 0) + (target.getWindowDecorVisible() ? target.getWindowDecorHeight() : 0)) * scale, + width: Math.max(1, (event.width ?? 1) * scale), height: Math.max(1, (event.height ?? 1) * scale) }; + const selection: inputMethod.Range = { start: event.anchor ?? 0, end: event.cursor ?? 0 }; + const inputType = event.inputType === 1 ? inputMethod.TextInputType.VISIBLE_PASSWORD : + event.inputType === 2 ? inputMethod.TextInputType.NUMBER : + event.inputType === 3 ? inputMethod.TextInputType.NUMBER_DECIMAL : inputMethod.TextInputType.TEXT; + if (TextInputHost.id !== id || event.kind === 1) { + if (TextInputHost.id !== 0) { TextInputHost.id = 0; await editor.detach(); } + if (revision !== TextInputHost.revision) return; + // Bind the input session to the UI context of its owning window. + await editor.attachWithUIContext(target.getUIContext(), { inputAttribute: { textInputType: inputType, + enterKeyType: inputMethod.EnterKeyType.DONE, consumeKeyEvents: true }, cursorInfo: cursor, selection, + windowId: target.getWindowProperties().id }, { showKeyboard: false }); + if (revision !== TextInputHost.revision) { await editor.detach(); return; } + TextInputHost.id = id; TextInputHost.preview = ''; + const current = (): boolean => TextInputHost.id === id && revision === TextInputHost.revision; + const deliver = (command: DesktopEvent): void => { + if (!current()) return; + command.imeSession = event.imeSession ?? 0; + send(command); + }; + const key = (code: number): void => { + deliver({ op: 'key', id, code, down: true }); deliver({ op: 'key', id, code, down: false }); + }; + // off() is required when the same process attaches a new text field. + editor.off('insertText'); editor.off('deleteLeft'); editor.off('deleteRight'); + editor.off('sendFunctionKey'); editor.off('setPreviewText'); editor.off('finishTextPreview'); + editor.off('handleExtendAction'); editor.off('moveCursor'); + editor.off('selectByRange'); editor.off('selectByMovement'); + editor.on('insertText', (text: string) => { + if (!current()) return; + TextInputHost.preview = ''; deliver({ op: 'composition', id, text, commit: true }); + }); + editor.on('setPreviewText', (text: string, _range: inputMethod.Range) => { + if (!current()) return; + TextInputHost.preview = text; deliver({ op: 'composition', id, text, commit: false }); + }); + editor.on('finishTextPreview', () => { + if (!current()) return; + if (TextInputHost.preview.length > 0) deliver({ op: 'composition', id, text: TextInputHost.preview, commit: true }); + TextInputHost.preview = ''; + }); + editor.on('deleteLeft', (length: number) => { for (let i = 0; i < Math.min(length, 65536); ++i) key(2055); }); + editor.on('deleteRight', (length: number) => { for (let i = 0; i < Math.min(length, 65536); ++i) key(2071); }); + editor.on('sendFunctionKey', () => key(2054)); + // The input method consumes these hardware shortcuts while attached. + // Deliver editing commands to the focused Slint TextInput, without + // synthesizing modifier presses that could disturb held physical keys. + editor.on('handleExtendAction', (action: inputMethod.ExtendAction) => { + let code = 0; + if (action === inputMethod.ExtendAction.SELECT_ALL) code = 2017; + else if (action === inputMethod.ExtendAction.CUT) code = 2040; + else if (action === inputMethod.ExtendAction.COPY) code = 2019; + else if (action === inputMethod.ExtendAction.PASTE) code = 2038; + if (code !== 0) deliver({ op: 'imeKey', id, code, control: true }); + }); + const move = (direction: inputMethod.Direction, shift: boolean): void => { + const code = direction === inputMethod.Direction.CURSOR_UP ? 2012 : + direction === inputMethod.Direction.CURSOR_DOWN ? 2013 : + direction === inputMethod.Direction.CURSOR_LEFT ? 2014 : + direction === inputMethod.Direction.CURSOR_RIGHT ? 2015 : 0; + if (code !== 0) deliver({ op: 'imeKey', id, code, shift }); + }; + editor.on('moveCursor', (direction: inputMethod.Direction) => move(direction, false)); + editor.on('selectByMovement', (movement: inputMethod.Movement) => move(movement.direction, true)); + editor.on('selectByRange', (range: inputMethod.Range) => { + deliver({ op: 'imeSelection', id, anchor: range.start, cursor: range.end }); + }); + } + await editor.changeSelection(event.text ?? '', selection.start, selection.end); + if (revision !== TextInputHost.revision) return; + await editor.updateCursor(cursor); + } +} diff --git a/apps/harmony/desktop-port/TrayHost.ets b/apps/harmony/desktop-port/TrayHost.ets new file mode 100644 index 00000000..ea22454b --- /dev/null +++ b/apps/harmony/desktop-port/TrayHost.ets @@ -0,0 +1,58 @@ +import { statusBarManager } from '@kit.DeskTopExtensionKit'; +import { common } from '@kit.AbilityKit'; +import { image } from '@kit.ImageKit'; +import { emitter } from '@kit.BasicServicesKit'; +import type { DesktopEvent } from './DesktopHost'; + +export class TrayHost { + private static active: boolean = false; + private static icon: image.PixelMap | undefined; + static async create(context: common.UIAbilityContext, event: DesktopEvent, + send: (event: DesktopEvent) => void): Promise { + if (TrayHost.active) return true; + if (!await statusBarManager.isStatusBarCapabilitySupported(context)) return false; + const data = await context.resourceManager.getMediaContent($r('app.media.app_icon').id); + const source = image.createImageSource(data.buffer as ArrayBuffer); + try { TrayHost.icon = await source.createPixelMap({ desiredSize: { width: 48, height: 48 } }); } + finally { source.release(); } + const action = (code: string): void => send({ op: 'tray', text: code }); + statusBarManager.on('statusBarIconClick', () => { + action('show'); + // A minimized main window cannot foreground its UIAbility with + // showWindow alone. This is an explicit click on our system tray icon. + context.startAbility({ bundleName: context.abilityInfo.bundleName, + abilityName: context.abilityInfo.name, moduleName: context.abilityInfo.moduleName }) + .catch((error: Error) => console.error(`Slint tray restore: ${error.message}`)); + }); + statusBarManager.on('rightMenuClick', (event: emitter.EventData) => { + const code: string | undefined = (event.data as Record | undefined)?.['menuCode']; + if (typeof code === 'string') action(code); + }); + const item: statusBarManager.StatusBarItem = { + icons: { white: TrayHost.icon, black: TrayHost.icon }, + hoverTips: event.title ?? 'CrossDesk', + quickOperation: { abilityName: '', moduleName: 'entry', title: 'CrossDesk', height: 30 }, + statusBarGroupMenu: [[ + { title: event.show ?? '', menuCode: 'show', menuAction: { abilityName: 'EntryAbility', moduleName: 'entry', notifyOnly: true } }, + { title: event.settings ?? '', menuCode: 'settings', menuAction: { abilityName: 'EntryAbility', moduleName: 'entry', notifyOnly: true } } + ]] + }; + try { + await new Promise((resolve, reject) => statusBarManager.addToStatusBar(context, item, + (error) => { if (error) reject(error); else resolve(); })); + TrayHost.active = true; + return true; + } catch (error) { TrayHost.release(); throw new Error(error.message); } + } + static async remove(context: common.UIAbilityContext): Promise { + if (!TrayHost.active) return; + try { + await new Promise((resolve, reject) => statusBarManager.removeFromStatusBar(context, + (error) => { if (error) reject(error); else resolve(); })); + } finally { TrayHost.active = false; TrayHost.release(); } + } + private static release(): void { + statusBarManager.off('statusBarIconClick'); statusBarManager.off('rightMenuClick'); + TrayHost.icon?.release(); TrayHost.icon = undefined; + } +} diff --git a/apps/harmony/desktop-port/bridge.h b/apps/harmony/desktop-port/bridge.h new file mode 100644 index 00000000..c49abf1f --- /dev/null +++ b/apps/harmony/desktop-port/bridge.h @@ -0,0 +1,21 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _BRIDGE_H_ +#define _BRIDGE_H_ + +#include +#include +namespace crossdesk::harmony::desktop { +using Json = nlohmann::json; +void Emit(Json command); +// Calls a platform service on the ArkTS thread. The Slint thread can wait while +// ArkUI keeps pumping window and picker events; shutdown cancels pending calls. +Json Request(Json command); +Json Environment(); +} + +#endif diff --git a/apps/harmony/desktop-port/cursor_shape.h b/apps/harmony/desktop-port/cursor_shape.h new file mode 100644 index 00000000..34d2b17b --- /dev/null +++ b/apps/harmony/desktop-port/cursor_shape.h @@ -0,0 +1,49 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _CURSOR_SHAPE_H_ +#define _CURSOR_SHAPE_H_ + +#include +#include + +namespace crossdesk::harmony::desktop { +inline RemoteCursorShape ShapeFromPointerStyle(Input_PointerStyle style) { + using Shape = RemoteCursorShape; + switch (style) { + case EAST: case MIDDLE_BTN_EAST: return Shape::e_resize; + case WEST: case MIDDLE_BTN_WEST: return Shape::w_resize; + case SOUTH: case MIDDLE_BTN_SOUTH: return Shape::s_resize; + case NORTH: case MIDDLE_BTN_NORTH: return Shape::n_resize; + case WEST_EAST: case MIDDLE_BTN_EAST_WEST: return Shape::ew_resize; + case NORTH_SOUTH: case MIDDLE_BTN_NORTH_SOUTH: return Shape::ns_resize; + case NORTH_EAST: case MIDDLE_BTN_NORTH_EAST: return Shape::ne_resize; + case NORTH_WEST: case MIDDLE_BTN_NORTH_WEST: return Shape::nw_resize; + case SOUTH_EAST: case MIDDLE_BTN_SOUTH_EAST: return Shape::se_resize; + case SOUTH_WEST: case MIDDLE_BTN_SOUTH_WEST: return Shape::sw_resize; + case NORTH_EAST_SOUTH_WEST: return Shape::nesw_resize; + case NORTH_WEST_SOUTH_EAST: return Shape::nwse_resize; + case CROSS: case CURSOR_CROSS: case COLOR_SUCKER: + case SCREENSHOT_CHOOSE: case SCREENSHOT_CURSOR: return Shape::crosshair; + case CURSOR_COPY: return Shape::copy; + case CURSOR_FORBID: return Shape::not_allowed; + case HAND_GRABBING: return Shape::grabbing; + case HAND_OPEN: return Shape::grab; + case HAND_POINTING: return Shape::pointer; + case HELP: return Shape::help; + case MOVE: case MIDDLE_BTN_NORTH_SOUTH_WEST_EAST: return Shape::move; + case RESIZE_LEFT_RIGHT: return Shape::col_resize; + case RESIZE_UP_DOWN: return Shape::row_resize; + case TEXT_CURSOR: case HORIZONTAL_TEXT_CURSOR: return Shape::text; + case LOADING: return Shape::wait; + case RUNNING: case RUNNING_LEFT: case RUNNING_RIGHT: return Shape::progress; + // As on the other PC platforms, unsupported/custom artwork uses an arrow. + default: return Shape::default_cursor; + } +} +} + +#endif diff --git a/apps/harmony/desktop-port/desktop.cpp b/apps/harmony/desktop-port/desktop.cpp new file mode 100644 index 00000000..9c9dabf8 --- /dev/null +++ b/apps/harmony/desktop-port/desktop.cpp @@ -0,0 +1,396 @@ +#include "bridge.h" +#include "keyboard.h" +#include "keyboard_hook.h" +#include "host_access.h" +#include "pointer.h" +#include "window_mode.h" +#include "platform/harmony/gui/tray/harmony_tray.h" +#include "platform/harmony/gui/window_state.h" +#include "platform/harmony/harmony_autostart.h" +#include "application/gui_application.h" +#include "SDL3/SDL.h" +#include "rd_log.h" +#include "platform/path_backend.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony::desktop { +namespace { +std::mutex bridge_mutex; +std::condition_variable wake; +Json environment; +std::deque incoming, outgoing; +std::map replies; +std::deque tasks; +std::atomic stopping{false}; +std::thread gui_thread; +uint64_t request_id = 0; +bool services_cancelled = false; +} +Json Environment() { std::lock_guard lock(bridge_mutex); return environment; } +void Emit(Json command) { std::lock_guard lock(bridge_mutex); outgoing.push_back(std::move(command)); } +Json Request(Json command) { + std::unique_lock lock(bridge_mutex); + if (stopping || services_cancelled) return Json::object(); + const auto id = ++request_id; + command["request"] = id; outgoing.push_back(std::move(command)); + // Picker dismissal and shutdown both reply/cancel; never block ArkUI. + wake.wait(lock, [id] { return stopping || services_cancelled || replies.count(id); }); + if (stopping || services_cancelled) return Json::object(); + auto result = std::move(replies.at(id)); replies.erase(id); return result; +} +namespace { +struct Surface; +class DesktopWindow final : public slint::platform::WindowAdapter { + public: + int id; + slint::platform::SoftwareRenderer software{slint::platform::SoftwareRenderer::RepaintBufferType::ReusedBuffer}; + slint::PhysicalSize extent; + slint::PhysicalPosition origin{{0, 0}}; + bool dirty = true, visible = false, active = false, explicitly_sized = false; + bool positioned = false; + float density = 1; + int decor = 0; + bool no_frame = false, topmost = false; + KeyboardState keyboard; + PointerState pointer; + const void* ime_adapter = nullptr; + uint64_t ime_session = 0; + std::string surface_id; + WindowMode window_mode; + uint64_t state_revision = 0; + bool scaled = false; + std::unique_ptr surface; + explicit DesktopWindow(int value); + ~DesktopWindow() override; + slint::PhysicalSize size() override { return extent; } + slint::platform::AbstractRenderer& renderer() override { return software; } + void request_redraw() override { dirty = true; } + void set_visible(bool value) override { + if (!scaled) SetDensity(Environment().value("density", 1.f)); + if (!value) ReleaseInput(); + visible = value; + Json command{{"op", "visible"}, {"id", id}, {"visible", value}, {"width", extent.width}, {"height", extent.height}}; + if (positioned) { command["x"] = origin.x; command["y"] = origin.y; } + Emit(std::move(command)); + } + void set_size(slint::PhysicalSize value) override { + explicitly_sized = true; extent = value; + Emit({{"op", "resize"}, {"id", id}, {"width", value.width}, {"height", value.height}}); + } + std::optional position() override { return origin; } + void set_position(slint::PhysicalPosition value) override { + positioned = true; origin = value; + Emit({{"op", "move"}, {"id", id}, {"x", value.x}, {"y", value.y}}); + } + void update_window_properties(const WindowProperties& properties) override { + const auto constraints = properties.layout_constraints(); + if (!scaled) SetDensity(Environment().value("density", 1.f)); + if (!explicitly_sized && !visible) + extent = slint::PhysicalSize({uint32_t(std::max(1.f, constraints.preferred.width * density)), + uint32_t(std::max(1.f, constraints.preferred.height * density))}); + Json command{{"op", "properties"}, {"id", id}, {"title", std::string(properties.title())}, + {"stateRevision", state_revision}, {"noFrame", no_frame}, {"topmost", topmost}}; + const Json requested{{"fullscreen", properties.is_fullscreen()}, {"maximized", properties.is_maximized()}, + {"minimized", properties.is_minimized()}}; + // Layout/title updates must not reapply an old window mode while the OS + // is asynchronously reporting a titlebar maximize/restore transition. + command.update(window_mode.Request(requested)); + if (constraints.min) { command["minWidth"] = constraints.min->width * density; command["minHeight"] = constraints.min->height * density; } + if (constraints.max) { command["maxWidth"] = constraints.max->width * density; command["maxHeight"] = constraints.max->height * density; } + Emit(std::move(command)); + } + void SetDensity(float value); + void ReleaseInput(); +}; +std::map windows; +#include "surface.inc" +DesktopWindow::DesktopWindow(int value) : id(value), extent{{1216, 903}}, surface(std::make_unique(*this)) { windows[id] = this; } +DesktopWindow::~DesktopWindow() { windows.erase(id); Emit({{"op", "destroy"}, {"id", id}}); } +void DesktopWindow::SetDensity(float value) { + if (!std::isfinite(value) || value <= 0 || (scaled && density == value)) return; + density = value; scaled = true; + surface->density = value; + surface->rendered_width = surface->rendered_height = 0; + window().dispatch_scale_factor_change_event(value); + dirty = true; +} +slint::PointerEventButton PointerButton(int button) { + return button == 1 ? slint::PointerEventButton::Right : button == 2 ? slint::PointerEventButton::Middle : slint::PointerEventButton::Left; +} +void DesktopWindow::ReleaseInput() { + pointer.Cancel([this](int button, float x, float y) { + window().dispatch_pointer_release_event(slint::LogicalPosition({x, y}), PointerButton(button)); + }); + window().dispatch_pointer_exit_event(); + for (const auto& [code, key] : keyboard.Pressed()) if (!key.empty()) window().dispatch_key_release_event(slint::SharedString(key)); + keyboard.Reset(); + std::lock_guard lock(bridge_mutex); + environment["buttons"] = 0; +} + +void OsRequest(void* data, const void* token, int kind, const uint8_t* text, size_t length, + size_t cursor, size_t anchor, float x, float y, float width, float height, int type) { + auto& adapter = *static_cast(data); + if (kind == 4) { adapter.no_frame = type & 1; adapter.topmost = type & 2; return; } + if (kind == 3) Emit({{"op", "cursor"}, {"id", adapter.id}, {"kind", type}}); + else { + if (kind == 0 || kind == 1) ++adapter.ime_session; + adapter.ime_adapter = kind == 0 ? nullptr : token; + Emit({{"op", "ime"}, {"id", adapter.id}, {"imeSession", adapter.ime_session}, {"kind", kind}, {"text", std::string(reinterpret_cast(text), length)}, + {"cursor", cursor}, {"anchor", anchor}, {"x", x}, {"y", y}, {"width", width}, {"height", height}, {"inputType", type}}); + } +} +void Key(DesktopWindow& adapter, const Json& command) { + const bool down = command.value("down", false); + const int code = command.value("code", 0); + const bool repeat = down && adapter.keyboard.IsPressed(code); + const auto lock = [&command](const char* name) -> std::optional { + return command.contains(name) ? std::optional{command.at(name).get()} : std::nullopt; + }; + const auto key = adapter.keyboard.Translate(code, down, command.value("unicode", 0u), lock("capsLock"), lock("numLock")); + if (key.empty()) return; + if (repeat) adapter.window().dispatch_key_press_repeat_event(slint::SharedString(key)); + else if (down) adapter.window().dispatch_key_press_event(slint::SharedString(key)); + else adapter.window().dispatch_key_release_event(slint::SharedString(key)); +} +void Dispatch(const Json& command) { + if (command.value("op", "") == "tray") { HarmonyTray::Dispatch(command.value("text", "")); return; } + const auto found = windows.find(command.value("id", 0)); + if (found == windows.end()) return; + auto& adapter = *found->second; + const auto op = command.value("op", ""); + // ArkUI can finish a callback after Slint has focused another editor in the + // same window. Window identity alone is not an input-session identity. + if (command.contains("imeSession") && command.value("imeSession", uint64_t{0}) != adapter.ime_session) return; + if (op != "attach" && command.contains("surface") && command.value("surface", "") != adapter.surface_id) return; + if (op == "attach") { + adapter.surface_id = command.at("surface").get(); + if (!adapter.surface->AttachSurface(std::stoull(command.at("surface").get()), command.at("density"))) + Emit({{"op", "error"}, {"message", "Slint surface initialization failed"}}); + adapter.dirty = true; + } else if (op == "detach") { adapter.surface_id.clear(); adapter.ReleaseInput(); adapter.surface->DetachSurface(); } + else if (op == "redraw") adapter.dirty = true; + else if (op == "restore") { adapter.window().set_minimized(false); adapter.window().show(); adapter.dirty = true; } + else if (op == "close") adapter.window().dispatch_close_requested_event(); + else if (op == "active") { + adapter.active = command.value("active", false); + if (!adapter.active) adapter.ReleaseInput(); + if (command.value("active", false) && adapter.scaled && !adapter.visible) { + adapter.window().set_minimized(false); + adapter.window().show(); + } + adapter.window().dispatch_window_active_changed_event(command.value("active", false)); + } else if (op == "state") { + adapter.state_revision = command.value("stateRevision", uint64_t{0}); + auto actual = Json::object(); + for (const auto* name : {"fullscreen", "maximized", "minimized"}) + if (command.contains(name)) actual[name] = command.at(name); + // Each setter synchronously calls update_window_properties. These are OS + // acknowledgements, not three independent requests back to the compositor. + adapter.window_mode.Acknowledge(actual, [&] { + if (command.contains("fullscreen")) adapter.window().set_fullscreen(command.at("fullscreen").get()); + if (command.contains("maximized")) adapter.window().set_maximized(command.at("maximized").get()); + if (command.contains("minimized")) adapter.window().set_minimized(command.at("minimized").get()); + }); + } else if (op == "metrics") { + adapter.positioned = true; + adapter.origin = slint::PhysicalPosition({command.value("x", 0), command.value("y", 0)}); + adapter.decor = command.value("decor", 0); + adapter.SetDensity(command.value("density", 1.f)); + adapter.dirty = true; + } else if (op == "pointerExit") { + adapter.window().dispatch_pointer_exit_event(); + } else if (op == "pointer") { + const int kind = command.value("kind", 2), button = command.value("button", 0); + const slint::LogicalPosition point{{command.value("x", 0.f), command.value("y", 0.f)}}; + { + std::lock_guard lock(bridge_mutex); + const auto density = adapter.density; + environment["pointerX"] = adapter.origin.x + point.x * density; + environment["pointerY"] = adapter.origin.y + adapter.decor + point.y * density; + adapter.pointer.Update(kind, button, point.x, point.y); + environment["buttons"] = adapter.pointer.Buttons(); + } + const auto b = PointerButton(button); + if (kind == 0) adapter.window().dispatch_pointer_press_event(point, b); + else if (kind == 1) adapter.window().dispatch_pointer_release_event(point, b); + else if (kind == 2) adapter.window().dispatch_pointer_move_event(point); + else adapter.ReleaseInput(); + } else if (op == "scroll") adapter.window().dispatch_pointer_scroll_event( + slint::LogicalPosition({command.value("x", 0.f), command.value("y", 0.f)}), command.value("dx", 0.f), command.value("dy", 0.f)); + else if (op == "key") Key(adapter, command); + else if (op == "imeKey" && adapter.ime_adapter) { + KeyboardState symbols; + const auto text = symbols.Translate(command.value("code", 0), true); + const int modifiers = (command.value("shift", false) ? 1 : 0) | (command.value("control", false) ? 2 : 0); + slint::cbindgen_private::crossdesk_slint_edit(adapter.ime_adapter, + reinterpret_cast(text.data()), text.size(), modifiers); + } else if (op == "imeSelection" && adapter.ime_adapter) { + slint::cbindgen_private::crossdesk_slint_selection(adapter.ime_adapter, + command.value("anchor", 0), command.value("cursor", 0)); + } + else if (op == "composition" && adapter.ime_adapter) { + const auto text = command.value("text", ""); + slint::cbindgen_private::crossdesk_slint_composition(adapter.ime_adapter, command.value("commit", true), reinterpret_cast(text.data()), text.size()); + } else if (op == "text") { + const slint::SharedString text(command.value("text", "")); + adapter.window().dispatch_key_press_event(text); adapter.window().dispatch_key_release_event(text); + } +} +class DesktopPlatform final : public slint::platform::Platform { + int next_id = 0; + public: + std::unique_ptr create_window_adapter() override { return std::make_unique(++next_id); } + void run_in_event_loop(Task task) override { + std::lock_guard lock(bridge_mutex); tasks.push_back(std::move(task)); wake.notify_all(); + } + void quit_event_loop() override { stopping = true; wake.notify_all(); } + void set_clipboard_text(const slint::SharedString& text, Clipboard kind) override { + if (kind == Clipboard::DefaultClipboard) SDL_SetClipboardText(text.data()); + } + std::optional clipboard_text(Clipboard kind) override { + if (kind != Clipboard::DefaultClipboard) return {}; + const auto result = Request({{"op", "clipboardPaste"}}); + if (!result.value("ok", false)) return {}; + return slint::SharedString(result.value("text", "")); + } + void run_event_loop() override { + while (!stopping) { + std::deque commands; + std::deque pending; + { + std::lock_guard lock(bridge_mutex); commands.swap(incoming); pending.swap(tasks); + } + for (auto& task : pending) std::move(task).run(); + for (auto& command : commands) { + try { Dispatch(command); } + catch (const std::exception& error) { Emit({{"op", "error"}, {"message", error.what()}}); } + } + slint::platform::update_timers_and_animations(); + for (const auto& [id, adapter] : windows) if (adapter->visible) adapter->surface->Render(); + std::unique_lock lock(bridge_mutex); + wake.wait_for(lock, std::chrono::milliseconds(8), [] { return stopping || !incoming.empty() || !tasks.empty(); }); + } + } +}; +std::string String(napi_env env, napi_value value) { + size_t length = 0; + if (napi_get_value_string_utf8(env, value, nullptr, 0, &length) != napi_ok || length > 1024 * 1024) return {}; + std::string result(length + 1, '\0'); + napi_get_value_string_utf8(env, value, result.data(), result.size(), &length); + result.resize(length); return result; +} +napi_value Undefined(napi_env env) { napi_value out; napi_get_undefined(env, &out); return out; } +napi_value Start(napi_env env, napi_callback_info info) { + if (gui_thread.joinable()) return Undefined(env); + napi_value arg; size_t count = 1; napi_get_cb_info(env, info, &count, &arg, nullptr, nullptr); + try { + environment = Json::parse(String(env, arg)); + if (environment.value("filesDir", "").empty()) throw std::runtime_error("Missing application storage directory"); + } catch (const std::exception& error) { napi_throw_error(env, nullptr, error.what()); return nullptr; } + stopping = false; + gui_thread = std::thread([] { + try { + setenv("SLINT_DEFAULT_FONT", "/system/fonts/HarmonyOS_Sans.ttf", 1); + setenv("SLINT_FONT_PATH", "/system/fonts/HarmonyOS_Sans_SC.ttf", 1); + const auto certificates = Environment().value("certificates", ""); + if (certificates.empty()) throw std::runtime_error("Missing system CA certificate directory"); + setenv("SSL_CERT_DIR", certificates.c_str(), 1); + slint::cbindgen_private::crossdesk_slint_set_os_callback(OsRequest); + slint::platform::set_platform(std::make_unique()); + // GuiRuntime constructs worker controllers before Run(). They may log + // immediately, so select an app-writable directory before construction. + crossdesk::InitLogger(crossdesk::platform::GetLogPath("CrossDesk").string()); + crossdesk::GuiApplication app; + app.Run(); + Emit({{"op", "exit"}}); + } catch (const std::exception& error) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Application failed: %{public}s", error.what()); + Emit({{"op", "error"}, {"message", error.what()}}); + } + }); + return Undefined(env); +} +napi_value Send(napi_env env, napi_callback_info info) { + napi_value arg; size_t count = 1; napi_get_cb_info(env, info, &count, &arg, nullptr, nullptr); + try { + auto command = Json::parse(String(env, arg)); + if (command.value("op", "") == "autostartState") { + SetHarmonyAutostartStatus(command.value("pending", false) ? HarmonyAutostartStatus::checking : + !command.value("ok", false) ? HarmonyAutostartStatus::failed : + !command.value("supported", false) ? HarmonyAutostartStatus::unsupported : + command.value("active", false) ? HarmonyAutostartStatus::enabled : HarmonyAutostartStatus::disabled); + return Undefined(env); + } + if (command.value("op", "") == "active") + SetKeyboardFocus(command.value("id", 0), command.value("active", false)); + else if (command.value("op", "") == "keyboardPermission") + SetKeyboardHookPermission(command.value("ok", false)); + else if (command.value("op", "") == "hostPermissions") { + SetHostPermissions(command.value("controlDevice", false), command.value("screenRecording", false)); + return Undefined(env); + } + else if (command.value("op", "") == "clipboardChanged") { + SDL_Event event{SDL_EVENT_CLIPBOARD_UPDATE}; SDL_PushEvent(&event); + return Undefined(env); + } + std::lock_guard lock(bridge_mutex); + // A synchronous native picker can be pending on the GUI thread when the + // system tray requests exit. Unblock it so the shared exit callback runs. + if (command.value("op", "") == "tray" && command.value("text", "") == "exit") + services_cancelled = true; + if (command.value("op", "") == "displays") environment["displays"] = command.at("displays"); + else if (command.value("op", "") == "reply") replies[command.at("request").get()] = command; + else incoming.push_back(std::move(command)); + wake.notify_all(); + } catch (...) { napi_throw_error(env, nullptr, "Invalid desktop event"); return nullptr; } + return Undefined(env); +} +napi_value Poll(napi_env env, napi_callback_info) { + Json result = Json::array(); + { std::lock_guard lock(bridge_mutex); for (auto& command : outgoing) result.push_back(std::move(command)); outgoing.clear(); } + const auto text = result.dump(); napi_value out; napi_create_string_utf8(env, text.c_str(), text.size(), &out); return out; +} +napi_value Init(napi_env env, napi_value exports) { + napi_property_descriptor properties[] = { + {"start", nullptr, Start, nullptr, nullptr, nullptr, napi_default, nullptr}, + {"send", nullptr, Send, nullptr, nullptr, nullptr, napi_default, nullptr}, + {"poll", nullptr, Poll, nullptr, nullptr, nullptr, napi_default, nullptr}}; + napi_define_properties(env, exports, 3, properties); + napi_add_env_cleanup_hook(env, [](void*) { stopping = true; wake.notify_all(); if (gui_thread.joinable()) gui_thread.join(); }, nullptr); + return exports; +} +napi_module module = {1, 0, nullptr, Init, "slint_desktop", nullptr, {0}}; +} +bool WindowActive(const slint::Window& window) { + const auto& handle = window.window_handle().handle(); + for (const auto& [id, adapter] : windows) { + const auto& candidate = adapter->window().window_handle().handle(); + if (handle._0 == candidate._0 && handle._1 == candidate._1) + return adapter->active && adapter->visible && !window.is_minimized(); + } + return false; +} +} +namespace crossdesk { +bool IsHarmonyWindowActive(const slint::Window& window) { + return harmony::desktop::WindowActive(window); +} +} +extern "C" __attribute__((constructor)) void RegisterSlintDesktop() { napi_module_register(&crossdesk::harmony::desktop::module); } diff --git a/apps/harmony/desktop-port/devices.cpp b/apps/harmony/desktop-port/devices.cpp new file mode 100644 index 00000000..b6785d87 --- /dev/null +++ b/apps/harmony/desktop-port/devices.cpp @@ -0,0 +1,403 @@ +#include "bridge.h" +#include "displays.h" +#include "capture.h" +#include "input.h" +#include "keyboard_hook.h" +#include "host_access.h" +#include "screen_capturer_factory.h" +#include "speaker_capturer_factory.h" +#include "device_controller_factory.h" +#include "platform/video_renderer.h" +#include "runtime/cursor_state_provider.h" +#include "runtime/cursor_position.h" +#include "cursor_shape.h" +#include "platform/harmony/input/system_shortcuts.h" +#include +#include +#include +#include +#include +#include + +namespace crossdesk { +namespace { +std::mutex input_mutex; +harmony::Input input; +std::atomic capture_active{false}; +std::atomic cursor_mode{harmony::CaptureCursorMode::unknown}; +std::atomic control_device_granted{false}; +std::atomic screen_recording_granted{false}; +std::mutex cursor_monitor_lifecycle, cursor_monitor_mutex; +bool cursor_monitor_requested = false, cursor_monitor_installed = false; +bool cursor_monitor_accepting = false; +struct MonitoredCursorPosition { + bool valid = false; + int32_t display = -1; + double x = 0, y = 0; +}; +MonitoredCursorPosition monitored_cursor; + +void ReceiveCursorPosition(const Input_MouseEvent* event) { + if (!event) return; + std::lock_guard lock(cursor_monitor_mutex); + if (!cursor_monitor_accepting || !capture_active) return; + if (OH_Input_GetMouseEventAction(event) == MOUSE_ACTION_CANCEL) { + monitored_cursor = {}; + return; + } + const auto display = OH_Input_GetMouseEventDisplayId(event); + const auto x = OH_Input_GetMouseEventDisplayX(event); + const auto y = OH_Input_GetMouseEventDisplayY(event); + monitored_cursor = {display >= 0 && x >= 0 && y >= 0, display, double(x), double(y)}; +} + +void SetCursorMonitoring(bool active) { + std::lock_guard operation(cursor_monitor_lifecycle); + if (cursor_monitor_requested == active) return; + cursor_monitor_requested = active; + { + std::lock_guard lock(cursor_monitor_mutex); + cursor_monitor_accepting = false; + monitored_cursor = {}; + } + // The callback only uses process-owned state. A failed removal remains + // inert, and must be removed before the next session can register again. + if (cursor_monitor_installed) { + const auto result = OH_Input_RemoveMouseEventMonitor(ReceiveCursorPosition); + // With this non-null callback, PARAMETER_ERROR means no listener remains + // (for example, the recording service already removed it on shutdown). + if (result != INPUT_SUCCESS && result != INPUT_PARAMETER_ERROR) { + OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Cursor monitor removal failed: %{public}d", int(result)); + return; + } + cursor_monitor_installed = false; + } + if (!active) return; + // Input Kit allows this listener only during recording. INPUT_MONITORING + // provides background positions without INPUT_DEVICE_CONFIGURATOR. + { + std::lock_guard lock(cursor_monitor_mutex); + cursor_monitor_accepting = true; + } + const auto result = OH_Input_AddMouseEventMonitor(ReceiveCursorPosition); + if (result == INPUT_SUCCESS) { + cursor_monitor_installed = true; + } else { + { + std::lock_guard lock(cursor_monitor_mutex); + cursor_monitor_accepting = false; + monitored_cursor = {}; + } + OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Cursor monitor unavailable: %{public}d", int(result)); + } +} + +bool InjectionAuthorized() { + if (!capture_active) return false; + // API 26's dialog status does not reflect CONTROL_DEVICE authorization. + if (control_device_granted) return true; + Input_InjectionStatus status = UNAUTHORIZED; + return OH_Input_QueryAuthorizedStatus(&status) == INPUT_SUCCESS && status == AUTHORIZED; +} +std::mutex speaker_mutex; +SpeakerCapturer::speaker_data_cb speaker_callback; +bool speaker_running = false; +std::vector Displays() { + std::vector result; + for (const auto& d : harmony::desktop::ReadDisplays(harmony::desktop::Environment())) + result.emplace_back(reinterpret_cast(uintptr_t(d.id)), d.name, d.primary, + d.x, d.y, d.x + d.width, d.y + d.height); + return result; +} +class HarmonyScreen final : public ScreenCapturer { + cb_desktop_data callback_; + harmony::Capture capture_; + mutable std::mutex mutex_; + std::vector displays_; + DisplayInfo current_{"", 0, 0, 0, 0}; + int selected_ = 0, fps_ = 60; + bool running_ = false, requested_ = false; + bool show_cursor_ = false; + + void StopCapture() { + capture_active = false; + cursor_mode = harmony::CaptureCursorMode::unknown; + SetCursorMonitoring(false); + capture_.Stop(); + { std::lock_guard lock(input_mutex); input.Release(); } + OH_Input_CancelInjection(); + } + void Refresh() { + const auto latest = Displays(); + // MiniRTC negotiates DisplayN at initialization. Preserve its slot/physical + // identity across hot-unplug; new displays become available on reinit. + for (auto& known : displays_) { + const auto found = std::find_if(latest.begin(), latest.end(), [&](const auto& d) { return d.handle == known.handle; }); + if (found != latest.end()) known = *found; + else { known.width = known.height = 0; known.right = known.left; known.bottom = known.top; } + } + int target = selected_; + if (target < 0 || target >= int(displays_.size()) || displays_[target].width <= 0 || displays_[target].height <= 0) { + const auto first = std::find_if(displays_.begin(), displays_.end(), [](const auto& d) { return d.width > 0 && d.height > 0; }); + target = first == displays_.end() ? -1 : int(first - displays_.begin()); + } + const bool changed = target != selected_ || (target >= 0 && + (displays_[target].width != current_.width || displays_[target].height != current_.height)); + selected_ = target; + if (requested_ && changed) { + StopCapture(); + running_ = StartCapture() == 0; + } + } + int StartCapture() { + if (selected_ < 0 || selected_ >= int(displays_.size()) || + displays_[selected_].width <= 0 || displays_[selected_].height <= 0) return -1; + current_ = displays_[selected_]; + const auto stream = MakeDisplayStreamId(selected_); + const bool started = capture_.Start(current_.width, current_.height, uintptr_t(current_.handle), + [this, stream](const MiniRtcVideoFrame& frame) { + if (callback_) callback_(reinterpret_cast(const_cast(frame.data)), + frame.size, frame.width, frame.height, stream.c_str(), nullptr); + }, [this](int state) { + const bool was_active = capture_active.exchange(harmony::CaptureActiveForState(state, capture_active)); + cursor_mode = capture_active ? capture_.CursorMode() : harmony::CaptureCursorMode::unknown; + SetCursorMonitoring(capture_active && cursor_mode == harmony::CaptureCursorMode::separate); + if (capture_active && !was_active && !InjectionAuthorized()) { + OH_Input_RequestInjection([](Input_InjectionStatus status) { + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Host input authorization: %{public}d", int(status)); + // Retry releases retained while authorization was unavailable + // before accepting a newly authorized controller's presses. + { std::lock_guard lock(input_mutex); input.Release(); } + }); + } else if (!capture_active) { std::lock_guard lock(input_mutex); input.Release(); } + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Host capture state: %{public}d", state); + }, + [](const char* data, size_t size) { + std::lock_guard lock(speaker_mutex); + if (speaker_running && speaker_callback) + speaker_callback(reinterpret_cast(const_cast(data)), size, "audio"); + }, fps_, screen_recording_granted, show_cursor_); + cursor_mode = started ? capture_.CursorMode() : harmony::CaptureCursorMode::unknown; + return started ? 0 : -1; + } + int Select(int index) { + if (index < 0 || index >= int(displays_.size()) || displays_[index].width <= 0 || displays_[index].height <= 0) return -1; + if (index == selected_) return 0; + const int previous = selected_; + if (running_) StopCapture(); + selected_ = index; + if (!requested_ || StartCapture() == 0) { running_ = requested_; return 0; } + // Do not acknowledge a failed switch or send input to the failed target. + selected_ = previous; + running_ = StartCapture() == 0; + return -1; + } + public: + ~HarmonyScreen() override { Stop(); } + int Init(int fps, cb_desktop_data cb) override { + std::lock_guard lock(mutex_); + fps_ = std::clamp(fps, 1, 60); callback_ = std::move(cb); displays_ = Displays(); return 0; + } + int Destroy() override { return Stop(); } + int Start(bool show_cursor) override { + std::lock_guard lock(mutex_); + if (running_ && show_cursor_ == show_cursor) return 0; + if (running_) StopCapture(); + requested_ = running_ = false; + show_cursor_ = show_cursor; + Refresh(); + running_ = StartCapture() == 0; + requested_ = running_; + return running_ ? 0 : -1; + } + int Stop() override { + std::lock_guard lock(mutex_); + requested_ = running_ = false; StopCapture(); return 0; + } + int Pause(int) override { return Stop(); } + int Resume(int index) override { + std::lock_guard lock(mutex_); + Refresh(); + if (Select(index) != 0) return -1; + if (!running_) running_ = StartCapture() == 0; + requested_ = running_; + return running_ ? 0 : -1; + } + std::vector GetDisplayInfoList() override { + std::lock_guard lock(mutex_); + Refresh(); + return displays_; + } + int SwitchTo(int index) override { std::lock_guard lock(mutex_); return Select(index); } + int ResetToInitialMonitor() override { return SwitchTo(0); } + int GetCurrentMonitorIndex() const override { std::lock_guard lock(mutex_); return selected_; } +}; +class HarmonyMouse final : public MouseController { + std::vector displays_; + void* display_ = nullptr; + int width_ = 0, height_ = 0; + bool Configure(int index) { + const auto& d = displays_[index]; + if (d.handle == display_ && d.width == width_ && d.height == height_) return true; + if (!input.Configure(d.width, d.height, uintptr_t(d.handle))) return false; + display_ = d.handle; width_ = d.width; height_ = d.height; + return true; + } + public: + int Init(std::vector displays) override { + if (displays.empty()) return -1; + std::lock_guard lock(input_mutex); + displays_ = std::move(displays); return Configure(0) ? 0 : -1; + } + void UpdateDisplayInfoList(const std::vector& displays) override { + std::lock_guard lock(input_mutex); + if (!std::any_of(displays.begin(), displays.end(), [this](const auto& d) { + return d.handle == display_ && d.width > 0 && d.height > 0; + })) input.Release(); + displays_ = displays; + } + int Destroy() override { + std::lock_guard lock(input_mutex); input.Release(); displays_.clear(); width_ = height_ = 0; return 0; + } + int SendMouseCommand(RemoteAction action, int index) override { + std::lock_guard lock(input_mutex); + if (index < 0 || index >= int(displays_.size()) || displays_[index].width <= 0 || displays_[index].height <= 0 || !InjectionAuthorized()) return -1; + if (!Configure(index)) return -1; + return input.Apply(action) ? 0 : -1; + } +}; +class HarmonyKeyboard final : public KeyboardCapturer { + harmony::desktop::KeyboardHook hook_; + int last_hook_result_ = -1; + public: + int Hook(OnKeyAction callback, void* context) override { + const int result = hook_.Start(callback, context); + if (result != last_hook_result_) { + last_hook_result_ = result; + OH_LOG_Print(LOG_APP, result == INPUT_SUCCESS ? LOG_INFO : LOG_WARN, 0, "SlintDesktop", + "Global keyboard hook result: %{public}d", result); + } + if (result == INPUT_PERMISSION_DENIED) + harmony::desktop::Emit({{"op", "keyboardPermission"}}); + return result; + } + // Local keyboard capture and remote input injection are independent. A + // controller-window focus change must not release another peer's host keys. + int Unhook() override { return hook_.Stop(); } + bool IsHookActive() const override { return hook_.Active(); } + bool ShouldRetryHook() const override { + return last_hook_result_ != INPUT_DEVICE_NOT_SUPPORTED && + (last_hook_result_ != INPUT_PERMISSION_DENIED || harmony::desktop::KeyboardHookPermissionGranted()); + } + int SendKeyboardCommand(int key, bool down, uint32_t scan, bool extended) override { + RemoteAction action{}; action.type = keyboard; + action.k.key_value = key; action.k.flag = down ? key_down : key_up; + action.k.scan_code = scan; action.k.extended = extended; + std::lock_guard lock(input_mutex); return InjectionAuthorized() && input.Apply(action) ? 0 : -1; + } +}; +// The common presenter owns its CPU image fallback. A GPU renderer can replace +// this without introducing a separate session, input, or window implementation. +class SoftwareVideo final : public VideoRenderer { + std::string selected_; + public: + bool IsReady() const override { return false; } + bool IsActive() const override { return false; } + bool SetSelectedStream(std::string id) override { bool changed = id != selected_; selected_ = std::move(id); return changed; } + void DiscardStream(std::string_view id) override { if (id == selected_) selected_.clear(); } + SubmitResult SubmitNv12(std::string_view, const uint8_t*, size_t, int, int, VideoLatencyFrame) override { return SubmitResult::failed; } + SubmitResult SubmitCachedNv12(std::string_view, const uint8_t*, size_t, int, int, VideoLatencyFrame) override { return SubmitResult::failed; } + bool CopyLatestNv12(std::string_view, std::vector*, int*, int*) const override { return false; } +}; +class HarmonySpeaker final : public SpeakerCapturer { + public: + int Init(speaker_data_cb callback) override { std::lock_guard lock(speaker_mutex); speaker_callback = std::move(callback); return 0; } + int Destroy() override { std::lock_guard lock(speaker_mutex); speaker_running = false; speaker_callback = {}; return 0; } + int Start() override { std::lock_guard lock(speaker_mutex); speaker_running = true; return 0; } + int Stop() override { std::lock_guard lock(speaker_mutex); speaker_running = false; return 0; } + bool IsRunning() const override { std::lock_guard lock(speaker_mutex); return speaker_running; } +}; +} +void harmony::desktop::SetHostPermissions(bool control_device, bool screen_recording) { + std::lock_guard lock(input_mutex); + if (control_device_granted.exchange(control_device) != control_device) input.Release(); + screen_recording_granted = screen_recording; +} +ScreenCapturer* ScreenCapturerFactory::Create() { return new HarmonyScreen(); } +bool SendHarmonySystemShortcut(ServiceCommandFlag command) { + // Windows secure attention has no public Harmony equivalent. Do not claim + // it succeeded, or turn a SAS request into an unrelated physical chord. + if (command != ServiceCommandFlag::lock_workstation) return false; + std::lock_guard lock(input_mutex); + return InjectionAuthorized() && input.LockWorkstation(); +} +SpeakerCapturer* SpeakerCapturerFactory::Create() { return new HarmonySpeaker(); } +DeviceController* DeviceControllerFactory::Create(Device device) { + return device == Mouse ? static_cast(new HarmonyMouse()) : new HarmonyKeyboard(); +} +std::unique_ptr CreateVideoRenderer() { return std::make_unique(); } +struct CursorStateProvider::Impl { + Input_CursorInfo* cursor = OH_Input_CursorInfo_Create(); + ~Impl() { if (cursor) OH_Input_CursorInfo_Destroy(&cursor); } +}; +CursorStateProvider::CursorStateProvider() : impl_(std::make_unique()) {} +CursorStateProvider::~CursorStateProvider() = default; +bool CursorStateProvider::Sample(const std::vector& displays, int, CursorState* state) { + if (!state) return false; + *state = {}; + ResetCursorPosition(state); + state->shape = RemoteCursorShape::none; + if (!capture_active) return false; + const auto mode = cursor_mode.load(); + if (mode == harmony::CaptureCursorMode::embedded) { + state->render_mode = CursorRenderMode::embedded; + return true; + } + if (mode != harmony::CaptureCursorMode::separate || !impl_->cursor || + OH_Input_GetCursorInfo(impl_->cursor, nullptr) != INPUT_SUCCESS || + OH_Input_CursorInfo_IsVisible(impl_->cursor, &state->visible) != INPUT_SUCCESS) return false; + + if (state->visible) { + Input_PointerStyle style = DEFAULT; + if (OH_Input_CursorInfo_GetStyle(impl_->cursor, &style) != INPUT_SUCCESS) return false; + state->shape = harmony::desktop::ShapeFromPointerStyle(style); + } + state->render_mode = state->visible ? CursorRenderMode::separate : CursorRenderMode::hidden; + state->hidden_reason = state->visible ? CursorHiddenReason::unspecified : CursorHiddenReason::system_hidden; + + int32_t display = -1; + double x = 0, y = 0; + auto location = OH_Input_GetPointerLocation(&display, &x, &y); + if (location != INPUT_SUCCESS) { + std::lock_guard lock(cursor_monitor_mutex); + if (location == INPUT_DEVICE_NO_POINTER) { + monitored_cursor = {}; + } else if (cursor_monitor_accepting && monitored_cursor.valid) { + display = monitored_cursor.display; + x = monitored_cursor.x; + y = monitored_cursor.y; + location = INPUT_SUCCESS; + } + } + if (location == INPUT_SUCCESS && display >= 0 && std::isfinite(x) && std::isfinite(y)) { + // SDK coordinates are relative to a physical display; the protocol uses + // negotiated display slots. Do not confuse a physical ID with a slot or + // select a different monitor when desktop rectangles overlap. + for (size_t index = 0; index < displays.size(); ++index) { + const auto& d = displays[index]; + if (uintptr_t(d.handle) != static_cast(display) || d.width <= 0 || d.height <= 0 || + x < 0 || y < 0 || x >= d.width || y >= d.height) continue; + state->display_id = static_cast(index); + state->x = static_cast(x / d.width); + state->y = static_cast(y / d.height); + state->position_valid = true; + break; + } + } else if (location == INPUT_DEVICE_NO_POINTER && !state->visible) { + state->hidden_reason = CursorHiddenReason::no_pointing_device; + } + // Without a successful query or a monitored event, report no position. + // This must not discard a successfully sampled shape/visibility. + return true; +} +} diff --git a/apps/harmony/desktop-port/displays.h b/apps/harmony/desktop-port/displays.h new file mode 100644 index 00000000..6198d58f --- /dev/null +++ b/apps/harmony/desktop-port/displays.h @@ -0,0 +1,59 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _DISPLAYS_H_ +#define _DISPLAYS_H_ + +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony::desktop { +struct DesktopDisplay { + int id; + std::string name; + bool primary; + int x, y, width, height; + int usable_x, usable_y, usable_width, usable_height; + // SDL reserves zero for failure; Harmony's primary display can be zero. + uint32_t SdlId() const { return uint32_t(id) + 1; } +}; + +inline std::vector ReadDisplays(const nlohmann::json& environment) { + std::vector result; + for (const auto& item : environment.value("displays", nlohmann::json::array())) { + const int64_t id = item.value("id", int64_t(-1)); + const int width = item.value("width", 0), height = item.value("height", 0); + if (id < 0 || id > INT32_MAX || width <= 0 || height <= 0) continue; + if (std::any_of(result.begin(), result.end(), [id](const auto& d) { return d.id == id; })) continue; + const int x = item.value("x", 0), y = item.value("y", 0); + result.push_back({int(id), item.value("name", "Harmony Display"), item.value("primary", false), + x, y, width, height, item.value("usableX", x), item.value("usableY", y), + item.value("usableWidth", width), item.value("usableHeight", height)}); + } + std::stable_sort(result.begin(), result.end(), [](const auto& a, const auto& b) { + return a.primary != b.primary ? a.primary : a.id < b.id; + }); + return result; +} + +inline const DesktopDisplay* DisplayAt(const std::vector& displays, int x, int y) { + const DesktopDisplay* nearest = nullptr; + int64_t distance = std::numeric_limits::max(); + for (const auto& d : displays) { + const int64_t dx = int64_t(x) - std::clamp(x, d.x, int64_t(d.x) + d.width - 1); + const int64_t dy = int64_t(y) - std::clamp(y, d.y, int64_t(d.y) + d.height - 1); + const int64_t next = dx * dx + dy * dy; + if (next < distance) { distance = next; nearest = &d; } + } + return nearest; +} +} + +#endif diff --git a/apps/harmony/desktop-port/font-discovery.patch b/apps/harmony/desktop-port/font-discovery.patch new file mode 100644 index 00000000..01244205 --- /dev/null +++ b/apps/harmony/desktop-port/font-discovery.patch @@ -0,0 +1,41 @@ +--- a/Cargo.toml ++++ b/Cargo.toml +@@ -200,7 +200,7 @@ + wgpu-29 = { package = "wgpu", version = "29.0.4", default-features = false } + input = { version = "0.10.0", default-features = false } + tr = { version = "0.1", default-features = false } +-fontique = { version = "0.10.0" } ++fontique = { version = "0.10.0", default-features = false, features = ["std"] } + # Pinned to match parley's version + read-fonts = { version = "0.39" } + # Pinned to match parley's version +--- a/internal/core/Cargo.toml ++++ b/internal/core/Cargo.toml +@@ -130,7 +130,7 @@ + unicode-script = { version = "0.5.7", optional = true } + icu_normalizer = { workspace = true, optional = true } + sys-locale = { version = "0.3.2", optional = true, features = ["js"] } +-parley = { version = "0.10.0", optional = true, features = ["complex-scripts"] } ++parley = { version = "0.10.0", optional = true, default-features = false, features = ["std", "complex-scripts"] } + + image = { workspace = true, optional = true, default-features = false } + clru = { workspace = true, optional = true } +--- a/Cargo.lock ++++ b/Cargo.lock +@@ -3960,16 +3960,9 @@ + "hashbrown 0.17.1", + "linebender_resource_handle", + "memmap2", +- "objc2 0.6.4", +- "objc2-core-foundation", +- "objc2-core-text", +- "objc2-foundation 0.3.2", + "parlance", + "read-fonts", +- "roxmltree", + "smallvec", +- "windows 0.62.2", +- "windows-core 0.62.2", + "yeslogic-fontconfig-sys", + ] + diff --git a/apps/harmony/desktop-port/host_access.h b/apps/harmony/desktop-port/host_access.h new file mode 100644 index 00000000..04b06ae2 --- /dev/null +++ b/apps/harmony/desktop-port/host_access.h @@ -0,0 +1,15 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _HOST_ACCESS_H_ +#define _HOST_ACCESS_H_ + +namespace crossdesk::harmony::desktop { +// Values come from AbilityKit's actual permission checks, never saved settings. +void SetHostPermissions(bool control_device, bool screen_recording); +} + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL.h b/apps/harmony/desktop-port/include/SDL3/SDL.h new file mode 100644 index 00000000..9cb81913 --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL.h @@ -0,0 +1,57 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_H_ +#define _SDL_H_ + +// The desktop's small SDL service surface, implemented with Harmony APIs. +// This header is private to the Harmony target; desktop SDL builds are unchanged. +#include +#include +using Uint8 = uint8_t; +using SDL_DisplayID = uint32_t; +struct SDL_Point { int x, y; }; +struct SDL_Rect { int x, y, w, h; }; +struct SDL_DisplayMode { int w, h; }; +struct SDL_Event { uint32_t type; }; +struct SDL_AudioSpec { int format, channels, freq; }; +struct SDL_AudioStream; +constexpr uint32_t SDL_INIT_VIDEO = 1, SDL_INIT_AUDIO = 2; +constexpr uint32_t SDL_EVENT_FIRST = 0, SDL_EVENT_LAST = UINT32_MAX; +constexpr uint32_t SDL_EVENT_QUIT = 0x100, SDL_EVENT_CLIPBOARD_UPDATE = 0x900; +constexpr int SDL_GETEVENT = 2, SDL_AUDIO_S16 = 0x8010; +constexpr uint32_t SDL_BUTTON_LMASK = 1, SDL_AUDIO_DEVICE_DEFAULT_PLAYBACK = 0; +bool SDL_Init(uint32_t); +bool SDL_InitSubSystem(uint32_t); +void SDL_Quit(); +void SDL_QuitSubSystem(uint32_t); +const char* SDL_GetError(); +uint64_t SDL_GetTicks(); +void SDL_Delay(uint32_t); +bool SDL_PollEvent(SDL_Event*); +int SDL_PeepEvents(SDL_Event*, int, int, uint32_t, uint32_t); +bool SDL_PushEvent(SDL_Event*); +char* SDL_GetClipboardText(); +bool SDL_SetClipboardText(const char*); +bool SDL_HasClipboardText(); +inline void SDL_free(void* p) { std::free(p); } +const char* SDL_GetCurrentVideoDriver(); +const SDL_DisplayMode* SDL_GetCurrentDisplayMode(SDL_DisplayID); +SDL_DisplayID SDL_GetPrimaryDisplay(); +SDL_DisplayID SDL_GetDisplayForPoint(const SDL_Point*); +SDL_DisplayID* SDL_GetDisplays(int*); +bool SDL_GetDisplayUsableBounds(SDL_DisplayID, SDL_Rect*); +uint32_t SDL_GetGlobalMouseState(float*, float*); +bool SDL_OpenURL(const char*); +uint32_t SDL_OpenAudioDevice(uint32_t, const SDL_AudioSpec*); +bool SDL_ResumeAudioDevice(uint32_t); +void SDL_CloseAudioDevice(uint32_t); +SDL_AudioStream* SDL_CreateAudioStream(const SDL_AudioSpec*, const SDL_AudioSpec*); +void SDL_DestroyAudioStream(SDL_AudioStream*); +bool SDL_BindAudioStream(uint32_t, SDL_AudioStream*); +bool SDL_PutAudioStreamData(SDL_AudioStream*, const void*, int); + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_audio.h b/apps/harmony/desktop-port/include/SDL3/SDL_audio.h new file mode 100644 index 00000000..049ca172 --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_audio.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_AUDIO_H_ +#define _SDL_AUDIO_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_clipboard.h b/apps/harmony/desktop-port/include/SDL3/SDL_clipboard.h new file mode 100644 index 00000000..3c1817e2 --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_clipboard.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_CLIPBOARD_H_ +#define _SDL_CLIPBOARD_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_error.h b/apps/harmony/desktop-port/include/SDL3/SDL_error.h new file mode 100644 index 00000000..805bc49a --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_error.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_ERROR_H_ +#define _SDL_ERROR_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_events.h b/apps/harmony/desktop-port/include/SDL3/SDL_events.h new file mode 100644 index 00000000..d0c0fc39 --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_events.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_EVENTS_H_ +#define _SDL_EVENTS_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_init.h b/apps/harmony/desktop-port/include/SDL3/SDL_init.h new file mode 100644 index 00000000..664b4faf --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_init.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_INIT_H_ +#define _SDL_INIT_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/SDL3/SDL_timer.h b/apps/harmony/desktop-port/include/SDL3/SDL_timer.h new file mode 100644 index 00000000..5dffb00b --- /dev/null +++ b/apps/harmony/desktop-port/include/SDL3/SDL_timer.h @@ -0,0 +1,12 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _SDL_TIMER_H_ +#define _SDL_TIMER_H_ + +#include "SDL.h" + +#endif diff --git a/apps/harmony/desktop-port/include/tinyfiledialogs.h b/apps/harmony/desktop-port/include/tinyfiledialogs.h new file mode 100644 index 00000000..5ec30f8e --- /dev/null +++ b/apps/harmony/desktop-port/include/tinyfiledialogs.h @@ -0,0 +1,14 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _TINYFILEDIALOGS_H_ +#define _TINYFILEDIALOGS_H_ + +const char* tinyfd_selectFolderDialog(const char* title, const char* path); +const char* tinyfd_openFileDialog(const char* title, const char* path, int, + const char* const*, const char*, int); + +#endif diff --git a/apps/harmony/desktop-port/index.d.ts b/apps/harmony/desktop-port/index.d.ts new file mode 100644 index 00000000..5638993e --- /dev/null +++ b/apps/harmony/desktop-port/index.d.ts @@ -0,0 +1,3 @@ +export const start: (environment: string) => void; +export const send: (event: string) => void; +export const poll: () => string; diff --git a/apps/harmony/desktop-port/keyboard.h b/apps/harmony/desktop-port/keyboard.h new file mode 100644 index 00000000..3f1ae73b --- /dev/null +++ b/apps/harmony/desktop-port/keyboard.h @@ -0,0 +1,93 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _KEYBOARD_H_ +#define _KEYBOARD_H_ + +#include +#include +#include +#include + +namespace crossdesk::harmony::desktop { +// Remember the symbol chosen on key-down. Releasing Shift before a letter must +// not turn the matching key-up into a different remote key. +class KeyboardState { + std::map pressed_; + bool caps_ = false; + bool num_ = true; + static std::string Utf8(std::u8string_view key) { return {reinterpret_cast(key.data()), key.size()}; } + public: + void Reset() { pressed_.clear(); } + bool IsPressed(int code) const { return pressed_.count(code); } + const std::map& Pressed() const { return pressed_; } + std::string Translate(int code, bool down, unsigned unicode = 0, + std::optional caps = {}, std::optional num = {}) { + // Lock keys may have changed in another application. Prefer the OS event + // snapshot, while keeping a down/up pair's original symbol unchanged. + if (caps) caps_ = *caps; + if (num) num_ = *num; + if (auto it = pressed_.find(code); it != pressed_.end()) { + auto key = it->second; + if (!down) pressed_.erase(it); + return key; + } + using namespace slint::platform::key_codes; + const bool shift = IsPressed(2047) || IsPressed(2048); + std::string key; + if (!num_ && code >= 2103 && code <= 2112) { + const std::u8string_view navigation[] = {Insert, End, DownArrow, PageDown, LeftArrow, u8"", RightArrow, Home, UpArrow, PageUp}; + key = Utf8(navigation[code - 2103]); + } else if (!num_ && code == 2117) key = Utf8(Delete); + else if (unicode >= 0x21 && unicode <= 0x7e && + ((code >= 2000 && code <= 2009) || (code >= 2017 && code <= 2044) || + (code >= 2056 && code <= 2064) || (code >= 2103 && code <= 2122))) + key = std::string(1, char(unicode)); + else if (code >= 2000 && code <= 2009) key = std::string(1, shift ? ")!@#$%^&*("[code - 2000] : char('0' + code - 2000)); + else if (code >= 2017 && code <= 2042) key = std::string(1, char(((shift != caps_) ? 'A' : 'a') + code - 2017)); + else if (code >= 2103 && code <= 2112) key = std::string(1, char('0' + code - 2103)); + else if (code >= 2090 && code <= 2101) { + const std::u8string_view functions[] = {F1, F2, F3, F4, F5, F6, F7, F8, F9, F10, F11, F12}; + key = Utf8(functions[code - 2090]); + } else if (code >= 2816 && code <= 2827) { + const std::u8string_view functions[] = {F13, F14, F15, F16, F17, F18, F19, F20, F21, F22, F23, F24}; + key = Utf8(functions[code - 2816]); + } else switch (code) { + case 2055: key = Utf8(Backspace); break; case 2054: case 2119: key = Utf8(Return); break; + case 2049: key = Utf8(Tab); break; case 2070: key = Utf8(Escape); break; + case 2012: key = Utf8(UpArrow); break; case 2013: key = Utf8(DownArrow); break; + case 2014: key = Utf8(LeftArrow); break; case 2015: key = Utf8(RightArrow); break; + case 2072: key = Utf8(Control); break; case 2073: key = Utf8(ControlR); break; + case 2047: key = Utf8(Shift); break; case 2048: key = Utf8(ShiftR); break; + case 2045: key = Utf8(Alt); break; case 2046: key = Utf8(AltGr); break; + case 2076: key = Utf8(Meta); break; case 2077: key = Utf8(MetaR); break; + case 2067: key = Utf8(Menu); break; + case 2050: key = " "; break; case 2071: key = Utf8(Delete); break; + case 2081: key = Utf8(Home); break; case 2082: key = Utf8(End); break; + case 2068: key = Utf8(PageUp); break; case 2069: key = Utf8(PageDown); break; + case 2083: key = Utf8(Insert); break; + case 2074: if (down && !caps) caps_ = !caps_; key = Utf8(CapsLock); break; + case 2102: if (down && !num) num_ = !num_; break; + case 2075: key = Utf8(ScrollLock); break; case 2080: key = Utf8(Pause); break; + case 2079: key = Utf8(SysReq); break; case 2113: key = "/"; break; + case 2114: key = "*"; break; case 2115: key = "-"; break; + case 2116: key = "+"; break; case 2117: key = "."; break; + case 2118: key = ","; break; case 2120: key = "="; break; + case 2121: key = "("; break; case 2122: key = ")"; break; + case 2043: key = shift ? "<" : ","; break; case 2044: key = shift ? ">" : "."; break; + case 2056: key = shift ? "~" : "`"; break; case 2057: key = shift ? "_" : "-"; break; + case 2058: key = shift ? "+" : "="; break; case 2059: key = shift ? "{" : "["; break; + case 2060: key = shift ? "}" : "]"; break; case 2061: key = shift ? "|" : "\\"; break; + case 2062: key = shift ? ":" : ";"; break; case 2063: key = shift ? "\"" : "'"; break; + case 2064: key = shift ? "?" : "/"; break; + } + if (down && (!key.empty() || code == 2102)) pressed_[code] = key; + return key; + } +}; +} + +#endif diff --git a/apps/harmony/desktop-port/keyboard_hook.cpp b/apps/harmony/desktop-port/keyboard_hook.cpp new file mode 100644 index 00000000..1b57b4da --- /dev/null +++ b/apps/harmony/desktop-port/keyboard_hook.cpp @@ -0,0 +1,115 @@ +#include "keyboard_hook.h" +#include "key_mapping.h" +#include "keyboard_state.h" +#include "windows_key_metadata.h" +#include + +namespace crossdesk::harmony::desktop { +namespace { +std::mutex lifecycle, delivery; +KeyboardHook* owner = nullptr; +OnKeyAction sink = nullptr; +void* sink_context = nullptr; +int focused_window = 0, capture_window = 0; +bool installed = false, permission = false; +PressedKeys pressed; + +void ReleaseCaptured() { + if (sink) for (const auto& key : ReconcileKeyboardState(pressed, {})) + sink(key.key_code, false, key.scan_code, key.extended, sink_context); + pressed.clear(); +} +} +void SetKeyboardFocus(int window, bool active) { + std::lock_guard lock(delivery); + if (active) focused_window = window; + else if (focused_window == window) focused_window = 0; + if (capture_window != focused_window) ReleaseCaptured(); +} +void SetKeyboardHookPermission(bool granted) { + std::lock_guard lock(delivery); + permission = granted; + if (!granted) ReleaseCaptured(); +} +bool KeyboardHookPermissionGranted() { std::lock_guard lock(delivery); return permission; } +KeyboardHook::~KeyboardHook() { Stop(); } +int KeyboardHook::Start(OnKeyAction callback, void* context) { + if (!callback) return INPUT_PARAMETER_ERROR; + std::lock_guard operation(lifecycle); + int target_window = 0; + { + std::lock_guard lock(delivery); + if (owner) return owner == this ? INPUT_SUCCESS : INPUT_REPEAT_INTERCEPTOR; + if (!focused_window) return INPUT_APP_NOT_FOCUSED; + target_window = focused_window; + } + // A failed removal leaves a pass-through callback installed. Retry removal + // before registering again; never interpret REPEAT_INTERCEPTOR as success. + if (installed) { + const auto result = OH_Input_RemoveKeyEventHook(Receive); + if (result != INPUT_SUCCESS) return result; + installed = false; + } + const auto result = OH_Input_AddKeyEventHook(Receive); + if (result != INPUT_SUCCESS) return result; + installed = true; + { + std::lock_guard lock(delivery); + owner = this; sink = callback; sink_context = context; + capture_window = target_window; permission = true; pressed.clear(); + } + return INPUT_SUCCESS; +} +int KeyboardHook::Stop() { + std::lock_guard operation(lifecycle); + { + std::lock_guard lock(delivery); + if (owner && owner != this) return INPUT_SUCCESS; + // Wait for an in-flight delivery, then detach the context before invoking + // the OS. Failed removal or a queued callback cannot access a dead owner. + owner = nullptr; sink = nullptr; sink_context = nullptr; + capture_window = 0; pressed.clear(); + } + if (!installed) return INPUT_SUCCESS; + const auto result = OH_Input_RemoveKeyEventHook(Receive); + if (result == INPUT_SUCCESS) installed = false; + return result; +} +bool KeyboardHook::Active() const { + std::lock_guard lock(delivery); + return owner == this && permission && capture_window != 0 && capture_window == focused_window; +} +void KeyboardHook::Receive(const Input_KeyEvent* event) { + if (!event) return; + bool consumed = false; + { + std::lock_guard lock(delivery); + if (owner && sink && permission && capture_window != 0 && capture_window == focused_window) { + const auto action = OH_Input_GetKeyEventAction(event); + const int code = OH_Input_GetKeyEventKeyCode(event); + const int key = ToVirtualKey(code); + if (action == KEY_ACTION_CANCEL) { + consumed = !pressed.empty(); ReleaseCaptured(); + } else if (key && (action == KEY_ACTION_DOWN || action == KEY_ACTION_UP)) { + const bool down = action == KEY_ACTION_DOWN; + // Do not swallow a release whose press belonged to the local app. + if (down || pressed.count(key)) { + uint32_t scan = 0; bool extended = false; + LookupWindowsKeyMetadataFromVk(key, &scan, &extended); + if (code == 2119) { scan = 0x1C; extended = true; } + if (down) pressed[key] = {key, scan, extended}; + else pressed.erase(key); + sink(key, down, scan, extended, sink_context); consumed = true; + } + } + } + } + // The SDK owns event storage. Return unhandled events in original order, + // without reinjecting them (which would re-enter the hook). + if (!consumed) { + int32_t id = 0; + if (OH_Input_GetKeyEventId(event, &id) == INPUT_SUCCESS) + OH_Input_DispatchToNextHandler(id); + } +} +} diff --git a/apps/harmony/desktop-port/keyboard_hook.h b/apps/harmony/desktop-port/keyboard_hook.h new file mode 100644 index 00000000..c58be81c --- /dev/null +++ b/apps/harmony/desktop-port/keyboard_hook.h @@ -0,0 +1,31 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _KEYBOARD_HOOK_H_ +#define _KEYBOARD_HOOK_H_ + +#include "device_controller.h" +#include + +namespace crossdesk::harmony::desktop { +// Process-wide OS hook; the shared SessionDeviceManager owns when it runs. +// ArkUI updates focus immediately, even while the Slint thread waits for an OS +// service. A late callback must never capture input for another application. +void SetKeyboardFocus(int window, bool active); +void SetKeyboardHookPermission(bool granted); +bool KeyboardHookPermissionGranted(); +class KeyboardHook { + public: + ~KeyboardHook(); + int Start(OnKeyAction callback, void* context); + int Stop(); + bool Active() const; + private: + static void Receive(const Input_KeyEvent* event); +}; +} + +#endif diff --git a/apps/harmony/desktop-port/os-services.patch b/apps/harmony/desktop-port/os-services.patch new file mode 100644 index 00000000..731ca524 --- /dev/null +++ b/apps/harmony/desktop-port/os-services.patch @@ -0,0 +1,118 @@ +--- a/api/cpp/platform.rs ++++ b/api/cpp/platform.rs +@@ -47,6 +47,11 @@ + } + + impl WindowAdapter for CppWindowAdapter { ++ #[cfg(target_env = "ohos")] ++ fn internal(&self, _: i_slint_core::InternalToken) -> Option<&dyn i_slint_core::window::WindowAdapterInternal> { ++ Some(self) ++ } ++ + fn window(&self) -> &Window { + &self.window + } +@@ -186,6 +191,103 @@ + } + } + ++ ++// Harmony's native editor and cursor services are provided by the C++ host. ++// Keep this extension target-local; the public C++ adapter ABI stays unchanged. ++#[cfg(target_env = "ohos")] ++mod harmony_services { ++ use super::*; ++ use core::cell::Cell; ++ use i_slint_core::window::{InputMethodRequest, WindowAdapterInternal, WindowInner}; ++ type HarmonyOsCallback = unsafe extern "C" fn(*mut c_void, *const c_void, i32, ++ *const u8, usize, usize, usize, f32, f32, f32, f32, i32); ++ std::thread_local! { static CALLBACK: Cell> = const { Cell::new(None) }; } ++ #[unsafe(no_mangle)] ++ pub extern "C" fn crossdesk_slint_set_os_callback(callback: HarmonyOsCallback) { ++ CALLBACK.set(Some(callback)); ++ } ++ impl WindowAdapterInternal for CppWindowAdapter { ++ fn input_method_request(&self, request: InputMethodRequest) { ++ let Some(callback) = CALLBACK.get() else { return }; ++ let (kind, properties) = match request { ++ InputMethodRequest::Enable(p) => (1, Some(p)), ++ InputMethodRequest::Update(p) => (2, Some(p)), ++ InputMethodRequest::Disable => (0, None), ++ _ => return, ++ }; ++ let p = properties.unwrap_or_default(); ++ let cursor = p.text[..p.cursor_position].encode_utf16().count(); ++ let anchor = p.text[..p.anchor_position.unwrap_or(p.cursor_position)].encode_utf16().count(); ++ unsafe { callback(self.user_data, self as *const Self as *const c_void, kind, p.text.as_ptr(), p.text.len(), cursor, anchor, ++ p.cursor_rect_origin.x, p.cursor_rect_origin.y, p.cursor_rect_size.width, ++ p.cursor_rect_size.height, p.input_type as i32); } ++ } ++ fn set_mouse_cursor(&self, cursor: i_slint_core::items::MouseCursor) { ++ if let Some(callback) = CALLBACK.get() { ++ unsafe { callback(self.user_data, self as *const Self as *const c_void, 3, core::ptr::null(), 0, 0, 0, ++ 0., 0., 0., 0., cursor as i32); } ++ } ++ } ++ } ++ // IME commands target the editor that owns the native input session. Use ++ // Slint's normal key handling (selection, clipboard, validation and undo) ++ // without mutating the window's physical modifier state. ++ #[unsafe(no_mangle)] ++ pub unsafe extern "C" fn crossdesk_slint_edit(adapter: *const c_void, ++ text: *const u8, length: usize, modifiers: i32) { ++ use i_slint_core::input::{InternalKeyEvent, KeyEvent, KeyEventType}; ++ let Ok(text) = core::str::from_utf8(unsafe { core::slice::from_raw_parts(text, length) }) else { return }; ++ let window = WindowInner::from_pub(&unsafe { &*(adapter as *const CppWindowAdapter) }.window); ++ let Some(item) = window.focus_item.borrow().upgrade() else { return }; ++ if item.downcast::().is_none() { return; } ++ let mut key_event = KeyEvent::default(); ++ key_event.text = text.into(); ++ key_event.modifiers.shift = modifiers & 1 != 0; ++ key_event.modifiers.control = modifiers & 2 != 0; ++ for event_type in [KeyEventType::KeyPressed, KeyEventType::KeyReleased] { ++ item.borrow().as_ref().key_event(&InternalKeyEvent { ++ key_event: key_event.clone(), event_type, ..Default::default() ++ }, &window.window_adapter(), &item); ++ } ++ } ++ #[unsafe(no_mangle)] ++ pub unsafe extern "C" fn crossdesk_slint_selection(adapter: *const c_void, anchor: i32, cursor: i32) { ++ let window = WindowInner::from_pub(&unsafe { &*(adapter as *const CppWindowAdapter) }.window); ++ let Some(item) = window.focus_item.borrow().upgrade() else { return }; ++ let Some(input) = item.downcast::() else { return }; ++ let input = input.as_pin_ref(); ++ let text = input.text(); ++ // Harmony uses UTF-16 indices; Slint uses UTF-8 byte offsets. Clamp ++ // malformed ranges to character boundaries, including surrogate pairs. ++ let byte_offset = |offset: i32| -> i32 { ++ let mut utf16 = 0; ++ for (byte, ch) in text.char_indices() { ++ if utf16 + ch.len_utf16() > offset.max(0) as usize { return byte as i32; } ++ utf16 += ch.len_utf16(); ++ } ++ text.len() as i32 ++ }; ++ input.set_selection_offsets(&window.window_adapter(), &item, byte_offset(anchor), byte_offset(cursor)); ++ } ++ // Called only on the Slint thread while the owning C++ window is alive. ++ #[unsafe(no_mangle)] ++ pub unsafe extern "C" fn crossdesk_slint_composition(adapter: *const c_void, ++ commit: bool, text: *const u8, length: usize) { ++ use i_slint_core::input::{InternalKeyEvent, KeyEvent, KeyEventType}; ++ let Ok(text) = core::str::from_utf8(unsafe { core::slice::from_raw_parts(text, length) }) else { return }; ++ let mut key_event = KeyEvent::default(); ++ key_event.text = text.into(); ++ let event = if commit { ++ InternalKeyEvent { event_type: KeyEventType::CommitComposition, ++ key_event, ..Default::default() } ++ } else { ++ InternalKeyEvent { event_type: KeyEventType::UpdateComposition, preedit_text: text.into(), ++ preedit_selection: Some(text.len() as i32..text.len() as i32), ..Default::default() } ++ }; ++ WindowInner::from_pub(&unsafe { &*(adapter as *const CppWindowAdapter) }.window).process_key_input(event); ++ } ++} ++ + type PlatformUserData = *mut c_void; + + struct CppPlatform { diff --git a/apps/harmony/desktop-port/pointer.h b/apps/harmony/desktop-port/pointer.h new file mode 100644 index 00000000..68d7b790 --- /dev/null +++ b/apps/harmony/desktop-port/pointer.h @@ -0,0 +1,34 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _POINTER_H_ +#define _POINTER_H_ + +#include +#include + +namespace crossdesk::harmony::desktop { +// Preserve each button's last position. Cancellation/focus loss must release +// every held button, while merely leaving the surface must preserve a drag. +class PointerState { + std::array pressed_{}; + float x_ = 0, y_ = 0; + public: + void Update(int kind, int button, float x, float y) { + x_ = x; y_ = y; + if (button >= 0 && button < 3 && (kind == 0 || kind == 1)) pressed_[button] = kind == 0; + } + unsigned Buttons() const { + return (pressed_[0] ? 1u : 0u) | (pressed_[1] ? 4u : 0u) | (pressed_[2] ? 2u : 0u); + } + template void Cancel(Release release) { + const auto pressed = std::exchange(pressed_, {}); + for (int i = 0; i < 3; ++i) if (pressed[i]) release(i, x_, y_); + } +}; +} + +#endif diff --git a/apps/harmony/desktop-port/services.cpp b/apps/harmony/desktop-port/services.cpp new file mode 100644 index 00000000..d34c36d2 --- /dev/null +++ b/apps/harmony/desktop-port/services.cpp @@ -0,0 +1,181 @@ +#include "bridge.h" +#include "displays.h" +#include "SDL3/SDL.h" +#include "audio.h" +#include "platform/path_backend.h" +#include "platform/autostart_backend.h" +#include "platform/harmony/harmony_autostart.h" +#include "platform.h" +#include +#include +#include +#include +#include +#include +#include + +using namespace crossdesk::harmony::desktop; +namespace { +std::mutex event_mutex; +std::deque events; +thread_local std::string service_error; +std::atomic autostart_status{crossdesk::HarmonyAutostartStatus::checking}; +bool AudioFormat(const SDL_AudioSpec* spec) { + return spec && spec->freq == 48000 && spec->channels == 1 && spec->format == SDL_AUDIO_S16; +} +} +bool SDL_Init(uint32_t) { return true; } +bool SDL_InitSubSystem(uint32_t) { return true; } +void SDL_Quit() { std::lock_guard lock(event_mutex); events.clear(); } +void SDL_QuitSubSystem(uint32_t) {} +const char* SDL_GetError() { return service_error.c_str(); } +uint64_t SDL_GetTicks() { + static const auto start = std::chrono::steady_clock::now(); + return std::chrono::duration_cast(std::chrono::steady_clock::now() - start).count(); +} +void SDL_Delay(uint32_t delay) { std::this_thread::sleep_for(std::chrono::milliseconds(delay)); } +bool SDL_PushEvent(SDL_Event* event) { + if (!event) return false; + std::lock_guard lock(event_mutex); events.push_back(*event); return true; +} +bool SDL_PollEvent(SDL_Event* event) { return SDL_PeepEvents(event, 1, SDL_GETEVENT, 0, UINT32_MAX) == 1; } +int SDL_PeepEvents(SDL_Event* out, int count, int, uint32_t first, uint32_t last) { + std::lock_guard lock(event_mutex); int copied = 0; + for (auto it = events.begin(); it != events.end() && copied < count;) { + if (it->type >= first && it->type <= last) { out[copied++] = *it; it = events.erase(it); } + else ++it; + } + return copied; +} +char* SDL_GetClipboardText() { + auto result = Request({{"op", "clipboardRead"}}); + if (!result.value("ok", false)) { + service_error = result.value("message", "Harmony clipboard read failed"); + return nullptr; + } + return strdup(result.value("text", "").c_str()); +} +bool SDL_SetClipboardText(const char* text) { + auto result = Request({{"op", "clipboardWrite"}, {"text", text ? text : ""}}); + if (result.value("ok", false)) return true; + service_error = result.value("message", "Harmony clipboard write failed"); + return false; +} +bool SDL_HasClipboardText() { return Request({{"op", "clipboardHas"}}).value("ok", false); } +const char* SDL_GetCurrentVideoDriver() { return "harmony"; } +const SDL_DisplayMode* SDL_GetCurrentDisplayMode(SDL_DisplayID id) { + static thread_local SDL_DisplayMode mode; + for (const auto& d : ReadDisplays(Environment())) if (d.SdlId() == id) { + mode = {d.width, d.height}; return &mode; + } + return nullptr; +} +SDL_DisplayID SDL_GetPrimaryDisplay() { + const auto displays = ReadDisplays(Environment()); + return displays.empty() ? 0 : displays.front().SdlId(); +} +SDL_DisplayID SDL_GetDisplayForPoint(const SDL_Point* point) { + if (!point) return 0; + const auto displays = ReadDisplays(Environment()); + const auto* found = DisplayAt(displays, point->x, point->y); + return found ? found->SdlId() : 0; +} +SDL_DisplayID* SDL_GetDisplays(int* count) { + const auto displays = ReadDisplays(Environment()); + auto* ids = static_cast(std::calloc(displays.size() + 1, sizeof(SDL_DisplayID))); + if (count) *count = ids ? int(displays.size()) : 0; + if (ids) for (size_t i = 0; i < displays.size(); ++i) ids[i] = displays[i].SdlId(); + return ids; +} +bool SDL_GetDisplayUsableBounds(SDL_DisplayID id, SDL_Rect* bounds) { + if (!bounds) return false; + for (const auto& d : ReadDisplays(Environment())) if (d.SdlId() == id) { + *bounds = {d.usable_x, d.usable_y, d.usable_width, d.usable_height}; return true; + } + return false; +} +uint32_t SDL_GetGlobalMouseState(float* x, float* y) { + auto env = Environment(); + double px = env.value("pointerX", 0.), py = env.value("pointerY", 0.); + // Touch-only devices and unfocused apps may not expose a global pointer. + // Keep the last window event coordinates intact and retry after hot-plug or + // focus changes without issuing a failing system call every render frame. + static thread_local std::chrono::steady_clock::time_point retry_after; + const auto now = std::chrono::steady_clock::now(); + if (now >= retry_after) { + int32_t display = 0; + double native_x = px, native_y = py; + if (OH_Input_GetPointerLocation(&display, &native_x, &native_y) == INPUT_SUCCESS) { + px = native_x; py = native_y; + for (const auto& d : ReadDisplays(env)) if (d.id == display) { px += d.x; py += d.y; break; } + } else retry_after = now + std::chrono::seconds(1); + } + if (x) *x = px; + if (y) *y = py; + return env.value("buttons", 0u); +} +bool SDL_OpenURL(const char* url) { return Request({{"op", "openUrl"}, {"url", url}}).value("ok", false); } +struct SDL_AudioStream { crossdesk::harmony::AudioPlayer player; }; +uint32_t SDL_OpenAudioDevice(uint32_t, const SDL_AudioSpec* spec) { + if (AudioFormat(spec)) return 1; + service_error = "Harmony playback requires 48 kHz mono PCM16"; return 0; +} +bool SDL_ResumeAudioDevice(uint32_t id) { return id == 1; } +void SDL_CloseAudioDevice(uint32_t) {} +SDL_AudioStream* SDL_CreateAudioStream(const SDL_AudioSpec* in, const SDL_AudioSpec* out) { + if (!AudioFormat(in) || !AudioFormat(out)) return nullptr; + return new SDL_AudioStream(); +} +void SDL_DestroyAudioStream(SDL_AudioStream* stream) { delete stream; } +bool SDL_BindAudioStream(uint32_t id, SDL_AudioStream* stream) { + if (id == 1 && stream && stream->player.Start()) return true; + service_error = "OH_AudioRenderer initialization failed"; return false; +} +bool SDL_PutAudioStreamData(SDL_AudioStream* stream, const void* data, int size) { + if (!stream || !data || size <= 0) return false; + stream->player.Push(static_cast(data), size); return true; +} +const char* tinyfd_selectFolderDialog(const char* title, const char* path) { + static thread_local std::string result; + result = Request({{"op", "selectFolder"}, {"title", title ? title : ""}, + {"path", path ? path : ""}}).value("path", ""); + return result.empty() ? nullptr : result.c_str(); +} +const char* tinyfd_openFileDialog(const char* title, const char*, int, const char* const*, const char*, int) { + static thread_local std::string result; + result = Request({{"op", "selectFile"}, {"title", title ? title : ""}}).value("path", ""); + return result.empty() ? nullptr : result.c_str(); +} +namespace crossdesk { +HarmonyAutostartStatus GetHarmonyAutostartStatus() { return autostart_status.load(); } +void SetHarmonyAutostartStatus(HarmonyAutostartStatus status) { autostart_status = status; } +void RefreshHarmonyAutostartStatus() { + autostart_status = HarmonyAutostartStatus::checking; + Emit({{"op", "autostartRefresh"}}); +} +std::string GetHostName() { return Environment().value("name", "Harmony PC"); } +std::string GetMac() { return {}; } // App sandbox does not expose a hardware MAC. +bool IsWaylandSession() { return false; } +namespace platform { +std::filesystem::path GetExecutableDirectory() { return Environment().value("filesDir", ""); } +std::filesystem::path GetConfigPath(const std::string&) { return GetCachePath(""); } +std::filesystem::path GetCachePath(const std::string&) { + auto path = GetExecutableDirectory() / "desktop"; + std::filesystem::create_directories(path); return path; +} +std::filesystem::path GetLogPath(const std::string&) { + auto path = GetCachePath("") / "logs"; + std::filesystem::create_directories(path); return path; +} +std::string GetAutostartExecutablePath() { return {}; } +bool IsAutostartEnabled(const std::string&) { + return GetHarmonyAutostartStatus() == HarmonyAutostartStatus::enabled; +} +// The public Harmony API can query this setting, but only the user can change +// it in system settings. Never report a registration change we did not make. +bool EnableAutostart(const std::string& app, const std::string&) { return IsAutostartEnabled(app); } +bool DisableAutostart(const std::string& app) { + return GetHarmonyAutostartStatus() == HarmonyAutostartStatus::disabled; +} +} +} diff --git a/apps/harmony/desktop-port/surface.inc b/apps/harmony/desktop-port/surface.inc new file mode 100644 index 00000000..e7239edb --- /dev/null +++ b/apps/harmony/desktop-port/surface.inc @@ -0,0 +1,124 @@ +// GLES presentation only; all UI and callbacks belong to GuiApplication. +struct Surface { + DesktopWindow& owner; + explicit Surface(DesktopWindow& window) : owner(window) {} + ~Surface() { DetachSurface(); } +EGLDisplay display = EGL_NO_DISPLAY; +EGLContext context = EGL_NO_CONTEXT; +EGLSurface surface = EGL_NO_SURFACE; +OHNativeWindow* native_window = nullptr; +GLuint program = 0, texture = 0, vao = 0; +float density = 1; +std::vector pixels; +uint64_t frames = 0; +uint32_t rendered_width = 0, rendered_height = 0; +void DetachSurface() { + if (display != EGL_NO_DISPLAY) { + if (surface != EGL_NO_SURFACE && context != EGL_NO_CONTEXT) { + eglMakeCurrent(display, surface, surface, context); + glDeleteTextures(1, &texture); glDeleteProgram(program); glDeleteVertexArrays(1, &vao); + } + eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + if (surface != EGL_NO_SURFACE) eglDestroySurface(display, surface); + if (context != EGL_NO_CONTEXT) eglDestroyContext(display, context); + // EGL display is shared by the application windows. + } + if (native_window) OH_NativeWindow_DestroyNativeWindow(native_window); + display = EGL_NO_DISPLAY; context = EGL_NO_CONTEXT; surface = EGL_NO_SURFACE; + native_window = nullptr; program = texture = vao = 0; + // ReusedBuffer's retained contents belong to the Slint window, not the EGL + // surface. Keep them when a hidden window gets a new XComponent; otherwise + // Slint repaints only dirty regions over a cleared (black) buffer. + frames = 0; + rendered_width = rendered_height = 0; +} + +GLuint Shader(GLenum type, const char* source) { + GLuint value = glCreateShader(type); + glShaderSource(value, 1, &source, nullptr); glCompileShader(value); + GLint ok = 0; glGetShaderiv(value, GL_COMPILE_STATUS, &ok); + if (!ok) { + char log[1024]{}; glGetShaderInfoLog(value, sizeof(log), nullptr, log); + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Shader failed: %{public}s", log); + glDeleteShader(value); return 0; + } + return value; +} + +bool AttachSurface(uint64_t id, float scale) { + DetachSurface(); + if (!id || !std::isfinite(scale) || scale <= 0) return false; + owner.SetDensity(scale); + density = scale; + if (OH_NativeWindow_CreateNativeWindowFromSurfaceId(id, &native_window) != 0) return false; + display = eglGetDisplay(EGL_DEFAULT_DISPLAY); + if (!eglInitialize(display, nullptr, nullptr) || !eglBindAPI(EGL_OPENGL_ES_API)) return false; + const EGLint attributes[] = {EGL_SURFACE_TYPE, EGL_WINDOW_BIT, EGL_RENDERABLE_TYPE, + EGL_OPENGL_ES3_BIT, EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_NONE}; + EGLConfig config; EGLint count = 0; + if (!eglChooseConfig(display, attributes, &config, 1, &count) || count == 0) return false; + const EGLint context_attributes[] = {EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE}; + context = eglCreateContext(display, config, EGL_NO_CONTEXT, context_attributes); + surface = eglCreateWindowSurface(display, config, reinterpret_cast(native_window), nullptr); + if (context == EGL_NO_CONTEXT || surface == EGL_NO_SURFACE || + !eglMakeCurrent(display, surface, surface, context)) return false; + const char* vertex = "#version 300 es\n" + "out vec2 uv; void main(){ vec2 p=vec2((gl_VertexID<<1)&2,gl_VertexID&2);" + "uv=vec2(p.x,1.-p.y); gl_Position=vec4(p*2.-1.,0.,1.);}"; + const char* fragment = "#version 300 es\nprecision mediump float;" + "in vec2 uv; uniform sampler2D frame; out vec4 color; void main(){color=texture(frame,uv);}"; + GLuint v = Shader(GL_VERTEX_SHADER, vertex), f = Shader(GL_FRAGMENT_SHADER, fragment); + if (!v || !f) { glDeleteShader(v); glDeleteShader(f); return false; } + program = glCreateProgram(); glAttachShader(program, v); glAttachShader(program, f); + glLinkProgram(program); glDeleteShader(v); glDeleteShader(f); + GLint linked = 0; glGetProgramiv(program, GL_LINK_STATUS, &linked); + if (!linked) return false; + glGenVertexArrays(1, &vao); glBindVertexArray(vao); + glGenTextures(1, &texture); glBindTexture(GL_TEXTURE_2D, texture); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Surface attached scale=%{public}.3f", density); + return true; +} + +void Render() { + if (surface == EGL_NO_SURFACE) return; + EGLint width = 0, height = 0; + eglQuerySurface(display, surface, EGL_WIDTH, &width); + eglQuerySurface(display, surface, EGL_HEIGHT, &height); + if (width <= 0 || height <= 0 || width > 8192 || height > 8192) return; + if (rendered_width != uint32_t(width) || rendered_height != uint32_t(height)) { + rendered_width = width; rendered_height = height; + owner.extent = slint::PhysicalSize({uint32_t(width), uint32_t(height)}); + owner.window().dispatch_resize_event(slint::LogicalSize({width / density, height / density})); + pixels.resize(size_t(width) * height); owner.dirty = true; + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Window %{public}d resize %{public}dx%{public}d", owner.id, width, height); + } + if (!owner.dirty && !owner.window().has_active_animations()) return; + if (pixels.size() != size_t(width) * height) pixels.resize(size_t(width) * height); + const auto started = std::chrono::steady_clock::now(); + owner.dirty = false; + owner.software.render(pixels, width); + if (!eglMakeCurrent(display, surface, surface, context)) return; + glViewport(0, 0, width, height); glUseProgram(program); + glActiveTexture(GL_TEXTURE0); glBindTexture(GL_TEXTURE_2D, texture); + glPixelStorei(GL_UNPACK_ALIGNMENT, 1); + glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB8, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, pixels.data()); + glUniform1i(glGetUniformLocation(program, "frame"), 0); + glBindVertexArray(vao); glDrawArrays(GL_TRIANGLES, 0, 3); + const GLenum error = glGetError(); + const bool swapped = eglSwapBuffers(display, surface); + eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + if (error || !swapped) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Present failed gl=%{public}x egl=%{public}x", error, eglGetError()); + } else if (++frames == 1 || frames % 120 == 0) { + double ms = std::chrono::duration(std::chrono::steady_clock::now() - started).count(); + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Presented frames=%{public}llu frame_ms=%{public}.2f", (unsigned long long)frames, ms); + } +} + +}; diff --git a/apps/harmony/desktop-port/tray.cpp b/apps/harmony/desktop-port/tray.cpp new file mode 100644 index 00000000..8c460bba --- /dev/null +++ b/apps/harmony/desktop-port/tray.cpp @@ -0,0 +1,39 @@ +#include "platform/harmony/gui/tray/harmony_tray.h" +#include "bridge.h" +#include "localization.h" +#include "rd_log.h" + +namespace crossdesk { +namespace { HarmonyTray* current = nullptr; } +HarmonyTray::HarmonyTray(std::function show, std::function hide, + std::function settings, std::function exit, + const std::string& tooltip, int language) + : show_(std::move(show)), hide_(std::move(hide)), + settings_(std::move(settings)), exit_(std::move(exit)) { + current = this; + language = localization::detail::ClampLanguageIndex(language); + auto result = harmony::desktop::Request({{"op", "trayCreate"}, {"title", tooltip}, + {"show", localization::show_main_window[language]}, + {"settings", localization::settings[language]}, + {"exit", localization::exit_program[language]}}); + active_ = result.value("ok", false); + if (!active_) LOG_WARN("Harmony tray unavailable: {}", result.value("message", "unsupported")); +} +HarmonyTray::~HarmonyTray() { RemoveTrayIcon(); if (current == this) current = nullptr; } +bool HarmonyTray::MinimizeToTray() { + if (!active_) return false; + hide_(); + return true; +} +void HarmonyTray::RemoveTrayIcon() { + if (!active_) return; + active_ = false; + harmony::desktop::Emit({{"op", "trayRemove"}}); +} +void HarmonyTray::Dispatch(const std::string& action) { + if (!current) return; + if (action == "show") current->show_(); + else if (action == "settings") current->settings_(); + else if (action == "exit") current->exit_(); +} +} diff --git a/apps/harmony/desktop-port/window-properties.patch b/apps/harmony/desktop-port/window-properties.patch new file mode 100644 index 00000000..27668dd6 --- /dev/null +++ b/apps/harmony/desktop-port/window-properties.patch @@ -0,0 +1,28 @@ +--- a/api/cpp/platform.rs ++++ b/api/cpp/platform.rs +@@ -98,6 +98,8 @@ + } + + fn update_window_properties(&self, properties: WindowProperties<'_>) { ++ #[cfg(target_env = "ohos")] ++ harmony_services::window_chrome(self); + unsafe { (self.update_window_properties)(self.user_data, &properties) } + } + } +@@ -206,6 +208,16 @@ + pub extern "C" fn crossdesk_slint_set_os_callback(callback: HarmonyOsCallback) { + CALLBACK.set(Some(callback)); + } ++ // Read these properties inside the normal property tracker so subsequent ++ // Slint changes propagate without a separate UI/window state machine. ++ pub(super) fn window_chrome(adapter: &CppWindowAdapter) { ++ let Some(callback) = CALLBACK.get() else { return }; ++ let Some(item) = WindowInner::from_pub(&adapter.window).window_item() else { return }; ++ let item = item.as_pin_ref(); ++ let flags = i32::from(item.no_frame()) | (i32::from(item.always_on_top()) << 1); ++ unsafe { callback(adapter.user_data, adapter as *const _ as *const c_void, 4, ++ core::ptr::null(), 0, 0, 0, 0., 0., 0., 0., flags); } ++ } + impl WindowAdapterInternal for CppWindowAdapter { + fn input_method_request(&self, request: InputMethodRequest) { + let Some(callback) = CALLBACK.get() else { return }; diff --git a/apps/harmony/desktop-port/window_mode.h b/apps/harmony/desktop-port/window_mode.h new file mode 100644 index 00000000..a304b624 --- /dev/null +++ b/apps/harmony/desktop-port/window_mode.h @@ -0,0 +1,38 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _WINDOW_MODE_H_ +#define _WINDOW_MODE_H_ + +#include + +namespace crossdesk::harmony::desktop { +// Slint setters synchronously notify the platform one property at a time. +// A native state snapshot must be acknowledged atomically without echoing +// those intermediate combinations back to the native compositor. +class WindowMode { + nlohmann::json known_ = nlohmann::json::object(); + bool acknowledging_ = false; + public: + nlohmann::json Request(const nlohmann::json& desired) { + auto changes = nlohmann::json::object(); + if (acknowledging_) return changes; + for (const auto& [name, value] : desired.items()) + if (!known_.contains(name) || known_.at(name) != value) changes[name] = value; + known_ = desired; + return changes; + } + template + void Acknowledge(const nlohmann::json& actual, Apply apply) { + known_.update(actual); + struct Reset { bool& flag; ~Reset() { flag = false; } } reset{acknowledging_}; + acknowledging_ = true; + apply(); + } +}; +} + +#endif diff --git a/apps/harmony/entry/build-profile.json5 b/apps/harmony/entry/build-profile.json5 new file mode 100644 index 00000000..bfe9a776 --- /dev/null +++ b/apps/harmony/entry/build-profile.json5 @@ -0,0 +1,18 @@ +{ + "apiType": "stageMode", + "buildOption": { + "externalNativeOptions": { + "path": "./src/main/cpp/CMakeLists.txt", + "arguments": "", + "cppFlags": "-std=c++17", + "abiFilters": [ + "arm64-v8a" + ] + } + }, + "targets": [ + { + "name": "default" + } + ] +} diff --git a/apps/harmony/entry/hvigorfile.ts b/apps/harmony/entry/hvigorfile.ts new file mode 100644 index 00000000..d178d79a --- /dev/null +++ b/apps/harmony/entry/hvigorfile.ts @@ -0,0 +1,2 @@ +import { hapTasks } from '@ohos/hvigor-ohos-plugin'; +export default { system: hapTasks, plugins: [] }; diff --git a/apps/harmony/entry/oh-package-lock.json5 b/apps/harmony/entry/oh-package-lock.json5 new file mode 100644 index 00000000..8c598839 --- /dev/null +++ b/apps/harmony/entry/oh-package-lock.json5 @@ -0,0 +1,19 @@ +{ + "meta": { + "stableOrder": true, + "enableUnifiedLockfile": false + }, + "lockfileVersion": 3, + "ATTENTION": "THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.", + "specifiers": { + "libcrossdesk.so@src/main/cpp/types/libcrossdesk": "libcrossdesk.so@src/main/cpp/types/libcrossdesk" + }, + "packages": { + "libcrossdesk.so@src/main/cpp/types/libcrossdesk": { + "name": "libcrossdesk.so", + "version": "0.1.0", + "resolved": "src/main/cpp/types/libcrossdesk", + "registryType": "local" + } + } +} \ No newline at end of file diff --git a/apps/harmony/entry/oh-package.json5 b/apps/harmony/entry/oh-package.json5 new file mode 100644 index 00000000..7bca0790 --- /dev/null +++ b/apps/harmony/entry/oh-package.json5 @@ -0,0 +1,7 @@ +{ + "name": "entry", + "version": "0.1.0", + "dependencies": { + "libcrossdesk.so": "file:./src/main/cpp/types/libcrossdesk" + } +} diff --git a/apps/harmony/entry/src/main/cpp/CMakeLists.txt b/apps/harmony/entry/src/main/cpp/CMakeLists.txt new file mode 100644 index 00000000..10d20071 --- /dev/null +++ b/apps/harmony/entry/src/main/cpp/CMakeLists.txt @@ -0,0 +1,18 @@ +cmake_minimum_required(VERSION 3.16) +project(crossdesk_harmony LANGUAGES CXX) +set(CMAKE_CXX_STANDARD 17) +get_filename_component(HARMONY_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/../../../.." ABSOLUTE) +set(CORE_LIBRARY "${HARMONY_ROOT}/entry/libs/${OHOS_ARCH}/libcrossdesk_harmony_core.so") +if(NOT EXISTS "${CORE_LIBRARY}") + message(FATAL_ERROR "Build MiniRTC first: python3 apps/harmony/scripts/build.py --mobile --native-only") +endif() +add_library(crossdesk_core SHARED IMPORTED) +set_target_properties(crossdesk_core PROPERTIES + IMPORTED_LOCATION "${CORE_LIBRARY}" + IMPORTED_SONAME "libcrossdesk_harmony_core.so") +add_library(crossdesk SHARED napi_init.cpp) +# HAP libraries are resolved by the app's linker namespace. The host build +# directory is not a useful runtime search path on a HarmonyOS device. +set_target_properties(crossdesk PROPERTIES SKIP_BUILD_RPATH TRUE) +target_include_directories(crossdesk PRIVATE "${HARMONY_ROOT}/native") +target_link_libraries(crossdesk PRIVATE crossdesk_core libace_napi.z.so) diff --git a/apps/harmony/entry/src/main/cpp/napi_init.cpp b/apps/harmony/entry/src/main/cpp/napi_init.cpp new file mode 100644 index 00000000..674e57b5 --- /dev/null +++ b/apps/harmony/entry/src/main/cpp/napi_init.cpp @@ -0,0 +1,37 @@ +#include +#include "mobile_session.h" +#include + +namespace { +napi_value Command(napi_env env, napi_callback_info info) { + size_t count = 1; napi_value argument; + if (napi_get_cb_info(env, info, &count, &argument, nullptr, nullptr) != napi_ok || count != 1) { + napi_throw_type_error(env, nullptr, "Expected a JSON command string"); return nullptr; + } + size_t size = 0; + if (napi_get_value_string_utf8(env, argument, nullptr, 0, &size) != napi_ok || size > 1024 * 1024) { + napi_throw_type_error(env, nullptr, "Invalid command"); return nullptr; + } + std::string value(size + 1, '\0'); + napi_get_value_string_utf8(env, argument, value.data(), value.size(), &size); value.resize(size); + crossdesk::harmony::MobileCommand(value); + napi_value result; napi_get_undefined(env, &result); return result; +} +napi_value Poll(napi_env env, napi_callback_info) { + const std::string data = crossdesk::harmony::MobilePoll(); + napi_value result; + napi_create_string_utf8(env, data.data(), data.size(), &result); return result; +} +napi_value Init(napi_env env, napi_value exports) { + napi_property_descriptor properties[] = { + {"mobileCommand", nullptr, Command, nullptr, nullptr, nullptr, napi_default, nullptr}, + {"mobilePoll", nullptr, Poll, nullptr, nullptr, nullptr, napi_default, nullptr}}; + napi_define_properties(env, exports, sizeof(properties) / sizeof(properties[0]), properties); + napi_add_env_cleanup_hook(env, [](void*) { + crossdesk::harmony::MobileShutdown(); + }, nullptr); + return exports; +} +napi_module module = {1, 0, nullptr, Init, "crossdesk", nullptr, {0}}; +} +extern "C" __attribute__((constructor)) void RegisterCrossDesk() { napi_module_register(&module); } diff --git a/apps/harmony/entry/src/main/cpp/types/libcrossdesk/index.d.ts b/apps/harmony/entry/src/main/cpp/types/libcrossdesk/index.d.ts new file mode 100644 index 00000000..ce9d0ee4 --- /dev/null +++ b/apps/harmony/entry/src/main/cpp/types/libcrossdesk/index.d.ts @@ -0,0 +1,2 @@ +export const mobileCommand: (json: string) => void; +export const mobilePoll: () => string; diff --git a/apps/harmony/entry/src/main/cpp/types/libcrossdesk/oh-package.json5 b/apps/harmony/entry/src/main/cpp/types/libcrossdesk/oh-package.json5 new file mode 100644 index 00000000..7bf930e2 --- /dev/null +++ b/apps/harmony/entry/src/main/cpp/types/libcrossdesk/oh-package.json5 @@ -0,0 +1,5 @@ +{ + "name": "libcrossdesk.so", + "version": "0.1.0", + "types": "index.d.ts" +} diff --git a/apps/harmony/entry/src/main/ets/mobile/AnnouncementInbox.ets b/apps/harmony/entry/src/main/ets/mobile/AnnouncementInbox.ets new file mode 100644 index 00000000..0b5dbd69 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/AnnouncementInbox.ets @@ -0,0 +1,147 @@ +import { announcementByteLength } from './AnnouncementText'; + +export interface AnnouncementRevision { id: number; revision: number; } +export interface AnnouncementItem extends AnnouncementRevision { title: string; body: string; updated_at: number; read?: boolean; } +export interface AnnouncementResponse { + type: string; request_id: string; summary_only: boolean; offset: number; total: number; + catalog_revision: number; items: AnnouncementItem[]; error?: string; +} +export interface AnnouncementRequest { type: string; request_id: string; summary_only: boolean; offset: number; } +interface LocalRevision { id: number; revision: number; } +interface LocalState { scope: string; reads: LocalRevision[]; dismissed: LocalRevision[]; } +// Revisions, pagination, stale replies and local dismissal follow AnnouncementInbox.java. +export class AnnouncementInbox { + items: AnnouncementItem[] = []; + total: number = 0; + unread: number = 0; + loaded: boolean = false; + loading: boolean = false; + failed: boolean = false; + saveFailed: boolean = false; + connected: boolean = false; + changed: () => void = () => {}; + private scope: string = ''; + private reads: Map = new Map(); + private dismissed: Map = new Map(); + private catalog: Map = new Map(); + private pending: Map = new Map(); + private pageIds: Map = new Map(); + private bodies: Map = new Map(); + private revision: number = -1; + private pendingRevision: number = -1; + private pendingTotal: number = 0; + private pageOffset: number = 0; + private limit: number = 20; + private summary: boolean = true; + private requestId: string = ''; + private sequence: number = 0; + private timeout: number = -1; + private send: (request: AnnouncementRequest) => void; + private load: (scope: string) => string; + private save: (scope: string, body: string) => void; + constructor(send: (request: AnnouncementRequest) => void, load: (scope: string) => string, save: (scope: string, body: string) => void) { + this.send = send; this.load = load; this.save = save; + } + configure(scope: string): void { + if (this.scope === scope) return; + this.reset(); this.scope = scope; + try { + const local = JSON.parse(this.load(scope)) as LocalState; + if (local.scope === scope) { + local.reads.forEach((v: LocalRevision) => { if (v.id > 0 && v.revision > 0) this.reads.set(v.id, v.revision); }); + local.dismissed.forEach((v: LocalRevision) => { if (v.id > 0 && v.revision > 0) this.dismissed.set(v.id, v.revision); }); + } + } catch { /* First use or corrupt local metadata. Content always comes from the server. */ } + } + reset(): void { + this.cancel(); this.connected = false; this.scope = ''; this.items = []; this.catalog.clear(); this.pending.clear(); + this.reads.clear(); this.dismissed.clear(); this.pageIds.clear(); this.bodies.clear(); this.revision = this.pendingRevision = -1; + this.total = this.unread = 0; this.limit = 20; this.loaded = this.failed = this.saveFailed = false; this.changed(); + } + connect(value: boolean): void { + if (value === this.connected) return; + this.connected = value; if (value) this.refresh(); else this.cancel(); this.changed(); + } + refresh(): void { this.cancel(); this.pending.clear(); this.pendingRevision = -1; this.pendingTotal = 0; this.summary = true; this.request(0); this.changed(); } + more(): void { if (!this.connected || this.loading || this.failed || this.limit >= this.total) return; this.limit += Math.min(20, this.total - this.limit); this.rebuild(); this.changed(); } + markRead(item: AnnouncementItem): void { + if (!this.items.some((v: AnnouncementItem) => v.id === item.id && v.revision === item.revision)) return; + const next = new Map(this.reads); next.set(item.id, item.revision); + if (this.persist(next, this.dismissed)) { this.reads = next; this.reading(); } this.changed(); + } + dismiss(item: AnnouncementItem): void { + if (!this.items.some((v: AnnouncementItem) => v.id === item.id && v.revision === item.revision)) return; + const next = new Map(this.dismissed); next.set(item.id, item.revision); + if (this.persist(this.reads, next)) { this.dismissed = next; this.cancel(); this.rebuild(); } this.changed(); + } + private persist(reads: Map, dismissed: Map): boolean { + const body: LocalState = { scope: this.scope, reads: [], dismissed: [] }; + reads.forEach((revision: number, id: number) => body.reads.push({ id: id, revision: revision })); + dismissed.forEach((revision: number, id: number) => body.dismissed.push({ id: id, revision: revision })); + try { this.save(this.scope, JSON.stringify(body)); this.saveFailed = false; return true; } + catch { this.saveFailed = true; return false; } + } + fail(id: string): void { if (id && id === this.requestId) { this.cancel(); this.failed = true; this.changed(); } } + receive(message: AnnouncementResponse): void { + if (!this.connected || !this.requestId || message.request_id !== this.requestId) return; + const request = this.requestId; + try { + if (message.type !== 'announcements' || message.error || message.summary_only !== this.summary || + !Number.isSafeInteger(message.offset) || !Number.isSafeInteger(message.total) || !Number.isSafeInteger(message.catalog_revision) || + message.offset < 0 || message.total < message.offset || message.total > 1000000 || message.catalog_revision < 0 || + !Array.isArray(message.items) || message.items.length !== Math.min(this.summary ? 200 : 20, message.total - message.offset)) throw new Error('Invalid announcement page'); + const ids: number[] = []; + for (const item of message.items) { + if (!Number.isSafeInteger(item.id) || !Number.isSafeInteger(item.revision) || item.id <= 0 || item.revision <= 0 || ids.includes(item.id)) throw new Error('Invalid revision'); + ids.push(item.id); + } + const expected = this.summary ? this.pendingRevision : this.revision; + if (expected >= 0 && expected !== message.catalog_revision) { this.refresh(); return; } + if (this.summary) { + if (message.offset !== this.pending.size || (this.pendingRevision >= 0 && this.pendingTotal !== message.total)) throw new Error('Invalid offset'); + for (const item of message.items) { if (this.pending.has(item.id)) throw new Error('Duplicate item'); this.pending.set(item.id, item.revision); } + this.pendingRevision = message.catalog_revision; this.pendingTotal = message.total; this.cancel(); + if (this.pending.size < this.pendingTotal) { this.request(this.pending.size); this.changed(); return; } + if (this.revision !== this.pendingRevision) { this.pageIds.clear(); this.bodies.clear(); } + this.catalog = new Map(this.pending); this.revision = this.pendingRevision; + } else { + if (message.offset !== this.pageOffset || message.total !== this.catalog.size) throw new Error('Invalid body page'); + for (const item of message.items) { + if (this.catalog.get(item.id) !== item.revision || typeof item.title !== 'string' || typeof item.body !== 'string' || + announcementByteLength(item.title) > 240 || announcementByteLength(item.body) > 8000 || !Number.isSafeInteger(item.updated_at) || item.updated_at <= 0) throw new Error('Invalid body'); + this.pageIds.forEach((entries: number[], offset: number) => { if (offset !== message.offset && entries.includes(item.id)) throw new Error('Duplicate page'); }); + } + this.cancel(); this.pageIds.set(message.offset, ids); this.bodies.set(message.offset, message.items); + } + this.failed = false; this.rebuild(); this.changed(); + } catch { this.fail(request); } + } + private reading(): void { + this.total = this.unread = 0; + this.catalog.forEach((revision: number, id: number) => { + if (this.dismissed.get(id) === revision) return; + this.total++; if (this.reads.get(id) !== revision) this.unread++; + }); + this.items.forEach((item: AnnouncementItem) => item.read = this.reads.get(item.id) === item.revision); + } + private rebuild(): void { + this.reading(); const wanted = Math.min(this.limit, this.total); const visible: AnnouncementItem[] = []; + this.summary = false; let next = -1; + for (let offset = 0; offset < this.catalog.size && visible.length < wanted; offset += 20) { + const ids = this.pageIds.get(offset); if (!ids) { next = offset; break; } + const selected = ids.filter((id: number) => this.dismissed.get(id) !== this.catalog.get(id)); + if (!selected.length) { this.bodies.delete(offset); continue; } + const body = this.bodies.get(offset); if (!body) { next = offset; break; } + for (const item of body) if (selected.includes(item.id) && visible.length < wanted) visible.push(item); + } + this.items = visible; this.loaded = true; this.reading(); + if (next >= 0) { this.pageOffset = next; this.request(next); } + } + private request(offset: number): void { + if (!this.connected || !this.scope) return; + this.requestId = `hm-${++this.sequence}-${Date.now()}`; this.loading = true; this.failed = false; + const id = this.requestId; this.timeout = setTimeout(() => this.fail(id), 12000); + this.send({ type: 'announcements_list', request_id: id, summary_only: this.summary, offset: offset }); + } + private cancel(): void { if (this.timeout >= 0) clearTimeout(this.timeout); this.timeout = -1; this.requestId = ''; this.loading = false; } +} diff --git a/apps/harmony/entry/src/main/ets/mobile/AnnouncementText.ets b/apps/harmony/entry/src/main/ets/mobile/AnnouncementText.ets new file mode 100644 index 00000000..3e15da57 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/AnnouncementText.ets @@ -0,0 +1,65 @@ +export interface AnnouncementSpan { text: string; url: string; } + +// Same restricted plain-text/link grammar as Android AnnouncementText.java. +export function isAnnouncementURL(address: string): boolean { + return !/[\x00-\x20\x7f\\<>\"]/.test(address) && + /^https?:\/\/(?:[^\/?#]*@)?(?:\[[0-9a-f:.]+\]|[^\/?#:@\[\]]+)(?::[0-9]+)?(?:[\/?#]|$)/i.test(address); +} +function hasScheme(text: string, start: number): boolean { + return /^https?:\/\//i.test(text.substring(start, start + 8)); +} +function urlEnd(text: string, start: number): number { + let parentheses = 0, brackets = 0, end = start; + for (; end < text.length; end++) { + const c = text.charAt(end); + if (c.charCodeAt(0) <= 0x20 || c.charCodeAt(0) === 0x7f || c === '\\' || c === '`' || + '<>\"\'{},。;:!?、()【】《》「」『』“”‘’'.includes(c)) break; + if (c === '(') parentheses++; + if (c === ')') { if (parentheses === 0) break; parentheses--; } + if (c === '[') brackets++; + if (c === ']') { if (brackets === 0) break; brackets--; } + } + return end; +} +export function announcementSpans(body: string): AnnouncementSpan[] { + const text = body.replace(/\r\n/g, '\n'), result: AnnouncementSpan[] = []; + let plain = ''; + for (let position = 0; position < text.length;) { + let label = '', address = '', next = position; + if (text.charAt(position) === '[') { + const labelEnd = text.indexOf(']', position + 1), start = labelEnd + 2; + if (labelEnd > position + 1 && start < text.length && text.charAt(labelEnd + 1) === '(' && + !/[\[\]\r\n]/.test(text.substring(position + 1, labelEnd)) && hasScheme(text, start)) { + const end = urlEnd(text, start), url = text.substring(start, end); + if (text.charAt(end) === ')' && isAnnouncementURL(url)) { + label = text.substring(position + 1, labelEnd); address = url; next = end + 1; + } + } + } + if (!address && hasScheme(text, position)) { + let end = urlEnd(text, position); + while (end > position && '.,;:!?'.includes(text.charAt(end - 1))) end--; + const url = text.substring(position, end); + if (isAnnouncementURL(url)) { label = address = url; next = end; } + } + if (address) { + if (plain) { result.push({ text: plain, url: '' }); plain = ''; } + result.push({ text: label, url: address }); position = next; + } else plain += text.charAt(position++); + } + if (plain) result.push({ text: plain, url: '' }); + return result; +} + +export function announcementByteLength(text: string): number { + let bytes = 0; + for (let i = 0; i < text.length; i++) { + const c = text.charCodeAt(i); + if (c < 0x80) bytes++; + else if (c < 0x800) bytes += 2; + else if (c >= 0xd800 && c <= 0xdbff && i + 1 < text.length && + text.charCodeAt(i + 1) >= 0xdc00 && text.charCodeAt(i + 1) <= 0xdfff) { bytes += 4; i++; } + else bytes += c >= 0xd800 && c <= 0xdfff ? 1 : 3; // Java UTF-8 replaces lone surrogates with '?'. + } + return bytes; +} diff --git a/apps/harmony/entry/src/main/ets/mobile/ControllerModels.ets b/apps/harmony/entry/src/main/ets/mobile/ControllerModels.ets new file mode 100644 index 00000000..b9868661 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/ControllerModels.ets @@ -0,0 +1,250 @@ +// Platform-independent controller rules ported from the Android client. +// Keep gesture geometry, acknowledgement ordering and expiry behavior identical. +export function clamp(value: number, low: number, high: number): number { + return Math.max(low, Math.min(high, value)); +} +export class RemoteViewport { + baseWidth: number = 1; + baseHeight: number = 1; + baseLeft: number = 0; + baseTop: number = 0; + scale: number = 1; + offsetX: number = 0; + offsetY: number = 0; + fit(width: number, height: number, videoWidth: number, videoHeight: number): void { + if (Math.min(width, height, videoWidth, videoHeight) <= 0) return; + const scale = Math.min(width / videoWidth, height / videoHeight); + this.baseWidth = Math.max(1, videoWidth * scale); this.baseHeight = Math.max(1, videoHeight * scale); + this.baseLeft = (width - this.baseWidth) / 2; this.baseTop = (height - this.baseHeight) / 2; + this.constrain(); + } + width(): number { return this.baseWidth * this.scale; } + height(): number { return this.baseHeight * this.scale; } + left(): number { return this.baseLeft + this.baseWidth * (1 - this.scale) / 2 + this.offsetX; } + top(): number { return this.baseTop + this.baseHeight * (1 - this.scale) / 2 + this.offsetY; } + contains(x: number, y: number): boolean { return x >= this.left() && x < this.left() + this.width() && y >= this.top() && y < this.top() + this.height(); } + x(value: number): number { return clamp((value - this.left()) / this.width(), 0, 1); } + y(value: number): number { return clamp((value - this.top()) / this.height(), 0, 1); } + zoom(factor: number, x: number, y: number): void { + if (!Number.isFinite(factor) || factor <= 0) return; + const next = clamp(this.scale * factor, 1, 10), ratio = next / this.scale; + const fx = x - this.baseLeft - this.baseWidth / 2, fy = y - this.baseTop - this.baseHeight / 2; + this.offsetX = fx - (fx - this.offsetX) * ratio; this.offsetY = fy - (fy - this.offsetY) * ratio; + this.scale = next; this.constrain(); + } + pan(x: number, y: number): void { this.offsetX += x; this.offsetY += y; this.constrain(); } + reset(): void { this.scale = 1; this.offsetX = this.offsetY = 0; } + finish(): void { if (this.scale <= 1.001) this.reset(); } + private constrain(): void { + this.offsetX = clamp(this.offsetX, -this.baseWidth * (this.scale - 1) / 2, this.baseWidth * (this.scale - 1) / 2); + this.offsetY = clamp(this.offsetY, -this.baseHeight * (this.scale - 1) / 2, this.baseHeight * (this.scale - 1) / 2); + } +} +export class MouseInput { + x: number = 0.5; + y: number = 0.5; + gesture: boolean = false; + left: boolean = false; + right: boolean = false; + revision: number = 0; + private sender: (x: number, y: number, flag: number, scroll: number) => void; + constructor(sender: (x: number, y: number, flag: number, scroll: number) => void) { this.sender = sender; } + held(): boolean { return this.left || this.right; } + move(x: number, y: number): void { this.x = clamp(x, 0, 1); this.y = clamp(y, 0, 1); this.send(0); } + drag(dx: number, dy: number, width: number, height: number): void { + if (this.held() && width > 0 && height > 0) this.move(this.x + dx / width, this.y + dy / height); + } + gestureLeft(down: boolean): void { + if ((down && this.held()) || this.gesture === down) return; + this.gesture = down; this.send(down ? 1 : 2); + } + button(left: boolean, down: boolean): void { + if ((left ? this.left : this.right) === down) return; + if (down) { this.gestureLeft(false); this.revision++; } + if (left) this.left = down; else this.right = down; + this.send(left ? (down ? 1 : 2) : (down ? 3 : 4)); + } + click(right: boolean = false): void { if (!this.held()) { this.send(right ? 3 : 1); this.send(right ? 4 : 2); } } + scroll(delta: number): void { if (delta !== 0) this.sender(this.x, this.y, 7, delta > 0 ? 1 : -1); } + release(): void { this.gestureLeft(false); this.button(true, false); this.button(false, false); this.revision++; } + private send(flag: number): void { this.sender(this.x, this.y, flag, 0); } +} +export interface VideoValues { quality: number; frameRate: number; preference: number; } +export class VideoSettings { + supported: boolean = false; + pending: boolean = false; + failed: boolean = false; + selection: VideoValues; + applied: VideoValues; + requestId: number = 0; + appliedRequestId: number = 0; + constructor(preference: number = 1, longEdge: number = 1920) { + this.selection = { quality: longEdge > 0 && longEdge <= 1280 ? 0 : 1, frameRate: 30, preference: clamp(preference, 0, 2) }; + this.applied = this.selection; + } + valid(v: VideoValues): boolean { return Number.isInteger(v.quality) && v.quality >= 0 && v.quality <= 2 && + (v.frameRate === 30 || v.frameRate === 60) && Number.isInteger(v.preference) && v.preference >= 0 && v.preference <= 2; } + support(value: boolean): void { this.supported = value; if (!value) { this.selection = this.applied; this.pending = this.failed = false; } } + begin(value: VideoValues): number { + if (!this.supported || !this.valid(value)) return -1; + this.requestId = (this.requestId + 1) >>> 0; this.selection = value; this.pending = true; this.failed = false; return this.requestId; + } + receive(value: VideoValues, id: number, accepted: boolean): void { + if (!this.supported || !this.valid(value) || !Number.isInteger(id) || id < 0 || id > 0xFFFFFFFF || ((this.requestId - id) | 0) < 0) return; + if (accepted && ((id - this.appliedRequestId) | 0) >= 0) { this.applied = value; this.appliedRequestId = id; } + if (id !== this.requestId) return; + this.pending = false; this.failed = !accepted; this.selection = this.applied; + } + expire(id: number): void { if (this.pending && id === this.requestId) { this.pending = false; this.failed = true; this.selection = this.applied; } } + feedback(): string { return !this.supported ? '连接建立后可调整,需被控端支持。' : this.failed ? '调整失败,请重试。' : this.pending ? '等待电脑端确认…' : ''; } +} +export interface Traffic { inbound: number; outbound: number; loss: number; } +export interface NetworkReport { rtt: number; mode: number; srtp: boolean; video: Traffic; audio: Traffic; data: Traffic; total: Traffic; } +export interface FrameSample { time: number; count: number; latency: number; latencyCount: number; } +export class NetworkStatistics { + report?: NetworkReport; + reportTime: number = -Infinity; + rttTime: number = -Infinity; + rtt: number = -1; + width: number = 0; + height: number = 0; + lastFrame: number = 0; + frames: FrameSample[] = []; + receive(report: NetworkReport, now: number): void { + this.report = report; this.reportTime = now; + if (Number.isFinite(report.rtt) && report.rtt >= 0 && report.rtt <= 2000) { + this.rtt = this.rtt >= 0 && now - this.rttTime < 3000 ? this.rtt + 0.25 * (report.rtt - this.rtt) : report.rtt; + this.rttTime = now; + } + } + frame(id: number, latency: number, width: number, height: number, now: number, latencyCount: number = -1): void { + if (id <= this.lastFrame || width <= 0 || height <= 0) return; + const count = Math.min(240, id - this.lastFrame); + this.frames.push({ time: now, count: count, latency: latency, latencyCount: latencyCount < 0 ? count : latencyCount }); + this.lastFrame = id; this.width = width; this.height = height; this.prune(now); + } + reset(): void { this.frames = []; this.width = this.height = 0; } + prune(now: number): void { this.frames = this.frames.filter((f: FrameSample) => now - f.time < 1000).slice(-240); } + fps(now: number): number { this.prune(now); return this.frames.reduce((n: number, f: FrameSample) => n + f.count, 0); } + latency(now: number): number { + this.prune(now); const valid = this.frames.filter((f: FrameSample) => Number.isFinite(f.latency) && f.latency >= 0); + const count = valid.reduce((n: number, f: FrameSample) => n + f.latencyCount, 0); + return count ? valid.reduce((n: number, f: FrameSample) => n + f.latency * f.latencyCount, 0) / count : -1; + } + current(now: number): NetworkReport | undefined { return now - this.reportTime < 3000 ? this.report : undefined; } + mode(now: number): string { const report = this.current(now); return !report ? '—' : report.mode === 0 ? 'P2P 直连' : report.mode === 1 ? 'TURN 中继' : '—'; } + static rate(n: number): string { return !Number.isFinite(n) || n < 0 ? '—' : n >= 1000000 ? `${(n / 1000000).toFixed(1)} Mbps` : n >= 1000 ? `${(n / 1000).toFixed(0)} Kbps` : `${n} bps`; } + static loss(n: number): string { return Number.isFinite(n) && n >= 0 ? `${(n * 100).toFixed(1)}%` : '—'; } + static delay(n: number): string { return !Number.isFinite(n) || n < 0 ? '—' : n < 1 ? '<1 ms' : `${n.toFixed(0)} ms`; } +} +export interface PresenceDevice { id: string; online: boolean; } +export interface PresenceMessage { type: string; devices?: PresenceDevice[]; id?: string; online?: boolean; } +export class RecentPresence { + watched: string[] = []; + connected: boolean = false; + awaiting: boolean = true; + lastRefresh: number = 0; + values: Map = new Map(); + times: Map = new Map(); + private sender: (ids: string[]) => void; + constructor(sender: (ids: string[]) => void) { this.sender = sender; } + watch(ids: string[], now: number): void { + if (JSON.stringify(ids) === JSON.stringify(this.watched)) return; + this.watched = ids; this.values.clear(); this.times.clear(); this.awaiting = true; this.refresh(now); + } + connect(value: boolean, now: number): void { + if (value === this.connected) return; + this.connected = value; this.values.clear(); this.times.clear(); this.awaiting = true; this.refresh(now); + } + online(id: string, now: number): boolean { return this.connected && !this.awaiting && this.values.get(id) === true && now - (this.times.get(id) ?? -Infinity) < 60000; } + receive(message: PresenceMessage, now: number): void { + if (!this.connected) return; + let devices: PresenceDevice[] = []; + if (message.type === 'presence' && Array.isArray(message.devices)) { + this.values.clear(); this.times.clear(); this.awaiting = false; devices = message.devices; + } else if (message.type === 'presence_update' && !this.awaiting && typeof message.id === 'string' && typeof message.online === 'boolean') { + devices = [{ id: message.id, online: message.online }]; + } + devices.forEach((d: PresenceDevice) => { + if (this.watched.includes(d.id) && typeof d.online === 'boolean') { this.values.set(d.id, d.online); this.times.set(d.id, now); } + }); + } + tick(now: number): void { if (this.connected && now - this.lastRefresh >= 30000) this.refresh(now); } + private refresh(now: number): void { if (this.connected) { this.lastRefresh = now; this.sender(this.watched.slice()); } } +} +export function asciiKey(character: string): number { + if (character.length !== 1) return 0; + const c = character.charCodeAt(0); + if (c >= 97 && c <= 122) return c - 32; + if (c >= 65 && c <= 90) return c | 0x100; + if (c >= 48 && c <= 57) return c; + if (c === 32 || c === 9 || c === 13) return c; + if (c === 10) return 13; + const plain = ';=,-./`[\\]\'', shifted = ':+<_>?~{|}"'; + const codes: number[] = [186, 187, 188, 189, 190, 191, 192, 219, 220, 221, 222]; + const index = plain.indexOf(character); if (index >= 0) return codes[index]; + const upper = shifted.indexOf(character); if (upper >= 0) return codes[upper] | 0x100; + const digit = ')!@#$%^&*('.indexOf(character); return digit >= 0 ? (48 + digit) | 0x100 : 0; +} +export interface TouchPoint { x: number; y: number; } +export class TouchInput { + relative: boolean = true; + private active: boolean = false; + private multi: boolean = false; + private moved: boolean = false; + private long: boolean = false; + private pinching: boolean = false; + private revision: number = 0; + private startX: number = 0; + private startY: number = 0; + private lastX: number = 0; + private lastY: number = 0; + private focusX: number = 0; + private focusY: number = 0; + private span: number = 0; + private startSpan: number = 0; + private scrollY: number = 0; + readonly viewport: RemoteViewport; + readonly mouse: MouseInput; + constructor(viewport: RemoteViewport, mouse: MouseInput) { this.viewport = viewport; this.mouse = mouse; } + cancel(): void { this.active = false; this.multi = true; this.mouse.release(); this.viewport.finish(); } + hold(): void { if (this.active && !this.multi && !this.moved && this.revision === this.mouse.revision && !this.mouse.held()) { this.mouse.click(true); this.long = true; } } + touch(points: TouchPoint[], kind: string): void { + if (kind === 'cancel') { this.cancel(); return; } + if (points.length >= 2) { + const x = (points[0].x + points[1].x) / 2, y = (points[0].y + points[1].y) / 2; + const span = Math.hypot(points[1].x - points[0].x, points[1].y - points[0].y); + if (!this.multi) { + this.mouse.gestureLeft(false); this.multi = true; this.startSpan = this.span = span; + this.focusX = x; this.scrollY = this.focusY = y; this.pinching = false; + } else if (kind === 'move' && !this.mouse.held()) { + if (!this.pinching && Math.abs(span - this.startSpan) > 6 && this.viewport.contains(x, y)) this.pinching = true; + if (this.pinching && this.span > 0) this.viewport.zoom(span / this.span, x, y); + if (this.viewport.scale > 1.001 || this.pinching) this.viewport.pan(x - this.focusX, y - this.focusY); + else if (this.viewport.contains(x, y) && Math.abs(y - this.scrollY) > 16) { this.mouse.scroll(y - this.scrollY); this.scrollY = y; } + this.span = span; this.focusX = x; this.focusY = y; + } + return; + } + if (this.multi) { + if (kind === 'up' && points.length === 0) { this.active = false; this.viewport.finish(); } + return; + } + if (kind === 'down' && points.length === 1) { + const p = points[0]; this.active = true; this.moved = this.long = false; this.pinching = false; + this.revision = this.mouse.revision; this.startX = this.lastX = p.x; this.startY = this.lastY = p.y; + if (!this.relative) { this.active = this.viewport.contains(p.x, p.y); if (this.active) this.mouse.move(this.viewport.x(p.x), this.viewport.y(p.y)); } + } else if (kind === 'move' && this.active && points.length) { + const p = points[0]; + if (Math.hypot(p.x - this.startX, p.y - this.startY) > 4) this.moved = true; + if (this.moved && !this.relative && !this.long && this.revision === this.mouse.revision && !this.mouse.held()) this.mouse.gestureLeft(true); + if (this.relative) this.mouse.move(this.mouse.x + (p.x - this.lastX) / this.viewport.width(), this.mouse.y + (p.y - this.lastY) / this.viewport.height()); + else this.mouse.move(this.viewport.x(p.x), this.viewport.y(p.y)); + this.lastX = p.x; this.lastY = p.y; + } else if (kind === 'up') { + if (this.active && !this.moved && !this.long && this.revision === this.mouse.revision) this.mouse.click(); + this.mouse.gestureLeft(false); this.active = false; + } + } + begin(): void { if (!this.active) this.multi = false; } +} diff --git a/apps/harmony/entry/src/main/ets/mobile/EntryAbility.ets b/apps/harmony/entry/src/main/ets/mobile/EntryAbility.ets new file mode 100644 index 00000000..f64f4ff1 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/EntryAbility.ets @@ -0,0 +1,15 @@ +import { UIAbility } from '@kit.AbilityKit'; +import { window } from '@kit.ArkUI'; +import { hilog } from '@kit.PerformanceAnalysisKit'; +import { MobileLifecycle } from './MobilePlatform'; + +export default class EntryAbility extends UIAbility { + onWindowStageCreate(stage: window.WindowStage): void { + stage.loadContent('pages/Mobile', (error) => { + if (error.code) hilog.error(0, 'CrossDesk', 'Unable to load the main page: %{public}d', error.code); + }); + } + onBackground(): void { MobileLifecycle.background(); } + onForeground(): void { MobileLifecycle.foreground(); } + onDestroy(): void { MobileLifecycle.destroy(); } +} diff --git a/apps/harmony/entry/src/main/ets/mobile/MobileController.ets b/apps/harmony/entry/src/main/ets/mobile/MobileController.ets new file mode 100644 index 00000000..9891c6e3 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/MobileController.ets @@ -0,0 +1,450 @@ +import bridge from 'libcrossdesk.so'; +import { common, bundleManager } from '@kit.AbilityKit'; +import { display, promptAction } from '@kit.ArkUI'; +import { util } from '@kit.ArkTS'; +import { pasteboard, systemDateTime } from '@kit.BasicServicesKit'; +import { MobileStore, RecentDevice } from './MobileStore'; +import { MobilePlatform, MobileLifecycle } from './MobilePlatform'; +import { AnnouncementInbox, AnnouncementItem, AnnouncementRequest, AnnouncementResponse } from './AnnouncementInbox'; +import { VideoSettings, VideoValues, NetworkStatistics, NetworkReport, RecentPresence, PresenceMessage, asciiKey } from './ControllerModels'; + +interface Command { + op: string; session?: number; connection?: number; server?: string; port?: number; identity?: string; + logs?: string; certificates?: string; directory?: string; previewDirectory?: string; remote?: string; password?: string; version?: string; + id?: string; x?: number; y?: number; flag?: number; scroll?: number; code?: number; down?: boolean; + index?: number; muted?: boolean; visible?: boolean; text?: string; message?: Object; quality?: number; frameRate?: number; + preference?: number; request?: number; path?: string; name?: string; body?: string; local?: boolean; +} +interface NativeEvent { + type: string; session: number; connection?: number; status?: number; controller?: boolean; generation?: number; + id?: string; identity?: string; message?: Object; report?: NetworkReport; width?: number; height?: number; + frame?: number; latency?: number; latencyCount?: number; text?: string; reason?: string; name?: string; progress?: number; + sending?: boolean; path?: string; ok?: boolean; request?: number | string; latest?: string; +} +export interface RemoteDisplay { name: string; } +interface HostInfo { host_name: string; platform: string; app_version: string; supports_video_settings: boolean; displays: RemoteDisplay[]; } +interface SettingAck { quality: number; frame_rate: number; preference: number; request_id: number; accepted: boolean; } +interface ServiceStatus { available: boolean; } +interface ControlData { type: number; host_info?: HostInfo; video_settings?: SettingAck; service_status?: ServiceStatus; } +interface PresenceRequest { type: string; devices: string[]; subscribe: boolean; } +interface SignalType { type: string; } + +@Observed +export class MobileState { + page: string = 'home'; + consent: boolean = false; + privacyAvailable: boolean = false; + modal: string = ''; + notice: string = ''; + server: string = 'api.crossdesk.cn'; + port: string = '9099'; + remote: string = ''; + password: string = ''; + remember: boolean = false; + signal: number = -1; + signalText: string = '未连接'; + identity: string = ''; + connecting: boolean = false; + ready: boolean = false; + relative: boolean = true; + muted: boolean = false; + previews: boolean = false; + videoPreference: number = 1; + version: string = '0.1.0'; + update: string = ''; + updateStatus: string = ''; + recent: RecentDevice[] = []; + width: number = 16; + height: number = 9; + hasFrame: boolean = false; + displays: RemoteDisplay[] = []; + selectedDisplay: number = 0; + viewportReset: number = 0; + platform: string = ''; + hostName: string = ''; + transfer: string = ''; + receivedPath: string = ''; + receivedName: string = ''; + remoteClipboard: string = ''; + clipboardText: string = ''; + videoQuality: number = 1; + videoRate: number = 30; + videoSelectedPreference: number = 1; + videoSupported: boolean = false; + videoFeedback: string = ''; + networkTime: number = 0; + unread: number = 0; + announcements: AnnouncementItem[] = []; + announcement?: AnnouncementItem; + documentTitle: string = ''; + documentText: string = ''; + documentBack: string = 'about'; +} +export class MobileController { + state: MobileState = new MobileState(); + store: MobileStore; + platform: MobilePlatform; + video: VideoSettings = new VideoSettings(); + network: NetworkStatistics = new NetworkStatistics(); + presence: RecentPresence; + inbox: AnnouncementInbox; + releaseInput: () => void = () => {}; + private foreground: boolean = true; + private context: common.UIAbilityContext; + private timer: number = -1; + private seconds: number = 0; + private session: number = 0; + private connection: number = 0; + private started: boolean = false; + private starting: boolean = false; + private signalGeneration: number = -1; + private pendingPassword: string = ''; + private connectingId: string = ''; + private sendingFile: boolean = false; + private serviceAvailable: boolean = false; + private settingsTimeout: number = -1; + private versionRequest: number = 0; + private localVersionRequest: number = 0; + private remoteVersionRequest: number = 0; + private checkingLocal: boolean = false; + private checkedRemote: boolean = false; + private remoteVersion: string = ''; + private receivedHostInfo: boolean = false; + private previewEligible: boolean = false; + private historySaved: boolean = false; + private previewPending: boolean = false; + private previewSaved: boolean = false; + private previewAttempts: number = 0; + private uiEpoch: number = 0; + private privacy: string = ''; + constructor(context: common.UIAbilityContext) { + this.context = context; this.store = new MobileStore(context); + this.state.consent = this.store.flag('consent'); this.state.server = this.store.text('server', 'api.crossdesk.cn'); + this.state.port = this.store.text('port', '9099'); this.state.remote = this.store.text('lastRemote'); + this.state.relative = this.store.flag('relative', true); this.state.videoPreference = this.store.number('videoPreference', 1); + this.state.previews = this.store.flag('previews'); + try { this.state.version = bundleManager.getBundleInfoForSelfSync(bundleManager.BundleFlag.GET_BUNDLE_INFO_DEFAULT).versionName; } catch {} + try { + this.privacy = util.TextDecoder.create('utf-8').decodeWithStream(context.resourceManager.getRawFileContentSync('PRIVACY.md')); + this.state.privacyAvailable = this.privacy.trim().length > 0; + } catch { this.state.privacyAvailable = false; } + this.platform = new MobilePlatform(context, (reason: string) => this.end(reason), () => this.release(), (text: string) => this.toast(text)); + this.platform.onMute = (muted: boolean) => { if (muted !== this.state.muted) this.mute(); }; + this.platform.onVisibility = (visible: boolean) => { if (this.state.ready) this.send({ op: 'visible', visible: visible }); }; + this.presence = new RecentPresence((ids: string[]) => { + const request: PresenceRequest = { type: 'recent_connections_presence', devices: ids, subscribe: true }; + this.send({ op: 'signal', message: request }); + }); + this.inbox = new AnnouncementInbox((request: AnnouncementRequest) => this.send({ op: 'signal', message: request }), + (scope: string) => this.store.text(`inbox:${scope}`, '{}'), (scope: string, body: string) => this.store.put(`inbox:${scope}`, body)); + this.inbox.changed = () => { this.state.announcements = this.inbox.items.slice(); this.state.unread = this.inbox.unread; }; + MobileLifecycle.foreground = () => this.resume(); MobileLifecycle.background = () => this.background(); MobileLifecycle.destroy = () => this.destroy(); + this.history(); this.timer = setInterval(() => this.poll(), 50); + if (!this.state.consent) this.state.modal = 'consent'; else this.resume(); + } + private send(command: Command): void { bridge.mobileCommand(JSON.stringify(command)); } + scope(): string { return MobileStore.scope(this.state.server, Number(this.state.port)); } + now(): number { return systemDateTime.getUptime(systemDateTime.TimeType.STARTUP, false); } + toast(text: string): void { if (this.foreground) promptAction.showToast({ message: text, duration: 3000 }); else this.state.notice = text; } + async consent(): Promise { + if (!this.state.privacyAvailable) { this.state.notice = '隐私政策无法读取,请重新安装应用'; return; } + try { this.store.put('consent', true); this.state.consent = true; this.state.modal = ''; await this.ensureSignal(); this.checkLocal(); } + catch { this.toast('无法保存联网授权,请重试'); } + } + revoke(): void { + this.end(''); this.stopSignal(); this.state.consent = false; this.state.previews = false; + this.store.put('consent', false); this.store.put('previews', false); this.store.clearPreviews(); this.history(); + this.state.update = ''; this.state.updateStatus = ''; this.state.page = 'home'; this.state.modal = 'consent'; + } + resume(): void { + this.foreground = true; this.platform.resume(); + if (this.state.consent) { this.ensureSignal(); this.checkLocal(); this.inbox.connect(this.state.signal === 1); } + else this.state.modal = 'consent'; + } + background(): void { + this.foreground = false; this.uiEpoch++; this.release(); this.platform.background(); + this.localVersionRequest = this.remoteVersionRequest = 0; this.checkingLocal = false; this.inbox.connect(false); + if (!this.state.ready && !this.platform.choosing) { if (this.state.connecting) this.end(''); this.stopSignal(); } + } + destroy(): void { this.foreground = false; this.end(''); this.stopSignal(); this.platform.dispose(); if (this.timer >= 0) clearInterval(this.timer); this.timer = -1; } + private async ensureSignal(): Promise { + if (!this.foreground || !this.state.consent || this.started || this.starting) return; + this.starting = true; const session = ++this.session, scope = this.scope(); + try { + let identity = await this.store.secret(`identity:${scope}`); + // A public ID alone is not a login credential. Discard legacy incomplete records. + if (identity && !/^[0-9]{9}@.+$/.test(identity)) { await this.store.removeSecret(`identity:${scope}`); identity = ''; } + if (session !== this.session || !this.foreground || !this.state.consent) return; + this.state.signalText = '正在连接…'; this.state.signal = 0; this.started = true; + this.send({ op: 'start', session: session, server: this.state.server, port: Number(this.state.port), identity: identity, + logs: this.context.cacheDir, certificates: '/etc/security/certificates', directory: this.store.directory, + previewDirectory: this.store.previewDirectory, version: this.state.version }); + } catch { this.state.signalText = '安全存储不可用,请解锁设备后重试'; this.state.signal = -1; } + finally { if (session === this.session) this.starting = false; } + } + private stopSignal(): void { + this.session++; this.starting = this.started = false; this.signalGeneration = -1; + this.send({ op: 'stop' }); this.state.signal = -1; this.state.identity = ''; this.state.signalText = '未连接'; + this.presence.connect(false, this.now()); this.inbox.connect(false); this.platform.cancelRequests(); + this.localVersionRequest = this.remoteVersionRequest = 0; this.checkingLocal = false; + } + reconnect(): void { if (!this.state.consent) { this.state.modal = 'consent'; return; } this.stopSignal(); this.ensureSignal(); } + saveSettings(server: string, port: string): void { + server = server.trim(); const number = Number(port); + if (!server || server.length > 253 || /[\s/@?#]/.test(server) || !Number.isInteger(number) || number < 1 || number > 65535) { + this.toast('请输入有效的服务器地址和端口'); return; + } + const changed = server !== this.state.server || port !== this.state.port; + this.store.put('server', server); this.store.put('port', port); this.state.server = server; this.state.port = port; + if (changed) { this.end(''); this.stopSignal(); this.inbox.reset(); this.history(); this.ensureSignal(); } + this.navigate('settings'); + } + navigate(page: string): void { this.state.page = page; this.state.modal = ''; } + history(): void { + const values = this.store.recent(this.scope()), now = this.now(); this.presence?.watch(values.map((d: RecentDevice) => d.id), now); + this.state.recent = values.filter((d: RecentDevice) => this.presence?.online(d.id, now)) + .concat(values.filter((d: RecentDevice) => !this.presence?.online(d.id, now))); + } + async deleteRecent(id: string): Promise { + try { await this.store.delete(this.scope(), id); this.history(); } catch { this.toast('记录或密码删除失败,请重试'); } + } + async requestConnection(id: string): Promise { + id = id.replace(/\s/g, ''); this.state.remote = id; + if (!/^[0-9]{9}$/.test(id)) { this.toast('请输入 9 位设备 ID'); return; } + if (!this.state.consent) { this.state.modal = 'consent'; return; } + const epoch = ++this.uiEpoch, scope = this.scope(); + this.state.password = ''; this.state.notice = ''; + this.state.remember = this.store.recent(scope).find((d: RecentDevice) => d.id === id)?.remember ?? false; + try { + const password = await this.store.secret(`password:${scope}:${id}`); + if (epoch !== this.uiEpoch || !this.foreground || !this.state.consent) return; + if (password) { this.state.remember = true; this.begin(password); return; } + } catch { if (epoch === this.uiEpoch) this.state.notice = '已保存密码无法读取,请重新输入'; } + if (epoch === this.uiEpoch) this.state.modal = 'password'; + } + begin(password: string): void { + if (!this.state.consent || !/^[0-9]{9}$/.test(this.state.remote) || !password || password.length > 128 || password.includes('@')) { this.toast('请输入有效的访问密码'); return; } + this.joinSent = false; this.connection++; this.connectingId = this.state.remote; this.pendingPassword = password; + this.state.connecting = true; this.state.ready = false; this.state.modal = 'connecting'; this.state.notice = ''; + this.state.muted = false; this.state.platform = ''; this.state.hostName = ''; this.state.receivedPath = this.state.receivedName = this.state.transfer = ''; + this.state.remoteClipboard = ''; this.state.displays = []; this.state.width = 16; this.state.height = 9; this.state.hasFrame = false; + this.state.selectedDisplay = 0; this.state.viewportReset++; this.remoteVersion = ''; this.receivedHostInfo = false; + this.serviceAvailable = this.checkedRemote = this.sendingFile = false; + this.previewEligible = this.state.previews; this.historySaved = false; this.previewSaved = this.previewPending = false; this.previewAttempts = 0; + this.network = new NetworkStatistics(); let edge = 1920; + try { const screen = display.getDefaultDisplaySync(); edge = Math.max(screen.width, screen.height); } catch {} + this.video = new VideoSettings(this.state.videoPreference, edge); this.syncVideo(); + this.store.put('lastRemote', this.state.remote); this.state.password = ''; + this.ensureSignal(); this.joinIfReady(); + } + private joinSent: boolean = false; + private joinIfReady(): void { + if (!this.state.connecting || this.joinSent || this.state.signal !== 1 || !this.state.identity) return; + this.joinSent = true; this.send({ op: 'connect', connection: this.connection, remote: this.connectingId, password: this.pendingPassword }); + } + end(reason: string): void { + this.release(); this.send({ op: 'disconnect' }); this.platform.disconnect(); + this.connection++; this.uiEpoch++; this.pendingPassword = ''; this.joinSent = false; this.sendingFile = false; + this.state.connecting = this.state.ready = false; this.state.page = 'home'; this.state.modal = ''; this.state.password = ''; + this.state.receivedPath = this.state.receivedName = this.state.remoteClipboard = this.state.clipboardText = ''; this.remoteVersionRequest = 0; + if (this.settingsTimeout >= 0) clearTimeout(this.settingsTimeout); this.settingsTimeout = -1; + this.history(); if (reason) { this.state.notice = reason; this.state.modal = 'notice'; } + if (this.foreground && this.state.consent) this.checkLocal(); + } + release(): void { this.releaseInput(); this.send({ op: 'release' }); } + pointer(x: number, y: number, flag: number, scroll: number): void { this.send({ op: 'pointer', x: x, y: y, flag: flag, scroll: scroll }); } + physicalKey(code: number, down: boolean): void { this.send({ op: 'key', code: code, down: down }); } + key(code: number, down: boolean): void { this.send({ op: 'virtualKey', code: code, down: down }); } + chord(code: number, modifiers: number[] = []): void { + modifiers.forEach((key: number) => this.key(key, true)); this.key(code, true); this.key(code, false); + modifiers.slice().reverse().forEach((key: number) => this.key(key, false)); + } + shortcut(code: number): void { this.chord(code, [this.state.platform === 'macos' ? 91 : 17]); } + text(text: string): void { + if (text.length > 2048 || Array.from(text).some((c: string) => asciiKey(c) === 0)) { this.toast('中文等文本请通过剪贴板发送,再点击粘贴'); return; } + for (const c of text) { const code = asciiKey(c); this.chord(code & 255, code & 256 ? [16] : []); } + } + setRelative(value: boolean): void { this.release(); this.state.relative = value; this.store.put('relative', value); } + mute(): void { this.state.muted = !this.state.muted; this.platform.setMuted(this.state.muted); this.send({ op: 'mute', muted: this.state.muted }); } + selectDisplay(index: number): void { + if (index < 0 || index >= this.state.displays.length) return; + this.release(); this.state.selectedDisplay = index; this.state.viewportReset++; this.state.hasFrame = false; this.network.reset(); + this.send({ op: 'display', index: index }); this.state.modal = ''; + } + updateVideo(field: number, value: number): void { + const old = this.video.selection; + const next: VideoValues = { quality: field === 0 ? value : old.quality, frameRate: field === 1 ? value : old.frameRate, preference: field === 2 ? value : old.preference }; + const id = this.video.begin(next); if (id < 0) return; + this.send({ op: 'videoSettings', quality: next.quality, frameRate: next.frameRate, preference: next.preference, request: id }); + if (this.settingsTimeout >= 0) clearTimeout(this.settingsTimeout); + this.settingsTimeout = setTimeout(() => { this.video.expire(id); this.syncVideo(); }, 5000); this.syncVideo(); + } + setPreference(value: number): void { this.store.put('videoPreference', value); this.state.videoPreference = value; } + setPreviews(value: boolean): void { + this.store.put('previews', value); this.state.previews = value; + if (!value) { this.send({ op: 'cancelPreview' }); this.previewEligible = false; this.store.clearPreviews(); this.history(); } + this.state.modal = ''; + } + clearPreviews(): void { this.send({ op: 'cancelPreview' }); this.previewEligible = false; this.store.clearPreviews(); this.history(); this.toast('已清除预览'); } + clipboard(): void { + const text = this.state.clipboardText; + if (new util.TextEncoder().encodeInto(text).length > 128 * 1024) { this.toast('文本超过 128 KB'); return; } + if (text) { this.send({ op: 'clipboard', text: text }); this.toast('已发送,请在远端粘贴'); } + } + async copyRemote(): Promise { + if (!this.state.remoteClipboard) { this.toast('尚未收到远端文本'); return; } + try { await pasteboard.getSystemPasteboard().setData(pasteboard.createData(pasteboard.MIMETYPE_TEXT_PLAIN, this.state.remoteClipboard)); this.toast('已复制远端文本'); } + catch { this.toast('复制失败,请重试'); } + } + sas(): void { + if (!this.serviceAvailable) { this.toast(this.state.platform === 'windows' ? '请在远端启用 CrossDesk 系统服务后重试' : 'Ctrl+Alt+Del 需要远端 Windows 系统服务支持'); return; } + this.send({ op: 'sas' }); + } + async chooseFile(): Promise { + if (this.sendingFile) { this.toast('请等待当前文件发送完成'); return; } + const connection = this.connection; const file = await this.platform.choose(this.store.directory); + if (connection !== this.connection || !this.state.ready || !file) { if (file) this.store.discardOutgoing(file.path); return; } + this.sendingFile = true; this.state.transfer = `发送 ${file.name} · 0%`; + this.send({ op: 'sendFile', path: file.path, name: file.name }); + } + saveFile(): void { if (this.state.receivedPath) this.platform.save(this.state.receivedPath, this.state.receivedName); } + surface(id: string, controller?: XComponentController): void { + this.send({ op: 'surface', id: id }); if (controller) this.platform.surface(controller, this.state.width, this.state.height); + } + async checkLocal(manual: boolean = false): Promise { + if (!this.state.consent || !this.foreground || this.checkingLocal) return; + this.checkingLocal = true; const request = ++this.versionRequest; this.localVersionRequest = request; + const epoch = this.uiEpoch; if (manual) this.state.updateStatus = '正在检查…'; + const body = await this.platform.version(); + if (request !== this.localVersionRequest || epoch !== this.uiEpoch || !this.foreground || !this.state.consent) return; + this.checkingLocal = false; + if (body) this.send({ op: 'version', body: body, local: true, version: this.state.version, request: request }); + else if (manual) this.state.updateStatus = '检查失败,请稍后重试'; + } + private async checkRemote(version: string): Promise { + if (this.checkedRemote || !this.foreground || !this.state.consent || !this.state.ready) return; + this.checkedRemote = true; + if (!version || !/^\d+\.\d+\.\d+/.test(version)) { this.state.notice = '被控端版本过低,请升级到最新版本。'; this.state.modal = 'upgrade'; return; } + const request = ++this.versionRequest, connection = this.connection; this.remoteVersionRequest = request; + const body = await this.platform.version(); + if (body && request === this.remoteVersionRequest && connection === this.connection && this.foreground && this.state.consent) + this.send({ op: 'version', body: body, version: version, request: request }); + } + document(name: string, title: string, back: string = 'about'): void { + try { + this.state.documentText = name === 'PRIVACY.md' ? this.privacy : util.TextDecoder.create('utf-8').decodeWithStream(this.context.resourceManager.getRawFileContentSync(name)); + this.state.documentTitle = title; this.state.documentBack = back; this.state.page = 'document'; this.state.modal = ''; + } catch { this.toast('无法读取文档'); } + } + private syncVideo(): void { + this.state.videoQuality = this.video.selection.quality; this.state.videoRate = this.video.selection.frameRate; + this.state.videoSelectedPreference = this.video.selection.preference; this.state.videoSupported = this.video.supported; + this.state.videoFeedback = this.video.feedback(); + } + private poll(): void { + try { const events = JSON.parse(bridge.mobilePoll()) as NativeEvent[]; events.forEach((event: NativeEvent) => this.event(event)); } + catch { this.state.signalText = '原生通信库不可用,请重新构建并安装'; } + if (++this.seconds % 20 === 0) { + this.state.networkTime = this.now(); this.presence.tick(this.state.networkTime); this.history(); + } + } + private event(e: NativeEvent): void { + if (e.type !== 'version' && (e.session !== this.session || (e.connection !== undefined && e.connection !== this.connection))) { + if (e.type === 'preview' && e.path) this.store.discardRaw(e.path); return; + } + if (e.type === 'identity' && e.identity) { + this.state.identity = e.identity.split('@')[0]; const scope = this.scope(); + if (e.identity.includes('@')) this.store.saveSecret(`identity:${scope}`, e.identity).catch(() => this.toast('设备身份保存失败,请检查安全存储')); + this.joinIfReady(); this.configureInbox(); + } else if (e.type === 'signal' && !e.controller) { + if (e.generation !== this.signalGeneration) { this.signalGeneration = e.generation ?? -1; this.presence.connect(false, this.now()); this.inbox.connect(false); } + this.state.signal = e.status ?? -1; this.state.identity = e.id || this.state.identity; + const labels = ['正在连接…', '已连接', '连接失败', '服务器已断开', '正在重连…', '服务器已断开', '服务器证书校验失败']; + this.state.signalText = labels[this.state.signal] ?? '未连接'; this.presence.connect(this.state.signal === 1, this.now()); this.configureInbox(); this.joinIfReady(); + } else if (e.type === 'signalMessage' && e.generation === this.signalGeneration && e.message) { + const type = (e.message as SignalType).type; + if (type === 'announcements_changed') this.inbox.refresh(); + else if (type === 'announcements') this.inbox.receive(e.message as AnnouncementResponse); + else this.presence.receive(e.message as PresenceMessage, this.now()); + } else if (e.type === 'requestFailed') this.inbox.fail(String(e.request ?? '')); + else if (e.type === 'connection') { + if (e.status === 1 && !this.state.ready) this.connected(); + else if (e.status === 6) { + this.end(''); this.state.remote = this.connectingId; this.state.notice = '访问密码错误,请重新输入'; this.state.modal = 'password'; + } else if ((e.status ?? 0) >= 3) { + const reasons = ['', '', '', '连接已断开', '连接失败', '远端已关闭连接', '访问密码错误', '找不到该设备 ID', '远端当前不可用']; + this.end(reasons[e.status ?? 0] ?? '连接已结束'); + } + } else if (e.type === 'ended') this.end(e.reason ?? ''); + else if (e.type === 'data' && e.message) this.data(e.message as ControlData); + else if (e.type === 'frame') { + this.state.hasFrame = true; + this.state.width = e.width ?? 16; this.state.height = e.height ?? 9; + this.network.frame(e.frame ?? 0, e.latency ?? -1, this.state.width, this.state.height, this.now(), e.latencyCount ?? -1); + this.platform.videoSize(this.state.width, this.state.height); + if (this.previewEligible && this.historySaved && !this.previewSaved && !this.previewPending && this.previewAttempts < 3) { + this.previewAttempts++; this.previewPending = true; this.send({ op: 'preview' }); + } + } else if (e.type === 'network' && e.report) this.network.receive(e.report, this.now()); + else if (e.type === 'clipboard') this.state.remoteClipboard = e.text ?? ''; + else if (e.type === 'feedback') this.toast(e.text ?? ''); + else if (e.type === 'file') { + if (e.sending && ((e.progress ?? 0) < 0 || (e.progress ?? 0) >= 1)) this.sendingFile = false; + this.state.transfer = (e.progress ?? 0) < 0 ? `${e.name} 传输失败` : `${e.sending ? '发送' : '接收'} ${e.name} · ${(e.progress ?? 0) >= 1 ? '已完成' : Math.floor((e.progress ?? 0) * 100) + '%'}`; + if (e.path) { this.state.receivedPath = e.path; this.state.receivedName = e.name ?? 'received-file'; } + } else if (e.type === 'settingsFailed') { this.video.expire(Number(e.request)); this.syncVideo(); } + else if (e.type === 'preview') { + this.previewPending = false; + if (e.ok && e.path && this.previewEligible && this.state.previews) { + this.previewSaved = true; this.store.savePreview(this.scope(), this.connectingId, e.path).then(() => this.history()).catch(() => this.previewSaved = false); + } else if (e.path) this.store.discardRaw(e.path); + } else if (e.type === 'version') { + if (e.request === this.localVersionRequest && this.foreground && this.state.consent) { + if (e.ok) { this.state.update = e.latest ?? ''; this.state.updateStatus = this.state.update ? `发现新版本 ${this.state.update}` : '已是最新版本'; } + else this.state.updateStatus = '暂无鸿蒙版本信息,请稍后重试'; + } else if (e.request === this.remoteVersionRequest && e.ok && e.latest && this.foreground && this.state.ready) { + this.state.notice = '被控端版本过低,请升级到最新版本。'; this.state.modal = 'upgrade'; + } + } else if (e.type === 'error') { + this.state.signalText = e.text ?? '连接失败'; if (this.state.connecting) { this.end(e.text ?? '连接失败'); this.stopSignal(); } + else if (!this.state.ready) { const text = e.text ?? '操作失败'; this.stopSignal(); this.state.signalText = text; } else this.toast(e.text ?? '操作失败'); + } + } + private configureInbox(): void { + if (this.state.identity) this.inbox.configure(`${this.scope()}:${this.state.identity}`); + this.inbox.connect(this.state.signal === 1 && this.foreground && this.state.consent); + } + private connected(): void { + this.state.ready = true; this.state.connecting = false; this.state.modal = ''; this.state.page = 'session'; + const device: RecentDevice = { id: this.connectingId, name: this.state.hostName || this.connectingId, + platform: this.state.platform, remember: this.state.remember, preview: '' }; + const password = this.pendingPassword; this.pendingPassword = ''; + const scope = this.scope(), connection = this.connection; + this.store.saveRecent(scope, device, password).then(() => { + if (connection === this.connection && scope === this.scope()) { + this.historySaved = true; + if (this.state.hostName) this.store.metadata(scope, device.id, this.state.hostName, this.state.platform); + this.history(); + } + }).catch(() => this.toast('密码保存失败,本次连接仍可使用')); + this.platform.connected(); + if (this.video.supported && this.video.requestId === 0) this.updateVideo(0, this.video.selection.quality); + if (this.receivedHostInfo) this.checkRemote(this.remoteVersion); + } + private data(message: ControlData): void { + if (message.type === 5) { this.serviceAvailable = message.service_status?.available === true; return; } + if (message.type === 12 && message.video_settings) { + const v = message.video_settings; + this.video.receive({ quality: v.quality, frameRate: v.frame_rate, preference: v.preference }, v.request_id, v.accepted === true); + if (!this.video.pending && this.settingsTimeout >= 0) clearTimeout(this.settingsTimeout); this.syncVideo(); return; + } + if (message.host_info) { + const info = message.host_info; this.state.platform = info.platform; this.state.hostName = info.host_name; + this.state.displays = Array.isArray(info.displays) ? info.displays.slice(0, 8) : []; + this.store.metadata(this.scope(), this.connectingId, info.host_name, info.platform); + const supported = this.video.supported; this.video.support(info.supports_video_settings === true); this.syncVideo(); + if (this.state.ready && this.video.supported && !supported) this.updateVideo(0, this.video.selection.quality); + this.receivedHostInfo = true; this.remoteVersion = info.app_version; this.checkRemote(info.app_version); + } + } +} diff --git a/apps/harmony/entry/src/main/ets/mobile/MobilePlatform.ets b/apps/harmony/entry/src/main/ets/mobile/MobilePlatform.ets new file mode 100644 index 00000000..fe59f879 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/MobilePlatform.ets @@ -0,0 +1,186 @@ +import { deviceInfo } from '@kit.BasicServicesKit'; +import { common, wantAgent } from '@kit.AbilityKit'; +import { PiPWindow, window, display } from '@kit.ArkUI'; +import { backgroundTaskManager } from '@kit.BackgroundTasksKit'; +import { picker, fileIo as fs, fileUri } from '@kit.CoreFileKit'; +import { http } from '@kit.NetworkKit'; +import { isAnnouncementURL } from './AnnouncementText'; + +export class MobileLifecycle { + static foreground: () => void = () => {}; + static background: () => void = () => {}; + static destroy: () => void = () => {}; +} +export class MobilePlatform { + private pip?: PiPWindow.PiPController; + private pipPending: number = -1; + private backgroundQueue: Promise = Promise.resolve(); + private muted: boolean = false; + onMute: (muted: boolean) => void = () => {}; + onVisibility: (visible: boolean) => void = () => {}; + private generation: number = 0; + private backgroundActive: boolean = false; + private active: boolean = false; + private foreground: boolean = true; + choosing: boolean = false; + pipActive: boolean = false; + private requests: http.HttpRequest[] = []; + readonly context: common.UIAbilityContext; + private ended: (reason: string) => void; + private release: () => void; + private feedback: (message: string) => void; + private backgroundCancelled: () => void = () => { + this.backgroundActive = false; + if (this.active && !this.foreground && !this.pipActive && !this.choosing) this.ended('系统已停止后台会话'); + }; + constructor(context: common.UIAbilityContext, ended: (reason: string) => void, release: () => void, feedback: (message: string) => void) { + this.context = context; this.ended = ended; this.release = release; this.feedback = feedback; + try { backgroundTaskManager.on('continuousTaskCancel', this.backgroundCancelled); } catch {} + } + async connected(): Promise { + this.active = true; const generation = ++this.generation; + this.muted = false; this.orientation(true); + this.backgroundQueue = this.backgroundQueue.then(async () => { + try { + const agent = await wantAgent.getWantAgent({ wants: [{ bundleName: this.context.abilityInfo.bundleName, + abilityName: this.context.abilityInfo.name }], operationType: wantAgent.OperationType.START_ABILITY, + requestCode: 0, wantAgentFlags: [wantAgent.WantAgentFlags.UPDATE_PRESENT_FLAG] }); + if (generation !== this.generation || !this.active) return; + await backgroundTaskManager.startBackgroundRunning(this.context, backgroundTaskManager.BackgroundMode.DATA_TRANSFER, agent); + if (generation !== this.generation || !this.active) { await backgroundTaskManager.stopBackgroundRunning(this.context); return; } + this.backgroundActive = true; + } catch { this.backgroundActive = false; } + }); + await this.backgroundQueue; + } + async surface(controller: XComponentController, width: number, height: number): Promise { + if (!this.active || this.pip || this.pipPending === this.generation) return; + const generation = this.generation; + this.pipPending = generation; + try { + if (!PiPWindow.isPiPEnabled()) return; + const pip = await PiPWindow.create({ context: this.context, componentController: controller, + templateType: PiPWindow.PiPTemplateType.VIDEO_LIVE, contentWidth: width, contentHeight: height, + controlGroups: [PiPWindow.VideoLiveControlGroup.MUTE_SWITCH] }); + if (generation !== this.generation || !this.active) return; + this.pip = pip; pip.setAutoStartEnabled(!this.choosing); + this.setMuted(this.muted); + pip.on('controlEvent', (event: PiPWindow.ControlEventParam) => { + if (generation === this.generation && event.controlType === PiPWindow.PiPControlType.MUTE_SWITCH) + this.onMute(event.status !== PiPWindow.PiPControlStatus.PLAY); + }); + pip.on('stateChange', (state: PiPWindow.PiPState, reason: string) => { + if (generation !== this.generation) return; + if (state === PiPWindow.PiPState.ABOUT_TO_START) this.release(); + if (state === PiPWindow.PiPState.STARTED) { this.pipActive = true; this.onVisibility(true); } + if (state === PiPWindow.PiPState.ABOUT_TO_RESTORE) { this.foreground = true; this.onVisibility(true); } + if (state === PiPWindow.PiPState.STOPPED) { + this.pipActive = false; + this.onVisibility(this.foreground); + if (!this.foreground && !this.choosing && this.active) this.ended('画中画已关闭'); + } + }); + } catch { /* A continuous data-transfer task remains the fallback. */ } + finally { if (this.pipPending === generation) this.pipPending = -1; } + } + setMuted(muted: boolean): void { + this.muted = muted; + try { this.pip?.updatePiPControlStatus(PiPWindow.PiPControlType.MUTE_SWITCH, + muted ? PiPWindow.PiPControlStatus.PAUSE : PiPWindow.PiPControlStatus.PLAY); } catch {} + } + videoSize(width: number, height: number): void { try { this.pip?.updateContentSize(width, height); } catch { /* Not supported on this device. */ } } + async background(): Promise { + this.foreground = false; this.release(); this.cancelRequests(); + this.onVisibility(this.pipActive); + if (!this.active) return; + const generation = this.generation; + if (!this.choosing && this.pip && !this.pipActive) { + try { await this.pip.startPiP(); return; } catch { /* Fall back to the registered task. */ } + } + await this.backgroundQueue; + if (generation === this.generation && !this.foreground && this.active && !this.backgroundActive && !this.pipActive && !this.choosing) + this.ended('系统未允许后台会话,连接已断开'); + } + resume(): void { this.foreground = true; this.dismissPiP(); this.orientation(this.active); this.onVisibility(true); } + private async dismissPiP(): Promise { + if (!this.pipActive) return; + this.pipActive = false; + try { await this.pip?.stopPiP(); } catch { /* Already restored or closed. */ } + } + dispose(): void { this.disconnect(); try { backgroundTaskManager.off('continuousTaskCancel', this.backgroundCancelled); } catch {} } + disconnect(): void { + this.active = false; this.choosing = false; this.generation++; this.cancelRequests(); + const pip = this.pip; this.pip = undefined; this.pipActive = false; + if (pip) { try { pip.setAutoStartEnabled(false); pip.stopPiP().catch(() => {}); } catch { /* Already stopped. */ } } + this.backgroundQueue = this.backgroundQueue.then(async () => { + if (this.backgroundActive) { this.backgroundActive = false; try { await backgroundTaskManager.stopBackgroundRunning(this.context); } catch {} } + }); + this.orientation(false); + } + private async orientation(session: boolean): Promise { + const generation = this.generation; + try { + const win = await window.getLastWindow(this.context); + if (generation !== this.generation) return; + await win.setWindowKeepScreenOn(session); + if (generation !== this.generation) return; + // Tablets and resizable windows follow the user orientation in both views. + const phone = deviceInfo.deviceType === 'phone'; + const narrow = win.getWindowProperties().windowRect.width / display.getDefaultDisplaySync().densityPixels < 600; + await win.setPreferredOrientation(!session && phone && narrow ? window.Orientation.PORTRAIT : window.Orientation.AUTO_ROTATION); + } catch { /* Window may already be going away. */ } + } + private setChoosing(value: boolean): void { + this.choosing = value; this.release(); + try { this.pip?.setAutoStartEnabled(!value); } catch { /* No PiP controller. */ } + } + async choose(directory: string): Promise { + if (this.choosing) return undefined; + this.setChoosing(true); const generation = this.generation; let path = ''; + try { + await this.dismissPiP(); + if (generation !== this.generation || !this.active) return undefined; + const uris = await new picker.DocumentViewPicker(this.context).select({ maxSelectNumber: 1 }); + if (!uris.length || generation !== this.generation || !this.active) return undefined; + const name = new fileUri.FileUri(uris[0]).name || 'received-file'; + path = `${directory}/outgoing/${Date.now()}-${Math.floor(Math.random() * 1000000)}`; + const input = await fs.open(uris[0], fs.OpenMode.READ_ONLY); + try { await fs.copyFile(input.fd, path); } finally { await fs.close(input); } + if (generation !== this.generation || !this.active) { await fs.unlink(path); return undefined; } + return { path: path, name: name }; + } catch { if (path) try { await fs.unlink(path); } catch {} this.feedback('无法读取所选文件或存储空间不足'); return undefined; } + finally { if (generation === this.generation) this.setChoosing(false); } + } + async save(path: string, name: string): Promise { + if (this.choosing || !this.active) return; + this.setChoosing(true); const generation = this.generation; + try { + await this.dismissPiP(); + if (generation !== this.generation || !this.active) return; + const uris = await new picker.DocumentViewPicker(this.context).save({ newFileNames: [name] }); + if (!uris.length || generation !== this.generation || !this.active) return; + const output = await fs.open(uris[0], fs.OpenMode.WRITE_ONLY | fs.OpenMode.TRUNC); + try { await fs.copyFile(path, output.fd); } finally { await fs.close(output); } + if (generation === this.generation) this.feedback('文件已保存'); + } catch { if (generation === this.generation) this.feedback('文件保存失败,请重试'); } + finally { if (generation === this.generation) this.setChoosing(false); } + } + async version(): Promise { + const request = http.createHttp(); this.requests.push(request); + try { + const response = await request.request('https://version.crossdesk.cn/version.json', { + method: http.RequestMethod.GET, connectTimeout: 15000, readTimeout: 15000, + expectDataType: http.HttpDataType.STRING, usingCache: false, maxLimit: 256 * 1024 + }); + if (response.responseCode !== 200 || typeof response.result !== 'string' || response.result.length > 256 * 1024) return undefined; + return response.result; + } catch { return undefined; } + finally { this.requests = this.requests.filter((r: http.HttpRequest) => r !== request); request.destroy(); } + } + cancelRequests(): void { for (const request of this.requests) request.destroy(); this.requests = []; } + openLink(url: string): void { + if (!isAnnouncementURL(url)) return; + this.context.startAbility({ action: 'ohos.want.action.viewData', uri: url }).catch(() => this.feedback('无法打开网页')); + } +} +export interface PreparedFile { path: string; name: string; } diff --git a/apps/harmony/entry/src/main/ets/mobile/MobileRemoteView.ets b/apps/harmony/entry/src/main/ets/mobile/MobileRemoteView.ets new file mode 100644 index 00000000..02ea237e --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/MobileRemoteView.ets @@ -0,0 +1,350 @@ +import { MobileController, MobileState } from './MobileController'; +import { RemoteViewport, MouseInput, TouchInput, TouchPoint, clamp, NetworkStatistics, Traffic, asciiKey } from './ControllerModels'; +import { MobileTheme, MobileSegment, MobileIcon, MobileIconButton } from './MobileWidgets'; + +interface KeyRow { keys: string[]; } +@Component +export struct MobileRemoteView { + @ObjectLink @Watch('stateChanged') state: MobileState; + controller?: MobileController; + @State private viewWidth: number = 1; + @State private viewHeight: number = 1; + @State private videoWidth: number = 1; + @State private videoHeight: number = 1; + @State private videoLeft: number = 0; + @State private videoTop: number = 0; + @State private zoom: number = 1; + @State private cursorX: number = 0; + @State private cursorY: number = 0; + @State private orbX: number = 12; + @State private orbY: number = 100; + @State private menu: string = ''; + @State private menuHeight: number = 390; + @State private keyboardHeight: number = 220; + private orbPositioned: boolean = false; + @State private keyboard: boolean = false; + @State private computerKeyboard: boolean = false; + @State private shift: boolean = false; + @State private modifiers: number[] = []; + @State private keyboardX: number = 8; + @State private keyboardY: number = 150; + @State private mouseVisible: boolean = false; + @State private leftHeld: boolean = false; + @State private rightHeld: boolean = false; + @State private wheelOffset: number = 0; + private surface: XComponentController = new XComponentController(); + private viewport: RemoteViewport = new RemoteViewport(); + private mouse: MouseInput = new MouseInput((x: number, y: number, flag: number, scroll: number) => { + this.controller?.pointer(x, y, flag, scroll); this.updateCursor(); + }); + private touches: TouchInput = new TouchInput(this.viewport, this.mouse); + private longTimer: number = -1; + private lastReset: number = -1; + private lastWidth: number = 0; + private lastHeight: number = 0; + private lastRelative: boolean = true; + private orbStartX: number = 0; + private orbStartY: number = 0; + private originX: number = 0; + private originY: number = 0; + private orbMoved: boolean = false; + private barX: number = 0; + private barY: number = 0; + private wheelY: number = 0; + private wheelLast: number = 0; + private keyboardOriginX: number = 0; + private keyboardOriginY: number = 0; + aboutToAppear(): void { if (this.controller) this.controller.releaseInput = () => this.release(); this.stateChanged(); } + aboutToDisappear(): void { this.release(); if (this.controller) this.controller.releaseInput = () => {}; this.controller?.surface('0'); } + private stateChanged(): void { + if (this.lastReset !== this.state.viewportReset || this.lastRelative !== this.state.relative) { + this.release(); + if (this.lastReset !== this.state.viewportReset) { this.viewport.reset(); this.applyViewport(); } + this.lastReset = this.state.viewportReset; this.lastRelative = this.state.relative; + } + this.touches.relative = this.state.relative; + if (this.lastWidth !== this.state.width || this.lastHeight !== this.state.height) { + this.lastWidth = this.state.width; this.lastHeight = this.state.height; this.fit(); + } + } + private release(): void { + this.touches.cancel(); this.leftHeld = this.rightHeld = false; this.modifiers = []; this.wheelOffset = 0; + if (this.longTimer >= 0) clearTimeout(this.longTimer); this.longTimer = -1; + } + private fit(): void { this.viewport.fit(this.viewWidth, this.viewHeight, this.state.width, this.state.height); this.applyViewport(); } + private applyViewport(): void { + this.videoWidth = this.viewport.baseWidth; this.videoHeight = this.viewport.baseHeight; this.zoom = this.viewport.scale; + this.videoLeft = this.viewport.baseLeft + this.viewport.offsetX; this.videoTop = this.viewport.baseTop + this.viewport.offsetY; this.updateCursor(); + } + private updateCursor(): void { + this.cursorX = this.viewport.left() + this.mouse.x * this.viewport.width(); this.cursorY = this.viewport.top() + this.mouse.y * this.viewport.height(); + } + private touch(event: TouchEvent): void { + if (event.source === SourceType.Mouse) return; + const kind = event.type === TouchType.Down ? 'down' : event.type === TouchType.Move ? 'move' : event.type === TouchType.Up ? 'up' : 'cancel'; + if (kind === 'down') { + this.touches.begin(); if (this.longTimer >= 0) clearTimeout(this.longTimer); + if (event.touches.length === 1) this.longTimer = setTimeout(() => this.touches.hold(), 500); + } + if (kind === 'up' || kind === 'cancel' || event.touches.length > 1) { if (this.longTimer >= 0) clearTimeout(this.longTimer); this.longTimer = -1; } + this.touches.touch(event.touches.map((t: TouchObject): TouchPoint => ({ x: t.x, y: t.y })), kind); this.applyViewport(); + } + private physicalMouse(event: MouseEvent): void { + if (!this.viewport.contains(event.x, event.y) && event.action !== MouseAction.Release) return; + this.mouse.move(this.viewport.x(event.x), this.viewport.y(event.y)); + const button = event.button === MouseButton.Left ? 0 : event.button === MouseButton.Right ? 1 : event.button === MouseButton.Middle ? 2 : -1; + if (button >= 0 && (event.action === MouseAction.Press || event.action === MouseAction.Release)) + this.controller?.pointer(this.mouse.x, this.mouse.y, (event.action === MouseAction.Press ? 1 : 2) + button * 2, 0); + } + private orbTouch(event: TouchEvent): void { + const p = event.changedTouches[0]; if (!p) return; + if (event.type === TouchType.Down) { this.release(); this.orbStartX = p.screenX; this.orbStartY = p.screenY; this.originX = this.orbX; this.originY = this.orbY; this.orbMoved = false; } + if (event.type === TouchType.Move) { + const dx = p.screenX - this.orbStartX, dy = p.screenY - this.orbStartY; + if (Math.hypot(dx, dy) > 4) { this.orbMoved = true; this.menu = ''; } + if (this.orbMoved) { this.orbX = clamp(this.originX + dx, 8, this.viewWidth - 62); this.orbY = clamp(this.originY + dy, 8, this.viewHeight - 62); } + } + if (event.type === TouchType.Up && !this.orbMoved) this.menu = this.menu ? '' : 'controls'; + } + private barTouch(event: TouchEvent, button: number): void { + const p = event.changedTouches[0]; if (!p) return; + if (event.type === TouchType.Down) { + this.barX = p.screenX; this.barY = this.wheelY = this.wheelLast = p.screenY; + if (button !== 1) this.mouse.button(button === 0, true); + } else if (event.type === TouchType.Move) { + if (button === 1) { + const delta = p.screenY - this.wheelY; this.wheelOffset = ((p.screenY - this.wheelLast) % 8 + 8) % 8; + if (Math.abs(delta) > 8) { this.mouse.scroll(delta); this.wheelY = p.screenY; } + } else this.mouse.drag(p.screenX - this.barX, p.screenY - this.barY, this.viewport.width(), this.viewport.height()); + this.barX = p.screenX; this.barY = p.screenY; + } else if (event.type === TouchType.Up || event.type === TouchType.Cancel) { + if (button !== 1) this.mouse.button(button === 0, false); this.wheelOffset = 0; + } + this.leftHeld = this.mouse.left; this.rightHeld = this.mouse.right; + } + private action(index: number): void { + if (index === 0) { this.keyboard = !this.keyboard; this.menu = ''; this.modifiers = []; if (this.keyboard) { this.keyboardX = (this.viewWidth - this.keyboardWidth()) / 2; this.keyboardY = Math.max(8, this.viewHeight - this.keyboardHeight - 20); } } + if (index === 1) this.controller?.setRelative(!this.state.relative); + if (index === 2) { this.release(); this.mouseVisible = !this.mouseVisible; this.menu = ''; } + if (index === 3) { this.state.modal = 'displays'; this.menu = ''; } + if (index === 4) this.menu = 'video'; + if (index === 5) this.controller?.mute(); + if (index === 6) this.controller?.chooseFile(); + if (index === 7) this.menu = 'network'; + if (index === 8) this.controller?.sas(); + if (index === 9) { this.state.modal = 'clipboard'; this.menu = ''; } + } + private labels(): string[] { return ['键盘', this.state.relative ? '相对鼠标' : '绝对鼠标', this.mouseVisible ? '收起鼠标' : '虚拟鼠标', '显示器', '画面设置', this.state.muted ? '静音' : '声音', '发送文件', '网络状态', 'Ctrl+Alt+Del', '剪贴板']; } + private icons(): string[] { return ['keyboard', 'mouse', 'mouse', 'display', 'sliders', this.state.muted ? 'muted' : 'audio', 'folder', 'chart', 'lock', 'clipboard']; } + private columns(): number { return this.viewHeight < 400 && this.viewWidth >= 400 ? 4 : 3; } + private menuWidth(): number { return Math.max(1, Math.min(this.viewWidth - 16, this.columns() === 4 ? 380 : Math.max(280, Math.min(340, this.viewWidth - 160)))); } + private menuX(): number { return clamp(this.orbX > this.viewWidth / 2 ? this.orbX - this.menuWidth() - 12 : this.orbX + 66, 8, this.viewWidth - this.menuWidth() - 8); } + private mousePosition(): Position { + let x = this.orbX >= 154 ? this.orbX - 146 : this.orbX + 60, y = this.orbY + 3; + if (x + 140 > this.viewWidth - 8) { x = this.orbX; y = this.orbY + 60; if (y + 48 > this.viewHeight - 8) y = this.orbY - 54; } + if (this.keyboard && x < this.keyboardX + this.keyboardWidth() && x + 140 > this.keyboardX && y < this.keyboardY + this.keyboardHeight && y + 48 > this.keyboardY) { + y = this.keyboardY - 54; if (y < 8) y = this.keyboardY + this.keyboardHeight + 6; + } + return { x: clamp(x, 8, this.viewWidth - 148), y: clamp(y, 8, this.viewHeight - 56) }; + } + private keyboardWidth(): number { return Math.max(1, Math.min(this.computerKeyboard ? 520 : 440, this.viewWidth - 16)); } + private keyboardRows(): KeyRow[] { + return this.computerKeyboard ? [ + { keys: ['Esc', 'F1', 'F2', 'F3', 'F4', 'F5', 'F6'] }, { keys: ['Tab', 'Ctrl', 'Alt', 'Win', '↑', '⌫'] }, + { keys: ['复制', '粘贴', '全选', '←', '↓', '→'] }, { keys: ['ABC', '空格', 'Enter'] } + ] : [{ keys: ['1','2','3','4','5','6','7','8','9','0'] }, { keys: ['q','w','e','r','t','y','u','i','o','p'] }, + { keys: ['a','s','d','f','g','h','j','k','l'] }, { keys: ['⇧','z','x','c','v','b','n','m','⌫'] }, + { keys: ['电脑','剪贴板','空格','.','Enter'] }]; + } + private keyboardKey(value: string): void { + if (value === '⇧') { this.shift = !this.shift; return; } + if (value === '电脑' || value === 'ABC') { this.computerKeyboard = value === '电脑'; return; } + if (value === '剪贴板') { this.state.modal = 'clipboard'; return; } + const modifier = value === 'Ctrl' ? 17 : value === 'Alt' ? 18 : value === 'Win' ? 91 : 0; + if (modifier) { this.modifiers = this.modifiers.includes(modifier) ? this.modifiers.filter((m: number) => m !== modifier) : [...this.modifiers, modifier]; return; } + if (value === '复制' || value === '粘贴' || value === '全选') { this.controller?.shortcut(value === '复制' ? 67 : value === '粘贴' ? 86 : 65); return; } + const names = ['空格','⌫','Enter','Esc','Tab','↑','↓','←','→']; const codes = [32,8,13,27,9,38,40,37,39]; + let code = names.includes(value) ? codes[names.indexOf(value)] : /^F[1-6]$/.test(value) ? 111 + Number(value.substring(1)) : 0; + if (!code && this.modifiers.length) { const key = asciiKey(this.shift ? value.toUpperCase() : value); code = key & 255; if (key & 256) this.modifiers = [...this.modifiers, 16]; } + if (code) { this.controller?.chord(code, this.modifiers); this.modifiers = []; } + else this.controller?.text(this.shift ? value.toUpperCase() : value); + } + private keyboardDrag(event: GestureEvent): void { + this.keyboardX = clamp(this.keyboardOriginX + event.offsetX, 8, this.viewWidth - this.keyboardWidth() - 8); + this.keyboardY = clamp(this.keyboardOriginY + event.offsetY, 8, Math.max(8, this.viewHeight - this.keyboardHeight - 8)); + } + @Builder + NetworkRow(title: string, traffic: Traffic | undefined) { + Row({ space: 4 }) { + Text(title).fontSize(11).fontWeight(title === '合计' ? 600 : 400).fontColor(MobileTheme.ink).width(28) + Text(NetworkStatistics.rate(traffic?.inbound ?? -1)).fontSize(11).minFontSize(8).maxLines(1).layoutWeight(1).textAlign(TextAlign.End) + Text(NetworkStatistics.rate(traffic?.outbound ?? -1)).fontSize(11).minFontSize(8).maxLines(1).layoutWeight(1).textAlign(TextAlign.End) + Text(NetworkStatistics.loss(traffic?.loss ?? -1)).fontSize(11).width(46).textAlign(TextAlign.End) + }.padding({ left: 4, right: 4 }).height(26).width('100%').borderRadius(5).backgroundColor(title === '合计' ? '#122E7ADB' : Color.Transparent) + } + @Builder + NetworkDetail(label: string, value: string, encrypted: boolean = false) { + Row({ space: 4 }) { + Text(label).fontSize(12).fontColor(MobileTheme.secondary).layoutWeight(1) + if (encrypted) { MobileIcon({ name: 'lock', iconExtent: 14, color: MobileTheme.green }) } + Text(value).fontSize(12).fontColor(encrypted ? MobileTheme.green : MobileTheme.ink).textAlign(TextAlign.End) + }.width('100%').constraintSize({ minHeight: 21 }) + } + @Builder + Menu() { + Column({ space: 10 }) { + Row({ space: 6 }) { + if (this.menu !== 'controls') { MobileIconButton({ icon: 'back', label: '返回控制栏', iconExtent: 30, iconSize: 19, action: () => this.menu = 'controls' }) } + Column({ space: 3 }) { + Row({ space: 6 }) { + if (this.menu === 'controls') { Row() {}.width(6).height(6).borderRadius(3).backgroundColor(MobileTheme.green) } + Text(this.menu === 'controls' ? '已连接' : this.menu === 'video' ? '画面设置' : '网络状态').fontSize(14).fontWeight(600).fontColor(MobileTheme.ink) + }.width('100%') + if (this.menu === 'controls') { + Text(`${this.controller?.network.mode(this.state.networkTime) ?? '—'} · ${this.state.width}×${this.state.height}`).fontSize(10).fontColor(MobileTheme.secondary).width('100%') + } + }.layoutWeight(1) + MobileIconButton({ icon: 'close', label: '关闭控制栏', iconExtent: 30, iconSize: 20, action: () => this.menu = '' }) + }.width('100%') + Scroll() { + Column({ space: 8 }) { + if (this.menu === 'controls') { + GridRow({ columns: this.columns(), gutter: 7 }) { + ForEach(this.labels(), (label: string, index: number) => { + GridCol() { + Column({ space: 4 }) { MobileIcon({ name: this.icons()[index], iconExtent: 19 }); Text(label).fontSize(10).fontColor(MobileTheme.ink).maxLines(1) } + .height(52).width('100%').justifyContent(FlexAlign.Center).backgroundColor('#F0F0F2').borderRadius(10) + .accessibilityText(label).accessibilityLevel('yes').onClick(() => this.action(index)) + } + }, (label: string) => label) + } + if (this.state.transfer) { Text(this.state.transfer).fontSize(11).fontColor(MobileTheme.secondary).width('100%') } + if (this.state.receivedPath) { Button('保存收到的文件', { type: ButtonType.Normal }).fontSize(14).fontColor(MobileTheme.blue).backgroundColor(Color.White).height(42).width('100%').onClick(() => this.controller?.saveFile()) } + } else if (this.menu === 'video') { + Text(this.state.videoFeedback).fontSize(12).fontColor(MobileTheme.secondary).width('100%') + Text('画面质量').fontSize(12).width('100%') + MobileSegment({ labels: ['低', '中', '高'], selection: this.state.videoQuality, + available: this.state.videoSupported, changed: (i: number) => this.controller?.updateVideo(0, i) }) + Text('画面采集帧率').fontSize(12).width('100%') + MobileSegment({ labels: ['30 fps', '60 fps'], selection: this.state.videoRate === 60 ? 1 : 0, + available: this.state.videoSupported, changed: (i: number) => this.controller?.updateVideo(1, i ? 60 : 30) }) + Text('画面偏好').fontSize(12).width('100%') + MobileSegment({ labels: ['帧率优先', '画质优先', '平衡'], selection: this.state.videoSelectedPreference, + available: this.state.videoSupported, changed: (i: number) => this.controller?.updateVideo(2, i) }) + } else { + Row({ space: 4 }) { + Text('').width(28) + Text('接收').fontSize(11).fontColor(MobileTheme.secondary).layoutWeight(1).textAlign(TextAlign.End) + Text('发送').fontSize(11).fontColor(MobileTheme.secondary).layoutWeight(1).textAlign(TextAlign.End) + Text('丢包率').fontSize(11).fontColor(MobileTheme.secondary).width(46).textAlign(TextAlign.End) + }.padding({ left: 4, right: 4 }).height(24).width('100%') + Column() { + this.NetworkRow('视频', this.controller?.network.current(this.state.networkTime)?.video) + this.NetworkRow('音频', this.controller?.network.current(this.state.networkTime)?.audio) + this.NetworkRow('数据', this.controller?.network.current(this.state.networkTime)?.data) + this.NetworkRow('合计', this.controller?.network.current(this.state.networkTime)?.total) + }.width('100%') + Divider().strokeWidth(0.5).color(MobileTheme.line).margin({ top: 2, bottom: 2 }) + this.NetworkDetail('帧率', `${this.controller?.network.fps(this.state.networkTime) ?? '—'} FPS`) + this.NetworkDetail('分辨率', this.state.hasFrame ? `${this.state.width} × ${this.state.height}` : '—') + this.NetworkDetail('画面延时', NetworkStatistics.delay(this.controller?.network.latency(this.state.networkTime) ?? -1)) + this.NetworkDetail('连接延时(RTT)', NetworkStatistics.delay(this.state.networkTime - (this.controller?.network.rttTime ?? 0) < 3000 ? this.controller?.network.rtt ?? -1 : -1)) + this.NetworkDetail('连接方式', this.controller?.network.mode(this.state.networkTime) ?? '—') + this.NetworkDetail('媒体加密', this.controller?.network.current(this.state.networkTime) === undefined ? '—' : this.controller?.network.report?.srtp ? 'SRTP 已启用' : '未启用', this.controller?.network.current(this.state.networkTime)?.srtp === true) + + } + }.width('100%') + }.constraintSize({ maxHeight: Math.max(48, this.viewHeight - 132) }).width('100%').scrollBar(BarState.Off) + Button('断开连接', { type: ButtonType.Normal }).fontSize(14).fontColor(Color.White).width('100%').height(34).borderRadius(8).backgroundColor(MobileTheme.red) + .onClick(() => { this.release(); this.state.modal = 'disconnect'; this.menu = ''; this.keyboard = false; }) + }.padding(12).width(this.menuWidth()).backgroundColor(Color.White).borderRadius(18) + .border({ width: 0.5, color: MobileTheme.line }).shadow({ radius: 24, color: '#33000000', offsetY: 6 }) + .onAreaChange((_old: Area, next: Area) => this.menuHeight = Number(next.height)) + .position({ x: this.menuX(), y: clamp(this.orbY, 8, Math.max(8, this.viewHeight - this.menuHeight - 8)) }) + } + @Builder + Keyboard() { + Column({ space: 3 }) { + Row() { + Stack() { Row() {}.width(32).height(4).backgroundColor(MobileTheme.secondary).borderRadius(2) }.layoutWeight(1).height(28).accessibilityText('拖动键盘') + .gesture(PanGesture().onActionStart(() => { this.keyboardOriginX = this.keyboardX; this.keyboardOriginY = this.keyboardY; }) + .onActionUpdate((e: GestureEvent) => this.keyboardDrag(e))) + MobileIconButton({ icon: 'close', label: '收起键盘', iconExtent: 28, iconSize: 18, action: () => { this.keyboard = false; this.modifiers = []; } }) + }.width('100%') + Scroll() { + Column({ space: 3 }) { + ForEach(this.keyboardRows(), (row: KeyRow) => { + Row({ space: 3 }) { + ForEach(row.keys, (key: string) => { + Text(this.shift && key.length === 1 ? key.toUpperCase() : key).fontSize(key.length > 2 ? 11 : 14) + .height(31).layoutWeight(key === '空格' ? 3 : 1).textAlign(TextAlign.Center).borderRadius(5).fontColor(MobileTheme.ink) + .shadow({ radius: 0.5, color: '#33000000', offsetY: 1 }) + .backgroundColor((key === '⇧' && this.shift) || this.modifiers.includes(key === 'Ctrl' ? 17 : key === 'Alt' ? 18 : key === 'Win' ? 91 : -1) ? '#9BC7F7' : key.length > 1 || key === '⇧' ? '#AFB8C5' : '#FFFFFF') + .accessibilityText(key).accessibilityLevel('yes').onClick(() => this.keyboardKey(key)) + }, (key: string) => key) + }.width('100%') + }) + }.width('100%') + }.width('100%').constraintSize({ maxHeight: Math.max(31, this.viewHeight - 60) }).scrollBar(BarState.Off) + }.padding(5).width(this.keyboardWidth()).backgroundColor('#D1D6DE').borderRadius(12) + .shadow({ radius: 20, color: '#33000000', offsetY: 5 }).onAreaChange((_old: Area, next: Area) => this.keyboardHeight = Number(next.height)) + .position({ x: clamp(this.keyboardX, 8, this.viewWidth - this.keyboardWidth() - 8), y: clamp(this.keyboardY, 8, Math.max(8, this.viewHeight - this.keyboardHeight - 8)) }) + } + @Builder + MouseBar() { + Row({ space: 2 }) { + ForEach([0, 1, 2], (button: number) => { + Stack() { + if (button === 1) { + Stack() { + Column({ space: 4 }) { + ForEach([0, 1, 2, 3, 4, 5, 6], (line: number) => { Row() {}.height(3).width(14).backgroundColor('#737378') }) + }.translate({ y: this.wheelOffset - 8 }) + }.width(18).height(30).clip(true) + } else { MobileIcon({ name: button === 0 ? 'mouse_left' : 'mouse_right', iconExtent: 22 }) } + }.width(44).height(44).borderRadius(7).clip(true) + .backgroundColor(button === 0 && this.leftHeld || button === 2 && this.rightHeld ? '#9BC7F7' : '#E9E9EE') + .accessibilityText(button === 0 ? '鼠标左键,按住拖动' : button === 1 ? '滚轮,上下滑动' : '鼠标右键,按住拖动') + .onTouch((event: TouchEvent) => this.barTouch(event, button)) + }) + }.padding(2).width(140).height(48).backgroundColor('#FFFFFF').borderRadius(10) + .border({ width: 0.5, color: MobileTheme.line }).shadow({ radius: 12, color: '#33000000', offsetY: 4 }).position(this.mousePosition()) + } + build() { + Stack({ alignContent: Alignment.TopStart }) { + XComponent({ id: 'mobileVideo', type: XComponentType.SURFACE, controller: this.surface }) + .width(this.videoWidth).height(this.videoHeight).position({ x: this.videoLeft, y: this.videoTop }) + .scale({ x: this.zoom, y: this.zoom }).onLoad(() => this.controller?.surface(this.surface.getXComponentSurfaceId(), this.surface)) + .onDestroy(() => this.controller?.surface('0')) + if (!this.state.hasFrame) { + Column({ space: 12 }) { + LoadingProgress().width(32).height(32).color(Color.White) + Text('等待远端画面…').fontSize(14).fontColor(Color.White) + }.padding(20).constraintSize({ minWidth: 200 }).backgroundColor('#B0202024').borderRadius(14) + .position({ x: (this.viewWidth - 200) / 2, y: (this.viewHeight - 104) / 2 }).hitTestBehavior(HitTestMode.None) + } + Column() {}.width('100%').height('100%').focusable(true).defaultFocus(true) + .onTouch((e: TouchEvent) => this.touch(e)).onMouse((e: MouseEvent) => this.physicalMouse(e)) + .onAxisEvent((event: AxisEvent) => { const delta = event.getVerticalAxisValue(); if (delta) this.mouse.scroll(-delta); }) + .onKeyEvent((e: KeyEvent) => { if (e.type === KeyType.Down || e.type === KeyType.Up) { this.controller?.physicalKey(e.keyCode, e.type === KeyType.Down); return true; } return false; }) + .onBlur(() => this.controller?.release()) + .accessibilityText('远程桌面:轻触点击、长按右键、双指捏合缩放、放大后双指拖动画面,原始比例下双指滚动') + MobileIcon({ name: 'cursor', iconExtent: 24, color: '#000000' }) + .position({ x: this.cursorX - 1, y: this.cursorY - 1 }).hitTestBehavior(HitTestMode.None) + if (this.menu) { Column() {}.width('100%').height('100%').onClick(() => this.menu = ''); this.Menu() } + if (this.keyboard) { this.Keyboard() } + if (this.mouseVisible && !this.menu && this.state.hasFrame) { this.MouseBar() } + Stack() { + Image($r('app.media.mobile_brand')).width(46).height(46).objectFit(ImageFit.Cover).borderRadius(23).draggable(false).hitTestBehavior(HitTestMode.None) + }.width(54).height(54).backgroundColor('#F5F5F7').borderRadius(27).shadow({ radius: 14, color: '#33000000', offsetY: 4 }) + .position({ x: this.orbX, y: this.orbY }).accessibilityText('远程控制菜单,拖动可移动').accessibilityLevel('yes') + .onTouch((e: TouchEvent) => this.orbTouch(e)) + }.width('100%').height('100%').backgroundColor(Color.Black).clip(true) + .onAreaChange((_old: Area, next: Area) => { + this.release(); this.viewWidth = Number(next.width); this.viewHeight = Number(next.height); this.fit(); + if (!this.orbPositioned && this.viewWidth > 100) { this.orbX = this.viewWidth - 66; this.orbY = 17; this.orbPositioned = true; } + this.orbX = clamp(this.orbX, 8, this.viewWidth - 62); this.orbY = clamp(this.orbY, 8, this.viewHeight - 62); + this.keyboardX = clamp(this.keyboardX, 8, this.viewWidth - this.keyboardWidth() - 8); + }) + } +} diff --git a/apps/harmony/entry/src/main/ets/mobile/MobileStore.ets b/apps/harmony/entry/src/main/ets/mobile/MobileStore.ets new file mode 100644 index 00000000..5bea4af7 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/MobileStore.ets @@ -0,0 +1,138 @@ +import { common } from '@kit.AbilityKit'; +import { preferences } from '@kit.ArkData'; +import { asset } from '@kit.AssetStoreKit'; +import { cryptoFramework } from '@kit.CryptoArchitectureKit'; +import { util } from '@kit.ArkTS'; +import { fileIo as fs } from '@kit.CoreFileKit'; +import { image } from '@kit.ImageKit'; + +export interface RecentDevice { id: string; name: string; platform: string; remember: boolean; preview: string; } +export class MobileStore { + private data: preferences.Preferences; + readonly directory: string; + readonly previewDirectory: string; + private previewEpoch: number = 0; + constructor(context: common.UIAbilityContext) { + this.data = preferences.getPreferencesSync(context, { name: 'crossdesk_mobile' }); + this.directory = `${context.filesDir}/mobile`; + this.previewDirectory = `${context.cacheDir}/mobile-previews`; + try { fs.mkdirSync(this.previewDirectory, true); } catch {} + for (const name of ['', '/outgoing', '/received']) { + try { fs.mkdirSync(this.directory + name, true); } catch { /* Existing directories are reusable. */ } + } + } + text(key: string, fallback: string = ''): string { const value = this.data.getSync(key, fallback); return typeof value === 'string' ? value : fallback; } + flag(key: string, fallback: boolean = false): boolean { const value = this.data.getSync(key, fallback); return typeof value === 'boolean' ? value : fallback; } + number(key: string, fallback: number): number { const value = this.data.getSync(key, fallback); return typeof value === 'number' ? value : fallback; } + put(key: string, value: string | number | boolean): void { this.data.putSync(key, value); this.data.flushSync(); } + static scope(server: string, port: number): string { return `${server.trim().toLowerCase().replace(/\.$/, '')}:${port}`; } + static async hash(value: string): Promise { + const md = cryptoFramework.createMd('SHA256'); await md.update({ data: new util.TextEncoder().encodeInto(value) }); + const digest = await md.digest(); return Array.from(digest.data).map((b: number) => b.toString(16).padStart(2, '0')).join(''); + } + private async query(key: string): Promise { + const query: asset.AssetMap = new Map(); + query.set(asset.Tag.ALIAS, new util.TextEncoder().encodeInto(await MobileStore.hash(`crossdesk-mobile:${key}`))); + return query; + } + async secret(key: string): Promise { + const query = await this.query(key); query.set(asset.Tag.RETURN_TYPE, asset.ReturnType.ALL); + try { + const found = await asset.query(query); const value = found.length ? found[0].get(asset.Tag.SECRET) : undefined; + return value instanceof Uint8Array ? util.TextDecoder.create('utf-8').decodeWithStream(value) : ''; + } catch (error) { + if ((error as ErrorWithCode).code === asset.ErrorCode.NOT_FOUND) return ''; + throw new Error('安全存储不可用,请解锁设备后重试'); + } + } + async saveSecret(key: string, value: string): Promise { + if (!value) { await this.removeSecret(key); return; } + const attributes = await this.query(key); + attributes.set(asset.Tag.SECRET, new util.TextEncoder().encodeInto(value)); + attributes.set(asset.Tag.ACCESSIBILITY, asset.Accessibility.DEVICE_FIRST_UNLOCKED); + attributes.set(asset.Tag.SYNC_TYPE, asset.SyncType.NEVER); + attributes.set(asset.Tag.CONFLICT_RESOLUTION, asset.ConflictResolution.OVERWRITE); + await asset.add(attributes); + } + async removeSecret(key: string): Promise { + try { await asset.remove(await this.query(key)); } + catch (error) { if ((error as ErrorWithCode).code !== asset.ErrorCode.NOT_FOUND) throw new Error('无法删除已保存的凭据'); } + } + recent(scope: string): RecentDevice[] { + try { + const entries = JSON.parse(this.text(`recent:${scope}`, '[]')) as RecentDevice[]; + return entries.filter((d: RecentDevice) => typeof d.id === 'string' && /^[0-9]{9}$/.test(d.id) && typeof d.name === 'string').slice(0, 20) + .map((d: RecentDevice) => { + try { if (typeof d.preview !== 'string' || !d.preview.startsWith(`${this.previewDirectory}/`) || !fs.accessSync(d.preview)) d.preview = ''; } + catch { d.preview = ''; } + return d; + }); + } catch { return []; } + } + async saveRecent(scope: string, device: RecentDevice, password: string): Promise { + if (device.remember) await this.saveSecret(`password:${scope}:${device.id}`, password); + else await this.removeSecret(`password:${scope}:${device.id}`); + const old = this.recent(scope), previous = old.find((d: RecentDevice) => d.id === device.id); + device.preview = previous?.preview ?? ''; + const next = [device, ...old.filter((d: RecentDevice) => d.id !== device.id)].slice(0, 20); + this.put(`recent:${scope}`, JSON.stringify(next)); + for (const dropped of old.filter((d: RecentDevice) => !next.some((n: RecentDevice) => n.id === d.id))) { + await this.removeSecret(`password:${scope}:${dropped.id}`); this.removePreview(dropped.preview); + } + } + metadata(scope: string, id: string, name: string, platform: string): void { + const entries = this.recent(scope); + entries.forEach((d: RecentDevice) => { if (d.id === id) { d.name = name || id; d.platform = platform; } }); + this.put(`recent:${scope}`, JSON.stringify(entries)); + } + async delete(scope: string, id: string): Promise { + await this.removeSecret(`password:${scope}:${id}`); + const old = this.recent(scope); this.put(`recent:${scope}`, JSON.stringify(old.filter((d: RecentDevice) => d.id !== id))); + this.removePreview(old.find((d: RecentDevice) => d.id === id)?.preview ?? ''); + } + private removePreview(path: string): void { + if (!path.startsWith(`${this.previewDirectory}/`)) return; + try { fs.unlinkSync(path); } catch { /* Already removed. */ } + } + clearPreviews(): void { + this.previewEpoch++; + let names: string[] = []; + try { names = fs.listFileSync(this.previewDirectory); } catch { /* The system may already have cleared the cache. */ } + for (const name of names) this.removePreview(`${this.previewDirectory}/${name}`); + } + discardRaw(path: string): void { + if (!path.startsWith(`${this.previewDirectory}/preview-`) || !/^preview-[0-9]+-[0-9]+\.rgba$/.test(path.substring(this.previewDirectory.length + 1))) return; + try { fs.unlinkSync(path); } catch { /* Already removed. */ } + } + discardOutgoing(path: string): void { + if (!path.startsWith(`${this.directory}/outgoing/`) || path.substring(this.directory.length + 10).includes('/')) return; + try { fs.unlinkSync(path); } catch { /* Already removed. */ } + } + async savePreview(scope: string, id: string, raw: string): Promise { + if (!raw.startsWith(`${this.previewDirectory}/preview-`) || !/^preview-[0-9]+-[0-9]+\.rgba$/.test(raw.substring(this.previewDirectory.length + 1))) return; + const epoch = this.previewEpoch; + let pixel: image.PixelMap | undefined; + const packer = image.createImagePacker(); + let output: fs.File | undefined; + let temporary = ''; + try { + if (!this.flag('previews') || !this.flag('consent') || !this.recent(scope).some((d: RecentDevice) => d.id === id)) return; + const file = fs.openSync(raw, fs.OpenMode.READ_ONLY); const bytes = new ArrayBuffer(640 * 360 * 4); + try { if (fs.readSync(file.fd, bytes) !== bytes.byteLength) return; } finally { fs.closeSync(file); } + pixel = await image.createPixelMap(bytes, { size: { width: 640, height: 360 }, pixelFormat: image.PixelMapFormat.RGBA_8888 }); + const destination = `${this.previewDirectory}/${await MobileStore.hash(`${scope}:${id}`)}.jpg`; + temporary = `${destination}.tmp`; + output = fs.openSync(temporary, fs.OpenMode.CREATE | fs.OpenMode.WRITE_ONLY | fs.OpenMode.TRUNC); + await packer.packToFile(pixel, output.fd, { format: 'image/jpeg', quality: 82 }); fs.closeSync(output); output = undefined; + if (epoch !== this.previewEpoch || !this.flag('previews') || !this.flag('consent')) return; + const entries = this.recent(scope), target = entries.find((d: RecentDevice) => d.id === id); if (!target) return; + fs.renameSync(temporary, destination); temporary = ''; target.preview = destination; + this.put(`recent:${scope}`, JSON.stringify(entries)); + } finally { + if (output) fs.closeSync(output); if (pixel) pixel.release(); packer.release(); + try { fs.unlinkSync(raw); } catch { /* Revocation may already have removed it. */ } + if (temporary) try { fs.unlinkSync(temporary); } catch { /* Best effort cleanup. */ } + } + } +} +interface ErrorWithCode { code: number; } diff --git a/apps/harmony/entry/src/main/ets/mobile/MobileWidgets.ets b/apps/harmony/entry/src/main/ets/mobile/MobileWidgets.ets new file mode 100644 index 00000000..1cf0af7c --- /dev/null +++ b/apps/harmony/entry/src/main/ets/mobile/MobileWidgets.ets @@ -0,0 +1,130 @@ +import { LengthMetricsUnit } from '@kit.ArkUI'; + +export class MobileTheme { + static readonly background: string = '#F2F2F7'; + static readonly ink: string = '#1C1C1E'; + static readonly secondary: string = '#737378'; + static readonly line: string = '#E2E2E7'; + static readonly blue: string = '#2E7ADB'; + static readonly red: string = '#FF3B30'; + static readonly green: string = '#248A3D'; +} +// Vector geometry follows Android MobileUi.Icon; no font-dependent emoji glyphs. +@Component +export struct MobileIcon { + @Prop @Watch('draw') name: string = 'info'; + @Prop @Watch('draw') color: string = MobileTheme.ink; + iconExtent: number = 22; + private context: CanvasRenderingContext2D = new CanvasRenderingContext2D(new RenderingContextSettings(true), LengthMetricsUnit.PX); + private paths(): string[] { + switch (this.name) { + case 'cursor': return ['M1 1L1 13L5 10L9 17L12 15L8 9L15 9Z']; + case 'folder': return ["M2 19V5h7l3 3h10v11z"]; + case 'keyboard': return ["M5.0 5.0H19.0Q22.0 5.0 22.0 8.0V16.0Q22.0 19.0 19.0 19.0H5.0Q2.0 19.0 2.0 16.0V8.0Q2.0 5.0 5.0 5.0Z", "M6 9h.01M10 9h.01M14 9h.01M18 9h.01M6 12h.01M10 12h.01M14 12h.01M18 12h.01M7 16h10"]; + case 'envelope': return ["M4.0 5.0H20.0Q22.0 5.0 22.0 7.0V17.0Q22.0 19.0 20.0 19.0H4.0Q2.0 19.0 2.0 17.0V7.0Q2.0 5.0 4.0 5.0Z", "M3 6l9 7 9-7"]; + case 'settings': return ["M9.807,4.828L10.264,2.152L13.736,2.152L14.193,4.828L15.521,5.378L17.736,3.808L20.192,6.264L18.622,8.479L19.172,9.807L21.848,10.264L21.848,13.736L19.172,14.193L18.622,15.521L20.192,17.736L17.736,20.192L15.521,18.622L14.193,19.172L13.736,21.848L10.264,21.848L9.807,19.172L8.479,18.622L6.264,20.192L3.808,17.736L5.378,15.521L4.828,14.193L2.152,13.736L2.152,10.264L4.828,9.807L5.378,8.479L3.808,6.264L6.264,3.808L8.479,5.378Z", "M8.8 12.0a3.2 3.2 0 1 0 6.4 0a3.2 3.2 0 1 0 -6.4 0"]; + case 'refresh': return ["M19.5 8A8 8 0 1 0 20 15M20 3v6h-6"]; + case 'chart': return ["M3 3v18h19M7 17v-4M12 17V8M18 17V4"]; + case 'arrow': return ["M4 12h16M14 6l6 6-6 6"]; + case 'muted': return ["M3 9h4l5-5v16l-5-5H3zM16 9l6 6m0-6-6 6"]; + case 'history': return ["M5.5 5.5a8.5 8.5 0 1 1-1 12M3 4v6h6M12 7v6l4 2"]; + case 'unlock': return ["M7.0 10.0H17.0Q19.0 10.0 19.0 12.0V20.0Q19.0 22.0 17.0 22.0H7.0Q5.0 22.0 5.0 20.0V12.0Q5.0 10.0 7.0 10.0Z", "M8 10V6a4 4 0 0 1 7.5-2M12 15v2"]; + case 'check': return ["M5 12l4 4L19 6"]; + case 'info': return ["M3.0 12.0a9.0 9.0 0 1 0 18.0 0a9.0 9.0 0 1 0 -18.0 0", "M12 10v7M12 6h.01"]; + case 'display': return ["M4.0 3.0H20.0Q22.0 3.0 22.0 5.0V15.0Q22.0 17.0 20.0 17.0H4.0Q2.0 17.0 2.0 15.0V5.0Q2.0 3.0 4.0 3.0Z", "M12 17v4M7 21h10"]; + case 'lock': return ["M7.0 10.0H17.0Q19.0 10.0 19.0 12.0V20.0Q19.0 22.0 17.0 22.0H7.0Q5.0 22.0 5.0 20.0V12.0Q5.0 10.0 7.0 10.0Z", "M8 10V6a4 4 0 0 1 8 0v4M12 15v2"]; + case 'eye': return ["M2 12c5-9 15-9 20 0-5 9-15 9-20 0z", "M9.0 12.0a3.0 3.0 0 1 0 6.0 0a3.0 3.0 0 1 0 -6.0 0"]; + case 'mouse_left': return ["M5 12v-3a7 7 0 0 1 7-7v10z", "M12.0 2.0H12.0Q19.0 2.0 19.0 9.0V15.0Q19.0 22.0 12.0 22.0H12.0Q5.0 22.0 5.0 15.0V9.0Q5.0 2.0 12.0 2.0Z", "M12 2v10M5 12h14"]; + case 'sliders': return ["M3 6h3m4 0h11M3 12h10m4 0h4M3 18h3m4 0h11", "M6.0 6.0a2.0 2.0 0 1 0 4.0 0a2.0 2.0 0 1 0 -4.0 0", "M13.0 12.0a2.0 2.0 0 1 0 4.0 0a2.0 2.0 0 1 0 -4.0 0", "M6.0 18.0a2.0 2.0 0 1 0 4.0 0a2.0 2.0 0 1 0 -4.0 0"]; + case 'close': return ["M6 6l12 12M18 6L6 18"]; + case 'eye_off': return ["M3 3l18 18M10 5c5-1 9 3 12 7a21 21 0 0 1-4 5M6 6a22 22 0 0 0-4 6c5 8 11 9 16 5M10 10a3 3 0 0 0 4 4"]; + case 'chevron': return ["M9 5l7 7-7 7"]; + case 'back': return ["M15 4l-8 8 8 8"]; + case 'mouse': return ["M12.0 2.0H12.0Q19.0 2.0 19.0 9.0V15.0Q19.0 22.0 12.0 22.0H12.0Q5.0 22.0 5.0 15.0V9.0Q5.0 2.0 12.0 2.0Z", "M12 3v7"]; + case 'clipboard': return ["M8 4H6a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V6a2 2 0 0 0-2-2h-2", "M10.0 2.0H14.0Q16.0 2.0 16.0 4.0V5.0Q16.0 7.0 14.0 7.0H10.0Q8.0 7.0 8.0 5.0V4.0Q8.0 2.0 10.0 2.0Z"]; + case 'audio': return ["M3 9h4l5-5v16l-5-5H3zM17 6a8 8 0 0 1 0 12M15 9a4 4 0 0 1 0 6"]; + case 'mouse_right': return ["M12 2a7 7 0 0 1 7 7v3h-7z", "M12.0 2.0H12.0Q19.0 2.0 19.0 9.0V15.0Q19.0 22.0 12.0 22.0H12.0Q5.0 22.0 5.0 15.0V9.0Q5.0 2.0 12.0 2.0Z", "M12 2v10M5 12h14"]; + case 'key': return ["M2.0 8.0a5.0 5.0 0 1 0 10.0 0a5.0 5.0 0 1 0 -10.0 0", "M10.5 11.5L19 20m-4-4 3-3m0 6 3-3M6 7h.01"]; + default: return ['M12 10v7M12 6h.01']; + } + } + private draw(): void { + const ctx = this.context; + // SVG path coordinates are pixels. Scale both geometry and stroke from the + // 24-unit viewport into physical pixels, including the display density. + const extent = this.getUIContext().vp2px(this.iconExtent); + ctx.clearRect(0, 0, extent, extent); ctx.save(); ctx.scale(extent / 24, extent / 24); + ctx.strokeStyle = this.color; ctx.fillStyle = this.color; ctx.lineWidth = 1.7; ctx.lineCap = 'round'; ctx.lineJoin = 'round'; + this.paths().forEach((data: string, index: number) => { + const path = new Path2D(data, LengthMetricsUnit.PX); + if (this.name === 'cursor') { ctx.fillStyle = '#FFFFFF'; ctx.lineWidth = 1.5; ctx.fill(path); } + if (index === 0 && (this.name === 'mouse_left' || this.name === 'mouse_right')) ctx.fill(path); + ctx.stroke(path); + }); + ctx.restore(); + } + build() { + Canvas(this.context).width(this.iconExtent).height(this.iconExtent).onReady(() => this.draw()) + .hitTestBehavior(HitTestMode.None).accessibilityLevel('no') + } +} +@Component +export struct MobileIconButton { + @Prop icon: string = 'close'; + @Prop label: string = ''; + iconExtent: number = 40; + iconSize: number = 22; + color: string = MobileTheme.ink; + buttonFill: string = '#FFFFFF'; + action: () => void = () => {}; + build() { + Button({ type: ButtonType.Normal, stateEffect: true }) { + MobileIcon({ name: this.icon, iconExtent: this.iconSize, color: this.color }) + }.width(this.iconExtent).height(this.iconExtent).padding(0).borderRadius(10).backgroundColor(this.buttonFill) + .accessibilityText(this.label).onClick(this.action) + } +} +@Component +export struct MobilePrimaryButton { + label: string = '连接'; + @Prop available: boolean = true; + arrow: boolean = false; + buttonHeight: number = 48; + action: () => void = () => {}; + build() { + Button({ type: ButtonType.Normal, stateEffect: true }) { + Row({ space: 8 }) { + Text(this.label).fontSize(16).fontWeight(600).fontColor(Color.White) + if (this.arrow) { MobileIcon({ name: 'arrow', iconExtent: 18, color: '#FFFFFF' }) } + }.justifyContent(FlexAlign.Center).width('100%') + }.height(this.buttonHeight).width('100%').padding({ left: 12, right: 12 }).borderRadius(12) + .linearGradient({ angle: 135, colors: [[MobileTheme.blue, 0], ['#1F61C7', 1]] }) + .enabled(this.available).opacity(this.available ? 1 : 0.45).accessibilityText(this.label).onClick(this.action) + } +} +@Component +export struct MobileSegment { + labels: string[] = []; + @Prop @Watch('selectionChanged') selection: number = 0; + @Prop available: boolean = true; + changed: (index: number) => void = () => {}; + @State private selected: number = 0; + @State private viewWidth: number = 1; + aboutToAppear(): void { this.selected = this.selection; } + private selectionChanged(): void { this.getUIContext().animateTo({ duration: 200, curve: Curve.EaseInOut }, () => this.selected = this.selection); } + build() { + Stack({ alignContent: Alignment.Start }) { + Row() {}.width(Math.max(0, this.viewWidth - 4) / Math.max(1, this.labels.length)).height(28) + .backgroundColor(Color.White).borderRadius(7).shadow({ radius: 2, color: '#18000000', offsetY: 1 }).translate({ x: 2 + this.selected * Math.max(0, this.viewWidth - 4) / Math.max(1, this.labels.length) }) + Row() { + ForEach(this.labels, (label: string, index: number) => { + Text(label).fontSize(12).fontWeight(this.selection === index ? 600 : 400).fontColor(MobileTheme.ink) + .textAlign(TextAlign.Center).height(32).layoutWeight(1).enabled(this.available) + .accessibilityLevel('yes').accessibilityText(label).accessibilitySelected(this.selection === index) + .onClick(() => this.changed(index)) + }, (label: string) => label) + }.padding({ left: 2, right: 2 }).width('100%') + }.height(32).width('100%').backgroundColor('#E9E9EE').borderRadius(9).opacity(this.available ? 1 : 0.5) + .onAreaChange((_old: Area, next: Area) => this.viewWidth = Number(next.width)) + } +} diff --git a/apps/harmony/entry/src/main/ets/pages/Mobile.ets b/apps/harmony/entry/src/main/ets/pages/Mobile.ets new file mode 100644 index 00000000..21eefa74 --- /dev/null +++ b/apps/harmony/entry/src/main/ets/pages/Mobile.ets @@ -0,0 +1,637 @@ +import { common } from '@kit.AbilityKit'; +import { MobileController, MobileState, RemoteDisplay } from '../mobile/MobileController'; +import { MobileRemoteView } from '../mobile/MobileRemoteView'; +import { MobileTheme, MobileSegment, MobileIcon, MobileIconButton, MobilePrimaryButton } from '../mobile/MobileWidgets'; +import { RecentDevice } from '../mobile/MobileStore'; +import { AnnouncementItem } from '../mobile/AnnouncementInbox'; +import { AnnouncementSpan, announcementSpans } from '../mobile/AnnouncementText'; +import { util } from '@kit.ArkTS'; +import { KeyboardAvoidMode } from '@kit.ArkUI'; + +interface LicenseDocument { title: string; asset: string; sha256?: string; } +interface LicenseComponent { id: string; name: string; license: string; version: string; sourceURL: string; documents: LicenseDocument[]; } +interface LicenseCatalog { components: LicenseComponent[]; } +interface SourceMetadata { revision: string; miniRTCRevision: string; isModified: boolean; sourceURL: string; buildDocuments: LicenseDocument[]; } +@Entry +@Component +struct Mobile { + @State private state: MobileState = new MobileState(); + @State private viewWidth: number = 400; + @State private viewHeight: number = 720; + @State private idFocused: boolean = false; + @State private serverExpanded: boolean = false; + @State private customServer: boolean = false; + @State private previewToggle: boolean = false; + @State private privacyText: string = ''; + @State private editingServer: string = ''; + @State private editingPort: string = ''; + @State private showPassword: boolean = false; + @State private accepted: boolean = false; + @State private deleteId: string = ''; + @State private refreshing: boolean = false; + @State private licenses: LicenseComponent[] = []; + @State private component: LicenseComponent | undefined = undefined; + @State private source: SourceMetadata | undefined = undefined; + private controller?: MobileController; + aboutToAppear(): void { + // Keep the entire password/clipboard sheet above the software keyboard. + this.getUIContext().setKeyboardAvoidMode(KeyboardAvoidMode.RESIZE); + const context = this.getUIContext().getHostContext() as common.UIAbilityContext; + this.controller = new MobileController(context); this.state = this.controller.state; + try { + const decoder = util.TextDecoder.create('utf-8'); + this.privacyText = decoder.decodeWithStream(context.resourceManager.getRawFileContentSync('PRIVACY.md')); + this.licenses = (JSON.parse(decoder.decodeWithStream(context.resourceManager.getRawFileContentSync('ThirdPartyLicenses.json'))) as LicenseCatalog).components; + this.source = JSON.parse(decoder.decodeWithStream(context.resourceManager.getRawFileContentSync('SourceMetadata.json'))) as SourceMetadata; + } catch { /* The package verification rejects missing legal resources. */ } + } + onBackPress(): boolean { + if (this.state.modal === 'connecting') { this.controller?.end(''); return true; } + if (this.state.modal) { if (this.state.modal !== 'consent') this.dismissModal(); return true; } + if (this.state.ready) { this.state.modal = 'disconnect'; return true; } + if (this.state.page === 'home') return false; + this.back(); return true; + } + private navigate(page: string): void { + if (page === 'settings') { + this.editingServer = this.state.server; this.editingPort = this.state.port; + this.customServer = this.state.server !== 'api.crossdesk.cn' || this.state.port !== '9099'; + this.previewToggle = this.state.previews; + } + this.getUIContext().animateTo({ duration: 200, curve: Curve.EaseInOut }, () => this.controller?.navigate(page)); + } + private back(): void { + const page = this.state.page; + if (page === 'settings' && this.customServer && this.serverExpanded && !this.saveServer()) return; + this.navigate(page === 'document' ? this.state.documentBack : page === 'announcement' ? 'announcements' : + page === 'component' ? 'licenses' : page === 'licenses' || page === 'source' ? 'about' : page === 'privacy' || page === 'about' ? 'settings' : 'home'); + if (this.state.page === 'home' && !this.state.consent) this.state.modal = 'consent'; + } + private title(): string { + switch (this.state.page) { + case 'settings': return '设置'; case 'privacy': return '隐私与授权'; + case 'about': return '关于 CrossDesk'; case 'licenses': return '开源组件'; case 'component': return this.component?.name ?? ''; + case 'source': return '源码与构建说明'; case 'announcements': return '通知公告'; case 'announcement': return '公告详情'; + case 'document': return this.state.documentTitle; default: return ''; + } + } + private deviceId(value: string): string { return value.replace(/[^0-9]/g, '').substring(0, 9); } + private formatId(value: string): string { return this.deviceId(value).replace(/(.{3})(?=.)/g, '$1 '); } + private policyParagraphs(body: string): string[] { + const text = body.replace(/\r\n/g, '\n'); + const start = text.indexOf('## 中文\n'), end = start < 0 ? -1 : text.indexOf('## English', start); + const localized = start >= 0 && end > start ? text.substring(start + '## 中文\n'.length, end) : text; + return localized.split(/\n\s*\n/).map((paragraph: string) => paragraph.trim()).filter((paragraph: string) => !!paragraph && paragraph !== '---'); + } + private openAnnouncement(item: AnnouncementItem): void { this.controller?.inbox.markRead(item); this.state.announcement = item; this.navigate('announcement'); } + private saveServer(): boolean { + this.controller?.saveSettings(this.editingServer, this.editingPort); + const saved = this.state.server === this.editingServer.trim() && this.state.port === this.editingPort; + if (saved) this.serverExpanded = false; + return saved; + } + private dismissModal(): void { + if (this.state.modal === 'password') { this.state.password = ''; this.showPassword = false; } + this.previewToggle = this.state.previews; this.state.modal = ''; + } + private confirmModal(): void { + const modal = this.state.modal; this.state.modal = ''; + if (modal === 'disconnect') this.controller?.end(''); + if (modal === 'revoke') this.controller?.revoke(); + if (modal === 'previews') { this.controller?.setPreviews(true); this.previewToggle = true; } + if (modal === 'clearPreviews') this.controller?.clearPreviews(); + if (modal === 'delete') this.controller?.deleteRecent(this.deleteId); + if (modal === 'deleteAnnouncement' && this.state.announcement) { this.controller?.inbox.dismiss(this.state.announcement); this.navigate('announcements'); } + } + @Builder + SectionTitle(label: string) { + Text(label).fontSize(13).fontColor(MobileTheme.secondary).width('100%').padding({ left: 16, top: 12, bottom: 8 }) + } + @Builder + Footer(label: string) { + Text(label).fontSize(12).lineHeight(18).fontColor(MobileTheme.secondary).width('100%') + .padding({ left: 16, right: 16, top: 8, bottom: 16 }) + } + @Builder + Rule() { Divider().strokeWidth(0.5).color(MobileTheme.line) } + @Builder + ValueRow(label: string, value: string) { + Row({ space: 16 }) { + Text(label).fontSize(16).fontColor(MobileTheme.ink) + Text(value).fontSize(15).fontColor(MobileTheme.secondary).layoutWeight(1).textAlign(TextAlign.End).copyOption(CopyOptions.LocalDevice) + }.constraintSize({ minHeight: 52 }).width('100%').padding({ left: 16, right: 16, top: 14, bottom: 14 }) + } + @Builder + Action(label: string, action: () => void, danger: boolean = false) { + Button(label, { type: ButtonType.Normal, stateEffect: true }).fontSize(16).fontWeight(500) + .fontColor(danger ? MobileTheme.red : MobileTheme.blue).backgroundColor(Color.White) + .height(48).width('100%').borderRadius(12).onClick(action) + } + @Builder + SettingRow(label: string, detail: string, action: () => void) { + Button({ type: ButtonType.Normal, stateEffect: true }) { + Row({ space: 8 }) { + Text(label).fontSize(16).fontColor(MobileTheme.ink).layoutWeight(1).textAlign(TextAlign.Start) + if (detail) { Text(detail).fontSize(15).fontColor(MobileTheme.secondary).maxLines(1).constraintSize({ maxWidth: '46%' }).textOverflow({ overflow: TextOverflow.Ellipsis }) } + MobileIcon({ name: 'chevron', iconExtent: 16, color: '#B7B7BD' }) + }.width('100%').constraintSize({ minHeight: 52 }).padding({ left: 16, right: 12, top: 12, bottom: 12 }) + }.width('100%').padding(0).backgroundColor(Color.White).borderRadius(0).onClick(action).accessibilityText(`${label} ${detail}`) + } + @Builder + ConnectionCard() { + Column({ space: 16 }) { + Text('远程桌面').fontSize(20).fontWeight(600).fontColor(MobileTheme.ink).width('100%') + Row({ space: 10 }) { + TextInput({ text: this.formatId(this.state.remote), placeholder: '对端 ID' }).type(InputType.Number).maxLength(11) + .fontSize(20).fontWeight(500).fontColor(MobileTheme.ink).placeholderColor(MobileTheme.secondary) + .height(54).layoutWeight(1).backgroundColor(MobileTheme.background).borderRadius(12) + .border({ width: this.idFocused ? 2 : 1, color: this.idFocused ? MobileTheme.blue : MobileTheme.line }) + .padding({ left: 14, right: 14 }).onFocus(() => this.idFocused = true).onBlur(() => this.idFocused = false) + .onChange((value: string) => this.state.remote = this.deviceId(value)).onSubmit(() => this.controller?.requestConnection(this.state.remote)) + Column() { + MobilePrimaryButton({ label: '连接', arrow: true, buttonHeight: 54, available: this.state.remote.length > 0, + action: () => this.controller?.requestConnection(this.state.remote) }) + }.width(96) + }.width('100%') + }.padding(18).width('100%').backgroundColor(Color.White).borderRadius(20) + .border({ width: 0.5, color: MobileTheme.line }).shadow({ radius: 14, color: '#0E000000', offsetY: 5 }) + } + @Builder + RecentCard(device: RecentDevice) { + Column() { + Stack() { + Text(device.platform === 'windows' ? 'Windows' : device.platform === 'macos' ? 'macOS' : device.platform === 'linux' ? 'Linux' : '远程电脑') + .fontSize(19).fontWeight(600).fontColor(Color.White).width('100%').height('100%').textAlign(TextAlign.Center) + if (this.state.previews && device.preview) { Image(`file://${device.preview}`).objectFit(ImageFit.Cover).width('100%').height('100%') } + }.aspectRatio(16 / 9).width('100%').backgroundColor(device.platform === 'windows' ? '#316AB7' : device.platform === 'macos' ? '#8057B9' : device.platform === 'linux' ? '#AE642B' : '#677587') + Column({ space: 4 }) { + Text(device.name || device.id).fontSize(12).fontWeight(600).maxLines(1).textOverflow({ overflow: TextOverflow.Ellipsis }).width('100%') + Row({ space: 4 }) { + Text(`ID ${device.id}`).fontSize(10).fontColor(MobileTheme.secondary).layoutWeight(1) + if (device.remember) { MobileIcon({ name: 'key', iconExtent: 11, color: MobileTheme.secondary }).accessibilityText('已保存密码') } + Text(this.controller?.presence.online(device.id, this.controller?.now() ?? 0) ? '在线' : '离线').fontSize(10) + .fontColor(this.controller?.presence.online(device.id, this.controller?.now() ?? 0) ? MobileTheme.green : MobileTheme.secondary) + }.width('100%') + }.padding(9).width('100%') + }.width('100%').backgroundColor(Color.White).border({ width: 0.5, color: MobileTheme.line }).borderRadius(14).clip(true) + .onClick(() => this.controller?.requestConnection(device.id)) + .gesture(LongPressGesture().onAction(() => { this.deleteId = device.id; this.state.modal = 'recentActions'; })) + } + @Builder + Recent() { + Column({ space: 14 }) { + Row() { + Text('最近连接').fontSize(20).fontWeight(600).fontColor(MobileTheme.ink).layoutWeight(1) + if (this.state.recent.length) { Text(`${this.state.recent.length} 台设备`).fontSize(12).fontColor(MobileTheme.secondary) } + }.width('100%') + if (this.state.recent.length === 0) { + Column() { + MobileIcon({ name: 'history', iconExtent: 34, color: '#B6B6BB' }) + Text('还没有连接记录').fontSize(17).fontWeight(600).fontColor(MobileTheme.ink).margin({ top: 12, bottom: 8 }) + Text('成功连接后,这里会显示最近的远程设备。').fontSize(13).lineHeight(19).fontColor(MobileTheme.secondary).textAlign(TextAlign.Center) + }.layoutWeight(1).width('100%').justifyContent(FlexAlign.Center).padding({ left: 6, right: 6 }) + } else { + Scroll() { + GridRow({ columns: { xs: 1, sm: 2, md: 3, lg: 4 }, breakpoints: { value: ['300vp', '564vp', '756vp'], reference: BreakpointsReference.ComponentSize }, gutter: 12 }) { + ForEach(this.state.recent, (device: RecentDevice) => { GridCol() { this.RecentCard(device) } }, (device: RecentDevice) => device.id) + }.width('100%') + }.layoutWeight(1).width('100%').align(Alignment.Top).scrollBar(BarState.Off) + } + }.padding(16).width('100%').height('100%').backgroundColor(Color.White).borderRadius(18).border({ width: 0.5, color: MobileTheme.line }) + } + @Builder + Home() { + Column() { + Row({ space: 12 }) { + Button({ type: ButtonType.Normal }) { + Row({ space: 6 }) { + Row() {}.width(6).height(6).borderRadius(3).backgroundColor(this.state.signal === 1 ? MobileTheme.green : MobileTheme.secondary) + Text(this.state.signalText).fontSize(12).fontWeight(600).fontColor(this.state.signal === 1 ? MobileTheme.green : MobileTheme.secondary) + } + }.padding({ left: 11, right: 11 }).height(34).backgroundColor(Color.White).borderRadius(17).onClick(() => this.controller?.reconnect()) + Blank() + Stack({ alignContent: Alignment.TopEnd }) { + MobileIconButton({ icon: 'envelope', label: '通知公告', action: () => this.navigate('announcements') }) + if (this.state.unread > 0) { + Text(this.state.unread > 99 ? '99+' : `${this.state.unread}`).fontSize(10).fontWeight(600).fontColor(Color.White) + .textAlign(TextAlign.Center).padding({ left: 4, right: 4 }).constraintSize({ minWidth: 16 }).height(16) + .backgroundColor(MobileTheme.red).borderRadius(8).translate({ x: 5, y: -3 }).hitTestBehavior(HitTestMode.None) + } + } + Stack({ alignContent: Alignment.TopEnd }) { + MobileIconButton({ icon: 'settings', label: '设置', action: () => this.navigate('settings') }) + if (this.state.update) { + Text('!').fontSize(10).fontWeight(600).fontColor(Color.White).textAlign(TextAlign.Center).width(16).height(16) + .backgroundColor(MobileTheme.red).borderRadius(8).translate({ x: 5, y: -3 }).hitTestBehavior(HitTestMode.None) + } + } + }.padding({ left: 20, right: 20 }).height(56).width('100%').constraintSize({ maxWidth: 1200 }) + Scroll() { + if (this.viewWidth >= 900) { + Row({ space: 24 }) { + Column() { this.ConnectionCard() }.width(360) + Column() { this.Recent() }.layoutWeight(1).height('100%') + }.alignItems(VerticalAlign.Top).width('100%').constraintSize({ maxWidth: 1200 }) + .height(Math.max(360, this.viewHeight - 56)).padding({ left: 20, right: 20, top: 24, bottom: 12 }) + } else { + Column({ space: 20 }) { + this.ConnectionCard() + Column() { this.Recent() }.layoutWeight(1).width('100%') + }.width('100%').height(Math.max(440, this.viewHeight - 56)).padding({ left: 20, right: 20, top: 24, bottom: 12 }) + } + }.layoutWeight(1).width('100%').align(Alignment.Top).scrollBar(BarState.Off) + }.width('100%').height('100%') + } + @Builder + PreviewRow() { + Row({ space: 12 }) { + Text('保存远程画面预览').fontSize(16).fontColor(MobileTheme.ink).layoutWeight(1) + Toggle({ type: ToggleType.Switch, isOn: this.previewToggle }).selectedColor(MobileTheme.green) + .onChange((value: boolean) => { + this.previewToggle = value; + if (value && !this.state.previews) this.state.modal = 'previews'; + else if (!value) this.controller?.setPreviews(false); + }).accessibilityText('保存远程画面预览') + }.constraintSize({ minHeight: 52 }).padding({ left: 16, right: 16, top: 8, bottom: 8 }).width('100%') + } + @Builder + Settings() { + Column() { + this.SectionTitle('鼠标控制') + Column() { + MobileSegment({ labels: ['相对位置', '绝对位置'], selection: this.state.relative ? 0 : 1, + changed: (i: number) => this.controller?.setRelative(i === 0) }) + }.padding(12).width('100%').backgroundColor(Color.White).borderRadius(26).border({ width: 0.5, color: MobileTheme.line }) + this.SectionTitle('画面偏好') + Column() { + MobileSegment({ labels: ['帧率优先', '画质优先', '平衡'], selection: this.state.videoPreference, + changed: (i: number) => this.controller?.setPreference(i) }) + }.padding(12).width('100%').backgroundColor(Color.White).borderRadius(26).border({ width: 0.5, color: MobileTheme.line }) + Column() { + this.SettingRow('服务器', this.customServer ? '自定义' : '默认', () => this.serverExpanded = !this.serverExpanded) + if (this.serverExpanded) { + this.Rule() + Row() { + Text('使用自定义服务器').fontSize(16).fontColor(MobileTheme.ink).layoutWeight(1) + Toggle({ type: ToggleType.Switch, isOn: this.customServer }).selectedColor(MobileTheme.green).onChange((value: boolean) => { + this.customServer = value; + if (!value) { this.editingServer = 'api.crossdesk.cn'; this.editingPort = '9099'; this.controller?.saveSettings(this.editingServer, this.editingPort); } + }).accessibilityText('使用自定义服务器') + }.padding({ left: 16, right: 16, top: 8, bottom: 8 }).width('100%') + if (this.customServer) { + Column({ space: 8 }) { + TextInput({ text: this.editingServer, placeholder: '服务器地址' }).fontSize(17).height(48).borderRadius(12) + .backgroundColor(MobileTheme.background).onChange((v: string) => this.editingServer = v) + TextInput({ text: this.editingPort, placeholder: '服务器端口' }).fontSize(17).height(48).borderRadius(12) + .backgroundColor(MobileTheme.background).type(InputType.Number).onChange((v: string) => this.editingPort = v) + this.Action('完成', () => { this.saveServer(); }) + }.padding({ left: 12, right: 12, bottom: 12 }).width('100%') + } + } + }.width('100%').backgroundColor(Color.White).borderRadius(26).clip(true).border({ width: 0.5, color: MobileTheme.line }).margin({ top: 28 }) + Text(this.state.identity ? `本机 ID ${this.formatId(this.state.identity)}` : this.state.signalText).fontSize(12).fontColor(MobileTheme.secondary) + .textAlign(TextAlign.Center).padding({ left: 16, right: 16, top: 12, bottom: 16 }).width('100%').copyOption(CopyOptions.LocalDevice) + this.SectionTitle('隐私') + Column() { + this.SettingRow('隐私与授权', this.state.consent ? '已授权' : '未授权', () => this.navigate('privacy')) + this.Rule() + this.PreviewRow() + this.Rule() + Button({ type: ButtonType.Normal }) { Text('清除预览图').fontSize(16).fontColor(MobileTheme.red).width('100%').textAlign(TextAlign.Start) } + .height(48).width('100%').padding({ left: 16 }).backgroundColor(Color.White) + .onClick(() => this.state.modal = 'clearPreviews') + }.width('100%').backgroundColor(Color.White).borderRadius(26).clip(true).border({ width: 0.5, color: MobileTheme.line }) + Column() { this.SettingRow('关于', this.state.update ? '新版本 ●' : '', () => this.navigate('about')) } + .width('100%').backgroundColor(Color.White).borderRadius(26).clip(true).border({ width: 0.5, color: MobileTheme.line }).margin({ top: 28 }) + }.width('100%') + } + @Builder + Privacy() { + Column() { + Column() { + this.ValueRow('授权状态', this.state.consent ? '已授权' : '未授权') + this.Rule() + this.SettingRow('隐私政策', '', () => this.controller?.document('PRIVACY.md', '隐私政策', 'privacy')) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + this.Footer('设备身份及选择保存的访问密码加密保存在本机。画面预览默认关闭,开启后从下一次连接起保存。剪贴板内容在结束会话时清除。离开应用后通过画中画或后台任务保持会话,可随时返回应用断开。') + if (this.state.consent) { this.Action('撤回联网授权', () => this.state.modal = 'revoke', true) } + else { MobilePrimaryButton({ label: '阅读并授权', action: () => this.state.modal = 'consent' }) } + }.width('100%') + } + @Builder + About() { + Column() { + Column() { + this.ValueRow('版本', this.state.version) + if (this.state.update) { this.Rule(); this.SettingRow('新版本可用', `v${this.state.update}`, () => this.controller?.platform.openLink('https://crossdesk.cn')) } + if (this.state.updateStatus) { this.Footer(this.state.updateStatus) } + this.Rule() + this.Action('检查更新', () => this.controller?.checkLocal(true)) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + Column() { + this.SettingRow('软件许可', 'GPL-3.0-only', () => this.controller?.document('LICENSE', '软件许可')) + this.Rule() + this.SettingRow('开源软件权利说明', '', () => this.controller?.document('OPEN_SOURCE_NOTICE.txt', '开源软件权利说明')) + this.Rule() + this.SettingRow('开源组件', '', () => this.navigate('licenses')) + this.Rule() + this.SettingRow('源码与构建说明', '', () => this.navigate('source')) + }.backgroundColor(Color.White).borderRadius(12).clip(true).width('100%').border({ width: 0.5, color: MobileTheme.line }).margin({ top: 28 }) + this.Footer('许可与版权声明可离线查阅。') + }.width('100%') + } + @Builder + Announcements() { + Column({ space: 12 }) { + if (!this.state.consent) { Text('同意联网后可查看公告').fontSize(14); this.Action('查看联网说明', () => this.state.modal = 'consent') } + if (this.controller?.inbox.failed) { this.Action('加载失败,点击重试', () => this.controller?.inbox.refresh()) } + if (this.controller?.inbox.loading) { LoadingProgress().width(28).height(28) } + if (this.controller?.inbox.saveFailed) { Text('已读或删除状态保存失败,请重试').fontSize(12).fontColor(MobileTheme.red) } + if (!this.state.announcements.length && this.controller?.inbox.loaded && !this.controller?.inbox.loading) { Text('暂无公告').fontSize(16).fontColor(MobileTheme.secondary).padding(40) } + ForEach(this.state.announcements, (item: AnnouncementItem) => { + Column({ space: 8 }) { + Row({ space: 8 }) { if (!item.read) { Row() {}.width(6).height(6).borderRadius(3).backgroundColor(MobileTheme.blue) } Text(item.title).fontSize(17).fontWeight(600).layoutWeight(1) }.width('100%') + Text(new Date(item.updated_at * 1000).toLocaleDateString('zh-CN')).fontSize(12).fontColor(MobileTheme.secondary).width('100%') + Text(item.body).fontSize(13).fontColor(MobileTheme.secondary).maxLines(2).textOverflow({ overflow: TextOverflow.Ellipsis }).width('100%') + }.padding(18).width('100%').backgroundColor(Color.White).borderRadius(18).border({ width: 0.5, color: MobileTheme.line }) + .onClick(() => this.openAnnouncement(item)).gesture(LongPressGesture().onAction(() => { this.state.announcement = item; this.state.modal = 'deleteAnnouncement'; })) + }, (item: AnnouncementItem) => `${item.id}:${item.revision}:${item.read}`) + if ((this.controller?.inbox.total ?? 0) > this.state.announcements.length) { this.Action('加载更多', () => this.controller?.inbox.more()) } + }.width('100%') + } + @Builder + Content() { + if (this.state.page === 'settings') { this.Settings() } + else if (this.state.page === 'privacy') { this.Privacy() } + else if (this.state.page === 'about') { this.About() } + else if (this.state.page === 'announcements') { this.Announcements() } + else if (this.state.page === 'announcement') { + Column({ space: 16 }) { + Text(this.state.announcement?.title ?? '').fontSize(24).fontWeight(600).width('100%').copyOption(CopyOptions.LocalDevice) + Text() { + ForEach(announcementSpans(this.state.announcement?.body ?? ''), (part: AnnouncementSpan) => { + if (part.url) { + Span(part.text).fontColor(MobileTheme.blue).onClick(() => this.controller?.platform.openLink(part.url)) + } else { Span(part.text) } + }) + }.fontSize(16).lineHeight(25).width('100%').copyOption(CopyOptions.LocalDevice) + this.Action('删除公告', () => this.state.modal = 'deleteAnnouncement', true) + }.width('100%') + } else if (this.state.page === 'document') { + if (this.state.documentTitle === '隐私政策') { this.PolicyText(this.state.documentText) } + else { + Column({ space: 12 }) { + ForEach(this.state.documentText.match(/[\s\S]{1,4000}/g) ?? [], (paragraph: string) => { + Text(paragraph).fontSize(15).lineHeight(24).width('100%').copyOption(CopyOptions.LocalDevice) + }) + }.width('100%') + } + } else if (this.state.page === 'licenses') { + Column() { + this.SectionTitle('OpenFEC 告知') + Column() { this.Footer('OpenFEC 提供前向纠错能力,遵循 CeCILL-C 1.0 许可。其许可规定了有限保证及责任限制,请阅读组件详情。') } + .width('100%').backgroundColor(Color.White).borderRadius(12).border({ width: 0.5, color: MobileTheme.line }) + Column() { + ForEach(this.licenses, (component: LicenseComponent, index: number) => { + if (index > 0) { this.Rule() } + Button({ type: ButtonType.Normal }) { + Row({ space: 12 }) { + Column({ space: 4 }) { + Text(component.name).fontSize(16).fontColor(MobileTheme.ink).width('100%') + Text(component.license).fontSize(12).fontColor(MobileTheme.secondary).width('100%') + }.layoutWeight(1).alignItems(HorizontalAlign.Start) + MobileIcon({ name: 'chevron', iconExtent: 16, color: '#B7B7BD' }) + }.width('100%').padding({ left: 16, right: 12, top: 12, bottom: 12 }) + }.width('100%').padding(0).backgroundColor(Color.White).onClick(() => { this.component = component; this.navigate('component'); }) + }, (component: LicenseComponent) => component.id) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }).margin({ top: 28 }) + this.Footer('各组件遵循所列开源许可证。许可与版权声明可离线查阅,查看源码需要网络连接。') + }.width('100%') + } else if (this.state.page === 'component') { + Column() { + Column() { + this.ValueRow('版本', this.component?.version ?? '') + this.Rule() + this.ValueRow('许可', this.component?.license ?? '') + this.Rule() + this.SettingRow('查看源码', '', () => this.controller?.platform.openLink(this.component?.sourceURL ?? '')) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + this.SectionTitle('许可与版权声明') + Column() { + ForEach(this.component?.documents ?? [], (document: LicenseDocument, index: number) => { + if (index > 0) { this.Rule() } + this.SettingRow(document.title, '', () => this.controller?.document(document.asset, document.title, 'component')) + }) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + }.width('100%') + } else if (this.state.page === 'source') { + Column() { + this.SectionTitle('本版本源码与构建说明') + Column() { + this.ValueRow('源码版本', (this.source?.revision ?? '').substring(0, 12)) + this.Rule() + this.ValueRow('MiniRTC', (this.source?.miniRTCRevision ?? '').substring(0, 12)) + if (this.source?.isModified) { this.Rule(); this.Footer('开发版本包含尚未发布的修改,以下链接为基础版本源码。') } + this.Rule() + this.SettingRow('查看源码', '', () => this.controller?.platform.openLink(this.source?.sourceURL ?? '')) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + this.SectionTitle('构建文档') + Column() { + ForEach(this.source?.buildDocuments ?? [], (document: LicenseDocument, index: number) => { + if (index > 0) { this.Rule() } + this.SettingRow(document.title, '', () => this.controller?.document(document.asset, document.title, 'source')) + }) + }.width('100%').backgroundColor(Color.White).borderRadius(12).clip(true).border({ width: 0.5, color: MobileTheme.line }) + }.width('100%') + } + } + @Builder + SheetHeader(title: string, subtitle: string = '') { + Row({ space: 12 }) { + Column({ space: 5 }) { + Text(title).fontSize(22).fontWeight(600).fontColor(MobileTheme.ink).width('100%') + if (subtitle) { Text(subtitle).fontSize(15).fontColor(MobileTheme.secondary).width('100%') } + }.layoutWeight(1) + MobileIconButton({ icon: 'close', label: '关闭', iconExtent: 36, iconSize: 20, action: () => this.dismissModal() }) + }.width('100%') + } + @Builder + PasswordSheet() { + Scroll() { + Column() { + Row() {}.width(36).height(5).backgroundColor('#C6C6CB').borderRadius(3).margin({ bottom: 20 }) + this.SheetHeader('连接远程桌面', `对端 ID ${this.formatId(this.state.remote)}`) + Row() { + TextInput({ text: this.state.password, placeholder: '访问密码' }).type(this.showPassword ? InputType.Normal : InputType.Password) + .showPasswordIcon(false).maxLength(128).fontSize(17).height(50).layoutWeight(1).backgroundColor(Color.Transparent) + .padding({ left: 14, right: 4 }).onChange((value: string) => this.state.password = value).onSubmit(() => this.controller?.begin(this.state.password)) + MobileIconButton({ icon: this.showPassword ? 'eye_off' : 'eye', label: this.showPassword ? '隐藏密码' : '显示密码', iconExtent: 44, + action: () => this.showPassword = !this.showPassword }) + }.width('100%').backgroundColor(Color.White).borderRadius(10).border({ width: 0.5, color: MobileTheme.line }).margin({ top: 18 }) + if (this.state.notice) { Text(this.state.notice).fontSize(13).fontColor(MobileTheme.red).width('100%').margin({ top: 8 }) } + Row({ space: 12 }) { + Column({ space: 3 }) { + Text('保存密码').fontSize(15).fontWeight(600).fontColor(MobileTheme.ink).width('100%') + Text('密码将加密保存在此设备上').fontSize(11).fontColor(MobileTheme.secondary).width('100%') + }.layoutWeight(1) + Toggle({ type: ToggleType.Switch, isOn: this.state.remember }).selectedColor(MobileTheme.green) + .onChange((value: boolean) => this.state.remember = value).accessibilityText('保存密码') + }.width('100%').margin({ top: 18, bottom: 18 }) + MobilePrimaryButton({ label: '连接', arrow: true, available: this.state.password.length > 0, action: () => this.controller?.begin(this.state.password) }) + }.padding({ left: 22, right: 22, top: 10, bottom: 22 }).width('100%') + }.width(this.viewWidth < 600 ? '100%' : Math.min(520, this.viewWidth - 48)).constraintSize({ maxHeight: this.viewHeight * 0.92 }) + .backgroundColor(MobileTheme.background).borderRadius(24).scrollBar(BarState.Off) + .shadow({ radius: 28, color: '#18000000', offsetY: 8 }) + } + @Builder + PolicyText(body: string) { + Column({ space: 14 }) { + ForEach(this.policyParagraphs(body), (paragraph: string) => { + Text(paragraph.replace(/^#+\s*/, '').replace(/\*\*/g, '')).fontSize(paragraph.startsWith('更新日期:') ? 12 : 16) + .fontWeight(paragraph.startsWith('#') ? 600 : 400).lineHeight(24) + .fontColor(paragraph.startsWith('更新日期:') ? MobileTheme.secondary : MobileTheme.ink).width('100%').copyOption(CopyOptions.LocalDevice) + }) + }.width('100%') + } + @Builder + ConsentSheet() { + Column() { + Text('隐私政策').fontSize(17).fontWeight(600).fontColor(MobileTheme.ink).height(56).width('100%').textAlign(TextAlign.Center) + this.Rule() + Scroll() { + Column({ space: 12 }) { + if (!this.state.privacyAvailable) { Text('隐私政策无法读取,请重新安装应用').fontSize(14).fontColor(MobileTheme.red) } + this.PolicyText(this.privacyText) + }.padding(20).width('100%') + }.layoutWeight(1).width('100%').align(Alignment.Top).scrollBar(BarState.Off) + this.Rule() + Column({ space: 12 }) { + Row({ space: 8 }) { + Checkbox({ name: 'consent' }).width(20).height(20).margin(0).select(this.accepted).selectedColor(MobileTheme.blue) + .onChange((value: boolean) => this.accepted = value).accessibilityText('我已阅读并同意隐私政策') + Text('我已阅读并同意《隐私政策》').fontSize(14).fontColor(MobileTheme.ink).layoutWeight(1).onClick(() => this.accepted = !this.accepted) + }.width('100%').constraintSize({ minHeight: 44 }) + MobilePrimaryButton({ label: '同意并继续', available: this.accepted && this.state.privacyAvailable, action: () => this.controller?.consent() }) + Button('暂不同意', { type: ButtonType.Normal }).fontSize(16).fontColor(MobileTheme.secondary).height(44).width('100%') + .backgroundColor(Color.Transparent).onClick(() => this.dismissModal()) + }.padding({ left: 20, right: 20, top: 12, bottom: 12 }).width('100%') + }.width(this.viewWidth < 600 ? '100%' : Math.min(640, this.viewWidth - 48)).height(Math.min(760, this.viewHeight * 0.9)) + .backgroundColor(Color.White).borderRadius(20).clip(true) + } + @Builder + DialogBody() { + if (this.state.modal === 'connecting') { + Column({ space: 14 }) { + LoadingProgress().width(34).height(34).color(MobileTheme.blue) + Text('正在连接').fontSize(20).fontWeight(600).fontColor(MobileTheme.ink) + Text(this.state.signal === 1 ? '正在建立安全连接…' : this.state.signalText).fontSize(14).fontColor(MobileTheme.secondary).textAlign(TextAlign.Center).constraintSize({ minHeight: 36 }) + }.padding({ left: 26, right: 26, top: 24, bottom: 20 }).width('100%') + this.Rule() + this.Action('取消连接', () => this.controller?.end(''), true) + } else if (this.state.modal === 'clipboard') { + Column({ space: 16 }) { + this.SheetHeader('剪贴板') + Text('发送文本后,点击“粘贴到远端”使用。').fontSize(13).fontColor(MobileTheme.secondary).width('100%') + TextArea({ text: this.state.clipboardText, placeholder: '要发送到远端的文本' }).fontSize(15).height(120).borderRadius(12).barState(BarState.Off) + .backgroundColor(MobileTheme.background).onChange((value: string) => this.state.clipboardText = value) + MobilePrimaryButton({ label: '发送文本', action: () => this.controller?.clipboard() }) + Row({ space: 8 }) { + Column() { this.Action('粘贴到远端', () => this.controller?.shortcut(86)) }.layoutWeight(1) + Column() { this.Action('复制远端文本', () => this.controller?.copyRemote()) }.layoutWeight(1) + }.width('100%') + }.padding(20).width('100%') + } else if (this.state.modal === 'displays') { + Column({ space: 12 }) { + this.SheetHeader('显示器') + if (!this.state.displays.length) { this.Footer('等待远端屏幕信息') } + ForEach(this.state.displays, (screen: RemoteDisplay, index: number) => { + Button({ type: ButtonType.Normal }) { + Row({ space: 12 }) { + MobileIcon({ name: 'display', iconExtent: 22, color: index === this.state.selectedDisplay ? MobileTheme.blue : MobileTheme.secondary }) + Text(`${index + 1} · ${screen.name || '显示器'}`).fontSize(16).fontColor(MobileTheme.ink).layoutWeight(1).textAlign(TextAlign.Start) + if (index === this.state.selectedDisplay) { MobileIcon({ name: 'check', iconExtent: 20, color: MobileTheme.blue }) } + }.padding(14).width('100%') + }.width('100%').padding(0).backgroundColor(index === this.state.selectedDisplay ? '#EDF4FC' : MobileTheme.background).borderRadius(12) + .onClick(() => this.controller?.selectDisplay(index)) + }) + }.padding(20).width('100%') + } else if (this.state.modal === 'recentActions') { + Text(this.state.recent.find((d: RecentDevice) => d.id === this.deleteId)?.name || this.formatId(this.deleteId)) + .fontSize(17).fontWeight(600).fontColor(MobileTheme.ink).padding(20).width('100%').textAlign(TextAlign.Center) + this.Rule() + this.Action('连接', () => { this.dismissModal(); this.controller?.requestConnection(this.deleteId); }) + this.Rule() + this.Action('删除记录', () => this.state.modal = 'delete', true) + this.Rule() + this.Action('取消', () => this.dismissModal()) + } else { + Column({ space: 12 }) { + Text(this.state.modal === 'notice' ? '连接提示' : this.state.modal === 'upgrade' ? '升级提示' : this.state.modal === 'disconnect' ? '断开远程连接?' : + this.state.modal === 'revoke' ? '撤回联网授权?' : this.state.modal === 'previews' ? '保存远程画面预览?' : this.state.modal === 'clearPreviews' ? '清除预览图?' : this.state.modal === 'deleteAnnouncement' ? '删除公告?' : '删除记录?') + .fontSize(20).fontWeight(600).fontColor(MobileTheme.ink).textAlign(TextAlign.Center).width('100%') + Text(this.state.modal === 'notice' || this.state.modal === 'upgrade' ? this.state.notice : this.state.modal === 'disconnect' ? '断开后将返回首页。' : + this.state.modal === 'previews' ? '预览可能包含远程设备上的敏感内容。仅保存在本机,从下一次连接起生效,可随时关闭并清除。' : + this.state.modal === 'clearPreviews' ? '将清除此设备保存的所有远程画面预览,保留连接记录和访问密码。' : + this.state.modal === 'revoke' ? '将停止远程连接、关闭画面预览保存并清除已有预览。连接记录和密码会保留,重新连接前需再次授权。' : + this.state.modal === 'delete' ? '同时删除该设备保存的密码和预览。' : '此公告将从列表中移除。') + .fontSize(14).lineHeight(21).fontColor(MobileTheme.secondary).textAlign(TextAlign.Center).width('100%') + }.padding(24).width('100%') + this.Rule() + if (this.state.modal === 'notice' || this.state.modal === 'upgrade') { this.Action('知道了', () => this.dismissModal()) } + else { + Row() { + Button('取消', { type: ButtonType.Normal }).fontSize(16).fontColor(MobileTheme.blue).height(50).layoutWeight(1) + .backgroundColor(Color.White).onClick(() => this.dismissModal()) + Divider().vertical(true).height(50).strokeWidth(0.5).color(MobileTheme.line) + Button(this.state.modal === 'previews' ? '开启' : this.state.modal === 'disconnect' ? '断开连接' : this.state.modal === 'revoke' ? '撤回授权' : this.state.modal === 'clearPreviews' ? '清除' : '删除', { type: ButtonType.Normal }) + .fontSize(16).fontWeight(600).fontColor(this.state.modal === 'previews' ? MobileTheme.blue : MobileTheme.red).height(50).layoutWeight(1) + .backgroundColor(Color.White).onClick(() => this.confirmModal()) + }.width('100%') + } + } + } + @Builder + Modal() { + Stack({ alignContent: this.viewWidth < 600 && (this.state.modal === 'password' || this.state.modal === 'consent') ? Alignment.Bottom : Alignment.Center }) { + Column() {}.width('100%').height('100%').backgroundColor(this.state.modal === 'connecting' ? '#14000000' : '#38000000') + .onClick(() => { if (this.state.modal !== 'consent' && this.state.modal !== 'connecting') this.dismissModal(); }) + if (this.state.modal === 'password') { this.PasswordSheet() } + else if (this.state.modal === 'consent') { this.ConsentSheet() } + else { + Scroll() { Column() { this.DialogBody() }.width('100%') } + .width(Math.min(this.state.modal === 'connecting' ? 292 : 420, this.viewWidth - 40)).constraintSize({ maxHeight: this.viewHeight * 0.9 }) + .backgroundColor(Color.White).borderRadius(24).clip(true).scrollBar(BarState.Off).shadow({ radius: 28, color: '#18000000', offsetY: 8 }) + } + }.width('100%').height('100%').transition(TransitionEffect.OPACITY.animation({ duration: 160 })) + } + build() { + Stack({ alignContent: Alignment.Center }) { + if (this.state.page === 'session' && this.state.ready) { + MobileRemoteView({ state: this.state, controller: this.controller }).transition(TransitionEffect.OPACITY.animation({ duration: 200 })) + } else if (this.state.page === 'home') { this.Home() } + else { + Column() { + Row({ space: 0 }) { + MobileIconButton({ icon: 'back', label: '返回', iconExtent: 44, iconSize: 22, action: () => this.back() }) + Text(this.title()).fontSize(17).fontWeight(600).fontColor(MobileTheme.ink).layoutWeight(1).textAlign(TextAlign.Center) + .maxLines(1).textOverflow({ overflow: TextOverflow.Ellipsis }) + if (this.state.page === 'announcements') { + MobileIconButton({ icon: 'refresh', label: '刷新公告', iconExtent: 44, iconSize: 22, action: () => this.controller?.inbox.refresh() }) + } else { Blank().width(44) } + }.width('100%').constraintSize({ maxWidth: 720 }).height(56).padding({ left: 12, right: 12 }) + if (this.state.page === 'announcements') { + Refresh({ refreshing: this.refreshing }) { + Scroll() { Column() { this.Content() }.width('100%').constraintSize({ maxWidth: 720 }).padding({ left: 20, right: 20, top: 16, bottom: 28 }) } + .width('100%').height('100%').align(Alignment.Top).scrollBar(BarState.Off) + }.layoutWeight(1).width('100%').onRefreshing(() => { this.controller?.inbox.refresh(); this.refreshing = false; }) + } else { + Scroll() { Column() { this.Content() }.width('100%').constraintSize({ maxWidth: 720 }).padding({ left: 20, right: 20, top: 16, bottom: 28 }) } + .layoutWeight(1).width('100%').align(Alignment.Top).scrollBar(BarState.Off) + } + }.width('100%').height('100%').transition(TransitionEffect.asymmetric(TransitionEffect.move(TransitionEdge.END), TransitionEffect.move(TransitionEdge.END)).animation({ duration: 200 })) + } + if (this.state.modal) { this.Modal() } + }.width('100%').height('100%').backgroundColor(MobileTheme.background) + .onAreaChange((_old: Area, next: Area) => { this.viewWidth = Number(next.width); this.viewHeight = Number(next.height); }) + } +} diff --git a/apps/harmony/entry/src/main/module.json5 b/apps/harmony/entry/src/main/module.json5 new file mode 100644 index 00000000..bc769edb --- /dev/null +++ b/apps/harmony/entry/src/main/module.json5 @@ -0,0 +1,48 @@ +{ + "module": { + "name": "entry", + "type": "entry", + "description": "$string:app_name", + "mainElement": "EntryAbility", + "deviceTypes": [ + "phone", + "tablet" + ], + "deliveryWithInstall": true, + "installationFree": false, + "pages": "$profile:main_pages", + "abilities": [ + { + "name": "EntryAbility", + "srcEntry": "./ets/mobile/EntryAbility.ets", + "description": "$string:app_name", + "icon": "$media:app_icon", + "label": "$string:app_name", + "startWindowIcon": "$media:app_icon", + "startWindowBackground": "$color:start_window_background", + "exported": true, + "skills": [ + { + "entities": [ + "entity.system.home" + ], + "actions": [ + "ohos.want.action.home" + ] + } + ], + "backgroundModes": [ + "dataTransfer" + ] + } + ], + "requestPermissions": [ + { + "name": "ohos.permission.INTERNET" + }, + { + "name": "ohos.permission.KEEP_BACKGROUND_RUNNING" + } + ] + } +} diff --git a/apps/harmony/entry/src/main/resources/base/element/color.json b/apps/harmony/entry/src/main/resources/base/element/color.json new file mode 100644 index 00000000..462c8172 --- /dev/null +++ b/apps/harmony/entry/src/main/resources/base/element/color.json @@ -0,0 +1,8 @@ +{ + "color": [ + { + "name": "start_window_background", + "value": "#F3F5F9" + } + ] +} diff --git a/apps/harmony/entry/src/main/resources/base/media/mobile_brand.png b/apps/harmony/entry/src/main/resources/base/media/mobile_brand.png new file mode 100644 index 00000000..a0d5a549 Binary files /dev/null and b/apps/harmony/entry/src/main/resources/base/media/mobile_brand.png differ diff --git a/apps/harmony/entry/src/main/resources/base/profile/main_pages.json b/apps/harmony/entry/src/main/resources/base/profile/main_pages.json new file mode 100644 index 00000000..79678284 --- /dev/null +++ b/apps/harmony/entry/src/main/resources/base/profile/main_pages.json @@ -0,0 +1 @@ +{"src": ["pages/Mobile"]} diff --git a/apps/harmony/hvigor/hvigor-config.json5 b/apps/harmony/hvigor/hvigor-config.json5 new file mode 100644 index 00000000..eab9ccb6 --- /dev/null +++ b/apps/harmony/hvigor/hvigor-config.json5 @@ -0,0 +1,4 @@ +{ + "modelVersion": "6.0.0", + "dependencies": {} +} diff --git a/apps/harmony/hvigorfile.ts b/apps/harmony/hvigorfile.ts new file mode 100644 index 00000000..58a73455 --- /dev/null +++ b/apps/harmony/hvigorfile.ts @@ -0,0 +1,2 @@ +import { appTasks } from '@ohos/hvigor-ohos-plugin'; +export default { system: appTasks, plugins: [] }; diff --git a/apps/harmony/native/audio.cpp b/apps/harmony/native/audio.cpp new file mode 100644 index 00000000..695e2b3d --- /dev/null +++ b/apps/harmony/native/audio.cpp @@ -0,0 +1,58 @@ +#include "audio.h" +#include +#include +#include +#include + +namespace crossdesk::harmony { +bool AudioPlayer::Start() { + Stop(); + OH_AudioStreamBuilder* builder = nullptr; + if (OH_AudioStreamBuilder_Create(&builder, AUDIOSTREAM_TYPE_RENDERER) != AUDIOSTREAM_SUCCESS) return false; + OH_AudioStreamBuilder_SetSamplingRate(builder, 48000); + OH_AudioStreamBuilder_SetChannelCount(builder, 1); + OH_AudioStreamBuilder_SetSampleFormat(builder, AUDIOSTREAM_SAMPLE_S16LE); + OH_AudioStreamBuilder_SetEncodingType(builder, AUDIOSTREAM_ENCODING_TYPE_RAW); + OH_AudioStreamBuilder_SetRendererInfo(builder, AUDIOSTREAM_USAGE_MUSIC); + OH_AudioRenderer_Callbacks callbacks{}; + callbacks.OH_AudioRenderer_OnWriteData = [](OH_AudioRenderer*, void* user, void* buffer, int32_t length) { + auto* self = static_cast(user); + if (!buffer || length <= 0) return 0; + std::memset(buffer, 0, length); + std::lock_guard lock(self->mutex_); + auto* output = static_cast(buffer); + const auto count = std::min(size_t(length), self->queue_.size()); + for (size_t i = 0; i < count; ++i) { + output[i] = self->queue_.front(); self->queue_.pop_front(); + } + if (count && !self->rendered_bytes_) + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "Audio playback received PCM: %{public}zu bytes at 48000 Hz mono", count); + self->rendered_bytes_ += count; + return 0; + }; + callbacks.OH_AudioRenderer_OnError = [](OH_AudioRenderer*, void*, OH_AudioStream_Result error) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "CrossDesk", "Audio playback error: %{public}d", int(error)); + return 0; + }; + OH_AudioStreamBuilder_SetRendererCallback(builder, callbacks, this); + auto result = OH_AudioStreamBuilder_GenerateRenderer(builder, &renderer_); + OH_AudioStreamBuilder_Destroy(builder); + if (result != AUDIOSTREAM_SUCCESS || OH_AudioRenderer_Start(renderer_) != AUDIOSTREAM_SUCCESS) { + Stop(); return false; + } + return true; +} +void AudioPlayer::Stop() { + if (renderer_) { OH_AudioRenderer_Stop(renderer_); OH_AudioRenderer_Release(renderer_); renderer_ = nullptr; } + std::lock_guard lock(mutex_); + if (rendered_bytes_) OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "Audio playback stopped after %{public}llu PCM bytes", (unsigned long long)rendered_bytes_); + queue_.clear(); rendered_bytes_ = 0; +} +void AudioPlayer::Push(const char* data, size_t size) { + if (!data || size == 0 || size > 19200 || (size & 1)) return; + std::lock_guard lock(mutex_); + if (queue_.size() + size > 19200) queue_.clear(); // At most 200 ms of mono PCM. + queue_.insert(queue_.end(), data, data + size); +} + +} diff --git a/apps/harmony/native/audio.h b/apps/harmony/native/audio.h new file mode 100644 index 00000000..88b9a7ce --- /dev/null +++ b/apps/harmony/native/audio.h @@ -0,0 +1,32 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _AUDIO_H_ +#define _AUDIO_H_ + +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +class AudioPlayer { + public: + ~AudioPlayer() { Stop(); } + bool Start(); + void Stop(); + void Push(const char* data, size_t size); + private: + OH_AudioRenderer* renderer_ = nullptr; + std::mutex mutex_; + std::deque queue_; + uint64_t rendered_bytes_ = 0; +}; + +} + +#endif diff --git a/apps/harmony/native/capture.cpp b/apps/harmony/native/capture.cpp new file mode 100644 index 00000000..841510f9 --- /dev/null +++ b/apps/harmony/native/capture.cpp @@ -0,0 +1,209 @@ +#include "capture.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +bool Capture::Start(int width, int height, uint64_t display, Frame frame, State state, + Audio audio, int fps, bool persistent_permission, bool show_cursor) { + Stop(); frame_ = std::move(frame); state_ = std::move(state); audio_ = std::move(audio); + show_cursor_ = show_cursor; + first_video_ = true; + first_data_ = true; + max_frame_rate_ = std::clamp(fps, 1, 60); + capture_ = OH_AVScreenCapture_Create(); + if (!capture_) return false; + window_ = gpu_surface_.Start(width, height, [this](const uint8_t* pixels, int width, int height, int stride) { + if (active_) ConvertRgba(pixels, width, height, stride); + }); + const bool surface_mode = window_ != nullptr || PrepareSurface(width, height); + const auto state_result = OH_AVScreenCapture_SetStateCallback(capture_, [](OH_AVScreenCapture* capture, OH_AVScreenCaptureStateCode code, void* user) { + auto* self = static_cast(user); + if (code == OH_SCREEN_CAPTURE_STATE_STARTED) { + // Both operations require a running capture. StartScreenCapture may + // return while the system is still awaiting the user's authorization. + const auto rate = OH_AVScreenCapture_SetMaxVideoFrameRate(capture, self->max_frame_rate_); + if (rate != AV_SCREEN_CAPTURE_ERR_OK) + OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Capture frame rate unavailable: %{public}d", int(rate)); + } + if (code == OH_SCREEN_CAPTURE_STATE_STARTED || code == OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE || + code == OH_SCREEN_CAPTURE_STATE_RESUMED_BY_USER || code == OH_SCREEN_CAPTURE_STATE_RESUMED_BY_APP) { + // Native viewers draw the separately transmitted cursor. Only advertise + // that mode once the recorder confirms it has removed the video cursor. + const auto result = OH_AVScreenCapture_ShowCursor(capture, self->show_cursor_); + if (result == AV_SCREEN_CAPTURE_ERR_OK) { + self->cursor_mode_ = self->show_cursor_ ? CaptureCursorMode::embedded : CaptureCursorMode::separate; + } else { + self->cursor_mode_ = !self->show_cursor_ && + OH_AVScreenCapture_ShowCursor(capture, true) == AV_SCREEN_CAPTURE_ERR_OK + ? CaptureCursorMode::embedded : CaptureCursorMode::unknown; + OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Capture cursor setting failed: %{public}d", int(result)); + } + } + self->active_ = CaptureActiveForState(code, self->active_); + self->state_(static_cast(code)); + }, this); + const auto error_result = OH_AVScreenCapture_SetErrorCallback(capture_, [](OH_AVScreenCapture*, int32_t error, void* user) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Capture error: %{public}d", error); + auto* self = static_cast(user); self->active_ = false; self->state_(-1); + }, this); + const auto data_result = OH_AVScreenCapture_SetDataCallback(capture_, [](OH_AVScreenCapture*, OH_AVBuffer* buffer, + OH_AVScreenCaptureBufferType type, int64_t, void* user) { + auto* self = static_cast(user); + if (self->first_data_.exchange(false)) + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Capture first data: type=%{public}d active=%{public}d", int(type), int(self->active_.load())); + if (!self->active_) return; + if (type == OH_SCREEN_CAPTURE_BUFFERTYPE_AUDIO_INNER && self->audio_) { + OH_AVCodecBufferAttr attr{}; + auto* data = OH_AVBuffer_GetAddr(buffer); + if (data && OH_AVBuffer_GetBufferAttr(buffer, &attr) == AV_ERR_OK && attr.offset >= 0 && attr.size > 0 && + int64_t(attr.offset) + attr.size <= OH_AVBuffer_GetCapacity(buffer)) + self->audio_(reinterpret_cast(data + attr.offset), attr.size); + return; + } + if (type != OH_SCREEN_CAPTURE_BUFFERTYPE_VIDEO) return; + const bool first_video = self->first_video_.exchange(false); + auto* native = OH_AVBuffer_GetNativeBuffer(buffer); + if (!native) { + if (first_video) OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Capture video has no native buffer"); + return; + } + self->Convert(native, first_video); + OH_NativeBuffer_Unreference(native); + }, this); + OH_AVScreenCaptureConfig config{}; + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Capture request: %{public}dx%{public}d display=%{public}llu fps=%{public}d", + width, height, static_cast(display), max_frame_rate_); + config.captureMode = OH_CAPTURE_SPECIFIED_SCREEN; config.dataType = OH_ORIGINAL_STREAM; + config.videoInfo.videoCapInfo = {display, nullptr, 0, width, height, OH_VIDEO_SOURCE_SURFACE_RGBA}; + if (audio_) config.audioInfo.innerCapInfo = {48000, 1, OH_ALL_PLAYBACK}; + const auto check = [](const char* stage, OH_AVSCREEN_CAPTURE_ErrCode result) { + if (result == AV_SCREEN_CAPTURE_ERR_OK) return true; + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Capture %{public}s failed: %{public}d", stage, int(result)); + return false; + }; + if (!check("state callback", state_result) || !check("error callback", error_result) || + !check("data callback", data_result) || !check("initialization", OH_AVScreenCapture_Init(capture_, config)) || + !check("microphone disable", OH_AVScreenCapture_SetMicrophoneEnabled(capture_, false))) { + Stop(); return false; + } + if (persistent_permission) { + // With CUSTOM_SCREEN_RECORDING granted, the system can reuse consent for + // this specified display. Disabling the picker alone never grants access. + auto* strategy = OH_AVScreenCapture_CreateCaptureStrategy(); + bool configured = false; + if (strategy) { + configured = OH_AVScreenCapture_StrategyForPickerPopUp(strategy, false) == AV_SCREEN_CAPTURE_ERR_OK && + OH_AVScreenCapture_SetCaptureStrategy(capture_, strategy) == AV_SCREEN_CAPTURE_ERR_OK; + OH_AVScreenCapture_ReleaseCaptureStrategy(strategy); + } + if (!configured) + OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "Persistent capture strategy unavailable; using system authorization"); + } + if (!check("start", surface_mode ? OH_AVScreenCapture_StartScreenCaptureWithSurface(capture_, window_) + : OH_AVScreenCapture_StartScreenCapture(capture_))) { + Stop(); return false; + } + return true; +} +bool Capture::PrepareSurface(int width, int height) { + image_ = OH_ConsumerSurface_Create(); + if (!image_) return false; + if (OH_ConsumerSurface_SetDefaultUsage(image_, NATIVEBUFFER_USAGE_CPU_READ | NATIVEBUFFER_USAGE_CPU_READ_OFTEN | + NATIVEBUFFER_USAGE_HW_RENDER) != 0 || + OH_ConsumerSurface_SetDefaultSize(image_, width, height) != 0 || + !(window_ = OH_NativeImage_AcquireNativeWindow(image_))) { + OH_NativeImage_Destroy(&image_); return false; + } + surface_pending_ = false; surface_stopping_ = false; + OH_OnFrameAvailableListener listener{this, [](void* context) { + auto* self = static_cast(context); + std::lock_guard lock(self->surface_mutex_); + self->surface_pending_ = true; self->surface_ready_.notify_one(); + }}; + if (OH_NativeImage_SetOnFrameAvailableListener(image_, listener) != 0) { + window_ = nullptr; // The native image owns its producer window. + OH_NativeImage_Destroy(&image_); return false; + } + surface_thread_ = std::thread([this] { ConsumeSurface(); }); + return true; +} +void Capture::ConsumeSurface() { + for (;;) { + { + std::unique_lock lock(surface_mutex_); + surface_ready_.wait(lock, [this] { return surface_stopping_ || surface_pending_; }); + if (surface_stopping_) return; + surface_pending_ = false; + } + // Notifications may coalesce while conversion is running. Drain the queue + // so a static desktop cannot leave its newest frame waiting indefinitely. + for (;;) { + { std::lock_guard lock(surface_mutex_); if (surface_stopping_) return; } + OHNativeWindowBuffer* buffer = nullptr; + int fence = -1; + if (OH_NativeImage_AcquireNativeWindowBuffer(image_, &buffer, &fence) != 0) break; + bool ready = true; + if (fence >= 0) { + pollfd descriptor{fence, POLLIN, 0}; + int result; + do { result = poll(&descriptor, 1, 1000); } while (result < 0 && errno == EINTR); + ready = result > 0 && (descriptor.revents & POLLIN); + } + OH_NativeBuffer* native = nullptr; + if (ready && active_ && OH_NativeBuffer_FromNativeWindowBuffer(buffer, &native) == 0 && native) + Convert(native, first_video_.exchange(false)); + // Return an unsignaled fence to the producer on timeout; never let it + // reuse memory while a GPU write is still outstanding. + OH_NativeImage_ReleaseNativeWindowBuffer(image_, buffer, ready ? -1 : fence); + if (ready && fence >= 0) close(fence); + } + } +} +void Capture::Convert(OH_NativeBuffer* native, bool first_video) { + OH_NativeBuffer_Config config{}; OH_NativeBuffer_GetConfig(native, &config); + if (first_video) + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Capture buffer: %{public}dx%{public}d format=%{public}d stride=%{public}d", + config.width, config.height, config.format, config.stride); + void* pixels = nullptr; + if (config.format == NATIVEBUFFER_PIXEL_FMT_RGBA_8888 && + config.width > 0 && config.height > 0 && config.width <= 4096 && config.height <= 4096 && + !(config.width & 1) && !(config.height & 1) && config.stride >= config.width * 4 && + OH_NativeBuffer_Map(native, &pixels) == 0 && pixels) { + ConvertRgba(static_cast(pixels), config.width, config.height, config.stride); + OH_NativeBuffer_Unmap(native); + } +} +void Capture::ConvertRgba(const uint8_t* pixels, int width, int height, int stride) { + std::lock_guard lock(data_mutex_); + const size_t area = size_t(width) * height; + nv12_.resize(area * 3 / 2); + if (libyuv::ABGRToNV12(pixels, stride, nv12_.data(), width, nv12_.data() + area, width, width, height) == 0 && active_) { + MiniRtcVideoFrame video{}; + video.data = reinterpret_cast(nv12_.data()); video.size = nv12_.size(); + video.width = width; video.height = height; + frame_(video); + } +} +void Capture::Stop() { + cursor_mode_ = CaptureCursorMode::unknown; + active_ = false; + if (capture_) { OH_AVScreenCapture_StopScreenCapture(capture_); OH_AVScreenCapture_Release(capture_); capture_ = nullptr; } + gpu_surface_.Stop(); + if (image_) { + OH_NativeImage_UnsetOnFrameAvailableListener(image_); + { std::lock_guard lock(surface_mutex_); surface_stopping_ = true; surface_ready_.notify_one(); } + if (surface_thread_.joinable()) surface_thread_.join(); + window_ = nullptr; // OH_NativeImage_Destroy releases the producer window. + OH_NativeImage_Destroy(&image_); + } + window_ = nullptr; +} +} diff --git a/apps/harmony/native/capture.h b/apps/harmony/native/capture.h new file mode 100644 index 00000000..3d88a881 --- /dev/null +++ b/apps/harmony/native/capture.h @@ -0,0 +1,82 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _CAPTURE_H_ +#define _CAPTURE_H_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "capture_surface.h" + +namespace crossdesk::harmony { +// Microphone notifications do not change video/input availability. +inline bool CaptureActiveForState(int state, bool current) { + if (state < 0) return false; + switch (state) { + case OH_SCREEN_CAPTURE_STATE_STARTED: + case OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE: + case OH_SCREEN_CAPTURE_STATE_RESUMED_BY_USER: + case OH_SCREEN_CAPTURE_STATE_RESUMED_BY_APP: return true; + case OH_SCREEN_CAPTURE_STATE_CANCELED: + case OH_SCREEN_CAPTURE_STATE_STOPPED_BY_USER: + case OH_SCREEN_CAPTURE_STATE_INTERRUPTED_BY_OTHER: + case OH_SCREEN_CAPTURE_STATE_STOPPED_BY_CALL: + case OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE: + case OH_SCREEN_CAPTURE_STATE_STOPPED_BY_USER_SWITCHES: + case OH_SCREEN_CAPTURE_STATE_PAUSED_BY_USER: + case OH_SCREEN_CAPTURE_STATE_PAUSED_BY_APP: return false; + default: return current; + } +} +enum class CaptureCursorMode { unknown, separate, embedded }; + +class Capture { + public: + using Frame = std::function; + using State = std::function; + using Audio = std::function; + ~Capture() { Stop(); } + bool Start(int width, int height, uint64_t display, Frame frame, State state, + Audio audio = {}, int fps = 30, bool persistent_permission = false, + bool show_cursor = true); + void Stop(); + CaptureCursorMode CursorMode() const { return cursor_mode_; } + bool CursorEmbedded() const { return CursorMode() == CaptureCursorMode::embedded; } + private: + bool PrepareSurface(int width, int height); + void ConsumeSurface(); + void Convert(OH_NativeBuffer* native, bool first_video); + void ConvertRgba(const uint8_t* pixels, int width, int height, int stride); + CaptureSurface gpu_surface_; + OH_NativeImage* image_ = nullptr; + OHNativeWindow* window_ = nullptr; + std::mutex surface_mutex_; + std::condition_variable surface_ready_; + bool surface_stopping_ = true, surface_pending_ = false; + std::thread surface_thread_; + OH_AVScreenCapture* capture_ = nullptr; + std::atomic cursor_mode_{CaptureCursorMode::unknown}; + bool show_cursor_ = true; + int max_frame_rate_ = 30; + Frame frame_; + State state_; + Audio audio_; + std::mutex data_mutex_; + std::vector nv12_; + std::atomic active_{false}; + std::atomic first_video_{true}; + std::atomic first_data_{true}; +}; +} + +#endif diff --git a/apps/harmony/native/capture_surface.cpp b/apps/harmony/native/capture_surface.cpp new file mode 100644 index 00000000..6ec5e009 --- /dev/null +++ b/apps/harmony/native/capture_surface.cpp @@ -0,0 +1,148 @@ +#include "capture_surface.h" +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +OHNativeWindow* CaptureSurface::Start(int width, int height, Frame frame) { + Stop(); + if (width <= 0 || height <= 0 || (width & 1) || (height & 1) || !frame) return nullptr; + { + std::lock_guard lock(mutex_); + prepared_ = false; stopping_ = false; pending_ = 0; window_ = nullptr; + } + thread_ = std::thread([this, width, height, frame = std::move(frame)] { Run(width, height, frame); }); + std::unique_lock lock(mutex_); + ready_.wait(lock, [this] { return prepared_; }); + auto* window = window_; + lock.unlock(); + if (!window) Stop(); // Finish failed EGL cleanup before trying the CPU path. + return window; +} +void CaptureSurface::Stop() { + { std::lock_guard lock(mutex_); stopping_ = true; ready_.notify_one(); } + if (thread_.joinable()) thread_.join(); +} +void CaptureSurface::Run(int width, int height, Frame frame) { + EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY); + EGLContext context = EGL_NO_CONTEXT; + EGLSurface target = EGL_NO_SURFACE; + OH_NativeImage* image = nullptr; + GLuint input = 0, output = 0, fbo = 0, program = 0, vao = 0; + bool current = false; + const auto prepare = [&]() -> bool { + if (!eglInitialize(display, nullptr, nullptr) || !eglBindAPI(EGL_OPENGL_ES_API)) return false; + const EGLint attributes[] = {EGL_SURFACE_TYPE, EGL_PBUFFER_BIT, EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT, + EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_ALPHA_SIZE, 8, EGL_NONE}; + EGLConfig config{}; EGLint count = 0; + if (!eglChooseConfig(display, attributes, &config, 1, &count) || count == 0) return false; + const EGLint context_attributes[] = {EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE}; + context = eglCreateContext(display, config, EGL_NO_CONTEXT, context_attributes); + const EGLint pbuffer_attributes[] = {EGL_WIDTH, 1, EGL_HEIGHT, 1, EGL_NONE}; + target = eglCreatePbufferSurface(display, config, pbuffer_attributes); + if (context == EGL_NO_CONTEXT || target == EGL_NO_SURFACE) return false; + if (!(current = eglMakeCurrent(display, target, target, context))) return false; + GLint limit = 0; glGetIntegerv(GL_MAX_TEXTURE_SIZE, &limit); + if (width > limit || height > limit) return false; + const char* vertex = R"(#version 300 es + uniform mat4 transform; out vec2 uv; + void main() { + vec2 p = vec2(float((gl_VertexID << 1) & 2), float(gl_VertexID & 2)); + uv = (transform * vec4(p, 0.0, 1.0)).xy; + gl_Position = vec4(p * 2.0 - 1.0, 0.0, 1.0); + })"; + const char* fragment = R"(#version 300 es + #extension GL_OES_EGL_image_external_essl3 : require + precision mediump float; + uniform samplerExternalOES source; in vec2 uv; out vec4 color; + void main() { color = texture(source, uv); })"; + const auto shader = [](GLenum type, const char* source) { + GLuint value = glCreateShader(type); glShaderSource(value, 1, &source, nullptr); glCompileShader(value); + GLint ok = 0; glGetShaderiv(value, GL_COMPILE_STATUS, &ok); + if (!ok) { glDeleteShader(value); return GLuint(0); } return value; + }; + GLuint v = shader(GL_VERTEX_SHADER, vertex), f = shader(GL_FRAGMENT_SHADER, fragment); + if (!v || !f) { glDeleteShader(v); glDeleteShader(f); return false; } + program = glCreateProgram(); glAttachShader(program, v); glAttachShader(program, f); glLinkProgram(program); + glDeleteShader(v); glDeleteShader(f); + GLint linked = 0; glGetProgramiv(program, GL_LINK_STATUS, &linked); if (!linked) return false; + glGenTextures(1, &input); glBindTexture(GL_TEXTURE_EXTERNAL_OES, input); + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + image = OH_NativeImage_Create(input, GL_TEXTURE_EXTERNAL_OES); + if (!image) return false; + if (OH_ConsumerSurface_SetDefaultSize(image, width, height) != 0) return false; + OH_OnFrameAvailableListener listener{this, [](void* context) { + auto* self = static_cast(context); + std::lock_guard lock(self->mutex_); + self->pending_ = std::min(self->pending_ + 1, 64u); self->ready_.notify_one(); + }}; + if (OH_NativeImage_SetOnFrameAvailableListener(image, listener) != 0) return false; + glGenTextures(1, &output); glBindTexture(GL_TEXTURE_2D, output); + glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr); + glGenFramebuffers(1, &fbo); glBindFramebuffer(GL_FRAMEBUFFER, fbo); + glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, output, 0); + if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) return false; + glGenVertexArrays(1, &vao); glBindVertexArray(vao); + window_ = OH_NativeImage_AcquireNativeWindow(image); + return window_ != nullptr; + }; + const bool success = prepare(); + { + std::lock_guard lock(mutex_); + if (!success) window_ = nullptr; + prepared_ = true; ready_.notify_one(); + } + if (success) { + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "SlintDesktop", "Capture uses GLES surface readback"); + std::vector pixels(size_t(width) * height * 4); + const auto source_location = glGetUniformLocation(program, "source"); + const auto transform_location = glGetUniformLocation(program, "transform"); + bool reported_error = false; + for (;;) { + { + std::unique_lock lock(mutex_); + ready_.wait(lock, [this] { return stopping_ || pending_ > 0; }); + if (stopping_) break; + --pending_; + } + if (OH_NativeImage_UpdateSurfaceImage(image) != 0) continue; + float transform[16]{}; + if (OH_NativeImage_GetTransformMatrixV2(image, transform) != 0) continue; + glBindFramebuffer(GL_FRAMEBUFFER, fbo); glViewport(0, 0, width, height); + glUseProgram(program); glBindVertexArray(vao); + glActiveTexture(GL_TEXTURE0); glBindTexture(GL_TEXTURE_EXTERNAL_OES, input); + glUniform1i(source_location, 0); + glUniformMatrix4fv(transform_location, 1, GL_FALSE, transform); + glDrawArrays(GL_TRIANGLES, 0, 3); + glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, pixels.data()); + const auto error = glGetError(); + if (error == GL_NO_ERROR) { + // NativeImage's V2 matrix already accounts for the producer's buffer + // orientation. Flipping glReadPixels rows again inverts the desktop. + frame(pixels.data(), width, height, width * 4); + } else if (!reported_error) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "SlintDesktop", "Capture GLES readback failed: %{public}u", error); + reported_error = true; + } + } + } else OH_LOG_Print(LOG_APP, LOG_WARN, 0, "SlintDesktop", "GLES capture surface unavailable"); + if (image) { OH_NativeImage_UnsetOnFrameAvailableListener(image); OH_NativeImage_Destroy(&image); } + if (current) { + glDeleteVertexArrays(1, &vao); glDeleteFramebuffers(1, &fbo); + glDeleteTextures(1, &input); glDeleteTextures(1, &output); glDeleteProgram(program); + eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + } + if (target != EGL_NO_SURFACE) eglDestroySurface(display, target); + if (context != EGL_NO_CONTEXT) eglDestroyContext(display, context); + // The default display is shared with Slint/ArkUI and other video windows. + // Terminating it here would invalidate their contexts on disconnect. + eglReleaseThread(); + { std::lock_guard lock(mutex_); window_ = nullptr; } +} +} diff --git a/apps/harmony/native/capture_surface.h b/apps/harmony/native/capture_surface.h new file mode 100644 index 00000000..8322afcd --- /dev/null +++ b/apps/harmony/native/capture_surface.h @@ -0,0 +1,37 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _CAPTURE_SURFACE_H_ +#define _CAPTURE_SURFACE_H_ + +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +// Owns the capture texture and its EGL context on one worker. Some graphics +// drivers expose GPU-backed capture buffers that cannot be read via mmap. +class CaptureSurface { + public: + using Frame = std::function; + ~CaptureSurface() { Stop(); } + OHNativeWindow* Start(int width, int height, Frame frame); + void Stop(); + private: + void Run(int width, int height, Frame frame); + std::mutex mutex_; + std::condition_variable ready_; + bool prepared_ = false, stopping_ = true; + unsigned pending_ = 0; + OHNativeWindow* window_ = nullptr; // Owned by the worker's OH_NativeImage. + std::thread thread_; +}; +} + +#endif diff --git a/apps/harmony/native/input.cpp b/apps/harmony/native/input.cpp new file mode 100644 index 00000000..ebc13ad1 --- /dev/null +++ b/apps/harmony/native/input.cpp @@ -0,0 +1,131 @@ +#include "input.h" +#include "key_mapping.h" +#include +#include +#include +#include + +namespace crossdesk::harmony { +namespace { +void ReportResult(const char* kind, int result, int& previous) { + if (result == previous) return; + previous = result; + OH_LOG_Print(LOG_APP, result == INPUT_SUCCESS ? LOG_INFO : LOG_WARN, 0, "SlintDesktop", + "Host %{public}s injection result: %{public}d", kind, result); +} +KeyboardKey NormalizeKey(int key, uint32_t scan, bool extended) { + const int native = FromVirtualKey(key, scan, extended); + return {native ? ToVirtualKey(native) : 0, scan, extended}; +} +} +bool Input::Configure(int width, int height, int display) { + if (width <= 0 || height <= 0 || display < 0 || !Release()) return false; + width_ = width; height_ = height; display_ = display; + return true; +} +bool Input::Key(const KeyboardKey& key, bool down) { + const int code = FromVirtualKey(key.key_code, key.scan_code, key.extended); + if (!code) return false; + auto* event = OH_Input_CreateKeyEvent(); + if (!event) return false; + OH_Input_SetKeyEventKeyCode(event, code); + OH_Input_SetKeyEventAction(event, down ? KEY_ACTION_DOWN : KEY_ACTION_UP); + const int result = OH_Input_InjectKeyEvent(event); + ReportResult("keyboard", result, last_key_result_); + const bool ok = result == INPUT_SUCCESS; + OH_Input_DestroyKeyEvent(&event); + if (ok) { if (down) keys_[key.key_code] = key; else keys_.erase(key.key_code); } + return ok; +} +bool Input::Pointer(int flag, float x, float y, int scroll) { + if (flag < move || flag > wheel_horizontal || width_ <= 0 || height_ <= 0 || + !std::isfinite(x) || !std::isfinite(y)) return false; + auto* event = OH_Input_CreateMouseEvent(); + if (!event) return false; + x_ = std::clamp(x, 0.f, 1.f); y_ = std::clamp(y, 0.f, 1.f); + OH_Input_SetMouseEventDisplayId(event, display_); + OH_Input_SetMouseEventDisplayX(event, std::lround(x_ * (width_ - 1))); + OH_Input_SetMouseEventDisplayY(event, std::lround(y_ * (height_ - 1))); + const int buttons[] = {MOUSE_BUTTON_LEFT, MOUSE_BUTTON_RIGHT, MOUSE_BUTTON_MIDDLE}; + int button = MOUSE_BUTTON_NONE; + // WMS rejects drag moves with BUTTON_NONE, even if a preceding down event + // populated the SDK's pressed-button set. Carry the held button on moves. + for (int i = 0; i < 3; ++i) if (buttons_ & (1u << i)) { button = buttons[i]; break; } + OH_Input_SetMouseEventButton(event, button); + bool ok = true; + const auto inject = [&] { + const int result = OH_Input_InjectMouseEvent(event); + ReportResult("mouse", result, last_mouse_result_); + return result == INPUT_SUCCESS; + }; + if (flag >= wheel_vertical) { + OH_Input_SetMouseEventAxisType(event, flag == wheel_vertical ? MOUSE_AXIS_SCROLL_VERTICAL : MOUSE_AXIS_SCROLL_HORIZONTAL); + for (int action : {MOUSE_ACTION_AXIS_BEGIN, MOUSE_ACTION_AXIS_UPDATE, MOUSE_ACTION_AXIS_END}) { + // Wire values are wheel steps. Only UPDATE carries a delta; repeating + // it on all three phases scrolls three times for one remote event. + OH_Input_SetMouseEventAxisValue(event, action == MOUSE_ACTION_AXIS_UPDATE ? std::clamp(scroll, -1200, 1200) : 0); + OH_Input_SetMouseEventAction(event, action); + ok = inject() && ok; + } + } else { + OH_Input_SetMouseEventAction(event, flag == move ? MOUSE_ACTION_MOVE : + (flag & 1) ? MOUSE_ACTION_BUTTON_DOWN : MOUSE_ACTION_BUTTON_UP); + if (flag != move) { + const int index = (flag - 1) / 2; + OH_Input_SetMouseEventButton(event, buttons[index]); + } + ok = inject(); + if (ok && flag != move) { + const unsigned bit = 1u << ((flag - 1) / 2); + if (flag & 1) buttons_ |= bit; else buttons_ &= ~bit; + } + } + OH_Input_DestroyMouseEvent(&event); + return ok; +} +bool Input::Apply(const RemoteAction& action) { + if (action.type == mouse) return Pointer(action.m.flag, action.m.x, action.m.y, action.m.s); + if (action.type == keyboard) { + if (action.k.flag != key_down && action.k.flag != key_up) return false; + return Key(NormalizeKey(static_cast(action.k.key_value), action.k.scan_code, action.k.extended), action.k.flag == key_down); + } + if (action.type == keyboard_state) { + if (action.ks.pressed_count > kMaxKeyboardStateKeys) return false; + if (action.ks.seq && have_key_sequence_ && static_cast(action.ks.seq - key_sequence_) <= 0) return true; + key_sequence_ = action.ks.seq; have_key_sequence_ = true; + PressedKeys next; + for (size_t i = 0; i < action.ks.pressed_count; ++i) { + const auto& wire = action.ks.pressed_keys[i]; + const auto key = NormalizeKey(wire.key_value, wire.scan_code, wire.extended); + if (key.key_code) next[key.key_code] = key; + } + bool ok = true; + for (const auto& change : ReconcileKeyboardState(keys_, next)) { + if (change.is_down && !ok) continue; + ok = Key({change.key_code, change.scan_code, change.extended}, change.is_down) && ok; + } + return ok; + } + return false; +} +bool Input::Release() { + for (const auto& change : ReconcileKeyboardState(keys_, {})) + Key({change.key_code, change.scan_code, change.extended}, false); + for (int button = 0; button < 3; ++button) + if (buttons_ & (1u << button)) Pointer(2 + button * 2, x_, y_, 0); + // Failed releases remain pending so a later cleanup can retry them. + have_key_sequence_ = false; + return keys_.empty() && buttons_ == 0; +} +bool Input::LockWorkstation() { + // The public Harmony shortcut is Meta+L. Never carry a prior remote drag or + // modifier into it, and retain failed releases for the normal cleanup path. + if (!Release()) return false; + const KeyboardKey meta{0x5B, 0x5B, true}, letter{0x4C, 0, false}; + bool ok = Key(meta, true); + if (ok) ok = Key(letter, true); + if (keys_.count(letter.key_code)) ok = Key(letter, false) && ok; + if (keys_.count(meta.key_code)) ok = Key(meta, false) && ok; + return ok; +} +} diff --git a/apps/harmony/native/input.h b/apps/harmony/native/input.h new file mode 100644 index 00000000..da95353d --- /dev/null +++ b/apps/harmony/native/input.h @@ -0,0 +1,33 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _INPUT_H_ +#define _INPUT_H_ + +#include +#include + +namespace crossdesk::harmony { +class Input { + public: + bool Configure(int width, int height, int display); + bool Apply(const RemoteAction& action); + bool Release(); + bool LockWorkstation(); + private: + bool Key(const KeyboardKey& key, bool down); + bool Pointer(int flag, float x, float y, int scroll); + int width_ = 0, height_ = 0, display_ = 0; + float x_ = 0.5f, y_ = 0.5f; + PressedKeys keys_; + int last_key_result_ = -1, last_mouse_result_ = -1; + unsigned buttons_ = 0; + uint32_t key_sequence_ = 0; + bool have_key_sequence_ = false; +}; +} + +#endif diff --git a/apps/harmony/native/key_mapping.h b/apps/harmony/native/key_mapping.h new file mode 100644 index 00000000..cf50a18a --- /dev/null +++ b/apps/harmony/native/key_mapping.h @@ -0,0 +1,66 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _KEY_MAPPING_H_ +#define _KEY_MAPPING_H_ + +#include + +namespace crossdesk::harmony { +// CrossDesk's keyboard wire format uses Windows virtual-key values. HarmonyOS +// key codes are shared by ArkUI KeyEvent and the native Input Kit. +inline constexpr std::pair kSpecialKeys[] = { + {2045, 0xA4}, {2046, 0xA5}, {2047, 0xA0}, {2048, 0xA1}, + {2049, 0x09}, {2050, 0x20}, {2054, 0x0D}, {2055, 0x08}, + {2056, 0xC0}, {2057, 0xBD}, {2058, 0xBB}, {2059, 0xDB}, + {2060, 0xDD}, {2061, 0xDC}, {2062, 0xBA}, {2063, 0xDE}, + {2064, 0xBF}, {2043, 0xBC}, {2044, 0xBE}, + {2070, 0x1B}, {2071, 0x2E}, {2072, 0xA2}, {2073, 0xA3}, + {2074, 0x14}, {2075, 0x91}, {2076, 0x5B}, {2077, 0x5C}, + {2079, 0x2C}, {2080, 0x13}, {2081, 0x24}, {2082, 0x23}, + {2083, 0x2D}, {2068, 0x21}, {2069, 0x22}, + {2012, 0x26}, {2013, 0x28}, {2014, 0x25}, {2015, 0x27}, + {2102, 0x90}, {2113, 0x6F}, {2114, 0x6A}, {2115, 0x6D}, + {2116, 0x6B}, {2117, 0x6E}, {2067, 0x5D}}; + +inline int ToVirtualKey(int key) { + if (key >= 2017 && key <= 2042) return key - 2017 + 0x41; + if (key >= 2000 && key <= 2009) return key - 2000 + 0x30; + if (key >= 2090 && key <= 2101) return key - 2090 + 0x70; + if (key >= 2816 && key <= 2827) return key - 2816 + 0x7C; + if (key >= 2103 && key <= 2112) return key - 2103 + 0x60; + if (key == 2119) return 0x0D; + for (auto [native, wire] : kSpecialKeys) if (key == native) return wire; + return 0; +} + +inline int FromVirtualKey(int key, unsigned scan = 0, bool extended = false) { + scan &= 0xFF; + if (key == 0x10) return scan == 0x36 ? 2048 : 2047; + if (key == 0x11) return extended ? 2073 : 2072; + if (key == 0x12) return extended ? 2046 : 2045; + if (key == 0x0D && extended) return 2119; + // Windows reports navigation VKs for the number pad with NumLock off. + // Preserve the physical key; the host's own NumLock state selects its action. + if (!extended) { + constexpr std::pair navigation[] = { + {0x2D, 0x52}, {0x23, 0x4F}, {0x28, 0x50}, {0x22, 0x51}, {0x25, 0x4B}, + {0x0C, 0x4C}, {0x27, 0x4D}, {0x24, 0x47}, {0x26, 0x48}, {0x21, 0x49}}; + for (int i = 0; i < 10; ++i) + if (key == navigation[i].first && scan == navigation[i].second) return 2103 + i; + if (key == 0x2E && scan == 0x53) return 2117; + } + if (key >= 0x41 && key <= 0x5A) return key - 0x41 + 2017; + if (key >= 0x30 && key <= 0x39) return key - 0x30 + 2000; + if (key >= 0x70 && key <= 0x7B) return key - 0x70 + 2090; + if (key >= 0x7C && key <= 0x87) return key - 0x7C + 2816; + if (key >= 0x60 && key <= 0x69) return key - 0x60 + 2103; + for (auto [native, wire] : kSpecialKeys) if (key == wire) return native; + return 0; +} +} // namespace crossdesk::harmony + +#endif diff --git a/apps/harmony/native/media.cpp b/apps/harmony/native/media.cpp new file mode 100644 index 00000000..3727b31f --- /dev/null +++ b/apps/harmony/native/media.cpp @@ -0,0 +1,180 @@ +#include "media.h" +#include +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +Renderer::Renderer(Presented presented) : presented_(std::move(presented)), thread_([this] { Run(); }) {} +Renderer::~Renderer() { + { std::lock_guard lock(mutex_); stop_ = true; ready_.notify_one(); } + thread_.join(); +} +void Renderer::Surface(uint64_t id) { + std::lock_guard lock(mutex_); + surface_ = id; changed_ = true; latest_.clear(); ready_.notify_one(); + rendered_ = 0; + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "Video surface attached=%{public}d", id != 0); +} +void Renderer::Clear() { + std::lock_guard lock(mutex_); + latest_.clear(); rendered_ = 0; +} +void Renderer::Video(const MiniRtcVideoFrame& frame, uint64_t generation) { + if (!frame.data || !frame.width || !frame.height || (frame.width & 1) || (frame.height & 1) || + frame.width > 8192 || frame.height > 8192 || + frame.size < size_t(frame.width) * frame.height * 3 / 2) return; + std::lock_guard lock(mutex_); + if (!surface_ || stop_) return; + width_ = frame.width; height_ = frame.height; + captured_ = frame.captured_timestamp; received_ = frame.received_timestamp; generation_ = generation; + latest_.assign(frame.data, frame.data + size_t(width_) * height_ * 3 / 2); + ready_.notify_one(); +} +void Renderer::Run() { + EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY); + EGLContext context = EGL_NO_CONTEXT; + EGLSurface surface = EGL_NO_SURFACE; + OHNativeWindow* window = nullptr; + GLuint program = 0, textures[2]{}, vertex_array = 0, vertex_buffer = 0; + EGLConfig config{}; + auto destroy = [&] { + if (display != EGL_NO_DISPLAY) { + eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + if (surface != EGL_NO_SURFACE) eglDestroySurface(display, surface); + } + surface = EGL_NO_SURFACE; + if (window) OH_NativeWindow_DestroyNativeWindow(window); + window = nullptr; + }; + if (display != EGL_NO_DISPLAY && eglInitialize(display, nullptr, nullptr)) { + eglBindAPI(EGL_OPENGL_ES_API); + // Use one ES3 dispatch library/context/shader pipeline. The emulator's + // GLESv2 entry points can return zero handles without a GL error. + const EGLint attrs[] = {EGL_SURFACE_TYPE, EGL_WINDOW_BIT, EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT, + EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_ALPHA_SIZE, 8, EGL_NONE}; + EGLint count = 0; + eglChooseConfig(display, attrs, &config, 1, &count); + const EGLint ctx[] = {EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE}; + if (count) context = eglCreateContext(display, config, EGL_NO_CONTEXT, ctx); + } + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "EGL initialized: context=%{public}d error=%{public}x", + context != EGL_NO_CONTEXT, eglGetError()); + bool reported_error = false; + bool shader_failed = false; + for (;;) { + std::vector pixels; + int width, height; uint64_t id, captured, received, generation; bool changed; + { + std::unique_lock lock(mutex_); + ready_.wait(lock, [&] { return stop_ || changed_ || !latest_.empty(); }); + if (stop_) break; + pixels.swap(latest_); width = width_; height = height_; + id = surface_; changed = changed_; changed_ = false; + captured = captured_; received = received_; generation = generation_; + } + if (changed) { + destroy(); + reported_error = false; + shader_failed = false; + int window_result = id ? OH_NativeWindow_CreateNativeWindowFromSurfaceId(id, &window) : -1; + if (id && context != EGL_NO_CONTEXT && window_result == 0) { + surface = eglCreateWindowSurface(display, config, reinterpret_cast(window), nullptr); + } + if (id) OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "EGL surface: window=%{public}d surface=%{public}d error=%{public}x", + window_result, surface != EGL_NO_SURFACE, eglGetError()); + } + if (pixels.empty()) continue; + if (surface == EGL_NO_SURFACE || !eglMakeCurrent(display, surface, surface, context)) { + if (!reported_error) OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "CrossDesk", "Cannot bind video surface: %{public}x", eglGetError()); + reported_error = true; continue; + } + if (shader_failed) continue; + if (!program) { + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "GL version: %{public}s", glGetString(GL_VERSION)); + const char* vs = "#version 300 es\nlayout(location=0) in vec2 position; layout(location=1) in vec2 uv; out vec2 coord; void main(){coord=uv;gl_Position=vec4(position,0.,1.);}"; + const char* fs = "#version 300 es\nprecision mediump float; in vec2 coord; uniform sampler2D yTex; uniform sampler2D uvTex; out vec4 color; void main(){float y=1.164383*(texture(yTex,coord).r-0.062745);vec2 c=texture(uvTex,coord).rg-vec2(0.5);color=vec4(y+1.596027*c.y,y-0.391762*c.x-0.812968*c.y,y+2.017232*c.x,1.);}"; + auto shader = [](GLenum type, const char* src) { + GLuint s = glCreateShader(type); glShaderSource(s, 1, &src, nullptr); glCompileShader(s); + GLint compiled = GL_FALSE; glGetShaderiv(s, GL_COMPILE_STATUS, &compiled); + if (!s || !compiled) { + char info[1024]{}; glGetShaderInfoLog(s, sizeof(info), nullptr, info); + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "CrossDesk", "Video shader compilation failed: id=%{public}u %{public}s", s, info); + glDeleteShader(s); return GLuint(0); + } + return s; + }; + GLuint v = shader(GL_VERTEX_SHADER, vs), f = shader(GL_FRAGMENT_SHADER, fs); + if (!v || !f) { + glDeleteShader(v); glDeleteShader(f); shader_failed = true; continue; + } + program = glCreateProgram(); glAttachShader(program, v); glAttachShader(program, f); + glBindAttribLocation(program, 0, "position"); glBindAttribLocation(program, 1, "uv"); + glLinkProgram(program); glDeleteShader(v); glDeleteShader(f); + GLint linked = GL_FALSE; glGetProgramiv(program, GL_LINK_STATUS, &linked); + if (!program || !linked) { + char info[1024]{}; glGetProgramInfoLog(program, sizeof(info), nullptr, info); + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "CrossDesk", "Video shader link failed: %{public}s", info); + glDeleteProgram(program); program = 0; shader_failed = true; continue; + } + const GLfloat vertices[] = {-1,-1,0,1, 1,-1,1,1, -1,1,0,0, 1,1,1,0}; + glGenVertexArrays(1, &vertex_array); glBindVertexArray(vertex_array); + glGenBuffers(1, &vertex_buffer); glBindBuffer(GL_ARRAY_BUFFER, vertex_buffer); + glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW); + glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 4 * sizeof(float), nullptr); + glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 4 * sizeof(float), reinterpret_cast(2 * sizeof(float))); + glEnableVertexAttribArray(0); glEnableVertexAttribArray(1); + glGenTextures(2, textures); + for (GLuint texture : textures) { + glBindTexture(GL_TEXTURE_2D, texture); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + } + } + EGLint sw = 0, sh = 0; + eglQuerySurface(display, surface, EGL_WIDTH, &sw); eglQuerySurface(display, surface, EGL_HEIGHT, &sh); + glViewport(0, 0, sw, sh); glClearColor(0.035f, 0.05f, 0.08f, 1); glClear(GL_COLOR_BUFFER_BIT); + const float scale = std::min(float(sw) / width, float(sh) / height); + const int rw = int(width * scale), rh = int(height * scale); + glViewport((sw - rw) / 2, (sh - rh) / 2, rw, rh); + glUseProgram(program); glPixelStorei(GL_UNPACK_ALIGNMENT, 1); + for (int i = 0; i < 2; ++i) { + glActiveTexture(GL_TEXTURE0 + i); glBindTexture(GL_TEXTURE_2D, textures[i]); + glTexImage2D(GL_TEXTURE_2D, 0, i ? GL_RG8 : GL_R8, + i ? width / 2 : width, i ? height / 2 : height, 0, + i ? GL_RG : GL_RED, GL_UNSIGNED_BYTE, + pixels.data() + (i ? size_t(width) * height : 0)); + } + glUniform1i(glGetUniformLocation(program, "yTex"), 0); + glUniform1i(glGetUniformLocation(program, "uvTex"), 1); + glBindVertexArray(vertex_array); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + const GLenum draw_error = glGetError(); + const bool swapped = eglSwapBuffers(display, surface); + if (draw_error == GL_NO_ERROR && swapped && sw > 0 && sh > 0) { + if (presented_) presented_(pixels.data(), width, height, captured, received, generation); + if (rendered_.fetch_add(1) == 0) + OH_LOG_Print(LOG_APP, LOG_INFO, 0, "CrossDesk", "First video frame presented: %{public}dx%{public}d on %{public}dx%{public}d", + width, height, sw, sh); + } else if (!reported_error) { + OH_LOG_Print(LOG_APP, LOG_ERROR, 0, "CrossDesk", "Video presentation failed: gl=%{public}x egl=%{public}x size=%{public}dx%{public}d", + draw_error, eglGetError(), sw, sh); + reported_error = true; + } + } + if (context != EGL_NO_CONTEXT && surface != EGL_NO_SURFACE) { + eglMakeCurrent(display, surface, surface, context); glDeleteTextures(2, textures); glDeleteProgram(program); + glDeleteBuffers(1, &vertex_buffer); glDeleteVertexArrays(1, &vertex_array); + } + destroy(); + if (context != EGL_NO_CONTEXT) eglDestroyContext(display, context); + // Keep the process-wide display alive for capture and other UI contexts. + eglReleaseThread(); +} + +} diff --git a/apps/harmony/native/media.h b/apps/harmony/native/media.h new file mode 100644 index 00000000..1c631c73 --- /dev/null +++ b/apps/harmony/native/media.h @@ -0,0 +1,46 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-14 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _MEDIA_H_ +#define _MEDIA_H_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +class Renderer { + public: + using Presented = std::function; + explicit Renderer(Presented presented = {}); + ~Renderer(); + void Surface(uint64_t id); + void Video(const MiniRtcVideoFrame& frame, uint64_t generation = 0); + void Clear(); + private: + void Run(); + std::mutex mutex_; + std::condition_variable ready_; + uint64_t surface_ = 0; + bool changed_ = false, stop_ = false; + int width_ = 0, height_ = 0; + std::vector latest_; + uint64_t captured_ = 0, received_ = 0, generation_ = 0; + Presented presented_; + std::atomic rendered_{0}; + // Start only after all state used by Run() has been initialized. + std::thread thread_; +}; + +} + +#endif diff --git a/apps/harmony/native/mobile_session.cpp b/apps/harmony/native/mobile_session.cpp new file mode 100644 index 00000000..e54615e2 --- /dev/null +++ b/apps/harmony/native/mobile_session.cpp @@ -0,0 +1,413 @@ +#include "mobile_session.h" +#include "media.h" +#include "audio.h" +#include "key_mapping.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace crossdesk::harmony { +namespace { +using json = nlohmann::json; +using Clock = std::chrono::steady_clock; +// A separate controller-only entry point. No Capture, Input, AddVideoStream, +// host input handler or screen-recording authorization exists in this session. +class MobileSession { + public: + MobileSession() : renderer_([this](const uint8_t* pixels, int w, int h, uint64_t captured, + uint64_t received, uint64_t generation) { + const auto session = session_.load(), connection = connection_.load(); + if (!connected_ || generation != video_generation_) return; + const auto now = std::chrono::duration_cast(Clock::now().time_since_epoch()).count(); + // MiniRTC uses its own monotonic clock; the offset is sampled on its worker. + const double latency = mobile_controller::VideoLatencyMilliseconds(captured, received, now + clock_offset_); + Event({{"type", "frame"}, {"width", w}, {"height", h}, {"latency", latency}, {"frame", ++frame_id_}, + {"videoGeneration", generation}, {"session", session}, {"connection", connection}}, true); + std::string path; + uint64_t preview_epoch; + { std::lock_guard lock(preview_mutex_); path.swap(preview_path_); preview_epoch = preview_epoch_; } + if (path.empty()) return; + // Only the decoded video is sampled, never the device screen or controls. + // Downscale YUV first: a 4K/8K desktop must not allocate a full-size RGBA copy on a phone. + std::vector scaled(640 * 360 * 3 / 2), preview(640 * 360 * 4); + const int cw = std::min(w, (h * 16 / 9) & ~1), ch = std::min(h, (w * 9 / 16) & ~1); + const int left = ((w-cw)/2)&~1, top = ((h-ch)/2)&~1; + bool ok = cw > 0 && ch > 0 && libyuv::NV12Scale(pixels+size_t(top)*w+left,w, + pixels+size_t(w)*h+size_t(top/2)*w+left,w,cw,ch,scaled.data(),640, + scaled.data()+640*360,640,640,360,libyuv::kFilterBilinear) == 0; + if (ok) ok = libyuv::NV12ToABGR(scaled.data(),640,scaled.data()+640*360,640,preview.data(),640*4,640,360) == 0; + std::lock_guard lock(preview_mutex_); + if (ok && connected_ && generation == video_generation_ && preview_epoch == preview_epoch_) { + std::ofstream out(path, std::ios::binary | std::ios::trunc); + out.write(reinterpret_cast(preview.data()), preview.size()); out.close(); ok = bool(out); + } else ok = false; + if (!ok) std::remove(path.c_str()); + Event({{"type", "preview"}, {"path", path}, {"ok", ok}, {"videoGeneration", generation}, + {"session", session}, {"connection", connection}}, true); + }), worker_([this] { Run(); }) {} + ~MobileSession() { + { std::lock_guard lock(queue_mutex_); stopping_ = true; pending_.clear(); ready_.notify_one(); } + worker_.join(); + } + void Queue(std::function fn) { + std::lock_guard lock(queue_mutex_); + if (!stopping_ && pending_.size() < 8192) { pending_.push_back(std::move(fn)); ready_.notify_one(); } + } + void Command(std::string text) { Queue([this, text = std::move(text)] { Execute(json::parse(text)); }); } + std::string Poll() { + std::lock_guard lock(events_mutex_); + auto value = events_.dump(-1, ' ', false, json::error_handler_t::replace); events_ = json::array(); return value; + } + private: + struct Context { MobileSession* owner; bool controller; uint64_t generation; }; + void Event(json value, bool controller = false) { + if (!value.contains("session")) value["session"] = session_.load(); + if (controller && !value.contains("connection")) value["connection"] = connection_.load(); + std::lock_guard lock(events_mutex_); + // Coalesce high-frequency samples, never displace identity/password or terminal events. + if (value["type"] == "frame") value["latencyCount"] = value["latency"].get() >= 0 ? 1 : 0; + if (value["type"] == "frame" || value["type"] == "network") { + for (auto it = events_.begin(); it != events_.end();) { + if ((*it)["type"] == value["type"]) { + if (value["type"] == "frame" && (*it)["session"] == value["session"] && + (*it)["connection"] == value["connection"] && (*it)["videoGeneration"] == value["videoGeneration"]) { + const int previous = it->value("latencyCount", 0), count = value["latencyCount"]; + if (previous + count > 0) value["latency"] = + (it->value("latency", 0.) * previous + value["latency"].get() * count) / (previous + count); + value["latencyCount"] = previous + count; + } + it = events_.erase(it); + } else ++it; + } + } + if (events_.size() < 1024) events_.push_back(std::move(value)); + } + bool Current(bool controller, uint64_t generation) const { + return generation == (controller ? connection_.load() : session_.load()); + } + template static void Dispatch(void* user, F fn) { + auto* c = static_cast(user); auto* self = c->owner; + self->Queue([self, controller = c->controller, generation = c->generation, fn] { + if (self->Current(controller, generation)) fn(self, controller); + }); + } + static void Signal(SignalStatus status, const char*, size_t, void* user) { + Dispatch(user, [status](MobileSession* s, bool controller) { + if (!controller) { + s->online_ = status == SignalConnected; + if (s->signal_status_ != status) { + ++s->signal_generation_; + if (s->signal_status_ == SignalConnected) s->signal_started_ = Clock::now(); + } + s->signal_status_ = status; + if (s->online_) { + constexpr char info[] = R"({"type":"client_info","version":"harmony-mobile","platform":"harmonyos"})"; + SendSignalMessage(s->identity_, info, sizeof(info)-1); + } + } + s->Event({{"type", "signal"}, {"controller", controller}, {"status", status}, + {"generation", s->signal_generation_}, {"id", s->login_.substr(0, s->login_.find('@'))}}); + if (controller && status >= SignalFailed && status != SignalReconnecting) s->Disconnect("远程会话信令连接失败"); + }); + } + static void SignalMessage(const char* data, size_t size, void* user) { + if (!data || size > 1024*1024) return; + const auto value = json::parse(data, data+size, nullptr, false); + if (!value.is_object()) return; + Dispatch(user, [value](MobileSession* s, bool controller) { + if (controller || !s->online_) return; + const auto type = value.value("type", ""); + if (type == "announcements" || type == "announcements_changed" || type == "presence" || type == "presence_update") + s->Event({{"type", "signalMessage"}, {"message", value}, {"generation", s->signal_generation_}}); + }); + } + static void Connection(ConnectionStatus status, const char* id, size_t size, void* user) { + const std::string remote(id ? id : "", id ? size : 0); + Dispatch(user, [status, remote](MobileSession* s, bool controller) { + if (!controller) { + // Defense in depth if a server tries to attach a guest to our identity. + if (status == Connected && s->identity_) + DisconnectPeerConnection(s->identity_, s->login_.substr(0,s->login_.find('@')).c_str(), remote.c_str()); + return; + } + s->Event({{"type", "connection"}, {"status", status}}, true); + if (status == Connected && !s->connected_) { + s->connecting_ = false; s->connected_ = true; + auto info = MakeHostInformation("CrossDesk HarmonyOS Mobile", {}, false, s->version_, HostPlatform::HarmonyOS); + s->Send(info); FreeRemoteAction(info); + RemoteAction audio{}; audio.type = audio_capture; audio.a = !s->muted_; s->Send(audio); + s->audio_.Start(); + } else if (status >= Disconnected) s->Disconnect(""); + }); + } + static void Network(const char* id, size_t size, TraversalMode mode, const MiniRtcNetTrafficStats* stats, + const char*, size_t, void* user) { + if (mode == UnknownMode && id && size) { + const std::string login(id, size); + Dispatch(user, [login](MobileSession* s, bool controller) { + if (!controller) { + // Reconnect reports may contain only the public ID. Never replace a + // usable ID@password with that public-only report (same as Android). + if (login.find('@') != std::string::npos || s->login_.rfind(login + "@", 0) != 0) s->login_ = login; + s->Event({{"type", "identity"}, {"identity", s->login_}}); + } + }); + } else if (stats) { + auto report = mobile_controller::NetworkReport(*stats, mode); + Dispatch(user, [report](MobileSession* s, bool controller) { + if (controller && s->connected_) s->Event({{"type", "network"}, {"report", report}}, true); + }); + } + } + static void Data(const char* data, size_t size, const char*, size_t, const char* stream, size_t length, void* user) { + if (!data || !stream || size > 1024*1024) return; + const std::string bytes(data,size), name(stream,length); + Dispatch(user, [bytes,name](MobileSession* s, bool controller) { + // The data channel can deliver host information/file chunks before the + // local ICE Connected notification. Keep those authenticated packets. + if (!controller || (!s->connected_ && !s->connecting_)) return; + if (name == kFileStream && s->files_) s->files_->Receive(bytes.data(), bytes.size()); + else if (name == kFileFeedbackStream && s->files_) s->files_->Ack(bytes.data(), bytes.size()); + else if (name == kClipboardStream && bytes.size() <= kMaxClipboardBytes) + s->Event({{"type", "clipboard"}, {"text", bytes}}, true); + else if (name == kControlStream) { + RemoteAction action{}; + if (action.from_json(bytes)) { + const auto value = json::parse(action.to_json()); FreeRemoteAction(action); + s->Event({{"type", "data"}, {"message", value}}, true); + } + } + }); + } + static void Video(const MiniRtcVideoFrame* frame, const char*, size_t, const char* stream, size_t size, void* user) { + auto* c = static_cast(user); auto* s = c->owner; + if (c->controller && c->generation == s->connection_ && s->connected_ && !s->suspended_ && frame && stream && + std::string(stream,size) == MakeDisplayStreamId(s->display_)) s->renderer_.Video(*frame, s->video_generation_); + } + static void Audio(const char* data, size_t size, const char*, size_t, const char*, size_t, void* user) { + auto* c = static_cast(user); auto* s = c->owner; + if (c->controller && c->generation == s->connection_ && s->connected_ && !s->muted_ && !s->suspended_) s->audio_.Push(data,size); + } + Params Parameters(bool controller) { + Params p{}; + std::snprintf(p.signal_server_ip,sizeof(p.signal_server_ip),"%s",server_.c_str()); + std::snprintf(p.log_path,sizeof(p.log_path),"%s",logs_.c_str()); + p.signal_server_port = port_; p.enable_srtp = true; p.turn_mode = TurnAutoUdpTcp; + p.video_content_type = VideoContentType::ScreenContent; p.video_quality = QualityMedium; p.video_frame_rate = 30; + p.video_degradation_preference = VideoDegradationPreference::MaintainResolution; + p.hardware_acceleration = true; + p.user_id = controller ? controller_login_.c_str() : login_.c_str(); + p.user_data = controller ? control_context_.get() : identity_context_.get(); + p.on_signal_status = Signal; p.on_signal_message = SignalMessage; p.on_connection_status = Connection; + p.on_net_status_report = Network; p.on_receive_data_buffer = Data; p.on_receive_video_frame = Video; p.on_receive_audio_buffer = Audio; + return p; + } + void Send(const RemoteAction& action, const char* stream = kControlStream) { + if (!controller_) return; + const auto bytes = action.to_json(); SendReliableDataFrame(controller_,bytes.data(),bytes.size(),stream); + } + void Release() { + for (int key : keys_) { RemoteAction a{}; a.type=keyboard; a.k={size_t(key),0,false,key_up}; Send(a,kKeyboardStream); } + for (int i=0;i<3;i++) if (buttons_ & (1u<(2+i*2)}; Send(a,kMouseStream); + } + keys_.clear(); buttons_=0; + } + void CancelPreview() { + std::lock_guard lock(preview_mutex_); preview_path_.clear(); ++preview_epoch_; + std::error_code error; + if (preview_directory_.empty()) return; + for (const auto& file : std::filesystem::directory_iterator(preview_directory_, error)) { + const auto name = file.path().filename().string(); + if (name.rfind("preview-", 0) == 0 && file.path().extension() == ".rgba") std::filesystem::remove(file.path(), error); + } + } + void Disconnect(const std::string& reason) { + const bool active = controller_ != nullptr; + Release(); connected_=false; connecting_=false; ++video_generation_; + CancelPreview(); + if (controller_) { LeaveConnection(controller_,remote_.c_str()); DestroyPeer(&controller_); } + files_.reset(); renderer_.Clear(); audio_.Stop(); + if (active) Event({{"type","ended"},{"reason",reason}},true); + ++connection_; display_=0; frame_id_=0; + } + void Stop() { + Disconnect(""); online_=false; + if (identity_) DestroyPeer(&identity_); + ++session_; login_.clear(); signal_status_=-1; + } + void Execute(const json& v) { + const auto op=v.at("op").get(); + if (op=="start") { + Stop(); session_=v.at("session").get(); + server_=v.at("server").get(); port_=v.at("port").get(); + logs_=v.at("logs").get(); login_=v.value("identity",""); version_=v.value("version","0.1.0"); + const auto roots=v.at("certificates").get(); directory_=v.at("directory").get(); + preview_directory_=v.at("previewDirectory").get(); + if (server_.empty()||server_.size()>=256||server_.find('@')!=std::string::npos||port_<1||port_>65535|| + logs_.size()>=256||roots.empty()||directory_.empty()) throw std::runtime_error("Invalid configuration"); + setenv("SSL_CERT_DIR",roots.c_str(),1); minirtc::InitLogger(logs_); minirtc::get_logger()->set_level(spdlog::level::off); + identity_context_=std::make_unique(Context{this,false,session_}); + auto p=Parameters(false); identity_=CreatePeer(&p); signal_started_=Clock::now(); + if (!identity_||Init(identity_)!=0) { throw std::runtime_error("Identity initialization failed"); } + } else if (op=="stop") Stop(); + else if (op=="version") { + const auto body=v.at("body").get(), current=v.value("version",""); + const bool local=v.value("local",false); + std::optional result; + if (local) result=CheckPlatformAppUpdate(current,body,"harmony-arm64"); + else if (const auto release=ParseVersionInfoJSON(body)) result=AvailableAppUpdate(current,*release).value_or(""); + Event({{"type","version"},{"request",v.at("request")},{"ok",result.has_value()},{"latest",result.value_or("")}}); + } else if (op=="surface") renderer_.Surface(std::stoull(v.value("id","0"))); + else if (op=="connect") { + if (!online_||login_.empty()) throw std::runtime_error("Not signed in"); + Disconnect(""); connection_=v.at("connection").get(); + remote_=v.at("remote").get(); const auto password=v.at("password").get(); + if (remote_.size()!=9||!std::all_of(remote_.begin(),remote_.end(),::isdigit)||password.size()>128||password.find('@')!=std::string::npos) + throw std::runtime_error("Invalid remote credentials"); + muted_=false; suspended_=false; controller_login_="C-"+login_.substr(0,login_.find('@')); + control_context_=std::make_unique(Context{this,true,connection_}); + auto p=Parameters(true); controller_=CreatePeer(&p); connecting_=true; connection_started_=Clock::now(); + if (!controller_) throw std::runtime_error("Peer creation failed"); + AddAudioStream(controller_,kAudioStream); AddDataStream(controller_,kDataStream,false); + for (auto stream : {kMouseStream,kKeyboardStream,kControlStream,kClipboardStream,kFileStream,kFileFeedbackStream}) + AddDataStream(controller_,stream,true); + files_=std::make_unique( + [this](const char* stream,const char* data,size_t size) { return controller_?SendReliableDataFrame(controller_,data,size,stream):-1; }, + [this](const std::string& name,double progress,bool sending,const std::string& path) { + Event({{"type","file"},{"name",name},{"progress",progress},{"sending",sending},{"path",path}},true); + }); + files_->SetDirectory(directory_+"/received"); + if (Init(controller_)!=0||JoinConnection(controller_,(remote_+"@"+password).c_str())!=0) Disconnect("无法发起连接"); + } else if (op=="disconnect") Disconnect(""); + else if (op=="release") Release(); + else if (op=="signal" && online_ && identity_) { + auto request=v.at("message"); const auto type=request.value("type",""); + if (type!="announcements_list"&&type!="recent_connections_presence") return; + request["user_id"]=login_.substr(0,login_.find('@')); + const auto bytes=request.dump(); const bool ok=SendSignalMessage(identity_,bytes.data(),bytes.size())==0; + if (!ok) Event({{"type","requestFailed"},{"request",request.value("request_id","")}}); + } else if (connected_) { + if (op=="pointer") { + const int flag=v.at("flag").get(); const float x=v.at("x").get(),y=v.at("y").get(); + if (flag<0||flag>8||!std::isfinite(x)||!std::isfinite(y)) return; + x_=std::clamp(x,0.f,1.f); y_=std::clamp(y,0.f,1.f); + if (flag>=1&&flag<=6) { unsigned bit=1u<<((flag-1)/2); if (flag&1) buttons_|=bit; else buttons_&=~bit; } + RemoteAction a{}; a.type=mouse; a.m={x_,y_,std::clamp(v.value("scroll",0),-120,120),static_cast(flag)}; Send(a,kMouseStream); + } else if (op=="key"||op=="virtualKey") { + const int key=op=="key"?ToVirtualKey(v.at("code").get()):v.at("code").get(); + if (key<=0||key>255) return; + const bool down=v.at("down").get(); if (down) keys_.insert(key); else keys_.erase(key); + RemoteAction a{}; a.type=keyboard; a.k={size_t(key),0,false,down?key_down:key_up}; Send(a,kKeyboardStream); + } else if (op=="display") { + const int index=v.at("index").get(); if(index<0||index>=32)return; + Release(); display_=index; ++video_generation_; renderer_.Clear(); + RemoteAction a{}; a.type=display_id; a.d=index; Send(a); + } else if (op=="visible") { + suspended_=!v.at("visible").get(); + if (suspended_) { renderer_.Clear(); audio_.Stop(); } else if (!muted_) audio_.Start(); + } else if (op=="mute") { + muted_=v.at("muted").get(); if (muted_||suspended_) audio_.Stop(); else audio_.Start(); + RemoteAction a{}; a.type=audio_capture; a.a=!muted_; Send(a); + } else if (op=="clipboard") { + const auto text=v.at("text").get(); + if(!text.empty()&&text.size()<=kMaxClipboardBytes) SendReliableDataFrame(controller_,text.data(),text.size(),kClipboardStream); + } else if (op=="videoSettings") { + const int q=v.at("quality"),fps=v.at("frameRate"),pref=v.at("preference"); + if(q<0||q>2||(fps!=30&&fps!=60)||pref<0||pref>2)return; + auto bytes=mobile_controller::VideoSettingsMessage(q,fps,pref,v.at("request").get()); + if(SendReliableDataFrame(controller_,bytes.data(),bytes.size(),kControlStream)!=0) + Event({{"type","settingsFailed"},{"request",v.at("request")}},true); + } else if (op=="sas") { + const auto bytes=mobile_controller::SecureAttentionMessage(); + const bool ok=SendReliableDataFrame(controller_,bytes.data(),bytes.size(),kControlStream)==0; + Event({{"type","feedback"},{"text",ok?"已发送 Ctrl+Alt+Del":"指令发送失败,请重试"}},true); + } else if (op=="sendFile") { + const auto path=v.at("path").get(),name=v.at("name").get(); + // Only disposable staged files owned by this session may be deleted by the transfer engine. + const auto parent=std::filesystem::weakly_canonical(std::filesystem::path(path).parent_path()); + const bool staged=parent==std::filesystem::weakly_canonical(directory_+"/outgoing"); + if(!staged||!files_||!files_->Start(path,name)) { + if (staged) { std::error_code error; std::filesystem::remove(path, error); } + Event({{"type","file"},{"name",name},{"progress",-1},{"sending",true},{"path",""}},true); + } + } else if (op=="preview") { + std::lock_guard lock(preview_mutex_); + preview_path_=preview_directory_+"/preview-"+std::to_string(video_generation_)+"-"+std::to_string(++preview_epoch_)+".rgba"; + } else if (op=="cancelPreview") CancelPreview(); + } + } + void Tick() { + const auto now=Clock::now(); + if(identity_&&!online_&&now-signal_started_>std::chrono::seconds(40)) { + Event({{"type","error"},{"text","服务器连接超时,请检查地址和网络"}}); Stop(); + } + if(connecting_&&now-connection_started_>std::chrono::seconds(40)) Disconnect("连接超时,请确认设备在线"); + if(!connected_)return; + if(files_)files_->Pump(); + if(now-keyboard_sent_>std::chrono::milliseconds(200)) { + RemoteAction a{}; a.type=keyboard_state; a.ks.seq=++keyboard_sequence_; + for(int key:keys_) { if(a.ks.pressed_count>=kMaxKeyboardStateKeys)break; a.ks.pressed_keys[a.ks.pressed_count++]={size_t(key),0,false}; } + Send(a,kKeyboardStream); keyboard_sent_=now; + const auto local=std::chrono::duration_cast(now.time_since_epoch()).count(); + const auto rtc=GetSystemTimeMicros(controller_); if(rtc>0)clock_offset_=rtc-local; + } + } + void Run() { + for(;;) { + std::function fn; + { std::unique_lock lock(queue_mutex_); ready_.wait_for(lock,std::chrono::milliseconds(10),[this]{return stopping_||!pending_.empty();}); + if(stopping_)break; if(!pending_.empty()){fn=std::move(pending_.front());pending_.pop_front();} } + try { if(fn)fn(); Tick(); } + catch(const std::exception&) { Event({{"type","error"},{"text","操作失败,请检查连接参数或重新连接"}}); } + } + Stop(); + } + std::mutex queue_mutex_,events_mutex_,preview_mutex_; + uint64_t preview_epoch_ = 0; + std::condition_variable ready_; + std::deque> pending_; + json events_=json::array(); + bool stopping_=false,online_=false,connecting_=false; + std::atomic connected_{false},muted_{false},suspended_{false}; + std::atomic session_{0},connection_{0},video_generation_{1},frame_id_{0}; + std::atomic clock_offset_{0}; + std::atomic display_{0}; + PeerPtr* identity_=nullptr; + PeerPtr* controller_=nullptr; + std::unique_ptr identity_context_,control_context_; + std::unique_ptr files_; + std::string server_,logs_,login_,controller_login_,remote_,directory_,preview_directory_,preview_path_,version_; + int port_=9099,signal_status_=-1; + uint64_t signal_generation_=0; + std::set keys_; + unsigned buttons_=0; + uint32_t keyboard_sequence_=0; + float x_=.5f,y_=.5f; + Clock::time_point signal_started_,connection_started_,keyboard_sent_; + AudioPlayer audio_; + Renderer renderer_; + std::thread worker_; +}; +std::mutex mobile_mutex; +std::unique_ptr mobile; +} +void MobileCommand(const std::string& text) { + if(text.size()>1024*1024)return; + std::lock_guard lock(mobile_mutex); + if(!mobile)mobile=std::make_unique(); + mobile->Command(text); +} +std::string MobilePoll() { std::lock_guard lock(mobile_mutex); return mobile?mobile->Poll():"[]"; } +void MobileShutdown() { std::lock_guard lock(mobile_mutex); mobile.reset(); } +} diff --git a/apps/harmony/native/mobile_session.h b/apps/harmony/native/mobile_session.h new file mode 100644 index 00000000..6be5342f --- /dev/null +++ b/apps/harmony/native/mobile_session.h @@ -0,0 +1,21 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _MOBILE_SESSION_H_ +#define _MOBILE_SESSION_H_ + +#include + +namespace crossdesk::harmony { +// Commands execute on one worker. MiniRTC callbacks copy their borrowed data +// before queuing it. Poll is the only bridge to the ArkTS thread. +// Mobile uses an identity-only signaling peer and can never accept a guest. +__attribute__((visibility("default"))) void MobileCommand(const std::string& json); +__attribute__((visibility("default"))) std::string MobilePoll(); +__attribute__((visibility("default"))) void MobileShutdown(); +} + +#endif diff --git a/apps/harmony/native/xmake.lua b/apps/harmony/native/xmake.lua new file mode 100644 index 00000000..2205fa44 --- /dev/null +++ b/apps/harmony/native/xmake.lua @@ -0,0 +1,38 @@ +set_project("crossdesk_harmony") +set_xmakever("3.1.1") +set_languages("c++17") +add_rules("mode.release", "mode.debug") +set_policy("package.precompiled", false) +add_cxflags("-fPIC") +includes("../../../deps/submodules/minirtc") + +target("crossdesk_harmony_mobile") + set_kind("shared") + set_default(false) + add_deps("minirtc") + add_packages("nlohmann_json", "libyuv", "spdlog") + add_files("mobile_session.cpp", "media.cpp", "audio.cpp", "../../../libs/wire/src/remote_action.cpp", "../../../libs/wire/src/file_transfer.cpp", "../../../libs/wire/src/app_version.cpp") + -- Export only the bridge API; bundled static dependencies are internal. + add_cxflags("-fvisibility=hidden", "-fvisibility-inlines-hidden") + add_shflags("-Wl,--exclude-libs,ALL", {force = true}) + set_targetdir("../.native/mobile-libs/arm64-v8a") + set_filename("libcrossdesk_harmony_core.so") + add_includedirs("../../../deps/submodules/minirtc/src/api", "../../../libs/wire/include") + add_includedirs("../../desktop/src/platform/common/input") + add_includedirs("../../../libs/controller/include") + add_shflags("-Wl,--no-undefined", {force = true}) + -- CMake imports this file by absolute path. An ELF SONAME keeps that build + -- machine path out of libcrossdesk.so's runtime DT_NEEDED entries. + add_shflags("-Wl,-soname,libcrossdesk_harmony_core.so", {force = true}) + add_syslinks("EGL", "GLESv3", "native_window", "ohaudio", "native_media_core", "hilog_ndk.z") +target_end() + +-- Opt-in integration fixture: synthetic NV12/PCM only, without screen capture/input APIs. +target("crossdesk_harmony_mobile_fixture") + set_default(false) + set_kind("binary") + add_deps("minirtc") + add_packages("nlohmann_json", "spdlog") + add_files("../tests/mobile_synthetic_host.cpp", "../../../libs/wire/src/remote_action.cpp", "../../../libs/wire/src/file_transfer.cpp") + add_includedirs("../../../deps/submodules/minirtc/src/api", "../../../libs/wire/include", "../../../libs/controller/include") + set_targetdir("../.native") diff --git a/apps/harmony/oh-package.json5 b/apps/harmony/oh-package.json5 new file mode 100644 index 00000000..c87f4403 --- /dev/null +++ b/apps/harmony/oh-package.json5 @@ -0,0 +1,7 @@ +{ + "modelVersion": "6.0.0", + "name": "crossdesk-harmony", + "version": "0.1.0", + "description": "CrossDesk for HarmonyOS PC and phone", + "dependencies": {} +} diff --git a/apps/harmony/scripts/build.py b/apps/harmony/scripts/build.py new file mode 100644 index 00000000..c3cc78ae --- /dev/null +++ b/apps/harmony/scripts/build.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +"""Build the Slint PC client (default), or the ArkUI phone/tablet client (--mobile).""" +import argparse +from contextlib import contextmanager +import fcntl +import json +import os +from pathlib import Path +import re +import shutil +import subprocess +import sys + +APP = Path(__file__).resolve().parents[1] +ROOT = APP.parents[1] +CACHE = APP / '.native' +PACKAGE_REVISION = '2eedd731d1e7e377bc0eea970b9f0c120ca5f00d' + + +def run(args, *, cwd=APP, env=None): + subprocess.run([str(arg) for arg in args], cwd=cwd, env=env, check=True) + + +def environment(): + studio = Path(os.environ.get('DEVECO_STUDIO_HOME', '/Applications/DevEco-Studio.app/Contents')) + sdk = Path(os.environ.get('DEVECO_SDK_HOME', studio / 'sdk')) + native = sdk / 'default/openharmony/native' + if not (native / 'llvm/bin/clang++').is_file(): + raise SystemExit('Set DEVECO_SDK_HOME to the DevEco SDK directory containing default/openharmony/native.') + xmake = os.environ.get('XMAKE_BIN') or shutil.which('xmake') + if not xmake: + raise SystemExit('Install Xmake >= 3.1.1 or set XMAKE_BIN.') + output = subprocess.check_output([xmake, '--version'], text=True) + version = re.search(r'xmake v(\d+)\.(\d+)\.(\d+)', output) + if not version or tuple(map(int, version.groups())) < (3, 1, 1): + raise SystemExit('Xmake >= 3.1.1 is required.') + env = os.environ.copy() + env.update(NODE_HOME=str(studio / 'tools/node'), DEVECO_SDK_HOME=str(sdk), + JAVA_HOME=str(studio / 'jbr/Contents/Home'), + XMAKE_PKG_INSTALLDIR=str(CACHE / 'packages'), + XMAKE_GLOBALDIR=str(CACHE / 'global'), NO_COLOR='1') + env['PATH'] = os.pathsep.join([str(CACHE / 'venv/bin'), str(studio / 'tools/node/bin'), + str(studio / 'tools/ohpm/bin'), + str(native / 'build-tools/cmake/bin'), env.get('PATH', '')]) + return studio, native, xmake, env + + +def bootstrap(env): + if sys.version_info < (3, 10): + raise SystemExit('Use Python >= 3.10 (for example /opt/homebrew/bin/python3).') + venv = CACHE / 'venv' + if not (venv / 'bin/python').exists(): + run([sys.executable, '-m', 'venv', venv]) + if not (venv / '.crossdesk-tools-v1').exists(): + run([venv / 'bin/python', '-m', 'pip', 'install', 'setuptools==80.9.0', + 'wheel==0.45.1', 'meson==1.12.1'], env=env) + (venv / '.crossdesk-tools-v1').touch() + repository = CACHE / 'global/.xmake/repositories/xmake-repo' + if not (repository / '.git').exists(): + repository.parent.mkdir(parents=True, exist_ok=True) + seed = Path.home() / '.xmake/repositories/xmake-repo' + source = seed if (seed / '.git').exists() else 'https://github.com/xmake-io/xmake-repo.git' + run(['git', 'clone', '--no-hardlinks', source, repository], env=env) + run(['git', 'remote', 'set-url', 'origin', 'https://github.com/xmake-io/xmake-repo.git'], cwd=repository, env=env) + revision = subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=repository, text=True).strip() + if revision != PACKAGE_REVISION: + run(['git', 'fetch', 'origin', PACKAGE_REVISION], cwd=repository, env=env) + run(['git', 'checkout', '--detach', PACKAGE_REVISION], cwd=repository, env=env) + + +def mobile_project(): + """Stage only the mobile source tree; never carry stale PC files forward.""" + project = CACHE / 'mobile-project' + project.mkdir(exist_ok=True) + for name in ['entry/src', 'entry/libs']: + shutil.rmtree(project / name, ignore_errors=True) + for name in ['AppScope', 'hvigor', 'entry/src/main/cpp', 'entry/src/main/ets/mobile']: + shutil.copytree(APP / name, project / name, dirs_exist_ok=True) + for name in ['element/color.json', 'media/mobile_brand.png']: + target = project / 'entry/src/main/resources/base' / name + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(APP / 'entry/src/main/resources/base' / name, target) + pages = project / 'entry/src/main/ets/pages' + pages.mkdir(parents=True) + shutil.copy2(APP / 'entry/src/main/ets/pages/Mobile.ets', pages / 'Mobile.ets') + shutil.copytree(CACHE / 'mobile-libs', project / 'entry/libs') + (project / 'native').mkdir(exist_ok=True) + (project / 'native/session.h').unlink(missing_ok=True) + shutil.copy2(APP / 'native/mobile_session.h', project / 'native/mobile_session.h') + for name in ['hvigorfile.ts', 'oh-package.json5', 'entry/hvigorfile.ts', 'entry/build-profile.json5', 'entry/oh-package.json5']: + shutil.copy2(APP / name, project / name) + profile = json.loads((APP / 'build-profile.json5').read_text()) + profile['app']['signingConfigs'] = [] + for product in profile['app']['products']: + product.pop('signingConfig', None) + (project / 'build-profile.json5').write_text(json.dumps(profile, indent=2)+'\n') + for name in ['entry/src/main/module.json5', 'entry/src/main/resources/base/profile/main_pages.json']: + target = project / name + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(APP / name, target) + return project + + +def build_native(native, xmake, env, *, mobile=False): + if not (ROOT / 'deps/submodules/minirtc/src/api/minirtc.h').exists(): + raise SystemExit('Run git submodule update --init --recursive first.') + bootstrap(env) + run([xmake, 'f', '-P', APP / 'native', '-y', '-p', 'harmony', '-a', 'arm64-v8a', + '-m', 'release', '--toolchain=hdk', '--sdk=' + str(native), + '--policies=package.precompiled:n', '-o', CACHE / 'build'], cwd=APP / 'native', env=env) + # The PC client links MiniRTC directly; it has no separate session core. + run([xmake, '-P', APP / 'native', '-y', 'crossdesk_harmony_mobile' if mobile else 'minirtc'], + cwd=APP / 'native', env=env) + + +@contextmanager +def build_lock(): + CACHE.mkdir(parents=True, exist_ok=True) + with (CACHE / 'build.lock').open('w') as lock: + fcntl.flock(lock, fcntl.LOCK_EX) + yield + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + epilog='PC options (including permissions/install): python3 apps/harmony/scripts/build_desktop.py --help') + group = parser.add_mutually_exclusive_group() + group.add_argument('--native-only', action='store_true') + group.add_argument('--hap-only', action='store_true', help='Use an already built native library') + parser.add_argument('--release', action='store_true') + parser.add_argument('--mobile', action='store_true', help='Build a controller-only phone/tablet HAP without PC permissions or signing material') + args, desktop_args = parser.parse_known_args() + if not args.mobile and not args.native_only: + run([sys.executable, APP / 'scripts/build_desktop.py', *sys.argv[1:]], cwd=ROOT) + return + if desktop_args: + parser.error('unrecognized arguments: ' + ' '.join(desktop_args)) + with build_lock(): + studio, native, xmake, env = environment() + if not args.hap_only: + build_native(native, xmake, env, mobile=args.mobile) + if not args.native_only: + if not (CACHE / 'mobile-libs/arm64-v8a/libcrossdesk_harmony_core.so').is_file(): + raise SystemExit('Build the mobile core first: python3 apps/harmony/scripts/build.py --mobile --native-only') + project = mobile_project() + run([sys.executable, APP / 'scripts/generate_mobile_resources.py', '--output', + project / 'entry/src/main/resources/rawfile'], env=env) + run([studio / 'tools/ohpm/bin/ohpm', 'install'], cwd=project, env=env) + run([studio / 'tools/hvigor/bin/hvigorw', '--mode', 'module', '-p', 'product=default', + '-p', 'module=entry@default', '-p', 'buildMode=' + ('release' if args.release else 'debug'), + 'assembleHap', '--no-daemon'], cwd=project, env=env) + for hap in sorted((project / 'entry/build/default/outputs').rglob('*.hap')): + run([sys.executable, APP / 'tests/verify_hap.py', hap, '--mobile'], env=env) + + +if __name__ == '__main__': + main() diff --git a/apps/harmony/scripts/build_desktop.py b/apps/harmony/scripts/build_desktop.py new file mode 100644 index 00000000..72041e09 --- /dev/null +++ b/apps/harmony/scripts/build_desktop.py @@ -0,0 +1,272 @@ +#!/usr/bin/env python3 +"""Build a separate HarmonyOS HAP using the unchanged desktop Slint UI.""" +import argparse +import hashlib +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys +import tarfile +import urllib.request + +import build as native_build + +APP = Path(__file__).resolve().parents[1] +ROOT = APP.parents[1] +PORT = APP / 'desktop-port' +CACHE = APP / '.native/slint-poc' +SOURCE = CACHE / 'slint-1.17.1' +PROJECT = CACHE / 'desktop-project' +TARGET = 'aarch64-unknown-linux-ohos' +SHA256 = '68222567f8c70ff677cd4a98cd94fb4765ac0f797eb8f8608a646911c908dc2a' + + +def run(args, *, cwd=ROOT, env=None): + subprocess.run([str(arg) for arg in args], cwd=cwd, env=env, check=True) + + +def write(path, text): + path.parent.mkdir(parents=True, exist_ok=True) + if not path.exists() or path.read_text() != text: + path.write_text(text) + + +def copy(source, dest): + dest.parent.mkdir(parents=True, exist_ok=True) + if not dest.exists() or source.read_bytes() != dest.read_bytes(): + shutil.copy2(source, dest) + + +def write_json(path, value): + write(path, json.dumps(value, ensure_ascii=False, indent=2) + '\n') + + +def main(): + bundle = json.loads((APP / 'AppScope/app.json5').read_text())['app']['bundleName'] + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--reuse-runtime', action='store_true', help='Skip an already completed Rust runtime build') + parser.add_argument('--install', action='store_true', help='Install/start the desktop app') + parser.add_argument('--hap-only', action='store_true', help='Reuse MiniRTC dependencies; rebuild the desktop HAP') + parser.add_argument('--release', action='store_true', help='Build a release HAP') + parser.add_argument('--version', default=os.environ.get('CROSSDESK_VERSION', '0.0.0')) + parser.add_argument('--device', default='127.0.0.1:5555') + parser.add_argument('--slint-compiler', default=os.environ.get('SLINT_COMPILER')) + parser.add_argument('--desktop-permissions', action='store_true', + help='Declare HOOK_KEY_EVENT/READ_PASTEBOARD (requires approved signing-profile ACLs)') + parser.add_argument('--host-permissions', action='store_true', + help='Declare CUSTOM_SCREEN_RECORDING/CONTROL_DEVICE for persistent host authorization (requires approved signing-profile ACLs)') + parser.add_argument('--cursor-position-permission', action='store_true', + help='Declare INPUT_MONITORING for host cursor position during recording (requires an approved signing-profile ACL)') + parser.add_argument('--signing-config', type=Path, + help='Private JSON file containing one DevEco signingConfigs entry; use absolute material paths') + parser.add_argument('--bundle-name', choices=[bundle], + help='Compatibility option; PC and mobile use the shared AppScope bundle name (requires --signing-config)') + args = parser.parse_args() + if sys.version_info < (3, 12): + parser.error('Use Python >= 3.12 (for example /opt/homebrew/bin/python3).') + if args.bundle_name and not args.signing_config: + parser.error('--bundle-name requires --signing-config') + signing = None + if args.signing_config: + signing = json.loads(args.signing_config.read_text()) + if not isinstance(signing, dict) or not signing.get('name') or not isinstance(signing.get('material'), dict): + parser.error('--signing-config must contain one named DevEco signingConfigs entry with material') + for name in ['certpath', 'storeFile', 'profile']: + path = Path(signing['material'].get(name, '')) + if not path.is_absolute() or not path.is_file(): + parser.error(f'signing material {name} must be an existing absolute path') + if not args.version or not all(char.isascii() and (char.isalnum() or char in '.-+') for char in args.version): + parser.error('Invalid application version') + studio, native, xmake, env = native_build.environment() + sdk = Path(env['DEVECO_SDK_HOME']) + env['OHOS_NATIVE'] = str(native) + env['PATH'] = str(Path.home() / '.cargo/bin') + os.pathsep + env['PATH'] + if not args.hap_only: + native_build.build_native(native, xmake, env) + compiler = Path(args.slint_compiler) if args.slint_compiler else next( + (Path.home() / '.xmake/packages/s/slint/1.17.1').glob('*/bin/slint-compiler'), None) + if not compiler or not compiler.is_file(): + raise SystemExit('Set SLINT_COMPILER to a host slint-compiler 1.17.1 executable.') + version = subprocess.check_output([compiler, '--version'], text=True).strip() + if version != 'slint-compiler 1.17.1': + raise SystemExit(f'Expected slint-compiler 1.17.1, got {version}') + CACHE.mkdir(parents=True, exist_ok=True) + archive = CACHE / 'slint-1.17.1.tar.gz' + if not archive.exists(): + urllib.request.urlretrieve('https://codeload.github.com/slint-ui/slint/tar.gz/refs/tags/v1.17.1', archive) + if hashlib.sha256(archive.read_bytes()).hexdigest() != SHA256: + raise SystemExit('Slint source checksum mismatch') + if not SOURCE.exists(): + with tarfile.open(archive) as tar: + tar.extractall(CACHE, filter='data') + patches = [PORT / name for name in ['font-discovery.patch', 'os-services.patch', 'window-properties.patch']] + # A changed patch cannot reverse its older installed version. Restore only + # our patch targets from the verified archive before applying the full set. + targets = {line[6:] for patch in patches for line in patch.read_text().splitlines() + if line.startswith('+++ b/')} + with tarfile.open(archive) as tar: + for relative in targets: + if Path(relative).is_absolute() or '..' in Path(relative).parts: + raise SystemExit('Invalid runtime patch path') + with tar.extractfile(f'slint-1.17.1/{relative}') as original: + (SOURCE / relative).write_bytes(original.read()) + for patch in patches: + run(['git', 'apply', patch], cwd=SOURCE) + linker = CACHE / 'linker.sh' + write(linker, '#!/bin/sh\nexec "${OHOS_NATIVE}/llvm/bin/clang" --target=aarch64-linux-ohos --sysroot="${OHOS_NATIVE}/sysroot" "$@"\n') + linker.chmod(0o755) + env.update(CARGO_TARGET_AARCH64_UNKNOWN_LINUX_OHOS_LINKER=str(linker), + CARGO_TARGET_AARCH64_UNKNOWN_LINUX_OHOS_RUSTFLAGS='-C link-arg=-Wl,-soname,libslint_cpp.so', + SLINT_GENERATED_INCLUDE_DIR=str(CACHE / 'include')) + runtime_signature = hashlib.sha256(b''.join(path.read_bytes() for path in patches)).hexdigest() + runtime_stamp = CACHE / 'runtime.sha256' + if not args.reuse_runtime or not runtime_stamp.exists() or runtime_stamp.read_text() != runtime_signature: + run(['rustup', 'target', 'add', TARGET], env=env) + run(['cargo', 'build', '--manifest-path', SOURCE / 'Cargo.toml', '--locked', '-p', 'slint-cpp', + '--target', TARGET, '--release', '--no-default-features', '--features', 'std,renderer-software'], env=env) + write(runtime_stamp, runtime_signature) + + # Reuse the actual entry point, including MainWindow/StreamWindow/ServerWindow + # and their imported components. No copied or rewritten Slint layout. + generated = CACHE / 'generated' + generated.mkdir(exist_ok=True) + cpp_files = [generated / f'crossdesk_ui_{index}.cpp' for index in range(4)] + command = [compiler, ROOT / 'apps/desktop/src/gui/ui/crossdesk_ui.slint', '-f', 'cpp', + '-o', generated / 'crossdesk_ui.h', '--style', 'fluent-light', + '--embed-resources=embed-files', '--cpp-namespace', 'crossdesk::ui'] + for path in cpp_files: + command += ['--cpp-file', path] + digest = hashlib.sha256('\0'.join(map(str, command)).encode()) + for path in sorted((ROOT / 'apps/desktop/src/gui').rglob('*')): + if '/ui/' not in str(path) and '/assets/' not in str(path): + continue + if path.is_file(): + digest.update(str(path).encode()) + digest.update(path.read_bytes()) + stamp = generated / 'input.sha256' + signature = digest.hexdigest() + if not stamp.exists() or stamp.read_text() != signature or not all(path.exists() for path in [generated / 'crossdesk_ui.h', *cpp_files]): + run(command, env=env) + write(stamp, signature) + deps = CACHE / 'desktop-deps' + deps.mkdir(exist_ok=True) + header = deps / 'httplib.h' + if not header.exists(): + urllib.request.urlretrieve('https://raw.githubusercontent.com/yhirose/cpp-httplib/v0.26.0/httplib.h', header) + if hashlib.sha256(header.read_bytes()).hexdigest() != '4a42da28ce477d06f53942c172197d3b83b4fe42b7e37b8af78c70e063c06da7': + raise SystemExit('cpp-httplib source checksum mismatch') + legacy_types = PROJECT / 'entry/src/main/cpp/types/libslint_probe' + if legacy_types.exists(): + run([studio / 'tools/hvigor/bin/hvigorw', 'clean', '--no-daemon'], cwd=PROJECT, env=env) + shutil.rmtree(legacy_types) + (PROJECT / 'entry/libs/arm64-v8a/libslint_probe.so').unlink(missing_ok=True) + for name in ['build-profile.json5', 'hvigorfile.ts', 'hvigor/hvigor-config.json5', 'entry/hvigorfile.ts', + 'entry/src/main/resources/base/profile/main_pages.json', + 'entry/src/main/resources/base/element/color.json', 'AppScope/resources/base/media/app_icon.png', + 'AppScope/app.json5', 'AppScope/resources/base/element/string.json']: + copy(APP / name, PROJECT / name) + profile = json.loads((APP / 'build-profile.json5').read_text()) + # The PC project contains only its own Index page. + write_json(PROJECT / 'entry/src/main/resources/base/profile/main_pages.json', {'src': ['pages/Index']}) + # The source project can have an IDE-managed signature for a different + # bundle. Only use signing material explicitly selected for this build. + profile['app']['signingConfigs'] = [signing] if signing else [] + for product in profile['app']['products']: + product.pop('signingConfig', None) + product['compatibleSdkVersion'] = '26.0.0' + if signing: + product['signingConfig'] = signing['name'] + write_json(PROJECT / 'build-profile.json5', profile) + (PROJECT / 'build-profile.json5').chmod(0o600) + write_json(PROJECT / 'oh-package.json5', {'modelVersion': '6.0.0', 'name': 'crossdesk-slint-desktop', + 'version': '0.1.0', 'dependencies': {}}) + write_json(PROJECT / 'entry/src/main/resources/base/element/string.json', + {'string': [{'name': 'keyboard_permission_reason', 'value': '控制远程桌面时将系统快捷键发送到远端'}, + {'name': 'screen_recording_permission_reason', 'value': '用于向已通过连接密码验证的远程设备共享本机屏幕'}, + {'name': 'control_device_permission_reason', 'value': '用于接收已通过连接密码验证的远程设备的键盘和鼠标操作'}, + {'name': 'clipboard_permission_reason', 'value': '在远程会话中同步本机和远端的剪贴板文本'}]}) + module = json.loads((APP / 'entry/src/main/module.json5').read_text()) + module['module']['deviceTypes'] = ['2in1'] + module['module']['requestPermissions'] = [ + {'name': 'ohos.permission.INTERNET'}, + {'name': 'ohos.permission.FILE_ACCESS_PERSIST'}, + {'name': 'ohos.permission.PREPARE_APP_TERMINATE'}] + for ability in module['module']['abilities']: + ability['srcEntry'] = './ets/entryability/EntryAbility.ets' + ability.pop('backgroundModes', None) + if args.desktop_permissions: + module['module']['requestPermissions'] += [ + {'name': 'ohos.permission.HOOK_KEY_EVENT', 'reason': '$string:keyboard_permission_reason', + 'usedScene': {'abilities': ['EntryAbility'], 'when': 'inuse'}}, + {'name': 'ohos.permission.READ_PASTEBOARD', 'reason': '$string:clipboard_permission_reason', + 'usedScene': {'abilities': ['EntryAbility'], 'when': 'always'}}] + if args.host_permissions: + module['module']['requestPermissions'] += [ + {'name': 'ohos.permission.CUSTOM_SCREEN_RECORDING', 'reason': '$string:screen_recording_permission_reason', + 'usedScene': {'abilities': ['EntryAbility'], 'when': 'always'}}, + {'name': 'ohos.permission.CONTROL_DEVICE', 'reason': '$string:control_device_permission_reason', + 'usedScene': {'abilities': ['EntryAbility'], 'when': 'always'}}] + if args.cursor_position_permission: + module['module']['requestPermissions'].append({'name': 'ohos.permission.INPUT_MONITORING'}) + write(PROJECT / 'entry/src/main/ets/pages/DesktopCapabilities.ets', + f'export const desktopPermissions: boolean = {str(args.desktop_permissions).lower()};\n' + f'export const hostPermissions: boolean = {str(args.host_permissions).lower()};\n') + write_json(PROJECT / 'entry/src/main/module.json5', module) + profile = json.loads((APP / 'entry/build-profile.json5').read_text()) + profile['buildOption']['externalNativeOptions']['cppFlags'] = '-std=c++20' + write_json(PROJECT / 'entry/build-profile.json5', profile) + write_json(PROJECT / 'entry/oh-package.json5', {'name': 'entry', 'version': '0.1.0', + 'dependencies': {'libslint_desktop.so': 'file:./src/main/cpp/types/libslint_desktop'}}) + types = PROJECT / 'entry/src/main/cpp/types/libslint_desktop' + write_json(types / 'oh-package.json5', {'name': 'libslint_desktop.so', 'version': '0.1.0', + 'types': './index.d.ts'}) + copy(PORT / 'index.d.ts', types / 'index.d.ts') + copy(PORT / 'Index.ets', PROJECT / 'entry/src/main/ets/pages/Index.ets') + copy(PORT / 'EntryAbility.ets', PROJECT / 'entry/src/main/ets/entryability/EntryAbility.ets') + for name in ['DesktopHost.ets', 'DesktopAccess.ets', 'AutostartHost.ets', 'TextInputHost.ets', 'KeyboardHost.ets', 'TrayHost.ets', 'FileHost.ets', 'DisplayHost.ets']: + copy(PORT / name, PROJECT / 'entry/src/main/ets/pages' / name) + runtime = PROJECT / 'entry/libs/arm64-v8a/libslint_cpp.so' + copy(SOURCE / f'target/{TARGET}/release/libslint_cpp.so', runtime) + def quote(path): + return '"' + str(path).replace('\\', '/') + '"' + cmake = f'''cmake_minimum_required(VERSION 3.16) +project(crossdesk_slint_desktop LANGUAGES CXX) +set(CMAKE_CXX_STANDARD 20) +add_library(slint_cpp SHARED IMPORTED) +set_target_properties(slint_cpp PROPERTIES IMPORTED_LOCATION {quote(runtime)} IMPORTED_SONAME "libslint_cpp.so") +add_library(slint_desktop SHARED {quote(PORT / 'desktop.cpp')} {' '.join(map(quote, cpp_files))}) +set_target_properties(slint_desktop PROPERTIES SKIP_BUILD_RPATH TRUE) +target_compile_options(slint_desktop PRIVATE -Os) +target_compile_definitions(slint_desktop PRIVATE _LIBCPP_ENABLE_EXPERIMENTAL) +target_include_directories(slint_desktop PRIVATE + {quote(SOURCE / 'api/cpp/include')} {quote(CACHE / 'include')} {quote(generated)} + {quote(ROOT / 'apps/desktop/src/gui/ui')} + {quote(ROOT / 'apps/desktop/src/gui/assets/localization')} + {quote(ROOT / 'apps/desktop/src/gui/assets/fonts')}) +target_link_libraries(slint_desktop PRIVATE slint_cpp ace_napi.z hilog_ndk.z EGL GLESv3 native_window) +''' + cmake += f'set(CROSSDESK_ROOT {quote(ROOT)})\nset(SLINT_DESKTOP_DEPS {quote(deps)})\nset(CROSSDESK_VERSION "{args.version}")\ninclude({quote(PORT / "CMakeLists.txt")})\n' + write(PROJECT / 'entry/src/main/cpp/CMakeLists.txt', cmake) + run([studio / 'tools/ohpm/bin/ohpm', 'install'], cwd=PROJECT, env=env) + run([studio / 'tools/hvigor/bin/hvigorw', '--mode', 'module', '-p', 'product=default', + '-p', 'module=entry@default', '-p', 'buildMode=' + ('release' if args.release else 'debug'), 'assembleHap', '--no-daemon'], cwd=PROJECT, env=env) + hap = PROJECT / ('entry/build/default/outputs/default/entry-default-' + + ('signed' if signing else 'unsigned') + '.hap') + verify = [sys.executable, APP / 'tests/verify_hap.py', hap, '--bundle-name', bundle] + for option in ['desktop_permissions', 'host_permissions', 'cursor_position_permission']: + if getattr(args, option): + verify.append('--' + option.replace('_', '-')) + run(verify, env=env) + print(f'Desktop HAP: {hap}') + if args.install: + hdc = sdk / 'default/openharmony/toolchains/hdc' + run([hdc, '-t', args.device, 'install', '-r', hap], env=env) + run([hdc, '-t', args.device, 'shell', 'aa', 'force-stop', bundle], env=env) + run([hdc, '-t', args.device, 'shell', 'aa', 'start', '-a', 'EntryAbility', '-b', bundle], env=env) + + +if __name__ == '__main__': + with native_build.build_lock(): + main() diff --git a/apps/harmony/scripts/generate_mobile_resources.py b/apps/harmony/scripts/generate_mobile_resources.py new file mode 100644 index 00000000..babd4089 --- /dev/null +++ b/apps/harmony/scripts/generate_mobile_resources.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python3 +"""Package source-backed, offline mobile legal and build documents.""" +import hashlib +import argparse +import importlib.util +import json +import os +from pathlib import Path +import subprocess +import shutil + +APP = Path(__file__).resolve().parents[1] +ROOT = APP.parents[1] +OUTPUT = APP / 'entry/src/main/resources/rawfile' + + +def git(*args, cwd=ROOT): + return subprocess.check_output(['git', *args], cwd=cwd, text=True).strip() + + +def main(output=OUTPUT): + spec = importlib.util.spec_from_file_location('android_notices', ROOT / 'apps/android/scripts/generate_notices.py') + android = importlib.util.module_from_spec(spec) + spec.loader.exec_module(android) + catalog = json.loads((ROOT / 'apps/ios/CrossDeskMobile/Resources/ThirdPartyLicenses.json').read_text()) + components = [dict(c) for c in catalog['components'] if c['id'] in android.COMPONENTS | {'crossdesk'}] + additions = json.loads((ROOT / 'apps/android/licenses/additional-notices.json').read_text()) + components += [c for c in additions if c['id'] != 'android-ndk'] + native = Path(os.environ.get('DEVECO_SDK_HOME', '/Applications/DevEco-Studio.app/Contents/sdk')) / 'default/openharmony/native' + components.append({'id': 'ohos-llvm', 'name': 'HarmonyOS LLVM runtime', 'license': 'Apache-2.0 WITH LLVM-exception', + 'version': 'SDK bundled', 'sourceURL': 'https://gitee.com/openharmony/third_party_llvm-project', + 'documents': [{'title': 'LLVM NOTICE', 'text': (native / 'llvm/NOTICE').read_text()}]}) + revision = git('rev-parse', 'HEAD') + mini = git('rev-parse', 'HEAD', cwd=ROOT / 'deps/submodules/minirtc') + source = f'https://github.com/kunkundi/crossdesk/tree/{revision}' + notice = ('CrossDesk HarmonyOS is GPL-3.0-only; MiniRTC is LGPL-3.0-only. Other components retain their licenses.\n' + 'You may modify, rebuild and relink these libraries, then install a HAP signed with your own HarmonyOS key.\n' + 'Open-source licenses govern use, modification and redistribution; their warranty limitations apply.\n') + output.mkdir(parents=True, exist_ok=True) + # The catalog points to individual, hash-checked documents. A second full + # concatenation was unused and duplicated every bundled license. + (output / 'THIRD_PARTY_NOTICES.txt').unlink(missing_ok=True) + shutil.rmtree(output / 'about', ignore_errors=True) + for filename in ['LICENSE', 'PRIVACY.md']: + (output / filename).write_text((ROOT / filename).read_text()) + (output / 'OPEN_SOURCE_NOTICE.txt').write_text(notice) + for component in components: + if component['id'] == 'crossdesk': + component['sourceURL'] = source + component['documents'] = [{'title': 'LICENSE', 'text': (ROOT / 'LICENSE').read_text()}, {'title': 'Open-source rights', 'text': notice}] + if component['id'] in {'minirtc', 'inih', 'webrtc'}: + component.update(version=mini, sourceURL=f'https://github.com/kunkundi/minirtc/tree/{mini}') + docs = [] + for index, doc in enumerate(component['documents']): + asset = f"about/licenses/{component['id']}/{index}.txt" + target = output / asset + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(doc['text']) + docs.append({'title': doc['title'], 'asset': asset, 'sha256': hashlib.sha256(doc['text'].encode()).hexdigest()}) + component['documents'] = docs + (output / 'ThirdPartyLicenses.json').write_text(json.dumps({'schemaVersion': 1, 'components': components}, ensure_ascii=False, indent=2)+'\n') + documents = [] + for i, (title, path) in enumerate([ + ('鸿蒙构建说明', APP / 'README.md'), ('移动端功能对照', APP / 'MOBILE_PARITY.md'), + ('原生构建配方', APP / 'native/xmake.lua'), ('构建脚本', APP / 'scripts/build.py'), + ('MiniRTC 构建配方', ROOT / 'deps/submodules/minirtc/xmake.lua'), + ('OpenSSL 构建配方', ROOT / 'deps/submodules/minirtc/thirdparty/openssl/xmake.lua')]): + if not path.exists(): continue + asset = f'about/build/{i}.txt'; target = output / asset; target.parent.mkdir(parents=True, exist_ok=True); target.write_text(path.read_text()) + documents.append({'title': title, 'asset': asset}) + (output / 'SourceMetadata.json').write_text(json.dumps({'revision': revision, 'miniRTCRevision': mini, 'sourceURL': source, + 'isModified': bool(git('status', '--porcelain', '--untracked-files=normal')), 'buildDocuments': documents}, ensure_ascii=False, indent=2)+'\n') + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--output', type=Path, default=OUTPUT) + main(parser.parse_args().output) diff --git a/apps/harmony/tests/autostart_host_test.cjs b/apps/harmony/tests/autostart_host_test.cjs new file mode 100644 index 00000000..1dc26f13 --- /dev/null +++ b/apps/harmony/tests/autostart_host_test.cjs @@ -0,0 +1,116 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const vm = require('node:vm'); +const ts = require(process.argv[2]); +const path = require('node:path'); +const source = process.argv[3]; +const sent = [], queries = [], timers = new Map(); +let supported = true, capabilityError = false, nextTimer = 0; +const sdk = { + isAutoStartupSupported() { if (capabilityError) throw Error('service unavailable'); return supported; }, + getAutoStartupStatusForSelf() { return new Promise((resolve, reject) => queries.push({ resolve, reject })); } +}; +const flush = () => new Promise(resolve => setImmediate(resolve)); +function load(file, imports, extra = {}) { + const module = {}; + vm.runInNewContext(ts.transpileModule(fs.readFileSync(file, 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } + }).outputText, { + exports: module, require: name => imports[name] ?? {}, + console: { warn() {}, error: e => { throw Error(e); } }, + setTimeout: fn => { timers.set(++nextTimer, fn); return nextTimer; }, + clearTimeout: id => timers.delete(id), ...extra + }); + return module; +} +const { AutostartHost: host } = load(source, { '@kit.AbilityKit': { autoStartupManager: sdk } }); +const notify = e => sent.push(JSON.parse(JSON.stringify(e))); +const latest = () => sent.at(-1); +const check = value => assert.deepEqual(latest(), { op: 'autostartState', ...value }); + +(async () => { + host.refresh(); assert.equal(sent.length, 0, 'no queries before startup'); + supported = false; host.start(notify); + check({ ok: true, supported: false }); assert.equal(queries.length, 0); + assert.equal(timers.size, 0); + supported = true; host.refresh(); check({ pending: true }); + queries.at(-1).resolve(true); await flush(); + check({ ok: true, supported: true, active: true }); + host.refresh(); queries.at(-1).resolve(false); await flush(); + check({ ok: true, supported: true, active: false }); + + host.refresh(); queries.at(-1).reject(Error('permission/service failure')); await flush(); + check({ ok: false }); // Failure must never be displayed as "Disabled". + capabilityError = true; host.refresh(); check({ ok: false }); + capabilityError = false; + + host.refresh(); const old = queries.at(-1); + host.refresh(); const fresh = queries.at(-1); + assert.equal(timers.size, 1, 'superseding a query clears its timeout'); + fresh.resolve(true); await flush(); const count = sent.length; + old.resolve(false); await flush(); assert.equal(sent.length, count); + check({ ok: true, supported: true, active: true }); + + host.refresh(); const timedOut = queries.at(-1); + [...timers.values()][0](); check({ ok: false }); + const afterTimeout = sent.length; + timedOut.resolve(true); await flush(); assert.equal(sent.length, afterTimeout); + host.refresh(); queries.at(-1).resolve(false); await flush(); + check({ ok: true, supported: true, active: false }); + + host.refresh(); const beforeStop = queries.at(-1); + host.stop(); const stopped = sent.length; + beforeStop.resolve(true); await flush(); host.refresh(); + assert.equal(sent.length, stopped); assert.equal(timers.size, 0); + host.start(notify); const previousRun = queries.at(-1); + host.start(notify); queries.at(-1).resolve(false); await flush(); + const restarted = sent.length; + previousRun.reject(Error('late failure')); await flush(); assert.equal(sent.length, restarted); + host.stop(); + + // Exercise the real desktop startup, window activation, EntryAbility and + // native-command routing, not only the standalone query helper. + const handlers = new Map(), commands = [], order = []; + const target = { + on: (name, fn) => handlers.set(name, fn), + getWindowDecorVisible: () => false, + getWindowProperties: () => ({ id: 1, windowRect: { left: 0, top: 0 } }) + }; + const { DesktopHost: desktop } = load(path.join(path.dirname(source), 'DesktopHost.ets'), { + 'libslint_desktop.so': { default: { + start: () => order.push('native'), send: text => notify(JSON.parse(text)), + poll: () => JSON.stringify(commands.splice(0)) + } }, + './AutostartHost': { AutostartHost: host }, + './FileHost': { FileHost: { prepare: async () => {} } }, + './DesktopAccess': { DesktopAccess: { start() {}, refresh() {}, stop() {} } }, + './DisplayHost': { DisplayHost: { + snapshot: async () => [], observe() {}, global: () => ({ x: 0, y: 0 }), screen: () => ({ densityPixels: 1 }) + } }, + '@kit.ArkUI': { display: { getDefaultDisplaySync: () => ({ width: 1280, height: 800, densityPixels: 1, id: 0 }) }, + window: { WindowEventType: { WINDOW_ACTIVE: 1, WINDOW_INACTIVE: 2 } } }, + '@kit.BasicServicesKit': { pasteboard: { getSystemPasteboard: () => ({ on() {}, off() {} }) } }, + '@kit.DeviceCertificateKit': { certificateManager: { getCertificateStorePath: () => '/test/ca', CertType: {} } } + }, { LocalStorage: class { setOrCreate() {} }, setInterval: () => 1, clearInterval() {} }); + desktop.foreground(); assert.equal(timers.size, 0); + desktop.start({ getMainWindowSync: () => target, loadContent: async () => {} }, + { filesDir: '/test/files', cacheDir: '/test/cache', terminateSelf: () => order.push('exit') }); + await flush(); assert.deepEqual(order, ['native']); check({ pending: true }); + queries.at(-1).resolve(false); await flush(); + check({ ok: true, supported: true, active: false }); + const { default: Ability } = load(path.join(path.dirname(source), 'EntryAbility.ets'), { + '@kit.AbilityKit': { UIAbility: class {} }, '../pages/DesktopHost': { DesktopHost: desktop } + }); + new Ability().onForeground(); check({ pending: true }); + queries.at(-1).resolve(true); await flush(); check({ ok: true, supported: true, active: true }); + handlers.get('windowEvent')(1); // Returning from system Settings refreshes. + queries.at(-1).resolve(false); await flush(); check({ ok: true, supported: true, active: false }); + commands.push({ op: 'autostartRefresh' }); desktop.poll(); check({ pending: true }); + const last = queries.at(-1); + commands.push({ op: 'exit' }); desktop.poll(); await flush(); + const exited = sent.length; + last.resolve(true); await flush(); new Ability().onForeground(); + assert.equal(sent.length, exited); assert.equal(timers.size, 0); + assert.deepEqual(order, ['native', 'exit']); + console.log('Autostart system state, failures, timeout, stale replies, startup/foreground and exit tests passed'); +})().catch(error => { console.error(error); process.exitCode = 1; }); diff --git a/apps/harmony/tests/build_mobile_fixture.py b/apps/harmony/tests/build_mobile_fixture.py new file mode 100644 index 00000000..8370f7b0 --- /dev/null +++ b/apps/harmony/tests/build_mobile_fixture.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python3 +"""Build the opt-in synthetic host after building the Harmony native core.""" +import importlib.util +from pathlib import Path +APP=Path(__file__).resolve().parents[1] +spec=importlib.util.spec_from_file_location('harmony_build',APP/'scripts/build.py') +build=importlib.util.module_from_spec(spec) +spec.loader.exec_module(build) +_, _, xmake, env=build.environment() +build.run([xmake,'-P',APP/'native','-y','crossdesk_harmony_mobile_fixture'],cwd=APP/'native',env=env) diff --git a/apps/harmony/tests/capture_test.cpp b/apps/harmony/tests/capture_test.cpp new file mode 100644 index 00000000..102e307d --- /dev/null +++ b/apps/harmony/tests/capture_test.cpp @@ -0,0 +1,330 @@ +#include "capture.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// Test the production capture adapter with asynchronous SDK callbacks, without +// opening a recorder or granting access to the workstation's screen. +struct OH_AVScreenCapture { + OH_AVScreenCapture_OnStateChange state{}; + OH_AVScreenCapture_OnError error{}; + OH_AVScreenCapture_OnBufferAvailable data{}; + void *state_user{}, *error_user{}, *data_user{}; + bool started = false; +}; +struct OH_NativeBuffer {}; +struct OH_AVScreenCapture_CaptureStrategy { bool picker = true; }; +struct OH_AVBuffer {}; +struct OH_NativeImage { + OH_OnFrameAvailableListener listener{}; + std::mutex mutex; + std::deque fences; +}; +OH_NativeImage* surface{}; +int window_token = 0, buffer_token = 0; +bool surface_available = false, reject_surface_listener = false; +int surface_starts = 0, surface_destroys = 0; +std::atomic surface_releases{0}, released_fence{-2}; +OH_AVScreenCapture* current{}; +OH_NativeBuffer native_buffer; +OH_AVBuffer av_buffer; +OH_NativeBuffer_Config buffer_config{4, 2, NATIVEBUFFER_PIXEL_FMT_RGBA_8888, 0, 16}; +OH_AVCodecBufferAttr audio_attr{}; +unsigned char pixels[32]{}; +int rate_calls = 0, rate = 0, starts = 0, releases = 0, unrefs = 0; +std::atomic unmaps{0}; +bool reject_init = false, no_native = false, reject_map = false; +bool gpu_available = false; +int gpu_stops = 0; +int strategy_creates = 0, strategy_releases = 0, strategy_applies = 0; +bool reject_strategy_create = false, reject_picker = false, reject_strategy_apply = false; +bool picker_disabled = false; +bool cursor_shown = false, reject_cursor_hide = false, reject_cursor_show = false; +crossdesk::harmony::CaptureSurface::Frame gpu_frame; + +// Only replace EGL/driver ownership here. Capture's authorization, conversion, +// SDK calls, fallback and cleanup remain the production implementation. +namespace crossdesk::harmony { +OHNativeWindow* CaptureSurface::Start(int width, int height, Frame frame) { + assert(width == 4 && height == 2); + if (!gpu_available) return nullptr; + gpu_frame = std::move(frame); + return reinterpret_cast(&window_token); +} +void CaptureSurface::Stop() { + if (gpu_frame) { ++gpu_stops; gpu_frame = {}; } +} +} + +extern "C" { +OH_NativeImage* OH_ConsumerSurface_Create() { return surface_available ? surface = new OH_NativeImage : nullptr; } +int32_t OH_ConsumerSurface_SetDefaultUsage(OH_NativeImage*, uint64_t usage) { + assert((usage & NATIVEBUFFER_USAGE_CPU_READ) && (usage & NATIVEBUFFER_USAGE_HW_RENDER)); return 0; +} +int32_t OH_ConsumerSurface_SetDefaultSize(OH_NativeImage*, int32_t width, int32_t height) { assert(width == 4 && height == 2); return 0; } +OHNativeWindow* OH_NativeImage_AcquireNativeWindow(OH_NativeImage*) { return reinterpret_cast(&window_token); } +int32_t OH_NativeImage_SetOnFrameAvailableListener(OH_NativeImage* image, OH_OnFrameAvailableListener listener) { + if (reject_surface_listener) return -1; + image->listener = listener; return 0; +} +int32_t OH_NativeImage_UnsetOnFrameAvailableListener(OH_NativeImage* image) { image->listener = {}; return 0; } +void OH_NativeImage_Destroy(OH_NativeImage** image) { + assert(!(*image)->listener.onFrameAvailable); + delete *image; *image = nullptr; surface = nullptr; ++surface_destroys; +} +int32_t OH_NativeImage_AcquireNativeWindowBuffer(OH_NativeImage* image, OHNativeWindowBuffer** buffer, int* fence) { + std::lock_guard lock(image->mutex); + if (image->fences.empty()) return 40601000; + *fence = image->fences.front(); image->fences.pop_front(); + *buffer = reinterpret_cast(&buffer_token); return 0; +} +int32_t OH_NativeImage_ReleaseNativeWindowBuffer(OH_NativeImage*, OHNativeWindowBuffer*, int fence) { + released_fence = fence; + if (fence >= 0) close(fence); + ++surface_releases; return 0; +} +int32_t OH_NativeBuffer_FromNativeWindowBuffer(OHNativeWindowBuffer*, OH_NativeBuffer** buffer) { + *buffer = &native_buffer; return 0; // Borrowed; must not decrement its reference count. +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_StartScreenCaptureWithSurface(OH_AVScreenCapture*, OHNativeWindow* window) { + assert(window == reinterpret_cast(&window_token)); + ++starts; ++surface_starts; return AV_SCREEN_CAPTURE_ERR_OK; +} +int OH_LOG_Print(LogType, LogLevel, unsigned int, const char*, const char*, ...) { return 0; } +OH_AVScreenCapture* OH_AVScreenCapture_Create() { return current = new OH_AVScreenCapture; } +OH_AVScreenCapture_CaptureStrategy* OH_AVScreenCapture_CreateCaptureStrategy() { + ++strategy_creates; + return reject_strategy_create ? nullptr : new OH_AVScreenCapture_CaptureStrategy; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_ReleaseCaptureStrategy(OH_AVScreenCapture_CaptureStrategy* strategy) { + ++strategy_releases; delete strategy; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_StrategyForPickerPopUp(OH_AVScreenCapture_CaptureStrategy* strategy, bool value) { + assert(!value); strategy->picker = value; + return reject_picker ? AV_SCREEN_CAPTURE_ERR_INVALID_VAL : AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetCaptureStrategy(OH_AVScreenCapture* p, OH_AVScreenCapture_CaptureStrategy* strategy) { + assert(p == current && !p->started); + ++strategy_applies; + if (reject_strategy_apply) return AV_SCREEN_CAPTURE_ERR_INVALID_VAL; + picker_disabled = !strategy->picker; + return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_Release(OH_AVScreenCapture* p) { + ++releases; delete p; current = nullptr; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetStateCallback(OH_AVScreenCapture* p, OH_AVScreenCapture_OnStateChange cb, void* user) { + p->state = cb; p->state_user = user; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetErrorCallback(OH_AVScreenCapture* p, OH_AVScreenCapture_OnError cb, void* user) { + p->error = cb; p->error_user = user; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetDataCallback(OH_AVScreenCapture* p, OH_AVScreenCapture_OnBufferAvailable cb, void* user) { + p->data = cb; p->data_user = user; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_Init(OH_AVScreenCapture*, OH_AVScreenCaptureConfig config) { + assert(config.captureMode == OH_CAPTURE_SPECIFIED_SCREEN); + assert(config.videoInfo.videoCapInfo.displayId == 42); + assert(config.videoInfo.videoCapInfo.videoFrameWidth == 4 && config.videoInfo.videoCapInfo.videoFrameHeight == 2); + return reject_init ? AV_SCREEN_CAPTURE_ERR_INVALID_VAL : AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetMicrophoneEnabled(OH_AVScreenCapture*, bool enabled) { + assert(!enabled); return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_StartScreenCapture(OH_AVScreenCapture*) { + ++starts; return AV_SCREEN_CAPTURE_ERR_OK; // Consent is still pending. +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_StopScreenCapture(OH_AVScreenCapture* p) { + p->started = false; p->state(p, OH_SCREEN_CAPTURE_STATE_STOPPED_BY_USER, p->state_user); + return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_SetMaxVideoFrameRate(OH_AVScreenCapture* p, int32_t fps) { + assert(p->started); ++rate_calls; rate = fps; return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_AVSCREEN_CAPTURE_ErrCode OH_AVScreenCapture_ShowCursor(OH_AVScreenCapture* p, bool visible) { + assert(p->started); + if (visible ? reject_cursor_show : reject_cursor_hide) return AV_SCREEN_CAPTURE_ERR_OPERATE_NOT_PERMIT; + cursor_shown = visible; + return AV_SCREEN_CAPTURE_ERR_OK; +} +OH_NativeBuffer* OH_AVBuffer_GetNativeBuffer(OH_AVBuffer*) { return no_native ? nullptr : &native_buffer; } +void OH_NativeBuffer_GetConfig(OH_NativeBuffer*, OH_NativeBuffer_Config* config) { *config = buffer_config; } +int32_t OH_NativeBuffer_Map(OH_NativeBuffer*, void** out) { *out = pixels; return reject_map ? -1 : 0; } +int32_t OH_NativeBuffer_Unmap(OH_NativeBuffer*) { ++unmaps; return 0; } +int32_t OH_NativeBuffer_Unreference(OH_NativeBuffer*) { ++unrefs; return 0; } +uint8_t* OH_AVBuffer_GetAddr(OH_AVBuffer*) { return pixels; } +int32_t OH_AVBuffer_GetCapacity(OH_AVBuffer*) { return sizeof(pixels); } +OH_AVErrCode OH_AVBuffer_GetBufferAttr(OH_AVBuffer*, OH_AVCodecBufferAttr* out) { *out = audio_attr; return AV_ERR_OK; } +} +namespace libyuv { +extern "C" int ABGRToNV12(const uint8_t* src, int stride, uint8_t* y, int y_stride, + uint8_t* uv, int uv_stride, int width, int height) { + assert(src == pixels && stride == 16 && width == 4 && height == 2); + assert(y_stride == 4 && uv_stride == 4 && uv == y + 8); + std::memset(y, 128, 12); return 0; +} +} +void State(OH_AVScreenCaptureStateCode state) { + if (state == OH_SCREEN_CAPTURE_STATE_STARTED) current->started = true; + current->state(current, state, current->state_user); +} +void Data(OH_AVScreenCaptureBufferType type = OH_SCREEN_CAPTURE_BUFFERTYPE_VIDEO) { + current->data(current, &av_buffer, type, 0, current->data_user); +} +void SurfaceData(std::initializer_list fences) { + { std::lock_guard lock(surface->mutex); surface->fences.insert(surface->fences.end(), fences); } + surface->listener.onFrameAvailable(surface->listener.context); +} +template void Wait(F condition) { + const auto limit = std::chrono::steady_clock::now() + std::chrono::seconds(3); + while (!condition() && std::chrono::steady_clock::now() < limit) + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + assert(condition()); +} +int main() { + crossdesk::harmony::Capture capture; + std::atomic frames{0}; + int audio_bytes = 0, last_state = -1; + const auto frame = [&](const MiniRtcVideoFrame& value) { + assert(value.width == 4 && value.height == 2 && value.size == 12); + assert(static_cast(value.data[0]) == 128); ++frames; + }; + const auto state = [&](int code) { last_state = code; }; + const auto audio = [&](const char*, size_t size) { audio_bytes += int(size); }; + assert(capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(starts == 1 && rate_calls == 0 && !capture.CursorEmbedded()); + Data(); assert(frames == 0); // Do not send data before consent completes. + State(OH_SCREEN_CAPTURE_STATE_STARTED); + assert(rate == 30 && rate_calls == 1 && capture.CursorEmbedded()); + Data(); assert(frames == 1 && unrefs == 1 && unmaps == 1); + State(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); Data(); assert(frames == 1); + State(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); Data(); assert(frames == 2); + State(OH_SCREEN_CAPTURE_STATE_PAUSED_BY_USER); Data(); assert(frames == 2); + State(OH_SCREEN_CAPTURE_STATE_RESUMED_BY_USER); Data(); assert(frames == 3); + State(OH_SCREEN_CAPTURE_STATE_MIC_MUTED_BY_USER); Data(); assert(frames == 4); + audio_attr.offset = 2; audio_attr.size = 4; + Data(OH_SCREEN_CAPTURE_BUFFERTYPE_AUDIO_INNER); assert(audio_bytes == 4); + audio_attr.offset = 31; audio_attr.size = 4; + Data(OH_SCREEN_CAPTURE_BUFFERTYPE_AUDIO_INNER); assert(audio_bytes == 4); + no_native = true; Data(); no_native = false; assert(frames == 4 && unrefs == 4); + reject_map = true; Data(); reject_map = false; assert(frames == 4 && unrefs == 5 && unmaps == 4); + buffer_config.width = 3; Data(); buffer_config.width = 4; assert(frames == 4 && unrefs == 6); + current->error(current, -10, current->error_user); Data(); assert(frames == 4 && last_state == -1); + capture.Stop(); assert(current == nullptr && releases == 1 && !capture.CursorEmbedded()); + capture.Stop(); assert(releases == 1); + reject_init = true; + assert(!capture.Start(4, 2, 42, frame, state, audio, 120)); + assert(starts == 1 && releases == 2 && current == nullptr); + reject_init = false; + assert(capture.Start(4, 2, 42, frame, state, audio, 120)); + State(OH_SCREEN_CAPTURE_STATE_STARTED); assert(rate == 60); + State(OH_SCREEN_CAPTURE_STATE_CANCELED); Data(); assert(frames == 4); + capture.Stop(); + + surface_available = true; + assert(capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(surface_starts == 1); + SurfaceData({-1}); Wait([] { return surface_releases == 1; }); assert(frames == 4); + State(OH_SCREEN_CAPTURE_STATE_STARTED); + const int references_before = unrefs; + SurfaceData({-1, -1, -1}); // One callback for three queued buffers must drain all three. + Wait([&] { return frames == 7 && surface_releases == 4; }); + assert(unrefs == references_before); + State(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + SurfaceData({-1}); Wait([] { return surface_releases == 5; }); assert(frames == 7); + State(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + int pipe_fd[2]; assert(pipe(pipe_fd) == 0); + assert(write(pipe_fd[1], "x", 1) == 1); + SurfaceData({pipe_fd[0]}); Wait([&] { return frames == 8 && surface_releases == 6; }); + Wait([&] { return fcntl(pipe_fd[0], F_GETFD) == -1; }); close(pipe_fd[1]); + assert(released_fence == -1); + assert(pipe(pipe_fd) == 0); + SurfaceData({pipe_fd[0]}); // Timeout must return the outstanding fence to its owner. + Wait([] { return surface_releases == 7; }); + assert(frames == 8 && released_fence == pipe_fd[0] && fcntl(pipe_fd[0], F_GETFD) == -1); + close(pipe_fd[1]); + capture.Stop(); assert(surface == nullptr && surface_destroys == 1); + capture.Stop(); assert(surface_destroys == 1); + reject_surface_listener = true; + assert(capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(surface == nullptr && surface_destroys == 2 && surface_starts == 1); + State(OH_SCREEN_CAPTURE_STATE_STARTED); Data(); assert(frames == 9); + capture.Stop(); + reject_surface_listener = false; + reject_init = true; + assert(!capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(surface == nullptr && surface_destroys == 3); // Worker exits even when capture never starts. + + reject_init = false; + gpu_available = true; + assert(capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(surface == nullptr && surface_starts == 2); // Prefer GPU over CPU mapping. + gpu_frame(pixels, 4, 2, 16); assert(frames == 9); + State(OH_SCREEN_CAPTURE_STATE_STARTED); + gpu_frame(pixels, 4, 2, 16); assert(frames == 10); + State(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + gpu_frame(pixels, 4, 2, 16); assert(frames == 10); + State(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + gpu_frame(pixels, 4, 2, 16); assert(frames == 11); + State(OH_SCREEN_CAPTURE_STATE_STOPPED_BY_USER); + gpu_frame(pixels, 4, 2, 16); assert(frames == 11); + capture.Stop(); capture.Stop(); assert(gpu_stops == 1 && !gpu_frame); + reject_init = true; + assert(!capture.Start(4, 2, 42, frame, state, audio, 30)); + assert(gpu_stops == 2 && !gpu_frame && current == nullptr); + assert(strategy_creates == 0); // Ordinary sessions keep the system's default authorization UI. + + reject_init = false; + assert(capture.Start(4, 2, 42, frame, state, audio, 30, true)); + assert(picker_disabled && strategy_applies == 1 && strategy_creates == strategy_releases); + gpu_frame(pixels, 4, 2, 16); assert(frames == 11); // A permission hint cannot replace STARTED. + State(OH_SCREEN_CAPTURE_STATE_STARTED); + gpu_frame(pixels, 4, 2, 16); assert(frames == 12); + capture.Stop(); + assert(capture.Start(4, 2, 42, frame, state, audio, 30, true)); + assert(strategy_applies == 2 && strategy_creates == strategy_releases); + State(OH_SCREEN_CAPTURE_STATE_CANCELED); // The OS can still reject a revoked grant. + gpu_frame(pixels, 4, 2, 16); assert(frames == 12); + capture.Stop(); + for (bool* failure : {&reject_strategy_create, &reject_picker, &reject_strategy_apply}) { + *failure = true; picker_disabled = false; + assert(capture.Start(4, 2, 42, frame, state, audio, 30, true)); + assert(!picker_disabled); // Failed optional configuration falls back to system authorization. + capture.Stop(); *failure = false; + } + assert(strategy_creates == strategy_releases + 1); // Failed allocation has no resource to release. + + using crossdesk::harmony::CaptureCursorMode; + assert(capture.Start(4, 2, 42, frame, state, {}, 30, false, false)); + assert(capture.CursorMode() == CaptureCursorMode::unknown); + State(OH_SCREEN_CAPTURE_STATE_STARTED); + assert(!cursor_shown && capture.CursorMode() == CaptureCursorMode::separate); + State(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + cursor_shown = true; + State(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + assert(!cursor_shown && capture.CursorMode() == CaptureCursorMode::separate); + State(OH_SCREEN_CAPTURE_STATE_PAUSED_BY_USER); + cursor_shown = true; + State(OH_SCREEN_CAPTURE_STATE_RESUMED_BY_USER); + assert(!cursor_shown && capture.CursorMode() == CaptureCursorMode::separate); + capture.Stop(); + assert(capture.CursorMode() == CaptureCursorMode::unknown); + + reject_cursor_hide = true; + assert(capture.Start(4, 2, 42, frame, state, {}, 30, false, false)); + State(OH_SCREEN_CAPTURE_STATE_STARTED); + assert(cursor_shown && capture.CursorMode() == CaptureCursorMode::embedded); + capture.Stop(); + reject_cursor_show = true; + assert(capture.Start(4, 2, 42, frame, state, {}, 30, false, false)); + State(OH_SCREEN_CAPTURE_STATE_STARTED); + assert(capture.CursorMode() == CaptureCursorMode::unknown); + capture.Stop(); +} diff --git a/apps/harmony/tests/desktop_devices_test.cpp b/apps/harmony/tests/desktop_devices_test.cpp new file mode 100644 index 00000000..3cb1bf70 --- /dev/null +++ b/apps/harmony/tests/desktop_devices_test.cpp @@ -0,0 +1,413 @@ +#include "bridge.h" +#include "capture.h" +#include "input.h" +#include "keyboard_hook.h" +#include "host_access.h" +#include "keyboard_hook_mock.h" +#include "platform/harmony/input/system_shortcuts.h" +#include "screen_capturer_factory.h" +#include "device_controller_factory.h" +#include "runtime/cursor_state_provider.h" +#include "runtime/cursor_presentation.h" +#include +#include +#include +#include +#include +#include + +// Exercise the production desktop adapter and SDK declarations without +// recording the workstation or requesting real input-injection permissions. +using crossdesk::harmony::desktop::Json; +Json environment; +crossdesk::harmony::Capture::Frame frame_callback; +crossdesk::harmony::Capture::State state_callback; +std::tuple capture_parameters; +int starts = 0, stops = 0, fail_display = -1, input_display = -1, releases = 0; +bool release_ok = true; +bool persistent_capture = false; +bool capture_show_cursor = true; +crossdesk::harmony::CaptureCursorMode captured_cursor_mode = crossdesk::harmony::CaptureCursorMode::separate; +struct Input_CursorInfo {}; +bool pointer_visible = true; +Input_PointerStyle pointer_style = DEFAULT; +Input_Result cursor_result = INPUT_SUCCESS, location_result = INPUT_SUCCESS, style_result = INPUT_SUCCESS; +int32_t pointer_display = 0; +double pointer_x = 960, pointer_y = 270; +struct Input_MouseEvent { int32_t action, display, x, y; }; +Input_MouseEventCallback mouse_monitor = nullptr; +Input_Result monitor_add_result = INPUT_PERMISSION_DENIED, monitor_remove_result = INPUT_SUCCESS; +int monitor_adds = 0, monitor_removes = 0; +const Input_MouseEvent* initial_monitor_event = nullptr; +int injection_requests = 0; +Input_InjectionStatus authorization = UNAUTHORIZED; +namespace crossdesk::harmony::desktop { +Json Environment() { return environment; } +void Emit(Json) {} +} +namespace crossdesk::harmony { +void CaptureSurface::Stop() {} +bool Capture::Start(int width, int height, uint64_t display, Frame frame, State state, Audio, int fps, bool persistent_permission, bool show_cursor) { + ++starts; + persistent_capture = persistent_permission; + capture_show_cursor = show_cursor; + if (int(display) == fail_display) return false; + capture_parameters = {width, height, int(display), fps}; + frame_callback = std::move(frame); + state_callback = state; + cursor_mode_ = show_cursor ? CaptureCursorMode::embedded : captured_cursor_mode; + state(OH_SCREEN_CAPTURE_STATE_STARTED); + return true; +} +void Capture::Stop() { ++stops; frame_callback = {}; } +bool Input::Configure(int, int, int display) { if (!Release()) return false; input_display = display; return true; } +bool Input::Release() { ++releases; return release_ok; } +bool Input::Apply(const RemoteAction&) { return true; } +bool Input::LockWorkstation() { return true; } +} +extern "C" { +int OH_LOG_Print(LogType, LogLevel, unsigned int, const char*, const char*, ...) { return 0; } +Input_Result OH_Input_QueryAuthorizedStatus(Input_InjectionStatus* result) { *result = authorization; return INPUT_SUCCESS; } +Input_Result OH_Input_RequestInjection(Input_InjectAuthorizeCallback callback) { + ++injection_requests; + authorization = AUTHORIZED; callback(authorization); return INPUT_SUCCESS; +} +void OH_Input_CancelInjection() { authorization = UNAUTHORIZED; } +Input_CursorInfo* OH_Input_CursorInfo_Create() { return new Input_CursorInfo; } +void OH_Input_CursorInfo_Destroy(Input_CursorInfo** info) { delete *info; *info = nullptr; } +Input_Result OH_Input_GetCursorInfo(Input_CursorInfo*, OH_PixelmapNative** pixels) { + assert(pixels == nullptr); return cursor_result; +} +Input_Result OH_Input_CursorInfo_IsVisible(Input_CursorInfo*, bool* visible) { + *visible = pointer_visible; return INPUT_SUCCESS; +} +Input_Result OH_Input_CursorInfo_GetStyle(Input_CursorInfo*, Input_PointerStyle* style) { + assert(pointer_visible); *style = pointer_style; return style_result; +} +Input_Result OH_Input_GetPointerLocation(int32_t* display, double* x, double* y) { + *display = pointer_display; *x = pointer_x; *y = pointer_y; return location_result; +} +Input_Result OH_Input_AddMouseEventMonitor(Input_MouseEventCallback callback) { + ++monitor_adds; + assert(!mouse_monitor); + if (monitor_add_result == INPUT_SUCCESS) { + mouse_monitor = callback; + if (initial_monitor_event) callback(initial_monitor_event); + } + return monitor_add_result; +} +Input_Result OH_Input_RemoveMouseEventMonitor(Input_MouseEventCallback callback) { + ++monitor_removes; + assert(mouse_monitor == callback || (!mouse_monitor && monitor_remove_result == INPUT_PARAMETER_ERROR)); + if (monitor_remove_result == INPUT_SUCCESS || monitor_remove_result == INPUT_PARAMETER_ERROR) mouse_monitor = nullptr; + return monitor_remove_result; +} +int32_t OH_Input_GetMouseEventAction(const Input_MouseEvent* event) { return event->action; } +int32_t OH_Input_GetMouseEventDisplayId(const Input_MouseEvent* event) { return event->display; } +int32_t OH_Input_GetMouseEventDisplayX(const Input_MouseEvent* event) { return event->x; } +int32_t OH_Input_GetMouseEventDisplayY(const Input_MouseEvent* event) { return event->y; } +} + +int main() { + using namespace crossdesk; + const auto primary = Json::parse(R"({"id":0,"primary":true,"width":1920,"height":1080})"); + const auto secondary = Json::parse(R"({"id":42,"x":-2560,"width":2560,"height":1440})"); + environment["displays"] = Json::array({secondary, primary}); + ScreenCapturerFactory factory; + std::unique_ptr screen(factory.Create()); + std::string stream; + screen->Init(30, [&](unsigned char*, int, int, int, const char* id, const MiniRtcNativeVideoFrame*) { stream = id; }); + assert(screen->GetDisplayInfoList().size() == 2); + assert(screen->Start(false) == 0); + assert((capture_parameters == std::tuple{1920, 1080, 0, 30})); + MiniRtcVideoFrame frame{}; frame_callback(frame); assert(stream == "Display1"); + CursorStateProvider cursor; + CursorState cursor_state{}; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(!capture_show_cursor && cursor_state.render_mode == CursorRenderMode::separate); + assert(cursor_state.visible && cursor_state.shape == RemoteCursorShape::default_cursor); + assert(cursor_state.position_update && cursor_state.position_valid && cursor_state.display_id == 0); + assert(cursor_state.x == 0.5f && cursor_state.y == 0.25f); + const std::pair shapes[] = { + {TEXT_CURSOR, RemoteCursorShape::text}, {HORIZONTAL_TEXT_CURSOR, RemoteCursorShape::text}, + {HAND_POINTING, RemoteCursorShape::pointer}, {HAND_OPEN, RemoteCursorShape::grab}, + {HAND_GRABBING, RemoteCursorShape::grabbing}, {CURSOR_FORBID, RemoteCursorShape::not_allowed}, + {NORTH_WEST_SOUTH_EAST, RemoteCursorShape::nwse_resize}, {NORTH_EAST_SOUTH_WEST, RemoteCursorShape::nesw_resize}, + {WEST_EAST, RemoteCursorShape::ew_resize}, {NORTH_SOUTH, RemoteCursorShape::ns_resize}, + {RESIZE_LEFT_RIGHT, RemoteCursorShape::col_resize}, {RESIZE_UP_DOWN, RemoteCursorShape::row_resize}, + {NORTH, RemoteCursorShape::n_resize}, {SOUTH, RemoteCursorShape::s_resize}, + {EAST, RemoteCursorShape::e_resize}, {WEST, RemoteCursorShape::w_resize}, + {NORTH_EAST, RemoteCursorShape::ne_resize}, {NORTH_WEST, RemoteCursorShape::nw_resize}, + {SOUTH_EAST, RemoteCursorShape::se_resize}, {SOUTH_WEST, RemoteCursorShape::sw_resize}, + {LOADING, RemoteCursorShape::wait}, {RUNNING, RemoteCursorShape::progress}, + {HELP, RemoteCursorShape::help}, {MOVE, RemoteCursorShape::move}, + {CROSS, RemoteCursorShape::crosshair}, {CURSOR_COPY, RemoteCursorShape::copy}, + {DEVELOPER_DEFINED_ICON, RemoteCursorShape::default_cursor}, + {static_cast(999), RemoteCursorShape::default_cursor} + }; + for (const auto& [native, remote] : shapes) { + pointer_style = native; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.shape == remote); + RemoteAction outgoing{}, incoming{}; + outgoing.type = ControlType::cursor_state; outgoing.cs = cursor_state; + assert(incoming.from_json(outgoing.to_json())); + assert(incoming.type == ControlType::cursor_state && incoming.cs.position_valid); + assert(incoming.cs.display_id == 0 && incoming.cs.x == 0.5f && incoming.cs.y == 0.25f); + // Mac and Windows both use this presentation policy for the Harmony host. + CursorPresentation presentation; + presentation.Observe(incoming.cs, 100); + assert(presentation.Resolve(101, HostPlatform::HarmonyOS) == remote); + } + pointer_visible = false; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(!cursor_state.visible && cursor_state.shape == RemoteCursorShape::none); + assert(cursor_state.render_mode == CursorRenderMode::hidden && cursor_state.hidden_reason == CursorHiddenReason::system_hidden); + location_result = INPUT_DEVICE_NO_POINTER; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(!cursor_state.position_valid && cursor_state.hidden_reason == CursorHiddenReason::no_pointing_device); + pointer_visible = true; pointer_style = TEXT_CURSOR; + location_result = INPUT_APP_NOT_FOCUSED; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.visible && cursor_state.shape == RemoteCursorShape::text && !cursor_state.position_valid); + cursor_result = INPUT_SERVICE_EXCEPTION; + assert(!cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + cursor_result = INPUT_SUCCESS; style_result = INPUT_SERVICE_EXCEPTION; + assert(!cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + style_result = INPUT_SUCCESS; location_result = INPUT_SUCCESS; + pointer_display = 42; pointer_x = 1280; pointer_y = 720; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.position_valid && cursor_state.display_id == 1 && cursor_state.x == 0.5f && cursor_state.y == 0.5f); + auto overlapping = screen->GetDisplayInfoList(); + overlapping[1].left = overlapping[1].top = 0; + overlapping[1].right = 2560; overlapping[1].bottom = 1440; + assert(cursor.Sample(overlapping, 0, &cursor_state) && cursor_state.display_id == 1); + for (double invalid : {-1., 2560., std::numeric_limits::infinity(), std::numeric_limits::quiet_NaN()}) { + pointer_x = invalid; + assert(cursor.Sample(overlapping, 0, &cursor_state) && !cursor_state.position_valid); + } + pointer_x = 1280; pointer_display = 99; + assert(cursor.Sample(overlapping, 0, &cursor_state) && !cursor_state.position_valid); + pointer_display = 42; + overlapping[1].width = 0; + assert(cursor.Sample(overlapping, 0, &cursor_state) && !cursor_state.position_valid); + assert(screen->SwitchTo(1) == 0 && screen->GetCurrentMonitorIndex() == 1); + assert(!capture_show_cursor); // Separate cursor policy survives a physical display switch. + assert((capture_parameters == std::tuple{2560, 1440, 42, 30})); + frame_callback(frame); assert(stream == "Display2"); + + DeviceControllerFactory input_factory; + std::unique_ptr device(input_factory.Create(DeviceControllerFactory::Mouse)); + auto* mouse = static_cast(device.get()); + mouse->Init(screen->GetDisplayInfoList()); + std::unique_ptr keyboard_device(input_factory.Create(DeviceControllerFactory::Keyboard)); + auto* keyboard = static_cast(keyboard_device.get()); + const int before_unhook = releases; + assert(keyboard->Unhook() == 0 && releases == before_unhook); + harmony::desktop::SetKeyboardFocus(2, true); + harmony::desktop::SetKeyboardHookPermission(false); + hook_add_result = INPUT_PERMISSION_DENIED; + assert(keyboard->Hook([](int, bool, uint32_t, bool, void*) {}, nullptr) == INPUT_PERMISSION_DENIED); + assert(!keyboard->ShouldRetryHook()); // Do not interrupt a held fallback chord every second. + harmony::desktop::SetKeyboardHookPermission(true); + assert(keyboard->ShouldRetryHook()); + hook_add_result = INPUT_SUCCESS; + RemoteAction action{}; action.type = ControlType::mouse; + release_ok = false; + assert(mouse->SendMouseCommand(action, 1) != 0 && input_display == 0); + release_ok = true; + const int before_input_switch = releases; + assert(mouse->SendMouseCommand(action, 1) == 0 && input_display == 42); + assert(releases == before_input_switch + 1); + assert(mouse->SendMouseCommand(action, 1) == 0 && releases == before_input_switch + 1); + assert(mouse->SendMouseCommand(action, -1) != 0 && mouse->SendMouseCommand(action, 2) != 0); + state_callback(OH_SCREEN_CAPTURE_STATE_MIC_MUTED_BY_USER); + assert(mouse->SendMouseCommand(action, 1) == 0); + state_callback(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + assert(mouse->SendMouseCommand(action, 1) != 0); + assert(!cursor.Sample(screen->GetDisplayInfoList(), 1, &cursor_state)); + state_callback(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + assert(mouse->SendMouseCommand(action, 1) == 0); + authorization = UNAUTHORIZED; + assert(mouse->SendMouseCommand(action, 1) != 0); + authorization = AUTHORIZED; + assert(SendHarmonySystemShortcut(ServiceCommandFlag::lock_workstation)); + assert(!SendHarmonySystemShortcut(ServiceCommandFlag::send_sas)); + + fail_display = 0; + assert(screen->SwitchTo(0) != 0 && screen->GetCurrentMonitorIndex() == 1); + frame_callback(frame); assert(stream == "Display2"); // Failed switch rolls back. + fail_display = -1; + environment["displays"] = Json::array({secondary}); + const int before_removal = starts; + auto displays = screen->GetDisplayInfoList(); + assert(displays.size() == 2 && displays[0].width == 0 && displays[1].width == 2560); + assert(screen->GetCurrentMonitorIndex() == 1 && starts == before_removal); + mouse->UpdateDisplayInfoList(displays); + assert(mouse->SendMouseCommand(action, 0) != 0); + assert(screen->SwitchTo(0) != 0); + + environment["displays"] = Json::array({primary}); + screen->GetDisplayInfoList(); + assert(screen->GetCurrentMonitorIndex() == 0); + frame_callback(frame); assert(stream == "Display1"); + environment["displays"].push_back(Json::parse(R"({"id":91,"width":1280,"height":720})")); + assert(screen->GetDisplayInfoList().size() == 2); // Unnegotiated streams must not be advertised as usable. + environment["displays"] = Json::array(); + screen->GetDisplayInfoList(); + assert(screen->GetCurrentMonitorIndex() == -1 && !frame_callback); + const int before_repeat = stops; + screen->GetDisplayInfoList(); assert(stops == before_repeat); + environment["displays"] = Json::array({secondary}); + screen->GetDisplayInfoList(); + assert(screen->GetCurrentMonitorIndex() == 1 && frame_callback); + frame_callback(frame); assert(stream == "Display2"); + assert(screen->Pause(1) == 0 && !frame_callback); + assert(screen->Resume(1) == 0 && frame_callback); + assert(screen->Stop() == 0); + assert(mouse->SendMouseCommand(action, 1) != 0); // Authorization alone is insufficient after capture ends. + assert(!SendHarmonySystemShortcut(ServiceCommandFlag::lock_workstation)); + + // A persistent grant is independent of dialog authorization and survives + // disconnect/reconnect. Capture must still be active before accepting input. + harmony::desktop::SetHostPermissions(true, true); + const int before_persistent = injection_requests; + assert(authorization == UNAUTHORIZED); + assert(screen->Resume(1) == 0 && persistent_capture); + assert(injection_requests == before_persistent && authorization == UNAUTHORIZED); + mouse->UpdateDisplayInfoList(screen->GetDisplayInfoList()); + assert(mouse->SendMouseCommand(action, 1) == 0); + assert(keyboard->SendKeyboardCommand(65, true) == 0); + assert(SendHarmonySystemShortcut(ServiceCommandFlag::lock_workstation)); + state_callback(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + assert(mouse->SendMouseCommand(action, 1) != 0); + assert(keyboard->SendKeyboardCommand(65, false) != 0); + state_callback(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + assert(injection_requests == before_persistent); + assert(screen->Stop() == 0 && mouse->SendMouseCommand(action, 1) != 0); + assert(screen->Resume(1) == 0 && persistent_capture && injection_requests == before_persistent); + const int before_revoke = releases; + harmony::desktop::SetHostPermissions(false, false); + assert(releases == before_revoke + 1); + assert(mouse->SendMouseCommand(action, 1) != 0); + assert(keyboard->SendKeyboardCommand(65, true) != 0); + assert(!SendHarmonySystemShortcut(ServiceCommandFlag::lock_workstation)); + assert(injection_requests == before_persistent); // Revocation never triggers a prompt loop. + assert(screen->Stop() == 0 && screen->Resume(1) == 0); + assert(!persistent_capture && injection_requests == before_persistent + 1); + assert(mouse->SendMouseCommand(action, 1) == 0); // Ordinary consent remains usable. + screen->Stop(); + assert(!cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(screen->Start(true) == 0 && capture_show_cursor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.render_mode == CursorRenderMode::embedded && !cursor_state.visible); + assert(cursor_state.shape == RemoteCursorShape::none && !cursor_state.position_valid); + CursorPresentation embedded; + embedded.Observe(cursor_state, 100); + assert(embedded.Resolve(101, HostPlatform::HarmonyOS) == RemoteCursorShape::none); + assert(screen->Start(false) == 0 && !capture_show_cursor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.render_mode == CursorRenderMode::separate); + screen->Stop(); + captured_cursor_mode = harmony::CaptureCursorMode::embedded; // Recorder refused to remove its cursor. + assert(screen->Start(false) == 0); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.render_mode == CursorRenderMode::embedded); + screen->Stop(); + captured_cursor_mode = harmony::CaptureCursorMode::unknown; + assert(screen->Start(false) == 0); + assert(!cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + screen->Stop(); + + // INPUT_MONITORING supplies screen-relative positions while the direct + // query is rejected in the background. No INPUT_DEVICE_CONFIGURATOR grant. + environment["displays"] = Json::array({primary, secondary}); + captured_cursor_mode = harmony::CaptureCursorMode::separate; + location_result = INPUT_APP_NOT_FOCUSED; + monitor_add_result = INPUT_SUCCESS; + assert(screen->Start(false) == 0 && mouse_monitor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 1, &cursor_state) && !cursor_state.position_valid); + Input_MouseEvent motion{MOUSE_ACTION_MOVE, 42, 640, 720}; + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.position_valid && cursor_state.display_id == 1); + assert(cursor_state.x == 0.25f && cursor_state.y == 0.5f && cursor_state.shape == RemoteCursorShape::text); + RemoteAction monitored_out{}, monitored_in{}; + monitored_out.type = ControlType::cursor_state; monitored_out.cs = cursor_state; + assert(monitored_in.from_json(monitored_out.to_json())); + assert(monitored_in.cs.position_valid && monitored_in.cs.display_id == 1 && monitored_in.cs.x == 0.25f); + const int before_notification = monitor_adds; + state_callback(OH_SCREEN_CAPTURE_STATE_MIC_MUTED_BY_USER); + assert(monitor_adds == before_notification); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && cursor_state.position_valid); + + const auto queued_monitor = mouse_monitor; + state_callback(OH_SCREEN_CAPTURE_STATE_ENTER_PRIVATE_SCENE); + assert(!mouse_monitor && !cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + queued_monitor(&motion); // Late events cannot repopulate a stopped session. + state_callback(OH_SCREEN_CAPTURE_STATE_EXIT_PRIVATE_SCENE); + assert(mouse_monitor && cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(!cursor_state.position_valid); + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && cursor_state.position_valid); + motion.action = MOUSE_ACTION_CANCEL; + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + motion.action = MOUSE_ACTION_MOVE; motion.x = -1; + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + motion.x = 640; + mouse_monitor(&motion); + assert(screen->SwitchTo(0) == 0 && mouse_monitor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + motion.display = 0; motion.x = 480; motion.y = 540; + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 1, &cursor_state)); + assert(cursor_state.position_valid && cursor_state.display_id == 0 && cursor_state.x == 0.25f && cursor_state.y == 0.5f); + + location_result = INPUT_DEVICE_NO_POINTER; pointer_visible = false; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(!cursor_state.position_valid && cursor_state.hidden_reason == CursorHiddenReason::no_pointing_device); + location_result = INPUT_APP_NOT_FOCUSED; pointer_visible = true; + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + + monitor_remove_result = INPUT_SERVICE_EXCEPTION; + screen->Stop(); + assert(mouse_monitor); + mouse_monitor(&motion); // Failed removal stays inert. + const int before_failed_restart = monitor_adds; + assert(screen->Start(false) == 0); + assert(monitor_adds == before_failed_restart); // Never install a duplicate callback. + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + monitor_remove_result = INPUT_SUCCESS; + screen->Stop(); + assert(!mouse_monitor); + assert(screen->Start(false) == 0 && mouse_monitor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && !cursor_state.position_valid); + mouse_monitor(&motion); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && cursor_state.position_valid); + // Capture can remove the monitor itself; "not registered" is clean removal. + mouse_monitor = nullptr; + monitor_remove_result = INPUT_PARAMETER_ERROR; + state_callback(OH_SCREEN_CAPTURE_STATE_PAUSED_BY_USER); + monitor_remove_result = INPUT_SUCCESS; + initial_monitor_event = &motion; + state_callback(OH_SCREEN_CAPTURE_STATE_RESUMED_BY_USER); + initial_monitor_event = nullptr; + assert(mouse_monitor); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state) && cursor_state.position_valid); + screen->Stop(); + assert(!mouse_monitor); + monitor_add_result = INPUT_PERMISSION_DENIED; + assert(screen->Start(false) == 0 && !mouse_monitor); + const int before_denied_notification = monitor_adds; + state_callback(OH_SCREEN_CAPTURE_STATE_MIC_MUTED_BY_USER); + assert(monitor_adds == before_denied_notification); + assert(cursor.Sample(screen->GetDisplayInfoList(), 0, &cursor_state)); + assert(cursor_state.visible && cursor_state.shape == RemoteCursorShape::text && !cursor_state.position_valid); + screen->Stop(); + const int before_embedded = monitor_adds; + assert(screen->Start(true) == 0 && !mouse_monitor && monitor_adds == before_embedded); + screen->Stop(); +} diff --git a/apps/harmony/tests/desktop_keyboard_test.cpp b/apps/harmony/tests/desktop_keyboard_test.cpp new file mode 100644 index 00000000..a0e2aa20 --- /dev/null +++ b/apps/harmony/tests/desktop_keyboard_test.cpp @@ -0,0 +1,64 @@ +#include "keyboard.h" +#include +#include +using crossdesk::harmony::desktop::KeyboardState; +int main() { + KeyboardState keys; + assert(keys.Translate(2017, true) == "a"); + assert(keys.Translate(2017, false) == "a"); + keys.Translate(2047, true); + assert(keys.Translate(2017, true) == "A"); + assert(keys.Translate(2017, true) == "A"); // repeat retains the original symbol + keys.Translate(2047, false); + assert(keys.Translate(2017, false) == "A"); // Shift was released first + keys.Translate(2047, true); keys.Translate(2048, true); keys.Translate(2047, false); + assert(keys.Translate(2001, true) == "!"); // the other Shift is still pressed + keys.Translate(2001, false); keys.Translate(2048, false); + assert(keys.Translate(2057, true) == "-"); keys.Translate(2057, false); + keys.Translate(2074, true); keys.Translate(2074, false); + assert(keys.Translate(2042, true) == "Z"); keys.Translate(2042, false); + keys.Translate(2047, true); + assert(keys.Translate(2042, true) == "z"); keys.Translate(2042, false); + keys.Translate(2072, true); keys.Reset(); + assert(keys.Pressed().empty()); + for (int code=2090; code<=2101; ++code) { + auto press=keys.Translate(code,true); assert(!press.empty()); assert(keys.Translate(code,false)==press); + } + assert(keys.Translate(2117, true) == "."); + assert(keys.Translate(99999, true).empty()); + keys.Reset(); + for (auto pair : {std::pair{2047, 2048}, std::pair{2072, 2073}, std::pair{2076, 2077}}) { + const auto left = keys.Translate(pair.first, true); + const auto right = keys.Translate(pair.second, true); + assert(!left.empty() && !right.empty() && left != right); + assert(keys.Translate(pair.first, false) == left && keys.IsPressed(pair.second)); + assert(keys.Translate(pair.second, false) == right); + } + for (int code = 2816; code <= 2827; ++code) { + const auto press = keys.Translate(code, true); + assert(!press.empty() && keys.Translate(code, false) == press); + } + assert(!keys.Translate(2067, true).empty()); + keys.Reset(); + // The lock state changed while another application had keyboard focus. + assert(keys.Translate(2017, true, 0, true) == "A"); + assert(keys.Translate(2017, false, 0, false) == "A"); + assert(keys.Translate(2017, true, 0, false) == "a"); + keys.Reset(); + // Prefer the OS layout result to the US fallback, preserving paired releases. + assert(keys.Translate(2042, true, 'y') == "y"); + assert(keys.Translate(2042, true, 'Y') == "y"); + assert(keys.Translate(2042, false, 'Y') == "y"); + assert(keys.Translate(2057, true, '?') == "?"); keys.Reset(); + using namespace slint::platform::key_codes; + const auto utf8 = [](std::u8string_view key) { return std::string(reinterpret_cast(key.data()), key.size()); }; + assert(keys.Translate(2104, true, '1', {}, false) == utf8(End)); + assert(keys.Translate(2104, false, '1', {}, true) == utf8(End)); + assert(keys.Translate(2104, true, '1', {}, true) == "1"); keys.Reset(); + assert(keys.Translate(2117, true, '.', {}, false) == utf8(Delete)); keys.Reset(); + assert(keys.Translate(2045, true) == utf8(Alt)); + assert(keys.Translate(2046, true) == utf8(AltGr)); + assert(keys.Translate(2045, false) == utf8(Alt) && keys.IsPressed(2046)); + assert(keys.Translate(2046, false) == utf8(AltGr)); + std::cout << "Slint keyboard chords, repeat, focus release, punctuation and function keys passed\n"; +} diff --git a/apps/harmony/tests/desktop_platform_test.cpp b/apps/harmony/tests/desktop_platform_test.cpp new file mode 100644 index 00000000..6200254a --- /dev/null +++ b/apps/harmony/tests/desktop_platform_test.cpp @@ -0,0 +1,69 @@ +#include "displays.h" +#include "pointer.h" +#include "window_mode.h" +#include +#include + +int main() { + using namespace crossdesk::harmony::desktop; + using Json = nlohmann::json; + const auto displays = ReadDisplays(Json::parse(R"({"displays":[ + {"id":42,"name":"Left","x":-2560,"y":-300,"width":2560,"height":1440}, + {"id":0,"name":"Main","primary":true,"width":1920,"height":1080,"usableY":30,"usableHeight":990}, + {"id":42,"width":2560,"height":1440}, + {"id":7,"width":0,"height":1080}, + {"id":-1,"width":1920,"height":1080} + ]})")); + assert(displays.size() == 2 && displays[0].id == 0 && displays[1].id == 42); + assert(displays[0].SdlId() == 1 && displays[1].SdlId() == 43); + assert(displays[0].usable_y == 30 && displays[0].usable_height == 990); + assert(displays[1].usable_x == -2560 && displays[1].usable_y == -300); + assert(DisplayAt(displays, -1, 0)->id == 42); + assert(DisplayAt(displays, 0, 0)->id == 0); + assert(DisplayAt(displays, -1280, -800)->id == 42); // Outside monitors: nearest display. + assert(DisplayAt(displays, 3000, 100)->id == 0); + const auto removed = ReadDisplays(Json::parse(R"({"displays":[{"id":42,"primary":true,"width":2560,"height":1440}]})")); + assert(removed[0].SdlId() == displays[1].SdlId()); // SDL identity survives primary changes. + assert(ReadDisplays(Json::object()).empty()); + assert(DisplayAt({}, 0, 0) == nullptr); + + PointerState pointer; + std::vector> releases; + const auto release = [&](int b, float x, float y) { releases.emplace_back(b, x, y); }; + pointer.Update(0, 0, 1, 2); + pointer.Update(0, 1, 10, 20); + pointer.Update(0, 2, 10, 20); + assert(pointer.Buttons() == 7); + pointer.Update(1, 1, 12, 22); // Right release does not clear left/middle drag. + assert(pointer.Buttons() == 3); + pointer.Update(2, 0, -15, 130); // Drag beyond the surface preserves held state. + pointer.Cancel(release); + assert(pointer.Buttons() == 0 && releases.size() == 2); + assert((releases[0] == std::tuple{0, -15.f, 130.f})); + assert((releases[1] == std::tuple{2, -15.f, 130.f})); + pointer.Cancel(release); + assert(releases.size() == 2); // Repeated focus/detach/cancel has no duplicate releases. + + WindowMode modes; + Json slint{{"fullscreen", false}, {"maximized", false}, {"minimized", false}}; + assert(modes.Request(slint) == slint); + assert(modes.Request(slint).empty()); + // Native titlebar maximize/restore: all three Slint setters synchronously + // call back, including the first one before the other flags have changed. + for (bool maximized : {true, false, true, false}) { + Json actual = slint; actual["maximized"] = maximized; + modes.Acknowledge(actual, [&] { + for (auto& [name, value] : actual.items()) { + slint[name] = value; + assert(modes.Request(slint).empty()); + } + }); + assert(modes.Request(slint).empty()); + } + slint["fullscreen"] = true; + assert(modes.Request(slint) == Json({{"fullscreen", true}})); + modes.Acknowledge(Json{{"minimized", true}}, [&] { slint["minimized"] = true; }); + assert(modes.Request(slint).empty()); // Minimize preserves fullscreen. + slint["minimized"] = false; + assert(modes.Request(slint) == Json({{"minimized", false}})); +} diff --git a/apps/harmony/tests/desktop_window_host_test.cjs b/apps/harmony/tests/desktop_window_host_test.cjs new file mode 100644 index 00000000..21e2286c --- /dev/null +++ b/apps/harmony/tests/desktop_window_host_test.cjs @@ -0,0 +1,144 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const vm = require('node:vm'); +const ts = require(process.argv[2]); + +// Exercise the production bridge with asynchronous native windows, including +// callbacks that outlive a destroyed XComponent/window with the same Slint id. +const states = { FLOATING: 1, MAXIMIZE: 2, FULL_SCREEN: 3, MINIMIZE: 4 }; +const modes = { FOLLOW_APP_IMMERSIVE_SETTING: 0, EXIT_IMMERSIVE: 1, ENTER_IMMERSIVE: 2 }; +const events = { WINDOW_ACTIVE: 1, WINDOW_INACTIVE: 2 }; +const sent = [], released = [], moduleExports = {}; +let commands = [], nextNativeId = 10; +class NativeWindow { + constructor(status = states.FLOATING) { + this.id = ++nextNativeId; this.status = status; this.previous = states.FLOATING; + this.handlers = new Map(); this.calls = []; this.decor = true; + } + getWindowStatus() { return this.status; } + getUIContext() { return {}; } + getWindowProperties() { return { id: this.id, windowRect: { left: 20, top: 30 } }; } + getWindowDecorVisible() { return this.decor; } + getWindowDecorHeight() { return 32; } + setWindowDecorVisible(value) { this.decor = value; } + async setWindowTitle() { if (this.rejectTitle) throw Error('unsupported'); } + async setSubWindowZLevel() { if (this.rejectStack) throw Error('unsupported'); } + async setWindowLimits(limits) { this.limits = limits; this.calls.push('limits'); } + async loadContent() { this.calls.push('load'); } + async resize(w, h) { this.calls.push(['resize', w, h]); } + async destroyWindow() { this.calls.push('destroy'); } + async minimize() { this.calls.push('minimize'); this.previous = this.status; this.status = states.MINIMIZE; } + async restore() { this.calls.push('restore'); this.status = this.previous; } + async showWindow() { + this.calls.push('show'); + // showWindow does not restore a minimized 2in1 main window. + if (this !== main && this.status === states.MINIMIZE) this.status = this.previous; + } + async recover() { this.calls.push('recover'); this.status = states.FLOATING; } + async maximize(mode) { + this.calls.push(['maximize', mode]); + this.status = mode === modes.EXIT_IMMERSIVE ? states.MAXIMIZE : states.FULL_SCREEN; + } + on(name, callback) { if (name === this.unsupportedEvent) throw Error('unsupported'); this.handlers.set(name, callback); } + off(name) { this.handlers.delete(name); } + async createSubWindowWithOptions() { return new NativeWindow(); } +} +const main = new NativeWindow(); +vm.runInNewContext(ts.transpileModule(fs.readFileSync(process.argv[3], 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } +}).outputText, { + exports: moduleExports, console: { warn: () => {}, error: e => { throw Error(e); } }, + LocalStorage: class { setOrCreate() {} }, + require: name => { + if (name === 'libslint_desktop.so') return { default: { + send: text => sent.push(JSON.parse(text)), poll: () => JSON.stringify(commands.splice(0)) + } }; + if (name === '@kit.ArkUI') return { window: { WindowStatusType: states, MaximizePresentation: modes, WindowEventType: events } }; + if (name === './DesktopAccess') return { DesktopAccess: { refresh() {}, requestHost() {} } }; + if (name === './TextInputHost') return { TextInputHost: { release: id => released.push(id) } }; + if (name === './DisplayHost') return { DisplayHost: { + screen: () => ({ densityPixels: 2 }), global: () => ({ x: 20, y: 30 }), + move: async (target, x, y) => target.calls.push(['move', x, y]) + } }; + return {}; + } +}); +const host = moduleExports.DesktopHost; +host.stage = { getMainWindowSync: () => main }; +host.windows.set(1, main); +const props = (id, changes = {}) => ({ op: 'properties', id, stateRevision: host.stateRevisions.get(id) ?? 0, + fullscreen: false, maximized: false, minimized: false, ...changes }); + +(async () => { + main.status = states.FULL_SCREEN; + await host.windowCommand(props(1, { maximized: true })); + assert.equal(main.status, states.MAXIMIZE, 'leaving fullscreen must explicitly exit immersive mode'); + await host.windowCommand(props(1, { fullscreen: true, minimized: true })); + assert.equal(main.status, states.MINIMIZE, 'minimized takes precedence over fullscreen'); + await host.windowCommand(props(1)); + assert.equal(main.status, states.FLOATING, 'unminimize must finish the requested mode transition'); + assert(main.calls.includes('restore')); assert(main.calls.includes('recover')); + + await host.windowCommand({ op: 'visible', id: 1, visible: true, width: 900, height: 600 }); + main.calls = []; main.status = states.MAXIMIZE; + await host.windowCommand({ op: 'visible', id: 1, visible: true, width: 3000, height: 1800 }); + assert.deepEqual(main.calls, ['show'], 'showing a visible maximized main window must not reset its restore size'); + main.rejectTitle = true; + await host.windowCommand(props(1, { title: 'new title', fullscreen: true })); + assert.equal(main.status, states.FULL_SCREEN, 'optional chrome failure cannot abort fullscreen'); + + await host.windowCommand(props(2, { topmost: true, noFrame: true, title: 'Shared panel' })); + await host.windowCommand({ op: 'visible', id: 2, visible: true, width: 400, height: 250, x: -100, y: 40 }); + const old = host.windows.get(2); + assert(old.calls.some(c => Array.isArray(c) && c[0] === 'move' && c[1] === -100)); + assert(!old.decor); + const late = new Map(old.handlers); + old.handlers.get('windowStatusDidChange')(states.MINIMIZE); + assert.deepEqual(sent.at(-1), { op: 'state', id: 2, stateRevision: 1, minimized: true }); + await host.windowCommand({ op: 'visible', id: 2, visible: false }); + assert.equal(old.handlers.size, 0); assert(released.includes(2)); + await host.windowCommand({ op: 'visible', id: 2, visible: true, width: 400, height: 250 }); + assert.notEqual(host.windows.get(2), old); + const before = sent.length; + late.get('windowEvent')(events.WINDOW_INACTIVE); + late.get('windowRectChange')(); late.get('displayIdChange')(); + late.get('windowStatusDidChange')(states.FULL_SCREEN); + assert.equal(await late.get('windowWillClose')(), false); + assert.equal(sent.length, before, 'destroyed native windows cannot update their replacement'); + + const child = host.windows.get(2); + child.rejectStack = true; + await host.windowCommand(props(2, { topmost: true, maximized: true })); + assert.equal(child.status, states.MAXIMIZE); + child.status = states.FLOATING; + child.handlers.get('windowStatusDidChange')(states.FLOATING); + await host.windowCommand(props(2, { maximized: true, stateRevision: 0 })); + assert.equal(child.status, states.FLOATING, 'an old mode command cannot undo a native titlebar restore'); + let finishLimits; + child.setWindowLimits = async () => { await new Promise(resolve => { finishLimits = resolve; }); }; + const pendingMode = host.windowCommand(props(2, { maximized: true })); + await new Promise(resolve => setImmediate(resolve)); + child.handlers.get('windowStatusDidChange')(states.FLOATING); + finishLimits(); await pendingMode; + assert.equal(child.status, states.FLOATING, 'a native transition during an awaited property operation also wins'); + child.setWindowLimits = async () => {}; + child.status = states.MAXIMIZE; + await host.windowCommand({ op: 'properties', id: 2, title: 'layout update' }); + assert.equal(child.status, states.MAXIMIZE, 'an unrelated property change must not recover the native window'); + child.unsupportedEvent = 'displayIdChange'; + host.observe(2, child); + assert(child.handlers.has('windowWillClose'), 'one unavailable observer must not disable close interception'); + commands = [{ op: 'destroy', id: 2 }, { op: 'destroy', id: 9 }]; + host.poll(); + await Promise.all([...host.queues.values()]); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(host.queues.size, 0); + assert.equal(host.properties.has(2), false); + assert.equal(host.windows.has(2), false); + const broken = new NativeWindow(); + broken.loadContent = async () => { throw Error('creation failed'); }; + main.createSubWindowWithOptions = async () => broken; + await assert.rejects(host.windowCommand({ op: 'visible', id: 3, visible: true }), /creation failed/); + assert.equal(host.windows.has(3), false); assert(broken.calls.includes('destroy')); + console.log('Slint window restore/fullscreen, optional services, recreation, stale callbacks and queue cleanup passed'); +})().catch(error => { console.error(error); process.exitCode = 1; }); diff --git a/apps/harmony/tests/input_test.cpp b/apps/harmony/tests/input_test.cpp new file mode 100644 index 00000000..b2ccf7c5 --- /dev/null +++ b/apps/harmony/tests/input_test.cpp @@ -0,0 +1,134 @@ +#include "input.h" +#include "key_mapping.h" +#include +#include +#include +#include +#include +#include + +// Execute the production input adapter against an in-memory Input Kit. These +// tests exercise releases and coordinate bounds without injecting real input. +struct Input_KeyEvent { int code = 0, action = 0; }; +struct Input_MouseEvent { int action = 0, button = -1, x = 0, y = 0, display = 0, axis = 0; double value = 0; }; +std::set pressed_keys; +std::set pressed_buttons; +std::vector pointer_events; +std::vector key_events; +bool allow = true; +int rejected_key = -1; +extern "C" { +int OH_LOG_Print(LogType, LogLevel, unsigned int, const char*, const char*, ...) { return 0; } +Input_KeyEvent* OH_Input_CreateKeyEvent() { return new Input_KeyEvent; } +void OH_Input_DestroyKeyEvent(Input_KeyEvent** e) { delete *e; *e = nullptr; } +void OH_Input_SetKeyEventKeyCode(Input_KeyEvent* e, int32_t code) { e->code = code; } +void OH_Input_SetKeyEventAction(Input_KeyEvent* e, int32_t action) { e->action = action; } +int32_t OH_Input_InjectKeyEvent(const Input_KeyEvent* e) { + if (!allow || e->code == rejected_key) return INPUT_PERMISSION_DENIED; + key_events.push_back(*e); + if (e->action == KEY_ACTION_DOWN) pressed_keys.insert(e->code); else pressed_keys.erase(e->code); + return INPUT_SUCCESS; +} +Input_MouseEvent* OH_Input_CreateMouseEvent() { return new Input_MouseEvent; } +void OH_Input_DestroyMouseEvent(Input_MouseEvent** e) { delete *e; *e = nullptr; } +void OH_Input_SetMouseEventAction(Input_MouseEvent* e, int32_t action) { e->action = action; } +void OH_Input_SetMouseEventButton(Input_MouseEvent* e, int32_t button) { e->button = button; } +void OH_Input_SetMouseEventDisplayId(Input_MouseEvent* e, int32_t display) { e->display = display; } +void OH_Input_SetMouseEventDisplayX(Input_MouseEvent* e, int32_t x) { e->x = x; } +void OH_Input_SetMouseEventDisplayY(Input_MouseEvent* e, int32_t y) { e->y = y; } +void OH_Input_SetMouseEventAxisType(Input_MouseEvent* e, int32_t axis) { e->axis = axis; } +void OH_Input_SetMouseEventAxisValue(Input_MouseEvent* e, float value) { e->value = value; } +int32_t OH_Input_InjectMouseEvent(const Input_MouseEvent* e) { + if (!allow) return INPUT_PERMISSION_DENIED; + pointer_events.push_back(*e); + if (e->action == MOUSE_ACTION_BUTTON_DOWN) pressed_buttons.insert(e->button); + if (e->action == MOUSE_ACTION_BUTTON_UP) pressed_buttons.erase(e->button); + return INPUT_SUCCESS; +} +} + +int main() { + using namespace crossdesk; + using namespace crossdesk::harmony; + for (auto [native, wire] : kSpecialKeys) { + assert(ToVirtualKey(native) == wire); assert(FromVirtualKey(wire) == native); + } + assert(ToVirtualKey(2017) == 'A' && FromVirtualKey('Z') == 2042); + assert(ToVirtualKey(2068) == 0x21 && ToVirtualKey(2069) == 0x22); + assert(ToVirtualKey(0) == 0 && FromVirtualKey(0xFFFF) == 0); + Input input; + input.Configure(1920, 1080, 7); + RemoteAction mouse_action{}; mouse_action.type = mouse; + mouse_action.m = {-2, 3, 0, left_down}; + assert(input.Apply(mouse_action)); + assert(pointer_events.back().x == 0 && pointer_events.back().y == 1079); + assert(pointer_events.back().display == 7 && pressed_buttons.count(MOUSE_BUTTON_LEFT)); + RemoteAction key{}; key.type = keyboard; key.k = {0xA2, 0, false, key_down}; + assert(input.Apply(key) && pressed_keys.count(2072)); + RemoteAction state{}; state.type = keyboard_state; state.ks.seq = 10; + state.ks.pressed_count = 1; state.ks.pressed_keys[0] = {0x41, 0, false}; + assert(input.Apply(state) && !pressed_keys.count(2072) && pressed_keys.count(2017)); + state.ks.seq = 9; state.ks.pressed_count = 0; + assert(input.Apply(state) && pressed_keys.count(2017)); // Ignore reordered snapshots. + input.Release(); + assert(pressed_keys.empty() && pressed_buttons.empty()); + input.Release(); // Repeated teardown is safe. + auto count = pointer_events.size(); + mouse_action.m.x = NAN; + assert(!input.Apply(mouse_action) && pointer_events.size() == count); + mouse_action.m = {.5f, .5f, 9000, wheel_vertical}; + assert(input.Apply(mouse_action)); + assert(pointer_events.size() == count + 3 && pointer_events.back().action == MOUSE_ACTION_AXIS_END); + assert(pointer_events[count].value == 0 && pointer_events[count + 1].value == 1200 && pointer_events[count + 2].value == 0); + mouse_action.m = {.25f, .75f, 0, left_down}; + assert(input.Apply(mouse_action)); + assert(input.Apply(key)); + input.Configure(2560, 1440, 42); // Switching screens releases the old screen first. + assert(pressed_keys.empty() && pressed_buttons.empty()); + assert(pointer_events.back().display == 7 && pointer_events.back().action == MOUSE_ACTION_BUTTON_UP); + mouse_action.m = {1.f, 1.f, 0, move}; + assert(input.Apply(mouse_action)); + assert(pointer_events.back().display == 42 && pointer_events.back().x == 2559 && pointer_events.back().y == 1439); + allow = false; + assert(!input.Apply(key) && pressed_keys.empty()); + allow = true; + assert(FromVirtualKey(0x10, 0x36) == 2048); + assert(FromVirtualKey(0x11, 0x1D, true) == 2073); + assert(FromVirtualKey(0x12, 0x38, true) == 2046); + assert(FromVirtualKey(0x0D, 0x1C, true) == 2119); + assert(FromVirtualKey(0x23, 0x4F, false) == 2104); + assert(FromVirtualKey(0x23, 0x4F, true) == 2082); + assert(FromVirtualKey(0x7C) == 2816 && ToVirtualKey(2827) == 0x87); + mouse_action.m = {.5f, .5f, 0, left_down}; assert(input.Apply(mouse_action)); + mouse_action.m.flag = move; assert(input.Apply(mouse_action)); + assert(pointer_events.back().button == MOUSE_BUTTON_LEFT); // Required by WMS for a drag. + mouse_action.m.flag = right_down; assert(input.Apply(mouse_action)); + mouse_action.m.flag = left_up; assert(input.Apply(mouse_action)); + mouse_action.m.flag = move; assert(input.Apply(mouse_action)); + assert(pointer_events.back().button == MOUSE_BUTTON_RIGHT); + assert(input.Release()); + state.ks.seq = 11; state.ks.pressed_count = 2; + state.ks.pressed_keys[0] = {0x41, 0x1E, false}; + state.ks.pressed_keys[1] = {0x11, 0x1D, true}; + key_events.clear(); assert(input.Apply(state)); + assert(key_events.size() == 2 && key_events[0].code == 2073 && key_events[1].code == 2017); + allow = false; assert(!input.Release()); + assert(!input.Configure(800, 600, 8)); // Do not change screen while a release is still pending. + allow = true; key_events.clear(); assert(input.Release()); + assert(key_events.size() == 2 && key_events[0].code == 2017 && key_events[1].code == 2073); + assert(pressed_keys.empty() && pressed_buttons.empty()); + assert(input.Release()); + state.ks.pressed_count = kMaxKeyboardStateKeys + 1; assert(!input.Apply(state)); + key_events.clear(); + assert(input.LockWorkstation()); + assert(key_events.size() == 4 && key_events[0].code == 2076 && key_events[1].code == 2028); + assert(key_events[2].action == KEY_ACTION_UP && key_events[3].action == KEY_ACTION_UP); + assert(pressed_keys.empty()); + allow = false; assert(!input.LockWorkstation()); + assert(pressed_keys.empty()); + allow = true; rejected_key = 2028; key_events.clear(); + assert(!input.LockWorkstation()); // Letter fails after Meta was already pressed. + assert(key_events.size() == 2 && key_events.back().code == 2076 && key_events.back().action == KEY_ACTION_UP); + assert(pressed_keys.empty()); + return 0; +} diff --git a/apps/harmony/tests/keyboard_hook_mock.h b/apps/harmony/tests/keyboard_hook_mock.h new file mode 100644 index 00000000..656b8465 --- /dev/null +++ b/apps/harmony/tests/keyboard_hook_mock.h @@ -0,0 +1,35 @@ +/* + * @Author: DI JUNKUN + * @Date: 2026-10-15 + * Copyright (c) 2026 by DI JUNKUN, All Rights Reserved. + */ + +#ifndef _KEYBOARD_HOOK_MOCK_H_ +#define _KEYBOARD_HOOK_MOCK_H_ + +#include +#include +struct Input_KeyEvent { int code = 0, action = 0, id = 0; }; +inline Input_KeyEventCallback hook_callback = nullptr; +inline Input_Result hook_add_result = INPUT_SUCCESS, hook_remove_result = INPUT_SUCCESS; +inline int hook_adds = 0, hook_removes = 0, focus_during_add = 0; +inline std::vector forwarded_events; +extern "C" { +Input_Result OH_Input_AddKeyEventHook(Input_KeyEventCallback cb) { + ++hook_adds; + if (focus_during_add) crossdesk::harmony::desktop::SetKeyboardFocus(focus_during_add, true); + if (hook_add_result == INPUT_SUCCESS) hook_callback = cb; + return hook_add_result; +} +Input_Result OH_Input_RemoveKeyEventHook(Input_KeyEventCallback) { + ++hook_removes; + if (hook_remove_result == INPUT_SUCCESS) hook_callback = nullptr; + return hook_remove_result; +} +int32_t OH_Input_GetKeyEventAction(const Input_KeyEvent* event) { return event->action; } +int32_t OH_Input_GetKeyEventKeyCode(const Input_KeyEvent* event) { return event->code; } +Input_Result OH_Input_GetKeyEventId(const Input_KeyEvent* event, int32_t* id) { *id = event->id; return INPUT_SUCCESS; } +Input_Result OH_Input_DispatchToNextHandler(int32_t id) { forwarded_events.push_back(id); return INPUT_SUCCESS; } +} + +#endif diff --git a/apps/harmony/tests/keyboard_hook_test.cpp b/apps/harmony/tests/keyboard_hook_test.cpp new file mode 100644 index 00000000..4801a71a --- /dev/null +++ b/apps/harmony/tests/keyboard_hook_test.cpp @@ -0,0 +1,83 @@ +#include "keyboard_hook.h" +#include "keyboard_hook_mock.h" +#include "keyboard_state.h" +#include +#include + +using namespace crossdesk; +using namespace crossdesk::harmony::desktop; +std::vector delivered; +void Sink(int key, bool down, uint32_t scan, bool extended, void* context) { + assert(context == &delivered); delivered.push_back({key, down, scan, extended}); +} +void Event(int code, int action) { + static int id = 0; + const Input_KeyEvent event{code, action, ++id}; + assert(hook_callback); hook_callback(&event); +} +int main() { + KeyboardHook hook; + assert(hook.Start(Sink, &delivered) == INPUT_APP_NOT_FOCUSED); + SetKeyboardFocus(2, true); + hook_add_result = INPUT_PERMISSION_DENIED; + assert(hook.Start(Sink, &delivered) == INPUT_PERMISSION_DENIED); + assert(!hook.Active() && hook.Stop() == INPUT_SUCCESS); + hook_add_result = INPUT_SUCCESS; + assert(hook.Start(Sink, &delivered) == INPUT_SUCCESS && hook.Active()); + const int installed_count = hook_adds; + assert(hook.Start(Sink, &delivered) == INPUT_SUCCESS && hook_adds == installed_count); + KeyboardHook other; + assert(other.Start(Sink, &delivered) == INPUT_REPEAT_INTERCEPTOR); + assert(other.Stop() == INPUT_SUCCESS && hook.Active()); + // Alt+Tab and Meta+L reach the same shared input queue as ordinary keys. + for (auto [modifier, key] : {std::pair{2045, 2049}, std::pair{2076, 2028}}) { + Event(modifier, KEY_ACTION_DOWN); Event(key, KEY_ACTION_DOWN); + Event(key, KEY_ACTION_UP); Event(modifier, KEY_ACTION_UP); + } + assert(delivered.size() == 8 && forwarded_events.empty()); + assert(delivered[0].key_code == 0xA4 && delivered[1].key_code == 0x09); + assert(delivered[4].key_code == 0x5B && delivered[4].extended && delivered[5].key_code == 'L'); + Event(2073, KEY_ACTION_DOWN); Event(2119, KEY_ACTION_DOWN); Event(2119, KEY_ACTION_DOWN); + assert(delivered.back().key_code == 0x0D && delivered.back().scan_code == 0x1C && delivered.back().extended); + Event(2119, KEY_ACTION_CANCEL); + assert(!delivered.back().is_down && delivered.back().key_code == 0xA3); + assert(delivered[delivered.size()-2].key_code == 0x0D); + const auto count = delivered.size(); + Event(9999, KEY_ACTION_DOWN); Event(2017, KEY_ACTION_UP); + assert(delivered.size() == count && forwarded_events.size() == 2); + Event(2072, KEY_ACTION_DOWN); + SetKeyboardFocus(2, false); + assert(!hook.Active() && !delivered.back().is_down); + Event(2017, KEY_ACTION_DOWN); + assert(forwarded_events.size() == 3); + SetKeyboardFocus(1, true); Event(2017, KEY_ACTION_DOWN); + assert(!hook.Active() && forwarded_events.size() == 4); + hook.Stop(); SetKeyboardFocus(2, true); + hook.Start(Sink, &delivered); Event(2076, KEY_ACTION_DOWN); + SetKeyboardHookPermission(false); + assert(!hook.Active() && !delivered.back().is_down); + Event(2017, KEY_ACTION_DOWN); assert(forwarded_events.size() == 5); + // Failed unregistration must leave a pass-through callback, never a dangling sink. + hook_remove_result = INPUT_SERVICE_EXCEPTION; + assert(hook.Stop() == INPUT_SERVICE_EXCEPTION); + const auto after_stop = delivered.size(); + Event(2017, KEY_ACTION_DOWN); assert(delivered.size() == after_stop); + assert(hook.Start(Sink, &delivered) == INPUT_SERVICE_EXCEPTION); + hook_remove_result = INPUT_SUCCESS; + assert(hook.Start(Sink, &delivered) == INPUT_SUCCESS && hook.Active()); + hook.Stop(); + // A focus change inside the OS registration call cannot bind the hook to + // a local dialog that replaced the control window during installation. + focus_during_add = 1; + hook.Start(Sink, &delivered); assert(!hook.Active()); + Event(2017, KEY_ACTION_DOWN); assert(delivered.size() == after_stop); + hook.Stop(); + focus_during_add = 0; + { + auto transient = std::make_unique(); + transient->Start(Sink, &delivered); + hook_remove_result = INPUT_SERVICE_EXCEPTION; + } + Event(2017, KEY_ACTION_DOWN); assert(delivered.size() == after_stop); + hook_remove_result = INPUT_SUCCESS; hook.Stop(); +} diff --git a/apps/harmony/tests/mobile_file_transfer_test.cpp b/apps/harmony/tests/mobile_file_transfer_test.cpp new file mode 100644 index 00000000..c3f143fc --- /dev/null +++ b/apps/harmony/tests/mobile_file_transfer_test.cpp @@ -0,0 +1,54 @@ +#include +#include +#include +#include +#include +using namespace crossdesk; +using mobile_controller::FileTransferSession; +int main() { + char pattern[] = "/tmp/crossdesk-mobile-files-XXXXXX"; + const std::filesystem::path root = mkdtemp(pattern); + std::deque chunks, acks; + std::string received; bool complete = false, rejected = false; + FileTransferSession sender([&](const char*, const char* p, size_t n) { chunks.emplace_back(p,n); return 0; }, + [&](const std::string&, double p, bool, const std::string&) { complete = p == 1; rejected |= p < 0; }); + FileTransferSession receiver([&](const char*, const char* p, size_t n) { acks.emplace_back(p,n); return 0; }, + [&](const std::string&, double p, bool, const std::string& path) { if (p == 1) received = path; }); + receiver.SetDirectory(root / "received"); + const std::string data(3*kFileChunkSize+37,'x'); + const auto staged = root / "outgoing"; + { std::ofstream out(staged,std::ios::binary); out << data; } + assert(sender.Start(staged,"../../test.bin")); + for(int i=0;i<4;i++) sender.Pump(); + assert(chunks.size()==4); + // Real KCP callbacks may be reordered: do not expose holes as a finished file. + for(int i : {3,1,0,2}) { const auto& c=chunks[i]; receiver.Receive(c.data(),c.size()); if(i!=2) assert(received.empty()); } + assert(!received.empty() && std::filesystem::path(received).filename()=="test.bin"); + for(auto& a:acks) sender.Ack(a.data(),a.size()); + assert(complete && !sender.Sending() && !std::filesystem::exists(staged)); + std::ifstream input(received,std::ios::binary); const std::string result((std::istreambuf_iterator(input)),{}); + assert(result == data); + // Duplicate ranges must be rejected and their partial file removed. + const std::string name="duplicate"; auto chunk=EncodeFileChunk(50,0,4,"ab",2,&name,true,false); + receiver.Receive(chunk.data(),chunk.size()); receiver.Receive(chunk.data(),chunk.size()); + FileTransferAck ack{}; assert(DecodeFileTransferAck(acks.back().data(),acks.back().size(),&ack)); assert(ack.flags==2); + // Sender timeout removes only the disposable staging copy. + { std::ofstream out(staged); out << "timeout"; } + const auto now=FileTransferSession::Clock::now(); assert(sender.Start(staged,"timeout",now)); + sender.Pump(now+std::chrono::seconds(31)); assert(rejected && !sender.Sending() && !std::filesystem::exists(staged)); + // Empty files still require terminal acknowledgement. + chunks.clear(); acks.clear(); complete=false; received.clear(); + { std::ofstream out(staged); } assert(sender.Start(staged,"empty")); sender.Pump(); + receiver.Receive(chunks[0].data(),chunks[0].size()); assert(!received.empty()); + sender.Ack(acks[0].data(),acks[0].size()); assert(complete && std::filesystem::file_size(received)==0); + const auto previous=received; + { + FileTransferSession fresh([](const char*,const char*,size_t){return 0;}, + [&](const std::string&,double p,bool,const std::string&path){if(p==1)received=path;}); + fresh.SetDirectory(root/"received"); + fresh.Receive(chunks[0].data(),chunks[0].size()); + assert(received!=previous && std::filesystem::exists(previous) && std::filesystem::exists(received)); + } + std::filesystem::remove_all(root); + std::cout << "PASS: shared Android/Harmony file transfer, reordered chunks, safe names, duplicate rejection, timeout cleanup and empty files\n"; +} diff --git a/apps/harmony/tests/mobile_lifecycle_test.cjs b/apps/harmony/tests/mobile_lifecycle_test.cjs new file mode 100644 index 00000000..f502e8ae --- /dev/null +++ b/apps/harmony/tests/mobile_lifecycle_test.cjs @@ -0,0 +1,84 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const vm = require('node:vm'); +const path = require('node:path'); +const ts = require(process.argv[2]), root = process.argv[3]; +const commands = [], events = [], timers = new Map(), intervals = new Map(); let timerId = 0, time = 100; +let preferences = new Map(), secrets = new Map(), records = new Map(), saves = [], platforms = []; +class Store { + constructor(){this.directory='/private/mobile';} + flag(k,f=false){return preferences.get(k)??f;} number(k,f){return preferences.get(k)??f;} text(k,f=''){return preferences.get(k)??f;} + put(k,v){preferences.set(k,v);} static scope(s,p){return `${s.trim().toLowerCase().replace(/\.$/,'')}:${p}`;} + async secret(k){return secrets.get(k)??'';} async saveSecret(k,v){secrets.set(k,v);} async removeSecret(k){secrets.delete(k);} + recent(s){return records.get(s)??[];} clearPreviews(){this.cleared=true;} + async saveRecent(s,d,p){saves.push({scope:s,device:{...d},password:p});records.set(s,[d]);if(d.remember)secrets.set(`password:${s}:${d.id}`,p);} + metadata(s,id,n,p){for(const d of this.recent(s))if(d.id===id){d.name=n;d.platform=p;}} + async delete(s,id){records.set(s,this.recent(s).filter(d=>d.id!==id));secrets.delete(`password:${s}:${id}`);} + async savePreview(){this.previewSaved=true;} discardRaw(){} discardOutgoing(){} +} +class Platform { + constructor(c,end,release,feedback){this.context=c;this.end=end;this.release=release;platforms.push(this);this.choosing=false;} + resume(){} async connected(){this.active=true;} async background(){this.release();} + disconnect(){this.active=false;} dispose(){this.disconnect();} setMuted(){} cancelRequests(){} orientation(){} videoSize(){} async surface(){} + async version(){this.versionCalls=(this.versionCalls??0)+1;return undefined;} + async choose(){return undefined;} save(){} openLink(){} +} +const modules={}; +function load(name){ + if(modules[name])return modules[name];const exports={}; modules[name]=exports; + vm.runInNewContext(ts.transpileModule(fs.readFileSync(path.join(root,name+'.ets'),'utf8'),{ + compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2020,experimentalDecorators:true} + }).outputText,{exports,console,Observed:c=>c,Date,Map,Set,Number,Math,JSON,Uint8Array,ArrayBuffer, + setTimeout:fn=>{timers.set(++timerId,fn);return timerId;},clearTimeout:id=>timers.delete(id), + setInterval:fn=>{intervals.set(++timerId,fn);return timerId;},clearInterval:id=>intervals.delete(id), + require:name=>{ + if(name==='libcrossdesk.so')return {default:{mobileCommand:s=>commands.push(JSON.parse(s)),mobilePoll:()=>JSON.stringify(events.splice(0))}}; + if(name==='@kit.AbilityKit')return {bundleManager:{BundleFlag:{GET_BUNDLE_INFO_DEFAULT:0},getBundleInfoForSelfSync:()=>({versionName:'1.6.1'})}}; + if(name==='@kit.ArkUI')return {display:{getDefaultDisplaySync:()=>({width:1080,height:2400})},promptAction:{showToast(){}}}; + if(name==='@kit.ArkTS')return {util:{TextDecoder:{create:()=>({decodeWithStream:()=> 'Privacy policy'})},TextEncoder:class{encodeInto(s){return Buffer.from(s);}}}}; + if(name==='@kit.BasicServicesKit')return {systemDateTime:{TimeType:{STARTUP:0},getUptime:()=>time},pasteboard:{}}; + if(name==='./MobileStore')return {MobileStore:Store}; + if(name==='./MobilePlatform')return {MobilePlatform:Platform,MobileLifecycle:{}}; + if(name.startsWith('./'))return load(name.slice(2));throw Error(name); + }});return exports; +} +const {MobileController}=load('MobileController'); +const context={cacheDir:'/private/cache',filesDir:'/private',resourceManager:{getRawFileContentSync:()=>new Uint8Array([1])}}; +const flush=async()=>{for(let i=0;i<8;i++)await Promise.resolve();}; +function feed(c,event){events.push({session:c.session,...event});c.poll();} +(async()=>{ + const c=new MobileController(context);await flush();assert.equal(commands.length,0);assert.equal(platforms[0].versionCalls,undefined); + assert.equal(c.state.modal,'consent');await c.consent();await flush(); + assert(commands.some(c=>c.op==='start'));const start=commands.find(c=>c.op==='start');assert.equal(start.identity,''); + feed(c,{type:'identity',identity:'123456789@secret'});await flush(); + assert.equal(secrets.get('identity:api.crossdesk.cn:9099'),'123456789@secret'); + feed(c,{type:'identity',identity:'123456789'});await flush(); + assert.equal(secrets.get('identity:api.crossdesk.cn:9099'),'123456789@secret'); // Public reconnect report cannot erase the password. + feed(c,{type:'signal',status:1,controller:false,generation:1,id:'123456789'}); + assert.equal(c.state.signal,1);assert(commands.some(c=>c.op==='signal'&&c.message.type==='announcements_list')); + await c.requestConnection('987654321');assert.equal(c.state.modal,'password');c.state.remember=true;c.begin('correct'); + const connect=commands.filter(c=>c.op==='connect').at(-1);assert.equal(connect.password,'correct');assert.equal(c.state.password,''); + feed(c,{type:'data',connection:connect.connection,message:{type:3,host_info:{host_name:'Early desktop',platform:'windows',app_version:'1.6.1',supports_video_settings:true,displays:[{name:'Early display'}]}}}); + assert.equal(c.state.displays.length,1);assert(!commands.some(c=>c.op==='videoSettings')); + feed(c,{type:'connection',connection:connect.connection,status:1});await flush();assert(c.state.ready);assert.equal(saves.length,1);assert.equal(saves[0].password,'correct'); + assert(commands.some(c=>c.op==='videoSettings'));assert.equal(saves[0].device.name,'Early desktop'); + feed(c,{type:'connection',connection:connect.connection,status:1});await flush();assert.equal(saves.length,1); + feed(c,{type:'data',connection:connect.connection,message:{type:3,host_info:{host_name:'Desktop',platform:'windows',app_version:'1.6.1',supports_video_settings:true,displays:[{name:'Display1'}]}}}); + assert(c.video.supported);assert(commands.some(c=>c.op==='videoSettings'));assert.equal(c.state.displays.length,1); + c.updateVideo(1,60);const settings=commands.filter(c=>c.op==='videoSettings').at(-1); + feed(c,{type:'data',connection:connect.connection,message:{type:12,video_settings:{quality:settings.quality,frame_rate:60,preference:settings.preference,request_id:settings.request,accepted:false}}}); + assert.equal(c.state.videoRate,30);assert(c.video.failed); + c.background();await flush();assert(c.state.ready);assert(!commands.slice(-2).some(c=>c.op==='disconnect'));assert(commands.some(c=>c.op==='release')); + c.resume();await flush();c.end('');await c.requestConnection('987654321'); + assert.equal(c.state.modal,'connecting'); // Encrypted remembered password, no prompt. + const second=commands.filter(c=>c.op==='connect').at(-1); + feed(c,{type:'connection',connection:connect.connection,status:1});assert(!c.state.ready); // Superseded session. + feed(c,{type:'connection',connection:second.connection,status:6});assert.equal(c.state.modal,'password');assert.match(c.state.notice,/密码错误/); + const previousSession=c.session;c.saveSettings('private.example','9443');await flush(); + assert.equal(c.scope(),'private.example:9443');assert.equal(c.state.recent.length,0); + feed(c,{session:previousSession,type:'identity',identity:'000000000@stale'});assert.notEqual(c.state.identity,'000000000'); + c.setPreviews(true);assert(c.state.previews);c.revoke();await flush();assert(!c.state.consent);assert(c.store.cleared); + const calls=platforms[0].versionCalls;c.resume();await c.checkLocal(true);assert.equal(platforms[0].versionCalls,calls); + c.destroy();assert.equal(intervals.size,0); + console.log('PASS: consent-before-network, encrypted identity/password use, duplicate/stale callbacks, controller-only lifecycle, ACK rollback, background retention, server isolation and revocation'); +})().catch(e=>{console.error(e);process.exitCode=1;}); diff --git a/apps/harmony/tests/mobile_models_test.cjs b/apps/harmony/tests/mobile_models_test.cjs new file mode 100644 index 00000000..3b67ca0e --- /dev/null +++ b/apps/harmony/tests/mobile_models_test.cjs @@ -0,0 +1,97 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const vm = require('node:vm'); +const ts = require(process.argv[2]); +const root = process.argv[3]; +let timers = new Map(), nextTimer = 0; +function load(name) { + const exports = {}; + vm.runInNewContext(ts.transpileModule(fs.readFileSync(path.join(root, name + '.ets'), 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } + }).outputText, { exports, console, setTimeout: fn => { timers.set(++nextTimer, fn); return nextTimer; }, + clearTimeout: id => timers.delete(id), Date, Map, Set, Number, Math, JSON, + require: name => { if (name.startsWith('./')) return load(name.slice(2)); throw Error(name); } }); + return exports; +} +const { RemoteViewport, MouseInput, TouchInput, VideoSettings, NetworkStatistics, RecentPresence, asciiKey } = load('ControllerModels'); +const viewport = new RemoteViewport(); viewport.fit(320, 200, 1920, 1080); +assert.equal(viewport.baseHeight, 180); assert.equal(viewport.top(), 10); +const before = viewport.x(100); viewport.zoom(2, 100, 100); assert.equal(viewport.x(100), before); +viewport.zoom(100, 160, 100); assert.equal(viewport.scale, 10); +viewport.pan(1e6, -1e6); assert.equal(viewport.offsetX, viewport.baseWidth * 4.5); +viewport.zoom(.00001, 160, 100); assert.equal(viewport.scale, 1); assert.equal(viewport.offsetX, 0); +assert(!viewport.contains(100, 0)); assert.equal(viewport.x(-100), 0); +const packets = []; +const mouse = new MouseInput((x, y, flag, wheel) => packets.push({ x, y, flag, wheel })); +const touch = new TouchInput(viewport, mouse); +function input(kind, points = []) { if (kind === 'down') touch.begin(); touch.touch(points.map(([x,y]) => ({x,y})), kind); } +input('down', [[100,100]]); input('move', [[120,100]]); input('up'); +assert(packets.every(p => p.flag === 0)); // Relative movement never drags. +packets.length = 0; input('down', [[100,100]]); input('up'); assert.deepEqual(packets.map(p => p.flag), [1,2]); +packets.length = 0; input('down', [[100,100]]); touch.hold(); input('up'); assert.deepEqual(packets.map(p => p.flag), [3,4]); +touch.relative = false; packets.length = 0; +input('down', [[100,100]]); input('move', [[120,100]]); input('move', [[400,300]]); input('up'); +assert.deepEqual(packets.map(p => p.flag), [0,1,0,0,2]); assert.equal(packets.at(-2).x, 1); +packets.length = 0; +input('down', [[100,100]]); input('down', [[100,100],[200,100]]); input('move', [[80,100],[220,100]]); +input('up', [[80,100]]); input('move', [[90,110]]); input('up'); +assert(viewport.scale > 1); assert(packets.every(p => p.flag === 0)); +viewport.reset(); packets.length = 0; +input('down', [[100,100]]); input('down', [[100,100],[200,100]]); input('move', [[100,125],[200,125]]); input('up'); +assert.equal(packets.filter(p => p.flag === 7).length, 1); +mouse.button(true, true); const revision = mouse.revision; +packets.length = 0; input('down', [[100,100]]); input('move', [[110,100]]); input('up'); +assert(mouse.left); assert(packets.every(p => p.flag === 0)); assert.equal(mouse.revision, revision); +mouse.button(false, true); mouse.drag(32,0,320,180); mouse.release(); +assert.equal(packets.at(-2).flag, 2); assert.equal(packets.at(-1).flag, 4); +assert.equal(asciiKey('a'), 65); assert.equal(asciiKey('A'), 321); assert.equal(asciiKey('!'), 305); assert.equal(asciiKey('中'), 0); +const settings = new VideoSettings(1, 1280); assert.equal(settings.selection.quality, 0); +assert.equal(settings.begin({quality:1,frameRate:30,preference:1}), -1); settings.support(true); +const one = {quality:2,frameRate:60,preference:0}, two = {quality:1,frameRate:30,preference:2}; +const id1 = settings.begin(one), id2 = settings.begin(two); +settings.receive(one, id1, true); assert(settings.pending); assert.equal(settings.applied.quality, 2); +settings.receive(two, id2, false); assert(!settings.pending); assert.equal(settings.selection.quality, 2); +const id3 = settings.begin(two); settings.expire(id2); assert(settings.pending); settings.expire(id3); assert.equal(settings.selection.quality, 2); +settings.receive(two,id3+1,true); assert.equal(settings.selection.quality,2); +settings.requestId = 0xffffffff; settings.appliedRequestId = 0xffffffff; +const wrapped = settings.begin(two); assert.equal(wrapped,0); settings.receive(two,wrapped,true); assert.equal(settings.selection.preference,2); +settings.support(false); assert(!settings.pending); +const network = new NetworkStatistics(); +const traffic = {inbound:8000000,outbound:10000,loss:.02}; +network.receive({rtt:40,mode:0,srtp:true,video:traffic,audio:traffic,data:traffic,total:traffic}, 100); +network.receive({rtt:80,mode:1,srtp:false,video:traffic,audio:traffic,data:traffic,total:traffic}, 200); +assert.equal(network.rtt,50); assert.equal(network.mode(201),'TURN 中继'); assert.equal(network.mode(3200),'—'); +network.frame(6,7000,1920,1080,200); network.frame(6,1,1920,1080,250); assert.equal(network.fps(250),6); assert.equal(network.latency(250),7000); +assert.equal(network.fps(1200),0); network.reset(); assert.equal(network.width,0); +const subscriptions = []; const presence = new RecentPresence(ids => subscriptions.push([...ids])); +presence.watch(['111111111'],0); presence.connect(true,10); assert.equal(subscriptions.length,1); +presence.receive({type:'presence_update',id:'111111111',online:true},20); assert(!presence.online('111111111',21)); +presence.receive({type:'presence',devices:[{id:'111111111',online:true},{id:'222222222',online:true}]},30); +assert(presence.online('111111111',31)); assert(!presence.online('222222222',31)); assert(!presence.online('111111111',60030)); +presence.tick(30010); assert.equal(subscriptions.length,2); presence.connect(false,40000); presence.connect(true,41000); assert(!presence.online('111111111',41001)); +const { AnnouncementInbox } = load('AnnouncementInbox'); +const { announcementSpans, announcementByteLength, isAnnouncementURL } = load('AnnouncementText'); +const spans = announcementSpans('说明\r\n[下载](HTTP://example.com/a(b));https://example.com/end. 普通文本'); +assert.equal(spans.map(p=>p.text).join(''),'说明\n下载;https://example.com/end. 普通文本'); +assert.deepEqual(Array.from(spans.filter(p=>p.url),p=>p.url),['HTTP://example.com/a(b)','https://example.com/end']); +assert(!isAnnouncementURL('javascript:alert(1)'));assert(!isAnnouncementURL('https://a\\b'));assert(!isAnnouncementURL('https://')); +assert.equal(announcementByteLength('中'.repeat(80)),240);assert.equal(announcementByteLength('😀'),4); +const requests = []; const saved = new Map(); let failSave = false; +const inbox = new AnnouncementInbox(r => requests.push({...r}), s => saved.get(s) ?? '{}', (s,v) => { if(failSave)throw Error(); saved.set(s,v); }); +inbox.configure('server:9099:identity'); inbox.connect(true); +function answer(request, entries, total, revision=1) { inbox.receive({type:'announcements',request_id:request.request_id,summary_only:request.summary_only,offset:request.offset,total,catalog_revision:revision,items:entries}); } +const entries = Array.from({length:21},(_,i)=>({id:i+1,revision:1,title:`Title ${i+1}`,body:'Body',updated_at:1700000000})); +const first = requests.at(-1); answer(first,entries,21); assert.equal(requests.at(-1).summary_only,false); +answer(requests.at(-1),entries.slice(0,20),21); assert.equal(inbox.items.length,20); assert.equal(inbox.unread,21); +inbox.markRead(inbox.items[0]); assert.equal(inbox.unread,20); +failSave = true; inbox.dismiss(inbox.items[0]); assert.equal(inbox.total,21); assert(inbox.saveFailed); +failSave = false; inbox.dismiss(inbox.items[0]); answer(requests.at(-1),entries.slice(20),21); assert.equal(inbox.items.length,20); assert.equal(inbox.total,20); +const stale = requests.at(-1); inbox.refresh(); answer(stale,[],0); assert(inbox.loading); answer(requests.at(-1),entries.map(e=>({...e,revision:2})),21,2); +answer(requests.at(-1),entries.slice(0,20).map(e=>({...e,revision:2})),21,2); assert.equal(inbox.unread,21); +inbox.more(); answer(requests.at(-1),entries.slice(20).map(e=>({...e,revision:2})),21,2); assert.equal(inbox.items.length,21); +inbox.connect(false); const count=requests.length; inbox.more(); assert.equal(requests.length,count); +inbox.connect(true); for(const timer of [...timers.values()])timer(); assert(inbox.failed); +inbox.configure('utf8');inbox.connect(true);answer(requests.at(-1),[entries[0]],1); +answer(requests.at(-1),[{...entries[0],title:'中'.repeat(81)}],1);assert(inbox.failed);assert.equal(inbox.items.length,0); +console.log('PASS: mobile viewport/gestures, shared button ownership, keyboard, video ACK ordering, network expiry, presence and announcement pagination/revisions'); diff --git a/apps/harmony/tests/mobile_platform_test.cjs b/apps/harmony/tests/mobile_platform_test.cjs new file mode 100644 index 00000000..c3783c12 --- /dev/null +++ b/apps/harmony/tests/mobile_platform_test.cjs @@ -0,0 +1,39 @@ +const assert=require('node:assert/strict'),fs=require('node:fs'),vm=require('node:vm'),path=require('node:path'); +const ts=require(process.argv[2]),root=process.argv[3], exports_={}; +const textExports={}; +vm.runInNewContext(ts.transpileModule(fs.readFileSync(path.join(root,'AnnouncementText.ets'),'utf8'),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2020}}).outputText,{exports:textExports}); +let agentResolve, startResolve, pipResolve, creating=0, activeTasks=0, calls=[], ended=[], releases=0; +const flush=async()=>{for(let i=0;i<12;i++)await Promise.resolve();}; +const pip={setAutoStartEnabled:v=>calls.push(['auto',v]),on:(type,fn)=>pip[type]=fn,updatePiPControlStatus:(t,s)=>calls.push(['mute',s]), + updateContentSize(){},startPiP:async()=>calls.push(['pip']),stopPiP:async()=>calls.push(['stopPip'])}; +const background={BackgroundMode:{DATA_TRANSFER:1},startBackgroundRunning:()=>new Promise(r=>{calls.push(['start']);startResolve=()=>{activeTasks++;r();};}), + stopBackgroundRunning:async()=>{calls.push(['stop']);activeTasks--;}}; +vm.runInNewContext(ts.transpileModule(fs.readFileSync(path.join(root,'MobilePlatform.ets'),'utf8'),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2020}}).outputText, + {exports:exports_,console,Promise,Date,Math,require:name=>{ + if(name==='./AnnouncementText')return textExports; + if(name==='@kit.BasicServicesKit')return {deviceInfo:{deviceType:'phone'}}; + if(name==='@kit.AbilityKit')return {wantAgent:{OperationType:{START_ABILITY:1},WantAgentFlags:{UPDATE_PRESENT_FLAG:1},getWantAgent:()=>new Promise(r=>agentResolve=r)}}; + if(name==='@kit.BackgroundTasksKit')return {backgroundTaskManager:background}; + if(name==='@kit.CoreFileKit')return {picker:{},fileIo:{},fileUri:{}}; + if(name==='@kit.NetworkKit')return {http:{}}; + if(name==='@kit.ArkUI')return {display:{getDefaultDisplaySync:()=>({densityPixels:1})},window:{Orientation:{PORTRAIT:1,AUTO_ROTATION:2}, + getLastWindow:async()=>({setWindowKeepScreenOn:async()=>{},getWindowProperties:()=>({windowRect:{width:360}}),setPreferredOrientation:async()=>{}})}, + PiPWindow:{PiPTemplateType:{VIDEO_LIVE:1},VideoLiveControlGroup:{MUTE_SWITCH:402},PiPControlType:{MUTE_SWITCH:8},PiPControlStatus:{PAUSE:0,PLAY:1}, + PiPState:{ABOUT_TO_START:1,STARTED:2,ABOUT_TO_RESTORE:3,STOPPED:4},isPiPEnabled:()=>true,create:()=>{creating++;return new Promise(r=>pipResolve=r);}}}; + throw Error(name); + }}); +(async()=>{ + const p=new exports_.MobilePlatform({abilityInfo:{name:'Entry',bundleName:'test'}},r=>ended.push(r),()=>releases++,()=>{}); + const first=p.connected();await flush();agentResolve({});await flush(); + // Disconnect during system start, then reconnect before it resolves. + p.disconnect();const second=p.connected();await flush();assert.equal(calls.filter(c=>c[0]==='start').length,1); + startResolve();await flush();assert.equal(activeTasks,0);agentResolve({});await flush();startResolve();await second;assert.equal(activeTasks,1); + const a=p.surface({},640,360), b=p.surface({},640,360);assert.equal(creating,1);pipResolve(pip);await Promise.all([a,b]); + let muted=false;p.onMute=v=>muted=v;pip.controlEvent({controlType:8,status:0});assert(muted);p.setMuted(true);assert(calls.some(c=>c[0]==='mute'&&c[1]===0)); + p.setChoosing(true);await p.background();assert(!calls.some(c=>c[0]==='pip'));assert.equal(ended.length,0); + p.resume();p.setChoosing(false);await p.background();assert(calls.some(c=>c[0]==='pip'));assert(releases>0); + pip.stateChange(2,'');assert(p.pipActive);p.resume();await flush();assert(!p.pipActive);assert.equal(ended.length,0); + assert(calls.some(c=>c[0]==='stopPip'));await p.background();pip.stateChange(2,'');pip.stateChange(4,'close');assert.equal(ended.length,1); + p.disconnect();await flush();assert.equal(activeTasks,0); + console.log('PASS: rapid reconnect serializes background tasks, PiP creation deduplicates, mute control, picker guard, background release and close cleanup'); +})().catch(e=>{console.error(e);process.exitCode=1;}); diff --git a/apps/harmony/tests/mobile_store_test.cjs b/apps/harmony/tests/mobile_store_test.cjs new file mode 100644 index 00000000..5e1d72d5 --- /dev/null +++ b/apps/harmony/tests/mobile_store_test.cjs @@ -0,0 +1,45 @@ +const assert=require('node:assert/strict'),fs=require('node:fs'),vm=require('node:vm'),path=require('node:path'),crypto=require('node:crypto'); +const ts=require(process.argv[2]),root=process.argv[3],exports_={}; +const values=new Map(),secrets=new Map(),files=new Map(),opened=new Map();let nextFd=0,finishPacking,cacheMissing=false; +const Tag={ALIAS:1,SECRET:2,RETURN_TYPE:3,ACCESSIBILITY:4,SYNC_TYPE:5,CONFLICT_RESOLUTION:6}; +const key=q=>Buffer.from(q.get(Tag.ALIAS)).toString(); +const asset={Tag,ReturnType:{ALL:1},Accessibility:{DEVICE_FIRST_UNLOCKED:2},SyncType:{NEVER:0},ConflictResolution:{OVERWRITE:1},ErrorCode:{NOT_FOUND:404}, + query:async q=>{const v=secrets.get(key(q));if(!v)throw {code:404};return [v];}, + add:async q=>{assert.equal(q.get(Tag.ACCESSIBILITY),2);assert.equal(q.get(Tag.SYNC_TYPE),0);secrets.set(key(q),new Map(q));}, + remove:async q=>{secrets.delete(key(q));}}; +const io={OpenMode:{READ_ONLY:1,WRITE_ONLY:2,CREATE:4,TRUNC:8},mkdirSync(){}, + accessSync:p=>{if(!files.has(p))throw Error('missing');return true;}, + listFileSync:p=>{if(cacheMissing)throw Error('cache removed');return [...files.keys()].filter(k=>k.startsWith(p+'/')).map(k=>k.slice(p.length+1));}, + unlinkSync:p=>files.delete(p),renameSync:(a,b)=>{assert(files.has(a));files.set(b,files.get(a));files.delete(a);}, + openSync:(p,mode)=>{if(mode&4)files.set(p,Buffer.alloc(0));if(!files.has(p))throw Error('missing');const file={fd:++nextFd};opened.set(file.fd,p);return file;}, + closeSync:f=>opened.delete(f.fd),readSync:(fd,b)=>{const content=files.get(opened.get(fd));new Uint8Array(b).set(content);return content.length;}}; +vm.runInNewContext(ts.transpileModule(fs.readFileSync(path.join(root,'MobileStore.ets'),'utf8'),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2020}}).outputText, + {exports:exports_,Uint8Array,ArrayBuffer,Map,console,require:name=>{ + if(name==='@kit.ArkData')return {preferences:{getPreferencesSync:()=>({getSync:(k,d)=>values.get(k)??d,putSync:(k,v)=>values.set(k,v),flushSync(){}})}}; + if(name==='@kit.AssetStoreKit')return {asset}; + if(name==='@kit.CryptoArchitectureKit')return {cryptoFramework:{createMd:()=>{const hash=crypto.createHash('sha256');return {update:async b=>hash.update(b.data),digest:async()=>({data:hash.digest()})};}}}; + if(name==='@kit.ArkTS')return {util:{TextEncoder:class{encodeInto(v){return Buffer.from(v);}},TextDecoder:{create:()=>({decodeWithStream:v=>Buffer.from(v).toString()})}}}; + if(name==='@kit.CoreFileKit')return {fileIo:io}; + if(name==='@kit.ImageKit')return {image:{PixelMapFormat:{RGBA_8888:1},createPixelMap:async()=>({release(){}}),createImagePacker:()=>({packToFile:()=>new Promise(r=>finishPacking=r),release(){}})}}; + throw Error(name); + }}); +const flush=async()=>{for(let i=0;i<12;i++)await Promise.resolve();}; +(async()=>{ + const Store=exports_.MobileStore,s=new Store({filesDir:'/private',cacheDir:'/cache'}),scope=Store.scope('EXAMPLE.COM.',9099); + assert.equal(scope,'example.com:9099'); + await s.saveSecret('identity:'+scope,'123456789@private');assert.equal(await s.secret('identity:'+scope),'123456789@private'); + await s.saveRecent(scope,{id:'987654321',name:'Desktop',platform:'windows',remember:true,preview:''},'password'); + assert(!JSON.stringify([...values]).includes('password'));assert(!JSON.stringify([...values]).includes('@private')); + assert.equal(await s.secret('password:'+scope+':987654321'),'password'); + s.put('consent',true);s.put('previews',true); + let raw='/cache/mobile-previews/preview-1-1.rgba';files.set(raw,Buffer.alloc(640*360*4)); + let save=s.savePreview(scope,'987654321',raw);await flush();assert(finishPacking);s.clearPreviews();finishPacking();await save; + assert.equal(files.size,0);assert.equal(s.recent(scope)[0].preview,'');assert.equal(opened.size,0); + raw='/cache/mobile-previews/preview-2-1.rgba';files.set(raw,Buffer.alloc(640*360*4)); + save=s.savePreview(scope,'987654321',raw);await flush();finishPacking();await save; + const preview=s.recent(scope)[0].preview;assert(preview.endsWith('.jpg'));assert(files.has(preview));assert(!files.has(raw)); + files.delete(preview);assert.equal(s.recent(scope).length,1);assert.equal(s.recent(scope)[0].preview,''); + cacheMissing=true;assert.doesNotThrow(()=>s.clearPreviews()); + await s.delete(scope,'987654321');assert.equal(s.recent(scope).length,0);assert.equal(await s.secret('password:'+scope+':987654321'),''); + console.log('PASS: Asset Store-only secrets without sync, private preview cache, asynchronous revocation, evicted cache and credential deletion'); +})().catch(e=>{console.error(e);process.exitCode=1;}); diff --git a/apps/harmony/tests/mobile_synthetic_host.cpp b/apps/harmony/tests/mobile_synthetic_host.cpp new file mode 100644 index 00000000..bee620c9 --- /dev/null +++ b/apps/harmony/tests/mobile_synthetic_host.cpp @@ -0,0 +1,70 @@ +// Manual simulator integration fixture. Generates pixels/PCM only; never captures a device. +// Run in a disposable directory: [av1]. Login is saved in fixture-login. +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +using namespace crossdesk; +struct Fixture { + PeerPtr* peer=nullptr; std::atomic connected{false}; std::atomic selected{0}; + std::mutex mutex; std::vector> messages; + std::string remote; std::atomic signal{-1}; int keys=0,mouse=0,settings=0,clipboard=0; + void Send(RemoteAction& a) {const auto s=a.to_json();SendReliableDataFrame(peer,s.data(),s.size(),kControlStream);} +}; +int main(int argc,char**argv) { + if(argc<3)return 2;umask(0077); +#if defined(__OHOS__) + setenv("SSL_CERT_DIR","/etc/security/certificates",1); +#endif + std::string login;{std::ifstream input("fixture-login");input>>login;} + Fixture f;Params p{};snprintf(p.signal_server_ip,sizeof(p.signal_server_ip),"%s",argv[1]);p.signal_server_port=atoi(argv[2]);p.user_id=login.c_str();p.user_data=&f; + p.turn_mode=TurnAutoUdpTcp;p.enable_srtp=true;p.av1_encoding=argc>3;p.video_frame_rate=30;p.video_quality=QualityMedium; + p.video_content_type=VideoContentType::ScreenContent;p.video_degradation_preference=VideoDegradationPreference::MaintainResolution; + p.on_signal_status=[](SignalStatus s,const char*,size_t,void*u){static_cast(u)->signal=s;}; + p.on_net_status_report=[](const char*id,size_t n,TraversalMode mode,const MiniRtcNetTrafficStats*,const char*,size_t,void*){ + if(mode==UnknownMode&&id&&n&&std::string(id,n).find('@')!=std::string::npos){std::ofstream out("fixture-login");out.write(id,n);} + }; + p.on_connection_status=[](ConnectionStatus s,const char*id,size_t n,void*u){auto*f=static_cast(u);std::lock_guard lock(f->mutex);f->remote.assign(id,n);f->connected=s==Connected;if(s==Connected)f->selected=0;}; + p.on_receive_data_buffer=[](const char*d,size_t n,const char*,size_t,const char*s,size_t l,void*u){auto*f=static_cast(u);std::lock_guard lock(f->mutex);f->messages.emplace_back(std::string(s,l),std::string(d,n));}; + f.peer=CreatePeer(&p);AddVideoStream(f.peer,"Display1");AddVideoStream(f.peer,"Display2");AddAudioStream(f.peer,kAudioStream);AddDataStream(f.peer,kDataStream,false); + for(auto s:{kKeyboardStream,kMouseStream,kControlStream,kClipboardStream,kFileStream,kFileFeedbackStream})AddDataStream(f.peer,s,true); + if(Init(f.peer))return 3; + mobile_controller::FileTransferSession files([&](const char*s,const char*b,size_t n){return SendReliableDataFrame(f.peer,b,n,s);}, + [](const std::string&,double p,bool,const std::string&path){if(p==1&&!path.empty())std::ofstream("received-result")< pixels(640*360*3/2,128);int16_t pcm[480]{}; + const auto end=std::chrono::steady_clock::now()+std::chrono::minutes(15); + while(std::chrono::steady_clock::now()> messages;{std::lock_guard lock(f.mutex);messages.swap(f.messages);} + if(f.connected&&!info){auto a=MakeHostInformation("Synthetic mobile fixture",{{"Test pattern",0,0,640,360},{"Second display",0,0,640,360}},true,"1.6.1",HostPlatform::Windows);a.i.supports_video_settings=true;f.Send(a);FreeRemoteAction(a);info=true; + const std::string clip="CrossDesk synthetic clipboard";SendReliableDataFrame(f.peer,clip.data(),clip.size(),kClipboardStream); + {std::ofstream out("sample.txt");out<<"Synthetic mobile file transfer\n";}files.Start("sample.txt","fixture.txt");} + if(!f.connected)info=false; + for(auto&[stream,bytes]:messages){ + if(stream==kFileStream)files.Receive(bytes.data(),bytes.size());else if(stream==kFileFeedbackStream)files.Ack(bytes.data(),bytes.size()); + else if(stream==kClipboardStream){f.clipboard++;SendReliableDataFrame(f.peer,bytes.data(),bytes.size(),kClipboardStream);} + else {RemoteAction a{};if(!a.from_json(bytes))continue; + if(a.type==host_infomation)info=false; + if(a.type==keyboard)f.keys++;if(a.type==mouse)f.mouse++;if(a.type==display_id)f.selected=a.d; + if(a.type==video_settings){f.settings++;a.type=video_settings_status;a.vs.accepted=true;f.Send(a);}FreeRemoteAction(a);} + } + if(f.connected){ + files.Pump();MiniRtcAudioFrame audio{};audio.data=reinterpret_cast(pcm);audio.size=sizeof(pcm);SendAudioFrame(f.peer,&audio,kAudioStream); + if(iteration%3==0){for(int y=0;y<360;y++)for(int x=0;x<640;x++)pixels[y*640+x]=uint8_t(35+(x/80)*24+((y+iteration)%180<12?30:0)); + std::fill(pixels.begin()+640*360,pixels.end(),128);for(size_t i=640*360;i(pixels.data()); + frame.captured_timestamp=std::chrono::duration_cast(std::chrono::steady_clock::now().time_since_epoch()).count(); + SendVideoFrame(f.peer,&frame,f.selected?"Display2":"Display1");} + } + if(iteration++%100==0)std::ofstream("fixture-status")<<"signal="< { + assert.equal(context, 17); assert.equal(options.showKeyboard, false); + assert.equal(config.inputAttribute.consumeKeyEvents, true); ++attached; + }, + detach: async () => { ++detached; }, + on: (type, callback) => { assert(!handlers.has(type)); handlers.set(type, callback); }, + off: type => handlers.delete(type), + changeSelection: async (...value) => { selection = value; }, updateCursor: async () => {} +}; +const inputMethod = { + getController: () => editor, + TextInputType: { TEXT: 0, VISIBLE_PASSWORD: 1, NUMBER: 2, NUMBER_DECIMAL: 3 }, + EnterKeyType: { DONE: 1 }, + Direction: { CURSOR_UP: 1, CURSOR_DOWN: 2, CURSOR_LEFT: 3, CURSOR_RIGHT: 4 }, + ExtendAction: { SELECT_ALL: 0, CUT: 3, COPY: 4, PASTE: 5 } +}; +const source = fs.readFileSync(process.argv[3], 'utf8'); +const compiled = ts.transpileModule(source, { compilerOptions: { + module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 +}}).outputText; +const exportsObject = {}; +vm.runInNewContext(compiled, { exports: exportsObject, console, require: name => { + if (name === '@kit.IMEKit') return { inputMethod }; + if (name === './DisplayHost') return { DisplayHost: { screen: () => ({ densityPixels: 2 }) } }; + throw Error(`Unexpected import ${name}`); +}}); +const host = exportsObject.TextInputHost; +const target = { getWindowProperties: () => ({ id: 17, windowRect: { left: 20, top: 30 } }), + getWindowDecorVisible: () => false, getWindowDecorHeight: () => 0, getUIContext: () => 17 }; +const sent = []; +const request = async event => { + host.request(event, target, value => sent.push(value)); + await new Promise(resolve => setImmediate(resolve)); +}; +const take = () => JSON.parse(JSON.stringify(sent.splice(0))); +const keyboardModule = {}; +vm.runInNewContext(ts.transpileModule(fs.readFileSync(process.argv[4], 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } +}).outputText, { exports: keyboardModule, KeyType: { Down: 0, Up: 1 } }); +const keyboard = new keyboardModule.KeyboardHost(); +const key = (code, down, ctrl = false) => keyboard.preIme({ keyCode: code, + type: down ? 0 : 1, getModifierKeyState: keys => { + assert(['Ctrl', 'Alt', 'Shift'].includes(keys[0])); + return keys[0] === 'Ctrl' && ctrl; + } }, 1, e => sent.push(e)); +assert(key(2072, true, true)); // Slint gets the modifier even if the IME consumes it. +assert(key(2017, true, true)); +assert(key(2017, true, true)); // Repeat is retained. +assert(key(2072, false)); +assert(key(2017, false)); // Ctrl was released first. +assert.equal(take().length, 5); +assert(!key(2017, true)); // Plain typing remains with the IME. +assert(!key(2050, true, true)); // Ctrl+Space can switch input methods. +assert(key(2019, true, true)); take(); keyboard.reset(); +assert(!key(2019, true)); // Lost focus must not consume the next plain letter. +assert(key(2076, true)); assert(key(2017, true)); +assert(key(2076, false)); assert(key(2017, false)); +assert.equal(take().length, 4); + +async function verifyClipboardQueue() { + const module = {}, replies = [], order = []; + let clipboardPermission = true; + let pending = [], releaseRead, clipboardText = 'old'; + const native = { poll: () => JSON.stringify(pending.splice(0)), + send: value => replies.push(JSON.parse(value)) }; + const read = new Promise(resolve => { releaseRead = resolve; }); + let readWait = read; + const board = { + hasDataType: () => true, + getData: async () => { order.push('read'); const text = clipboardText; await readWait; return { getPrimaryText: () => text }; }, + setData: async data => { order.push(data); if (data === 'fail') throw Error('denied'); } + }; + vm.runInNewContext(ts.transpileModule(fs.readFileSync(process.argv[5], 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } + }).outputText, { exports: module, console, require: name => { + if (name === 'libslint_desktop.so') return { default: native }; + if (name === './DesktopAccess') return { DesktopAccess: { requestClipboard: () => {} } }; + if (name === '@kit.BasicServicesKit') return { pasteboard: { + getSystemPasteboard: () => board, MIMETYPE_TEXT_PLAIN: 'text/plain', createData: (_, text) => text + } }; + if (name === '@kit.AbilityKit') return { abilityAccessCtrl: { + GrantStatus: { PERMISSION_GRANTED: 0 }, createAtManager: () => ({ + checkAccessTokenSync: () => clipboardPermission ? 0 : -1 + }) + } }; + return {}; // Other OS services are not entered by this test. + }}); + const host = module.DesktopHost; + host.context = { applicationInfo: { accessTokenId: 17 } }; + pending = [{ op: 'clipboardRead', request: 1 }, { op: 'clipboardWrite', request: 2, text: 'new' }]; + host.poll(); + await new Promise(resolve => setImmediate(resolve)); + assert.deepEqual(order, ['read']); // Write must wait for the outstanding read. + pending = [{ op: 'unknown', request: 3 }]; host.poll(); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(replies[0].request, 3); // Unrelated services remain responsive. + releaseRead(); await host.clipboardQueue; + assert.deepEqual(order, ['read', 'new']); + assert.deepEqual(replies.slice(1).map(r => r.request), [1, 2]); + pending = [{ op: 'clipboardWrite', request: 4, text: 'fail' }, { op: 'clipboardWrite', request: 5, text: 'recovered' }]; + host.poll(); await host.clipboardQueue; + assert.equal(replies[3].ok, false); assert.equal(replies[4].ok, true); + clipboardPermission = false; + const before = order.length; + pending = [{ op: 'clipboardRead', request: 6 }, { op: 'clipboardHas', request: 7 }]; + host.poll(); await host.clipboardQueue; + assert.equal(order.length, before); // Polling cannot consume Ctrl+V's one-read grant. + assert.equal(replies[5].ok, false); assert.equal(replies[6].ok, false); + pending = [{ op: 'clipboardPaste', request: 8 }]; + host.poll(); await host.clipboardQueue; + assert.equal(order.length, before + 1); assert.equal(replies[7].text, 'old'); + // No active window is required: an OS update invalidates the cache and + // enters the same SDL notification path while the application is hidden. + host.started = true; host.clipboardObserved = true; clipboardPermission = true; + const query = async id => { + pending = [{ op: 'clipboardRead', request: id }]; host.poll(); await host.clipboardQueue; + return replies.find(r => r.request === id); + }; + host.clipboardChanged(); + assert.equal((await query(9)).text, 'old'); + const afterFirst = order.length; + assert.equal((await query(10)).text, 'old'); assert.equal(order.length, afterFirst); + clipboardText = '外部应用复制 😀'; host.clipboardChanged(); + assert.equal((await query(11)).text, clipboardText); assert.equal(order.length, afterFirst + 1); + // A change during an asynchronous read must not be cached as the new value. + let releaseRace; + readWait = new Promise(resolve => { releaseRace = resolve; }); + clipboardText = 'first'; host.clipboardChanged(); + const racingRead = query(12); + await new Promise(resolve => setImmediate(resolve)); + clipboardText = 'second'; host.clipboardChanged(); releaseRace(); + assert.equal((await racingRead).text, 'first'); + assert.equal((await query(13)).text, 'second'); + // Revocation during getData must not expose or cache the result. + readWait = new Promise(resolve => { releaseRace = resolve; }); + host.clipboardChanged(); const revokedRead = query(14); + await new Promise(resolve => setImmediate(resolve)); + clipboardPermission = false; host.clipboardChanged(); releaseRace(); + assert.equal((await revokedRead).ok, false); + const afterRevocation = order.length; + assert.equal((await query(15)).ok, false); assert.equal(order.length, afterRevocation); + assert.equal(host.clipboardCache, ''); + host.terminating = true; + const notifications = replies.length; host.clipboardChanged(); + assert.equal(replies.length, notifications); // Late callbacks after shutdown are ignored. +} + +async function verifyDesktopAccess() { + const module = {}, notifications = [], calls = [], grants = new Set(); + let change, releasePrompt, watched; + const manager = { + checkAccessTokenSync: (_, permission) => grants.has(permission) ? 0 : -1, + on: (type, permissions, callback) => { assert.equal(type, 'selfPermissionStateChange'); watched = Array.from(permissions); change = callback; }, + off: (type, permissions, callback) => { assert.equal(callback, change); change = undefined; }, + requestPermissionsFromUser: async () => { calls.push('clipboard'); await new Promise(resolve => { releasePrompt = resolve; }); }, + openPermissionOnSetting: async () => { calls.push('keyboard'); throw Error('user dismissed'); } + }; + vm.runInNewContext(ts.transpileModule(fs.readFileSync(process.argv[6], 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } + }).outputText, { exports: module, console: { warn: () => {} }, require: name => { + assert.equal(name, '@kit.AbilityKit'); return { abilityAccessCtrl: { + GrantStatus: { PERMISSION_GRANTED: 0 }, createAtManager: () => manager + } }; + }}); + let selection = 1, blockGuide = false, failSettings = false; + const dialogs = [], routes = []; + const ui = { getPromptAction: () => ({ showDialog: async options => { + calls.push('guide'); dialogs.push(options); + if (blockGuide) return await new Promise(resolve => { releasePrompt = resolve; }); + return { index: selection }; + } }) }; + const access = module.DesktopAccess, context = { + applicationInfo: { accessTokenId: 17, name: 'com.chunkdash.crossdesk' }, + startAbility: async want => { + calls.push('settings'); routes.push(JSON.parse(JSON.stringify(want))); + if (failSettings) throw Error('settings unavailable'); + } + }; + access.start(context, false, e => notifications.push(e)); + access.requestKeyboard(); access.requestClipboard(); access.requestHost(); await access.requests; + assert.equal(calls.length, 0); assert.equal(change, undefined); // Ordinary HAP never opens unusable prompts. + access.stop(); access.start(context, true, e => notifications.push(e)); + access.requestClipboard(); access.requestClipboard(); access.requestKeyboard(); + await new Promise(resolve => setImmediate(resolve)); + assert.deepEqual(calls, ['clipboard']); // Permission UIs must not overlap. + releasePrompt(); await access.requests; + assert.deepEqual(calls, ['clipboard', 'keyboard']); + access.requestKeyboard(); access.requestClipboard(); await access.requests; + assert.equal(calls.length, 2); // Denial/dismissal is not a retry loop. + grants.add('ohos.permission.HOOK_KEY_EVENT'); change({}); + assert.equal(notifications.filter(e => e.op === 'keyboardPermission').at(-1).ok, true); + grants.clear(); change({}); + assert.equal(notifications.filter(e => e.op === 'keyboardPermission').at(-1).ok, false); + const late = change; access.stop(); + const before = notifications.length; late({}); + assert.equal(notifications.length, before); + + calls.length = 0; + const screen = 'ohos.permission.CUSTOM_SCREEN_RECORDING', control = 'ohos.permission.CONTROL_DEVICE'; + blockGuide = true; + access.start(context, false, e => notifications.push(e), true); + assert.deepEqual(watched, [screen, control]); // Host-only builds do not require controller ACLs. + access.requestHost(ui); access.requestHost(ui); access.requestKeyboard(); access.requestClipboard(); + await new Promise(resolve => setImmediate(resolve)); + assert.deepEqual(calls, ['guide']); + assert.match(dialogs.at(-1).message, /屏幕共享/); assert.match(dialogs.at(-1).message, /辅助操作/); + releasePrompt({ index: 1 }); await access.requests; + assert.deepEqual(calls, ['guide', 'settings']); + assert.deepEqual(routes[0], { + bundleName: 'com.huawei.hmos.settings', abilityName: 'com.huawei.hmos.settings.MainAbility', + uri: 'application_info_entry', parameters: { pushParams: 'com.chunkdash.crossdesk' } + }); + const latest = () => notifications.filter(e => e.op === 'hostPermissions').at(-1); + // Navigating to Settings is not a grant. Only system state changes enable access. + assert.equal(latest().controlDevice, false); assert.equal(latest().screenRecording, false); + grants.add(screen); grants.add(control); change({}); + assert.equal(latest().controlDevice, true); assert.equal(latest().screenRecording, true); + access.stop(); access.start(context, false, e => notifications.push(e), true); + access.requestHost(ui); await access.requests; + assert.deepEqual(calls, ['guide', 'settings']); // Existing system grants survive an app restart. + grants.delete(control); change({}); + assert.equal(latest().controlDevice, false); assert.equal(latest().screenRecording, true); + access.stop(); + + // Only explain the missing switch. Declining never launches Settings or loops. + calls.length = 0; blockGuide = false; selection = 0; + access.start(context, false, e => notifications.push(e), true); + access.requestHost(ui); await access.requests; + access.requestHost(ui); await access.requests; + assert.deepEqual(calls, ['guide']); + assert.match(dialogs.at(-1).message, /辅助操作/); assert.doesNotMatch(dialogs.at(-1).message, /屏幕共享/); + assert.equal(latest().controlDevice, false); assert.equal(latest().screenRecording, true); + grants.delete(screen); change({}); assert.equal(latest().screenRecording, false); + access.stop(); + + // A response from an old guide cannot navigate or publish state after restart. + calls.length = 0; blockGuide = true; selection = 1; + access.start(context, false, e => notifications.push(e), true); + access.requestHost(ui); await new Promise(resolve => setImmediate(resolve)); + access.stop(); access.start(context, false, e => notifications.push(e), false); + const beforeCompletion = notifications.length; + releasePrompt({ index: 1 }); await access.requests; + assert.deepEqual(calls, ['guide']); assert.equal(notifications.length, beforeCompletion); + access.stop(); + + // A failed Settings route gives manual guidance without granting access or + // falling back to the system dialog with the misleading navigation path. + calls.length = 0; blockGuide = false; failSettings = true; + access.start(context, false, e => notifications.push(e), true); + access.requestHost(ui); await access.requests; + assert.deepEqual(calls, ['guide', 'settings', 'guide']); + assert.equal(dialogs.at(-1).title, '未能打开应用设置'); + assert.match(dialogs.at(-1).message, /屏幕共享/); assert.match(dialogs.at(-1).message, /辅助操作/); + assert.equal(latest().controlDevice, false); assert.equal(latest().screenRecording, false); + access.stop(); +} + +(async () => { + await verifyClipboardQueue(); + await verifyDesktopAccess(); + await request({ op: 'ime', id: 1, kind: 1, text: 'a😀中', anchor: 1, cursor: 4 }); + assert.deepEqual(selection, ['a😀中', 1, 4]); + for (const [action, code] of [[0, 2017], [3, 2040], [4, 2019], [5, 2038]]) { + handlers.get('handleExtendAction')(action); + assert.deepEqual(take(), [{ op: 'imeKey', id: 1, imeSession: 0, code, control: true }]); + } + handlers.get('moveCursor')(3); + handlers.get('selectByMovement')({ direction: 4 }); + handlers.get('selectByRange')({ start: 1, end: 3 }); + assert.deepEqual(take(), [ + { op: 'imeKey', id: 1, imeSession: 0, code: 2014, shift: false }, + { op: 'imeKey', id: 1, imeSession: 0, code: 2015, shift: true }, + { op: 'imeSelection', id: 1, imeSession: 0, anchor: 1, cursor: 3 } + ]); + handlers.get('insertText')('你好'); + assert.deepEqual(take(), [{ op: 'composition', id: 1, imeSession: 0, text: '你好', commit: true }]); + const oldInsert = handlers.get('insertText'), oldPreview = handlers.get('setPreviewText'); + const oldFinish = handlers.get('finishTextPreview'); + await request({ op: 'ime', id: 1, kind: 1 }); + oldInsert('must not enter the new field'); oldPreview('stale'); oldFinish(); + assert.equal(take().length, 0); + assert.equal(host.preview, ''); + assert.equal(attached, 2); assert.equal(detached, 1); + handlers.get('handleExtendAction')(0); + assert.equal(take().length, 1); + const lateInsert = handlers.get('insertText'); + await request({ op: 'ime', id: 1, kind: 0 }); + lateInsert('late'); handlers.get('deleteLeft')(1); + assert.equal(take().length, 0); + // A native attach may finish after the window is destroyed. No editor or + // callbacks may be installed after release, even with the same window id. + let finishAttach; + editor.attachWithUIContext = async () => { await new Promise(resolve => { finishAttach = resolve; }); }; + host.request({ op: 'ime', id: 2, kind: 1, imeSession: 7 }, target, e => sent.push(e)); + await new Promise(resolve => setImmediate(resolve)); + host.release(2); finishAttach(); await host.queue; + assert.equal(host.id, 0); + editor.attachWithUIContext = async () => {}; + await request({ op: 'ime', id: 2, kind: 1, imeSession: 9 }); + handlers.get('insertText')('current'); + assert.deepEqual(take(), [{ op: 'composition', id: 2, imeSession: 9, text: 'current', commit: true }]); + const currentSelection = selection; + await request({ op: 'ime', id: 2, kind: 2, imeSession: 7, text: 'old update' }); + assert.equal(selection, currentSelection); + host.release(1); await host.queue; // Destroying another window keeps this editor. + assert.equal(host.id, 2); + host.release(2); await host.queue; + assert.equal(host.id, 0); + console.log('Slint IME editing, selection, composition, detached callbacks and clipboard serialization passed'); +})().catch(error => { console.error(error); process.exitCode = 1; }); diff --git a/apps/harmony/tests/verify_hap.py b/apps/harmony/tests/verify_hap.py new file mode 100644 index 00000000..e42ab058 --- /dev/null +++ b/apps/harmony/tests/verify_hap.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +"""Check the packaged app, rather than only the files in the source project.""" +import argparse +import json +import hashlib +import os +from pathlib import Path +import re +import subprocess +import tempfile +import zipfile + +APP = Path(__file__).resolve().parents[1] +parser = argparse.ArgumentParser(description=__doc__) +parser.add_argument('hap', nargs='?', type=Path) +parser.add_argument('--mobile', action='store_true') +parser.add_argument('--bundle-name', default=json.loads((APP / 'AppScope/app.json5').read_text())['app']['bundleName']) +parser.add_argument('--desktop-permissions', action='store_true') +parser.add_argument('--host-permissions', action='store_true') +parser.add_argument('--cursor-position-permission', action='store_true') +args = parser.parse_args() +if args.hap is None: + project = 'mobile-project' if args.mobile else 'slint-poc/desktop-project' + args.hap = APP / '.native' / project / 'entry/build/default/outputs/default/entry-default-unsigned.hap' +sdk = Path(os.environ.get('DEVECO_SDK_HOME', '/Applications/DevEco-Studio.app/Contents/sdk')) +native = sdk / 'default/openharmony/native' +readelf = native / 'llvm/bin/llvm-readelf' +system_libraries = native / 'sysroot/usr/lib/aarch64-linux-ohos' + + +def require(condition, message): + if not condition: + raise SystemExit(message) + + +with zipfile.ZipFile(args.hap) as hap, tempfile.TemporaryDirectory(prefix='crossdesk-hap-') as temporary: + require(hap.testzip() is None, 'Corrupt HAP archive') + manifest = json.loads(hap.read('module.json')) + expected_app = (json.loads((APP / 'AppScope/app.json5').read_text())['app']['bundleName'] + if args.mobile else args.bundle_name) + require(manifest['app']['bundleName'] == expected_app, 'Unexpected bundle name') + require(set(manifest['module']['deviceTypes']) == ({'phone', 'tablet'} if args.mobile else {'2in1'}), 'Unexpected device types') + permissions = {item['name'] for item in manifest['module'].get('requestPermissions', [])} + if args.mobile: + require(permissions == {'ohos.permission.INTERNET', 'ohos.permission.KEEP_BACKGROUND_RUNNING'}, 'Mobile must not declare screen capture or input-injection permissions') + require(all('dataTransfer' in a.get('backgroundModes', []) for a in manifest['module']['abilities']), 'Missing mobile background mode') + else: + expected_permissions = {'ohos.permission.INTERNET', 'ohos.permission.FILE_ACCESS_PERSIST', + 'ohos.permission.PREPARE_APP_TERMINATE'} + for enabled, names in [ + (args.desktop_permissions, ['ohos.permission.HOOK_KEY_EVENT', 'ohos.permission.READ_PASTEBOARD']), + (args.host_permissions, ['ohos.permission.CUSTOM_SCREEN_RECORDING', 'ohos.permission.CONTROL_DEVICE']), + (args.cursor_position_permission, ['ohos.permission.INPUT_MONITORING'])]: + if enabled: + expected_permissions.update(names) + require(permissions == expected_permissions, 'Desktop permissions do not match the selected build options') + require(all(not a.get('backgroundModes') for a in manifest['module']['abilities']), + 'Desktop must not inherit mobile background modes') + arkts = hap.read('ets/modules.abc') + require(len(arkts) > 0, 'Missing compiled ArkTS') + if args.mobile: + require(b'pages/Mobile' in arkts and b'pages/Index' not in arkts and b'NativeSession' not in arkts, + 'Mobile package must contain only the mobile entry and controller') + prefix = 'resources/rawfile/' + require(prefix + 'THIRD_PARTY_NOTICES.txt' not in hap.namelist(), 'Redundant aggregate license file') + for name in ['PRIVACY.md', 'LICENSE', 'OPEN_SOURCE_NOTICE.txt', 'SourceMetadata.json']: + require(len(hap.read(prefix + name)) > 0, 'Missing mobile document: ' + name) + catalog = json.loads(hap.read(prefix + 'ThirdPartyLicenses.json')) + require({'minirtc', 'openh264', 'dav1d', 'openfec', 'ohos-llvm'} <= {c['id'] for c in catalog['components']}, 'Incomplete legal catalog') + for component in catalog['components']: + for document in component['documents']: + require(hashlib.sha256(hap.read(prefix + document['asset'])).hexdigest() == document['sha256'], 'License content mismatch') + metadata = json.loads(hap.read(prefix + 'SourceMetadata.json')) + for document in metadata['buildDocuments']: + require(len(hap.read(prefix + document['asset'])) > 0, 'Missing offline build document') + else: + require(b'pages/Index' in arkts and b'pages/Mobile' not in arkts and b'NativeSession' not in arkts, + 'Desktop must contain only the shared Slint entry') + require('resources/rawfile/crossdesk-icons.ttf' not in hap.namelist(), + 'Unused ArkUI desktop icon font must not be packaged') + libraries = [name for name in hap.namelist() if name.endswith('.so')] + require(all(name.startswith('libs/arm64-v8a/') for name in libraries), 'Unexpected library ABI') + packaged = {Path(name).name for name in libraries} + expected_libraries = ({'libcrossdesk.so', 'libcrossdesk_harmony_core.so', 'libc++_shared.so'} if args.mobile + else {'libslint_desktop.so', 'libslint_cpp.so', 'libc++_shared.so'}) + require(packaged == expected_libraries, 'Missing native library or stale library from a retired entry') + for name in libraries: + data = hap.read(name) + require(data[:6] == b'\x7fELF\x02\x01', f'{name}: expected ELF64 little endian') + require(int.from_bytes(data[18:20], 'little') == 183, f'{name}: expected AArch64') + require(int.from_bytes(data[16:18], 'little') == 3, f'{name}: expected shared object') + path = Path(temporary) / Path(name).name + path.write_bytes(data) + dynamic = subprocess.check_output([readelf, '-d', path], text=True) + for dependency in re.findall(r'\(NEEDED\).*?\[(.*?)\]', dynamic): + # An absolute dependency could exist on this build machine and + # still be impossible to resolve inside the device's app sandbox. + require(dependency and '/' not in dependency and '\\' not in dependency, + f'{name}: runtime dependency must be a library name, not a path: {dependency}') + require(dependency in packaged or (system_libraries / dependency).is_file(), + f'{name}: dependency is neither packaged nor in the HarmonyOS SDK: {dependency}') + if args.mobile: + require(dependency not in {'libnative_avscreen_capture.so', 'libohinput.so', 'libnative_image.so'}, + f'{name}: unexpected PC capture/input dependency: {dependency}') + if args.mobile and path.name == 'libcrossdesk_harmony_core.so': + symbols = subprocess.check_output([readelf, '--dyn-syms', '--demangle', '--wide', path], text=True) + for api in ['Command(', 'Poll(', 'Shutdown(', 'Capture::', 'Input::']: + require('crossdesk::harmony::' + api not in symbols, 'Mobile core exports PC API: ' + api) + for api in ['MobileCommand(', 'MobilePoll(', 'MobileShutdown(']: + require('crossdesk::harmony::' + api in symbols, 'Missing mobile bridge API: ' + api) + if not args.mobile and path.name == 'libslint_desktop.so': + symbols = subprocess.check_output([readelf, '--dyn-syms', '--demangle', '--wide', path], text=True) + for api in ['Renderer::', 'MobileCommand(', 'MobilePoll(']: + require('crossdesk::harmony::' + api not in symbols, 'Desktop bundles unused mobile API: ' + api) + sonames = re.findall(r'\(SONAME\).*?\[(.*?)\]', dynamic) + require(sonames == [path.name], f'{name}: expected SONAME {path.name}, got {sonames}') + for search_path in re.findall(r'\((?:RUNPATH|RPATH)\).*?\[(.*?)\]', dynamic): + require(all(not item.startswith('/') for item in search_path.split(':')), + f'{name}: absolute runtime search path: {search_path}') + print(f'HAP verified: {args.hap}') + print(f'{"phone / tablet controller only" if args.mobile else "2in1 shared Slint desktop"}, compiled ArkTS, {len(libraries)} AArch64 libraries, resolved dependencies') diff --git a/deps/submodules/minirtc b/deps/submodules/minirtc index f5b53e92..18f12c34 160000 --- a/deps/submodules/minirtc +++ b/deps/submodules/minirtc @@ -1 +1 @@ -Subproject commit f5b53e920db0713b456e23af6a0fe9907d94a612 +Subproject commit 18f12c342de38066d36e6743c8e405470027d689 diff --git a/docs/BUILD.md b/docs/BUILD.md index 36b53ad2..89ed9117 100644 --- a/docs/BUILD.md +++ b/docs/BUILD.md @@ -9,6 +9,8 @@ - Git、[Xmake](https://xmake.io/guide/quick-start.html)(桌面端和移动端均要求 3.1.1 或更新版本)及下文对应平台的编译工具链。 - 构建脚本通过 Xmake 管理 **CMake 3.x(≥ 3.21、< 4.0)**、**Rust 1.92.0** 等依赖;首次构建需要下载工具与源码。具体约束见 [Slint 包配置](../deps/thirdparty/slint/xmake.lua)。 +鸿蒙 PC 的 Slint 移植直接编译桌面公共界面和业务代码,见 [Slint 桌面构建与一致性验收](../apps/harmony/desktop-port/README.md)(API 26)。独立 ArkUI 手机 / 平板工程见 [HarmonyOS 构建与真机验证](../apps/harmony/README.md)。请使用各自说明中的脚本,不要通过桌面根工程切换鸿蒙配置。 + ## Windows 安装 Visual Studio 2022 的“使用 C++ 的桌面开发”工作负载、Windows SDK 和“适用于 Windows 的 C++ Clang 编译器”组件,并安装 Xmake。Windows x64 的 libyuv 使用 clang-cl 编译,以启用 SIMD 图像转换和缩放;主程序仍使用 MSVC。在开发者 PowerShell 中执行: diff --git a/docs/BUILD_EN.md b/docs/BUILD_EN.md index da9fcee5..7a7c21cf 100644 --- a/docs/BUILD_EN.md +++ b/docs/BUILD_EN.md @@ -9,6 +9,8 @@ After cloning, run build and packaging commands from the repository root. The [X - Git, [Xmake](https://xmake.io/guide/quick-start.html) (3.1.1 or newer for both desktop and mobile), and the platform toolchain listed below. - The build scripts use Xmake to manage **CMake 3.x (≥ 3.21, < 4.0)**, **Rust 1.92.0**, and other dependencies. The first build downloads tools and source archives. See the [Slint package configuration](../deps/thirdparty/slint/xmake.lua) for the constraints. +The HarmonyOS Slint desktop port compiles the shared desktop UI and application logic; see its [build and parity validation guide (Chinese)](../apps/harmony/desktop-port/README.md) (API 26). The separate ArkUI phone / tablet project is documented in the [HarmonyOS guide (Chinese)](../apps/harmony/README.md). Use the respective scripts instead of reconfiguring the desktop root project. + ## Windows Install Visual Studio 2022 with the **Desktop development with C++** workload, Windows SDK, and **C++ Clang Compiler for Windows** component, plus Xmake. Windows x64 builds use clang-cl for libyuv to enable SIMD conversion and scaling; the application still uses MSVC. In a developer PowerShell, run: diff --git a/libs/wire/include/remote_action.h b/libs/wire/include/remote_action.h index 4693b60a..6b6916cc 100644 --- a/libs/wire/include/remote_action.h +++ b/libs/wire/include/remote_action.h @@ -116,7 +116,7 @@ struct CursorState { CursorHiddenReason hidden_reason; }; -enum class HostPlatform { Unknown, Windows, MacOS, Linux, IOS, Android }; +enum class HostPlatform { Unknown, Windows, MacOS, Linux, IOS, Android, HarmonyOS }; const char* HostPlatformName(HostPlatform platform); HostPlatform ParseHostPlatform(const std::string& platform); diff --git a/libs/wire/src/remote_action.cpp b/libs/wire/src/remote_action.cpp index 840795f4..ce0d123b 100644 --- a/libs/wire/src/remote_action.cpp +++ b/libs/wire/src/remote_action.cpp @@ -61,6 +61,7 @@ const char* HostPlatformName(HostPlatform platform) { case HostPlatform::Linux: return "linux"; case HostPlatform::IOS: return "ios"; case HostPlatform::Android: return "android"; + case HostPlatform::HarmonyOS: return "harmonyos"; default: return "unknown"; } } @@ -71,6 +72,7 @@ HostPlatform ParseHostPlatform(const std::string& platform) { if (platform == "linux") return HostPlatform::Linux; if (platform == "ios") return HostPlatform::IOS; if (platform == "android") return HostPlatform::Android; + if (platform == "harmonyos") return HostPlatform::HarmonyOS; return HostPlatform::Unknown; } diff --git a/libs/wire/tests/wire_test.cpp b/libs/wire/tests/wire_test.cpp index d0a6165d..3c53f250 100644 --- a/libs/wire/tests/wire_test.cpp +++ b/libs/wire/tests/wire_test.cpp @@ -166,7 +166,7 @@ int main() { // The platform is optional metadata, not a requirement for connecting. for (const auto& platform : std::vector{ - "windows", "linux", "macos", "ios", "android", "unknown"}) { + "windows", "linux", "macos", "ios", "android", "harmonyos", "unknown"}) { advertised = crossdesk::MakeHostInformation( "Host", {}, true, "1.5.3-20260928", crossdesk::ParseHostPlatform(platform)); diff --git a/xmake.lua b/xmake.lua index d0871b74..36aea904 100644 --- a/xmake.lua +++ b/xmake.lua @@ -4,7 +4,7 @@ set_license("GPL-3.0-only") local source_dir = os.getenv("CROSSDESK_SOURCE_DIR") or os.scriptdir() local desktop_platform = is_plat("windows", "macosx", "linux") or - (not is_plat("iphoneos", "android") and is_host("windows", "macosx", "linux")) + (not is_plat("iphoneos", "android", "harmony") and is_host("windows", "macosx", "linux")) add_rules("mode.release", "mode.debug") set_languages("c++17")