[feat] keep mobile sessions connected in background

This commit is contained in:
dijunkun
2026-10-10 00:29:07 +08:00
parent be639eaf97
commit 1dccbe42a3
17 changed files with 693 additions and 37 deletions
+8 -4
View File
@@ -1,12 +1,12 @@
# CrossDesk Privacy Policy
Last updated: October 2, 2026
Last updated: October 10, 2026
[中文](#中文) | [English](#english)
## 中文
更新日期:2026 年 10 月 2 日
更新日期:2026 年 10 月 10 日
CrossDesk 重视您的个人信息与隐私保护。本政策说明我们在提供远程桌面服务时如何处理相关信息,以及您可以如何管理授权、保存的内容和数据请求。请在使用前阅读并了解本政策。
@@ -60,7 +60,9 @@ Android 默认信令连接同样进行 TLS 证书校验。
**系统权限与会话功能。** iOS 客户端作为控制端接收远程电脑的画面与音频,不采集本机屏幕、摄像头或麦克风。局域网访问权限用于连接本地网络中的设备,您可在系统设置中管理。发送文件通过系统文件选择器选择,不要求开放整个文件库。当前 iOS 界面不读取或发送本机剪贴板文本;远端发送的剪贴板文本可在会话中写入本机剪贴板。
Android 客户端同样作为控制端接收远程画面与音频,不采集本机屏幕、摄像头或麦克风,当前不支持文件传输。剪贴板文本仅在您主动发送或复制远端文本时传输或写入本机,会话结束时清除应用内缓存的远端剪贴板文本。
Android 客户端同样作为控制端接收远程画面与音频,不采集本机屏幕、摄像头或麦克风。发送及保存文件通过系统文件选择器完成,不要求开放整个文件库。剪贴板文本仅在您主动发送或复制远端文本时传输或写入本机,会话结束时清除应用内缓存的远端剪贴板文本。
Android 远程会话优先使用系统画中画在切换应用后继续显示画面。画中画不可用时,通过前台服务和连接通知保持您已建立的远程连接;后台暂停本地画面渲染和音频播放,不申请悬浮窗权限。通知权限用于提供返回及断开入口,CPU 唤醒锁用于会话后台期间维持连接,返回前台或会话结束时释放。您可以在应用或连接通知中断开,移除应用任务也会结束会话及保活服务。
应用及通信组件使用本地文件元数据、应用偏好和计时接口实现文件传输、设置保存及网络计时,不使用这些信息制作设备指纹或进行广告追踪。
@@ -138,7 +140,9 @@ Default signaling connections on Android also validate TLS certificates.
**System permissions and session features.** The iOS client receives remote computer screens and audio as a controller; it does not capture the local screen, camera or microphone. Local network access supports connections to devices on your network and can be managed in system settings. File sending uses the system document picker without requiring access to the entire file library. The current iOS interface does not read or send local clipboard text. Text sent by the remote device may be written to the local clipboard during a session.
The Android client also receives remote screens and audio as a controller and does not capture the local screen, camera or microphone. It currently does not support file transfer. Clipboard text is transmitted or written locally only when you explicitly send text or copy remote text. The app clears cached remote clipboard text when the session ends.
The Android client also receives remote screens and audio as a controller and does not capture the local screen, camera or microphone. Sending and saving files uses the system document picker without requiring access to the entire file library. Clipboard text is transmitted or written locally only when you explicitly send text or copy remote text. The app clears cached remote clipboard text when the session ends.
Android remote sessions prefer system picture-in-picture to continue displaying video when you switch apps. If picture-in-picture is unavailable, a foreground service and connection notification maintain the remote connection you have established. Local video rendering and audio playback pause in the background; no overlay permission is requested. Notification permission provides return and disconnect actions. A CPU wake lock helps maintain the connection while the session is backgrounded and is released on foreground return or session end. You can disconnect in the app or its connection notification; removing the app task also ends the session and its foreground service.
The app and its communication components use local file metadata, preferences and timing APIs for file transfer, settings and network timing, not for device fingerprinting or advertising tracking.
+21 -3
View File
@@ -4,7 +4,7 @@
## 当前功能
- 官方或自托管 WSS 信令服务器;授权后进入前台即自动注册或登录本机身份,首页实时显示连接、重连及失败状态。发起桌面连接时复用该身份;切换服务器会重新登录,撤回授权或进入后台会关闭连接。
- 官方或自托管 WSS 信令服务器;授权后进入前台即自动注册或登录本机身份,首页实时显示连接、重连及失败状态。发起桌面连接时复用该身份;切换服务器会重新登录,撤回授权会关闭连接。已有远程会话优先通过系统画中画保持连接,画中画不可用时由带连接通知的前台服务兜底;首页空闲信令在后台关闭。
- 优先使用 MediaCodec 硬件解码 H.264 / AV1,硬件缓冲区直接提交 Surface;设备无对应硬件、分辨率超限或运行失败时,按原编码格式回退到 OpenH264 / dav1d。软件输出由独立显示线程将 NV12 转 RGBA;只保留最新待显示帧。MiniRTC 原有的通用压缩帧解码队列保持不变。
- 48 kHz 单声道 PCM 播放,支持静音,音频队列有容量限制。
- 相对鼠标(触控板)和绝对位置模式;单指点击、绝对模式按住拖动、长按右键,以及外接键盘、鼠标。画面手势与 iOS 一致:双指捏合以触点为中心缩放(1–10 倍),放大后双指拖动画面,原始比例下双指上下滑动控制滚轮。
@@ -14,7 +14,7 @@
- 可拖动的悬浮控制球、三列控制菜单和浮动 ASCII / 电脑键盘、常用快捷键;通过剪贴板发送中文等文本,再手动执行远端粘贴。
- 悬浮菜单中的“虚拟鼠标”展开 140 × 48 dp 横向左键/滚轮/右键条,跟随悬浮球移动,可随时收起,不压缩远程画面。按住左键或右键后,同一根手指拖动即可拖拽,滑出按键范围仍继续,松开结束;也支持另一指在画面上移动光标。采用无文字的扁平分体按键,中间滚轮外观为 18 × 30 dp,三键均保留 44 × 44 dp 触控范围,通过留白分隔。无阴影、凹凸或按压位移,按压仅变色。等色横条随手指上下循环移动,松手停止;上下滑动可超出按键范围,在画面放大后仍控制远端滚动。切换模式、显示器、进入后台或取消触摸会释放按键。状态菜单按内容高度完整展开,只有安全区域空间不足时才滚动。
- 悬浮控制栏支持选择并发送文件、接收电脑端文件、查看确认进度及保存收到的文件;Ctrl+Alt+Del 使用远端 Windows 系统服务,未启用服务时显示提示。
- 设备身份与选择保存的连接密码使用 Android Keystore / AES-GCM 加密;首次联网需同意,后台切换会结束会话。
- 设备身份与选择保存的连接密码使用 Android Keystore / AES-GCM 加密;首次联网需同意。远程会话在切换应用时自动使用系统画中画,不可用时自动使用通知保活。
- 与 iOS 一致的首页卡片、顶部状态/公告/设置入口、密码弹层及分组设置页;最近连接按服务器隔离,长按可删除记录及密码。
- 平板首页支持横竖屏和分屏,按当前可用宽度自动适配:900 dp 起连接与最近连接并排,较窄窗口纵向排列;最近连接随宽度显示 1–4 列,手机常规宽度仍为两列。首页最大宽度为 1200 dp,设置、公告及阅读页最大宽度为 720 dp,宽窗口中的密码和隐私弹窗居中限宽。旋转或调整分屏直接重排现有页面,保留输入、设置编辑和当前会话;手机全屏首页保持竖屏。
- 页面转场沿用 iOS 的交互方式:设置、隐私、关于及公告页面进入时从右滑入,返回时反向滑出,背景页面轻微跟随;进入和退出远程会话使用 0.2 秒淡入淡出。遵循 Android 系统动画开关及速度设置。
@@ -155,7 +155,21 @@ gh secret set ANDROID_KEY_PASSWORD --repo kunkundi/crossdesk
“发送文件”调用系统文件选择器,后台读取所选文件后使用桌面端相同的 `file` / `file_feedback` 协议传输,支持空文件和中文文件名。每次发送一个文件,以 64 KiB 分块、最多 1 MiB 未确认数据限制发送队列;进度来自接收端确认,30 秒没有有效进展则报告失败。接收时校验大小并追踪完整字节范围,允许跨线程回调乱序,未完成的文件不会提供保存入口。同名文件分开存储,不覆盖已有文件;可通过“保存收到的文件”导出最近收到的文件,连续接收多个文件时应逐个保存。
选取或保存文件期间保留会话,取消选择不会断开;文件选择器停留超过 5 分钟且应用仍不在前台时结束会话。其他进入后台的操作仍结束会话。发送副本和收到的文件暂存于应用私有缓存,结束会话后清除,需要保留的接收文件应在断开前导出;不申请全盘存储权限。
选取或保存文件期间保留会话并暂时禁用自动画中画,取消选择不会断开;停留超过 5 分钟且应用仍不在前台时结束会话。发送副本和收到的文件暂存于应用私有缓存,结束会话后清除,需要保留的接收文件应在断开前导出;不申请全盘存储权限。
## 后台连接与画中画
与 iOS 一样,通过系统画中画显示真实远程视频并维持会话。连接成功且已显示远程画面后,Android 12 及以上启用系统自动画中画;Android 8–11 在离开应用时尝试进入。设备需支持画中画,系统需允许 CrossDesk 使用该功能。未连接、尚无画面或正在打开系统文件选择器时不自动进入。
Android 版本满足要求不代表固件提供画中画,部分低内存模式设备会禁用该功能。以系统的 `FEATURE_PICTURE_IN_PICTURE` 检测为准,单独允许画中画权限不能补齐系统能力。无系统画中画或权限被禁止时,连接建立后自动启动前台服务兜底;画中画进入失败也尝试使用该兜底。控制菜单不提供“画中画”或“后台保活”按钮,两者均在切换应用时自动处理。
画中画仅显示完整远程画面,隐藏本地光标、悬浮球、鼠标条和键盘,并释放远程按键。使用原有视频 Surface 和 RTC 连接,进入小窗时不会关闭音视频;通过系统小窗返回全屏继续操作。画面比例根据远端尺寸更新,超宽或超长画面在系统允许的比例范围内完整显示并留黑边。
正常画中画不启动兜底服务,可见的小窗使 Activity 保持在 started 状态,进入时仅暂停交互。关闭小窗会结束会话,不自动转为隐藏的后台连接。
兜底模式通过前台服务保留原 RTC/信令连接,并显示带“返回”和“断开”操作的常驻通知。后台停止本地画面渲染及音频播放,不创建悬浮窗;视频传输和解码不做协议层暂停。返回应用后重新绑定 Surface 并请求关键帧,恢复原静音状态、显示器和画面设置,不重新登录或连接远端。会话状态独立于 Activity,页面被系统回收后可恢复,整个进程被系统终止则不能恢复原连接。
服务仅用于用户主动建立且已成功连接的远程会话,不用于首页待机。Android 14 及以上使用 `specialUse` 前台服务类型并声明远程控制连接用途;Android 13 及以上在兜底首次启用时请求通知权限,拒绝后系统仅在活动应用管理中显示服务。CPU 部分唤醒锁仅在兜底会话后台期间持有,返回前台、断开、移除任务或服务销毁时释放。通知断开、远端断开、移除任务、撤回授权和强制停止均结束会话及服务,不自动重连。系统文件选择器仍有上述五分钟限制;系统强杀或网络中断仍可能结束连接。
首页设置里的“画面偏好”保存在本机,作为下一次连接的默认偏好;会话内画面设置与 iOS 一样只影响当前连接。初始质量按屏幕长边选择低(不超过 1280 像素)或中,采集帧率为 30 fps,电脑端报告支持后自动下发;旧版电脑未报告支持时禁用控件。网络页的 FPS 统计实际提交给 Android Surface 的帧,画面延时使用 MiniRTC 校准后的采集时间至提交时刻的估计值,未完成校准时显示“—”;RTT 使用 0.25 权重平滑,3 秒未更新的数据会过期。切换屏幕清除旧画面的帧率、分辨率和画面延时,保留网络统计。
@@ -210,6 +224,10 @@ UI 验证统一由人工完成,不维护页面导航、布局、动画、悬
控制栏联调需在真实远程会话中验证文件选择及取消、收发文件内容一致性、接收后导出、传输中断线;Windows 远端分别检查系统服务开启与关闭时的 Ctrl+Alt+Del 行为。
画中画需在真实会话中检查切换应用自动进入、其他应用前台时画面持续更新、返回全屏、关闭小窗、静音、超宽及竖屏画面、文件选择器取消、系统禁用画中画和锁屏。
前台服务兜底需在无原生画中画的实机上验证后台往返、Activity 重建时复用会话、通知断开、旧通知隔离、远端断开时释放唤醒锁,以及实际网络和画面恢复。设备测试前后应备份、核对并恢复服务器等应用设置,避免中断的测试留下本地测试地址。
## MediaCodec 编解码
实现参照 MiniRTC 的 NVCodec 分层:`src/media/mediacodec/` 负责硬件后端、能力查询和 JNI/Surface 资源;`src/media/video/encode/mediacodec/` 与 `src/media/video/decode/mediacodec/` 分别接入编码器、解码器接口,并负责同编码格式的软件回退。Android 工程直接编译这些源码。通用 API 仅需启用 `hardware_acceleration` 和 `native_video_output`。JavaVM 由 Android JNI 适配层在 `JNI_OnLoad` 中设置到后端私有实现;该桥接不安装为 SDK 头文件,也不导出为动态库符号。当前能力查询和 SurfaceTexture 创建依赖 JNI,缺少 JavaVM 时仍可使用软件编解码。独立输出线程在画面停止更新后仍会取出最后一帧,空闲时休眠。
@@ -2,12 +2,23 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_SPECIAL_USE" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<application android:theme="@style/AppTheme" android:label="CrossDesk"
android:icon="@mipmap/ic_launcher" android:allowBackup="false"
android:fullBackupContent="false"
android:dataExtractionRules="@xml/data_extraction_rules"
android:supportsRtl="true" android:usesCleartextTraffic="false">
<service android:name=".RemoteSessionService" android:exported="false"
android:foregroundServiceType="specialUse">
<property android:name="android.app.PROPERTY_SPECIAL_USE_FGS_SUBTYPE"
android:value="Maintain a user-started remote desktop RTC/control session when picture-in-picture is unavailable. Background rendering and audio are suspended; the user can return or disconnect from the notification." />
</service>
<activity android:name=".MainActivity" android:exported="true"
android:launchMode="singleTop"
android:supportsPictureInPicture="true"
android:configChanges="orientation|screenSize|smallestScreenSize|screenLayout|keyboard|keyboardHidden"
android:resizeableActivity="true"
android:windowSoftInputMode="adjustResize">
@@ -0,0 +1,85 @@
package cn.crossdesk.mobile;
import android.content.Context;
import org.json.JSONArray;
import org.json.JSONObject;
import java.io.File;
import java.util.UUID;
/** Main-thread session state; never retains a stopped Activity or its Views. */
final class BackgroundSession implements NativeSession.Listener {
final Context context;
final NativeSession session;
final String token = UUID.randomUUID().toString();
final String host, remote;
final int port;
NativeSession.Listener view;
String identity = "", platform = "", clipboard = "", fileStatus = "", endReason = "";
JSONArray displays = new JSONArray();
RemoteVideoSettings videoSettings;
int signalState, selectedDisplay, width, height;
boolean muted, remoteServiceAvailable, background, ended;
File receivedFile;
BackgroundSession(Context context, NativeSession session, String host, int port, String remote) {
this.context = context.getApplicationContext();
this.session = session; this.host = host; this.port = port; this.remote = remote;
}
void attach(NativeSession.Listener listener) { view = listener; }
void detach(NativeSession.Listener listener) { if (view == listener) view = null; }
void setBackground(boolean value) {
if (background == value || ended) return;
background = value;
// With no visible window, release rendering buffers and stop remote
// audio. The RTC/control connection and file transfers remain alive.
if (value) session.surface(null);
session.control(2, value || muted ? 0 : 1);
RemoteSessionService.refresh(this);
}
void finish(String reason) {
if (ended) return;
ended = true; endReason = reason; clipboard = "";
session.close();
RemoteSessionService.release(context, this);
NativeSession.Listener listener = view; view = null;
if (listener != null) listener.ended(reason);
}
@Override public void ended(String reason) { finish(reason); }
@Override public void passwordRejected() { finish("远端访问密码已失效,请重新连接"); }
@Override public void connected() { /* Already connected before the service takes ownership. */ }
@Override public void status(String message) { if (view != null) view.status(message); }
@Override public void signaling(int state, String id) {
signalState = state; identity = id;
if (view != null) view.signaling(state, id);
}
@Override public void data(JSONObject message) {
if (view != null) { view.data(message); return; }
if (message.optInt("type", -1) == 12 && videoSettings != null) videoSettings.receive(message);
if (message.optInt("type", -1) == 5) {
JSONObject service = message.optJSONObject("service_status");
remoteServiceAvailable = service != null && service.optBoolean("available", false);
}
JSONObject info = message.optJSONObject("host_info");
if (info != null) {
platform = info.optString("platform", "");
JSONArray values = info.optJSONArray("displays"); if (values != null) displays = values;
if (videoSettings != null) videoSettings.setSupported(info.optBoolean("supports_video_settings", false));
}
}
@Override public void videoSize(int w, int h) {
width = w; height = h;
if (view != null) view.videoSize(w, h);
}
@Override public void clipboard(String text) { clipboard = text; if (view != null) view.clipboard(text); }
@Override public void statistics(RemoteNetworkStatistics.Snapshot value) { if (view != null) view.statistics(value); }
@Override public void fileTransfer(String name, double progress, boolean sending, File received) {
if (received != null) receivedFile = received;
fileStatus = progress < 0 ? name + " 传输失败" : (sending ? "发送 " : "接收 ") + name
+ (progress >= 1 ? " · 已完成" : " · " + (int)(progress * 100) + "%");
if (view != null) view.fileTransfer(name, progress, sending, received);
}
@Override public void controlFeedback(String text) { if (view != null) view.controlFeedback(text); }
@Override public void presence(JSONObject message) { if (view != null) view.presence(message); }
@Override public void announcement(JSONObject message) { if (view != null) view.announcement(message); }
@Override public void announcementFailed(String id) { if (view != null) view.announcementFailed(id); }
}
@@ -66,10 +66,17 @@ public final class MainActivity extends Activity implements NativeSession.Listen
private String signalingServer="", localIdentity="", signalError="";
private int signalState=-1;
private boolean foreground;
private boolean pictureInPictureUi;
private boolean enteredPictureInPicture;
private BackgroundSession backgroundSession;
private int videoWidth,videoHeight;
private final RemotePictureInPicture pictureInPicture=new RemotePictureInPicture();
private RemoteVideoView video;
private SessionControls controls;
private Dialog passwordDialog, progressDialog, privacyDialog;
private AlertDialog disconnectDialog;
private AlertDialog connectionDialog;
private String pendingConnectionMessage="";
private AlertDialog remoteUpdateDialog;
private RemoteVersionCheck remoteVersionCheck;
private String host, connectedRemote="", hostPlatform="", remoteClipboard="", pendingPassword="", page="home";
@@ -217,7 +224,15 @@ public final class MainActivity extends Activity implements NativeSession.Listen
});renderRecentConnections();
}
ensureSignaling();
if(!message.isEmpty())new AlertDialog.Builder(this).setTitle("连接提示").setMessage(message).setPositiveButton("确定",null).show();
if(!message.isEmpty())pendingConnectionMessage=message;
showConnectionMessage();
}
private void showConnectionMessage(){
if(pendingConnectionMessage.isEmpty()||!foreground||isFinishing()||isDestroyed())return;
if(connectionDialog!=null)connectionDialog.dismiss();
connectionDialog=new AlertDialog.Builder(this).setTitle("连接提示").setMessage(pendingConnectionMessage).setPositiveButton("确定",null).create();
connectionDialog.setOnDismissListener(dialog->connectionDialog=null);
pendingConnectionMessage="";connectionDialog.show();
}
private String id(){return remote.getText().toString().replace(" ","");}
static String formatId(String value){String digits=value.replaceAll("[^0-9]","");if(digits.length()>9)digits=digits.substring(0,9);return digits.replaceAll("(.{3})(?=.)","$1 ");}
@@ -283,6 +298,8 @@ public final class MainActivity extends Activity implements NativeSession.Listen
window.setGravity(width>=600?Gravity.CENTER:Gravity.BOTTOM);
}
private void begin(String id,String secret){
videoWidth=videoHeight=0;enteredPictureInPicture=false;
pictureInPicture.disconnect();updatePictureInPicture();
remoteServiceAvailable=false;receivedFile=null;
resetRemoteVersionCheck();
resetVideoSettings();
@@ -307,17 +324,26 @@ public final class MainActivity extends Activity implements NativeSession.Listen
public void mute(){muted=!muted;if(session!=null)session.control(2,muted?0:1);controls.setMuted(muted);}
public void mouse(){boolean relative=!preferences.getBoolean("relativeMouse",true);preferences.edit().putBoolean("relativeMouse",relative).apply();video.setRelative(relative);controls.setRelative(relative);}
public void videoSettings(int field,int value){RemoteVideoSettings.Values old=MainActivity.this.videoSettings.selection;updateVideoSettings(new RemoteVideoSettings.Values(field==0?value:old.quality,field==1?value:old.frameRate,field==2?value:old.preference));}
});controls.setRelative(preferences.getBoolean("relativeMouse",true));controls.videoSettings(videoSettings);root.addView(controls,new LinearLayout.LayoutParams(-1,-1));getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
});controls.setRelative(preferences.getBoolean("relativeMouse",true));controls.setMuted(muted);controls.videoSettings(videoSettings);root.addView(controls,new LinearLayout.LayoutParams(-1,-1));getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
controls.attachMouseBar(canvas,new VirtualMouseBar(ui,video.mouseInput));
video.addOnLayoutChangeListener((view,left,top,right,bottom,oldLeft,oldTop,oldRight,oldBottom)->{
if(!pictureInPictureUi&&!isInPictureInPictureMode())updatePictureInPicture();
});
pictureInPicture.connected();updatePictureInPicture();
}
private void confirmDisconnect(){disconnectDialog=new AlertDialog.Builder(this).setTitle("断开远程连接?").setMessage("断开后将返回首页").setNegativeButton("取消",null).setPositiveButton("断开连接",(d,w)->end("")).show();}
private void end(String reason){
if(backgroundSession!=null){BackgroundSession retained=backgroundSession;backgroundSession=null;retained.detach(this);retained.finish(reason);}
boolean closeWindow=isInPictureInPictureMode();
pictureInPicture.disconnect();updatePictureInPicture();
if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();
mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;receivedFile=null;remoteServiceAvailable=false;
resetRemoteVersionCheck();
resetVideoSettings();
previewCapture=null;previewCopying=false;
if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}if(session!=null){releaseKeys();session.close();session=null;}stopSignaling();pendingPassword="";remoteClipboard="";hostPlatform="";displays=new JSONArray();ready=false;video=null;controls=null;getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);home(reason);
if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}if(session!=null){releaseKeys();session.close();session=null;}stopSignaling();pendingPassword="";remoteClipboard="";hostPlatform="";displays=new JSONArray();ready=false;video=null;controls=null;getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
if(closeWindow){finish();return;}
home(reason);
appUpdates.checkNow();
}
private void settings(){
@@ -409,7 +435,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
@Override public void announcementFailed(String requestId){announcementInbox.failed(requestId);}
private void privacyPage(){
LinearLayout content=formPage("privacy","隐私与授权",this::settings);LinearLayout card=section(content,"");boolean consent=preferences.getBoolean("networkConsent",false);link(card,"授权状态",consent?"已授权":"未授权",()->{});ui.divider(card);link(card,"隐私政策","",this::privacyPolicyPage);
footer(content,"设备身份及选择保存的访问密码使用 Android Keystore 加密。连接记录仅保存在本机,不纳入设备备份。剪贴板内容在结束会话时清除。离开应用会结束当前会话。");
footer(content,"设备身份及选择保存的访问密码使用 Android Keystore 加密。连接记录仅保存在本机,不纳入设备备份。剪贴板内容在结束会话时清除。离开应用后通过画中画或连接通知保持会话,可在应用或通知中断开。");
if(consent)content.addView(ui.action("撤回联网授权",RED,()->new AlertDialog.Builder(this).setTitle("撤回联网授权?").setMessage("撤回后将停止远程连接、关闭画面预览保存并清除已有预览。连接记录和密码会保留,重新连接前需再次授权。").setNegativeButton("取消",null).setPositiveButton("撤回",(d,w)->{preferences.edit().putBoolean("networkConsent",false).apply();appUpdates.setEnabled(false);previews.setEnabled(false);stopSignaling();announcementInbox.reset();privacyPage();}).show()));else content.addView(ui.primary("阅读并授权",()->consent(this::privacyPage)));
}
private void privacyPolicyPage(){
@@ -460,14 +486,16 @@ public final class MainActivity extends Activity implements NativeSession.Listen
private void openDocument(Intent intent,int request){
if(documentSession!=null)return;
documentSession=session;releaseKeys();
pictureInPicture.choosingDocument(true);updatePictureInPicture();
try{startActivityForResult(intent,request);mainHandler.postDelayed(documentTimeout,5*60*1000);}
catch(android.content.ActivityNotFoundException error){documentSession=null;documentToSave=null;controlFeedback("此设备没有可用的文件选择器");}
catch(android.content.ActivityNotFoundException error){documentSession=null;documentToSave=null;pictureInPicture.choosingDocument(false);updatePictureInPicture();controlFeedback("此设备没有可用的文件选择器");}
}
@Override protected void onActivityResult(int request,int result,Intent data){
super.onActivityResult(request,result,data);
if(request!=PICK_FILE&&request!=SAVE_FILE)return;
NativeSession source=documentSession;java.io.File saved=documentToSave;
documentSession=null;documentToSave=null;mainHandler.removeCallbacks(documentTimeout);
pictureInPicture.choosingDocument(false);updatePictureInPicture();
if(source==null||source!=session||!ready||result!=RESULT_OK||data==null||data.getData()==null)return;
if(request==PICK_FILE)source.sendFile(data.getData());else if(saved!=null)source.saveFile(saved,data.getData());
}
@@ -515,6 +543,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
renderSignaling();signaling.start();
}
private void stopSignaling(){
if(backgroundSession!=null&&backgroundSession.session==signaling){BackgroundSession retained=backgroundSession;backgroundSession=null;retained.detach(this);retained.finish("");}
mainHandler.removeCallbacks(presenceTick);recentPresence.setConnected(false,android.os.SystemClock.elapsedRealtime());
announcementInbox.setConnected(false);
if(signaling!=null)signaling.close();
@@ -550,6 +579,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
// The first success consumes pendingPassword; a duplicate must not save it again.
if(session==null||ready)return;ready=true;if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}
try{history.save(serverKey(),connectedRemote,connectedRemote,hostPlatform,rememberPassword,pendingPassword);}catch(Exception error){toast("密码保存失败,本次连接仍可使用");}pendingPassword="";sessionScreen();
if(!canUsePictureInPicture())prepareBackgroundSession(true);
if(videoSettings.supported&&videoSettings.requestId==0)updateVideoSettings(videoSettings.selection);
if(foreground&&preferences.getBoolean("networkConsent",false))remoteVersionCheck.connected();
}
@@ -566,7 +596,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
}
private void cancelVideoSettingsTimeout(){if(videoSettingsTimeout!=null)mainHandler.removeCallbacks(videoSettingsTimeout);videoSettingsTimeout=null;}
private void showRemoteUpdate(){
if(!ready||session==null||!foreground||isFinishing()||!preferences.getBoolean("networkConsent",false))return;
if(!ready||session==null||!foreground||pictureInPictureUi||isInPictureInPictureMode()||isFinishing()||!preferences.getBoolean("networkConsent",false))return;
remoteUpdateDialog=new AlertDialog.Builder(this).setTitle("升级提示").setMessage(RemoteVersionCheck.NOTICE).setPositiveButton("知道了",null).create();
remoteUpdateDialog.setOnDismissListener(dialog->remoteUpdateDialog=null);
remoteUpdateDialog.setCanceledOnTouchOutside(false);
@@ -601,11 +631,11 @@ public final class MainActivity extends Activity implements NativeSession.Listen
if(foreground&&preferences.getBoolean("networkConsent",false))remoteVersionCheck.hostInfo(info);
}
}
@Override public void videoSize(int width,int height){if(video!=null)video.videoSize(width,height);if(controls!=null)controls.videoSize(width,height);}
@Override public void videoSize(int width,int height){videoWidth=width;videoHeight=height;if(video!=null)video.videoSize(width,height);if(controls!=null)controls.videoSize(width,height);pictureInPicture.videoSize(width,height);updatePictureInPicture();}
@Override public void clipboard(String value){remoteClipboard=value;}
@Override public void statistics(RemoteNetworkStatistics.Snapshot value){if(controls!=null)controls.statistics(value);if(value.fps>0)capturePreviewIfNeeded();}
@Override public boolean dispatchKeyEvent(KeyEvent event){
if(ready&&session!=null&&hasWindowFocus()&&!(getCurrentFocus() instanceof EditText)&&event.getKeyCode()!=KeyEvent.KEYCODE_BACK){int code=KeyMap.windowsCode(event.getKeyCode());if(code!=0){boolean down=event.getAction()==KeyEvent.ACTION_DOWN;if(down)pressed.add(code);else pressed.remove(code);session.key(code,down);return true;}}return super.dispatchKeyEvent(event);
if(ready&&session!=null&&!pictureInPictureUi&&hasWindowFocus()&&!(getCurrentFocus() instanceof EditText)&&event.getKeyCode()!=KeyEvent.KEYCODE_BACK){int code=KeyMap.windowsCode(event.getKeyCode());if(code!=0){boolean down=event.getAction()==KeyEvent.ACTION_DOWN;if(down)pressed.add(code);else pressed.remove(code);session.key(code,down);return true;}}return super.dispatchKeyEvent(event);
}
private void releaseKeys(){if(session!=null)for(int code:pressed)session.key(code,false);pressed.clear();}
private void navigateBack(){if(backAction==null)return;navigatingBack=true;try{backAction.run();}finally{navigatingBack=false;}}
@@ -618,14 +648,124 @@ public final class MainActivity extends Activity implements NativeSession.Listen
if(!page.equals("session"))updateHomeOrientation();
sizePasswordDialog();
if(privacyDialog instanceof PrivacyConsentDialog)((PrivacyConsentDialog)privacyDialog).updateSize();
if(controls!=null)controls.post(controls::constrainPanels);
if(controls!=null&&!pictureInPictureUi)controls.post(controls::constrainPanels);
if(video!=null)video.post(this::updatePictureInPicture);
}
private boolean supportsPictureInPicture(){return getPackageManager().hasSystemFeature(android.content.pm.PackageManager.FEATURE_PICTURE_IN_PICTURE);}
private boolean canUsePictureInPicture(){
if(!supportsPictureInPicture())return false;
android.app.AppOpsManager ops=getSystemService(android.app.AppOpsManager.class);
int mode=ops.checkOpNoThrow(android.app.AppOpsManager.OPSTR_PICTURE_IN_PICTURE,android.os.Process.myUid(),getPackageName());
return mode==android.app.AppOpsManager.MODE_ALLOWED||mode==android.app.AppOpsManager.MODE_DEFAULT;
}
private void captureBackgroundSession(){
BackgroundSession retained=backgroundSession;if(retained==null)return;
retained.identity=localIdentity;retained.signalState=signalState;retained.platform=hostPlatform;retained.displays=displays;
retained.muted=muted;retained.selectedDisplay=selectedDisplay;retained.remoteServiceAvailable=remoteServiceAvailable;
retained.clipboard=remoteClipboard;retained.videoSettings=videoSettings;retained.receivedFile=receivedFile;
retained.width=videoWidth;retained.height=videoHeight;
}
private boolean prepareBackgroundSession(boolean inform){
if(!ready||session==null||session.isClosed()||isFinishing()||!hasNetworkConsent())return false;
if(backgroundSession!=null&&!backgroundSession.ended)return RemoteSessionService.current()==backgroundSession;
BackgroundSession retained=new BackgroundSession(this,session,host,port,connectedRemote);
backgroundSession=retained;captureBackgroundSession();retained.attach(this);session.setListener(retained);
if(!RemoteSessionService.retain(this,retained)){
session.setListener(this);retained.detach(this);backgroundSession=null;
if(inform)toast("后台保活未能启动,请保持应用在前台");return false;
}
if(inform){
toast("已启用后台保活,可从连接通知返回或断开");
requestBackgroundNotifications();
}
return true;
}
private void requestBackgroundNotifications(){
if(android.os.Build.VERSION.SDK_INT>=33&&checkSelfPermission(android.Manifest.permission.POST_NOTIFICATIONS)!=android.content.pm.PackageManager.PERMISSION_GRANTED&&!preferences.getBoolean("notificationPermissionAsked",false)){
preferences.edit().putBoolean("notificationPermissionAsked",true).apply();
requestPermissions(new String[]{android.Manifest.permission.POST_NOTIFICATIONS},43);
}
}
private boolean restoreBackgroundSession(){
BackgroundSession retained=backgroundSession!=null?backgroundSession:RemoteSessionService.current();
if(retained==null)return false;
if(retained.ended||retained.session.isClosed()||!hasNetworkConsent()||!host.equals(retained.host)||port!=retained.port){
retained.detach(this);retained.finish("后台连接已结束,请重新连接");backgroundSession=null;
if(session==retained.session)end(retained.endReason);return false;
}
boolean rebuild=session!=retained.session||video==null||!page.equals("session");
backgroundSession=retained;session=signaling=retained.session;signalingServer=serverKey();
connectedRemote=retained.remote;localIdentity=retained.identity;signalState=retained.signalState;signalError="";
hostPlatform=retained.platform;displays=retained.displays;muted=retained.muted;selectedDisplay=retained.selectedDisplay;
remoteServiceAvailable=retained.remoteServiceAvailable;remoteClipboard=retained.clipboard;receivedFile=retained.receivedFile;
videoSettings=retained.videoSettings;ready=true;retained.attach(this);
if(rebuild)sessionScreen();
if(retained.width>0&&retained.height>0)videoSize(retained.width,retained.height);
if(controls!=null){controls.setMuted(muted);controls.videoSettings(videoSettings);controls.fileStatus(retained.fileStatus,receivedFile!=null);}
retained.setBackground(false);signaling(signalState,localIdentity);
return true;
}
private android.app.PictureInPictureParams pictureInPictureParams(){
return pictureInPicture.parameters(video==null||pictureInPictureUi||isInPictureInPictureMode()?null:video.pictureInPictureBounds());
}
private void updatePictureInPicture(){
if(!supportsPictureInPicture()||isDestroyed())return;
try{setPictureInPictureParams(pictureInPictureParams());}
catch(IllegalArgumentException|IllegalStateException error){android.util.Log.w("CrossDesk","Picture in Picture parameters unavailable",error);}
}
private void enterRemotePictureInPicture(){
if(isInPictureInPictureMode())return;
if(!canUsePictureInPicture()){
prepareBackgroundSession(false);return;
}
if(!pictureInPicture.canEnter()||session==null||session.isClosed()||isFinishing())return;
android.app.PictureInPictureParams params=pictureInPictureParams();
setPictureInPictureUi(true);
boolean entered=false;
try{entered=enterPictureInPictureMode(params);}
catch(IllegalArgumentException|IllegalStateException error){android.util.Log.w("CrossDesk","Picture in Picture could not start",error);}
if(!entered){setPictureInPictureUi(false);prepareBackgroundSession(false);}
}
private void setPictureInPictureUi(boolean value){
pictureInPictureUi=value;
if(value){releaseKeys();hideKeyboard();pageHost.finish();if(disconnectDialog!=null)disconnectDialog.dismiss();if(remoteUpdateDialog!=null)remoteUpdateDialog.dismiss();}
if(controls!=null)controls.setPictureInPicture(value);
if(video!=null)video.setPictureInPicture(value);
}
@Override protected void onUserLeaveHint(){
super.onUserLeaveHint();
if(documentSession!=null)return;
if(!canUsePictureInPicture()||!pictureInPicture.canEnter())prepareBackgroundSession(false);
else if(android.os.Build.VERSION.SDK_INT<31)enterRemotePictureInPicture();
}
@Override public void onPictureInPictureModeChanged(boolean active,Configuration config){
super.onPictureInPictureModeChanged(active,config);if(active)enteredPictureInPicture=true;setPictureInPictureUi(active);
if(!active&&video!=null)video.post(this::updatePictureInPicture);
}
@android.annotation.TargetApi(31)
@Override public void onPictureInPictureUiStateChanged(android.app.PictureInPictureUiState state){
super.onPictureInPictureUiStateChanged(state);
if(android.os.Build.VERSION.SDK_INT>=35&&state.isTransitioningToPip())setPictureInPictureUi(true);
}
@Override protected void onResume(){super.onResume();if(!isInPictureInPictureMode())enteredPictureInPicture=false;setPictureInPictureUi(isInPictureInPictureMode());updatePictureInPicture();if(ready&&!canUsePictureInPicture())prepareBackgroundSession(true);if(backgroundSession!=null)requestBackgroundNotifications();}
// onStart marks a new visit; focus/resume callbacks must not reopen a refused notice.
@Override protected void onPause(){if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();super.onPause();}
@Override protected void onStart(){super.onStart();foreground=true;appUpdates.setEnabled(preferences.getBoolean("networkConsent",false));if(!preferences.getBoolean("networkConsent",false))consent(this::renderSignaling);ensureSignaling();}
@Override protected void onPause(){releaseKeys();if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();super.onPause();}
@Override protected void onStart(){super.onStart();foreground=true;restoreBackgroundSession();appUpdates.setEnabled(preferences.getBoolean("networkConsent",false));if(!preferences.getBoolean("networkConsent",false))consent(this::renderSignaling);ensureSignaling();showConnectionMessage();}
@Override protected void onStop(){foreground=false;appUpdates.setEnabled(false);if(privacyDialog!=null)privacyDialog.dismiss();pageHost.finish();
// The system document picker is part of this session's explicit file action.
if(documentSession==null||documentSession!=session||isFinishing()){if(session!=null)end(isFinishing()?"":"应用已进入后台,会话已结束");stopSignaling();}
if(connectionDialog!=null)connectionDialog.dismiss();
// Closing native PiP is an explicit end, not a request for invisible
// fallback. An unavailable/failed PiP can hand its live session off.
if(!isFinishing()&&!enteredPictureInPicture&&ready&&(backgroundSession!=null||documentSession==null)&&prepareBackgroundSession(false)){
captureBackgroundSession();backgroundSession.detach(this);backgroundSession.setBackground(true);
mainHandler.removeCallbacks(presenceTick);recentPresence.setConnected(false,android.os.SystemClock.elapsedRealtime());announcementInbox.setConnected(false);
cancelVideoSettingsTimeout();if(videoSettings.pending)videoSettings.expire(videoSettings.requestId);
resetRemoteVersionCheck();if(disconnectDialog!=null)disconnectDialog.dismiss();
}else if(documentSession==null||documentSession!=session||isFinishing()){if(session!=null)end(isFinishing()?"":"远程窗口已关闭,或后台保活未能启动,会话已结束");stopSignaling();}
super.onStop();}
@Override protected void onDestroy(){mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;appUpdates.setEnabled(false);pageHost.finish();resetRemoteVersionCheck();cancelVideoSettingsTimeout();previews.removeListener(previewChanged);if(privacyDialog!=null)privacyDialog.dismiss();if(previewDialog!=null)previewDialog.dismiss();announcementInbox.close();if(disconnectDialog!=null)disconnectDialog.dismiss();if(passwordDialog!=null)passwordDialog.dismiss();if(progressDialog!=null)progressDialog.dismiss();if(session!=null)session.close();stopSignaling();super.onDestroy();}
@Override protected void onDestroy(){mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;appUpdates.setEnabled(false);pageHost.finish();resetRemoteVersionCheck();cancelVideoSettingsTimeout();previews.removeListener(previewChanged);if(privacyDialog!=null)privacyDialog.dismiss();if(previewDialog!=null)previewDialog.dismiss();announcementInbox.close();if(disconnectDialog!=null)disconnectDialog.dismiss();if(passwordDialog!=null)passwordDialog.dismiss();if(progressDialog!=null)progressDialog.dismiss();
if(connectionDialog!=null)connectionDialog.dismiss();
boolean retained=!isFinishing()&&backgroundSession!=null&&!backgroundSession.ended&&RemoteSessionService.current()==backgroundSession;
if(backgroundSession!=null)backgroundSession.detach(this);
if(!retained){if(session!=null)session.close();stopSignaling();}else{mainHandler.removeCallbacks(presenceTick);session=signaling=null;}
super.onDestroy();}
}
@@ -40,7 +40,7 @@ final class NativeSession implements AutoCloseable {
private static final ExecutorService RTC = Executors.newSingleThreadExecutor(r -> new Thread(r, "CrossDesk RTC"));
private final Handler main = new Handler(Looper.getMainLooper());
private final Context context;
private final Listener listener;
private volatile Listener listener;
private final SecretStore secrets;
private final String host, scope;
private volatile String remote;
@@ -108,6 +108,9 @@ final class NativeSession implements AutoCloseable {
});
}
boolean isClosed() { return closed; }
// UI callbacks run on the main thread. A service-owned state can replace
// the Activity listener without recreating either native peer.
void setListener(Listener value) { listener = java.util.Objects.requireNonNull(value); }
private void publishSignaling() {
int state = signalState;
// Only expose the public device ID, never the identity's @password suffix.
@@ -0,0 +1,39 @@
package cn.crossdesk.mobile;
import android.app.PictureInPictureParams;
import android.graphics.Rect;
import android.os.Build;
import android.util.Rational;
/** Builds platform PiP parameters only for a live, visible desktop video. */
final class RemotePictureInPicture {
private boolean connected, choosingDocument;
private int width, height;
void connected() { connected = true; }
void videoSize(int width, int height) { this.width = width; this.height = height; }
void choosingDocument(boolean value) { choosingDocument = value; }
void clearVideo() { width = height = 0; }
void disconnect() { connected = choosingDocument = false; clearVideo(); }
boolean canEnter() { return connected && !choosingDocument && width > 0 && height > 0; }
PictureInPictureParams parameters(Rect source) {
PictureInPictureParams.Builder builder = new PictureInPictureParams.Builder()
.setAspectRatio(aspectRatio(width, height));
if (source != null && !source.isEmpty()) builder.setSourceRectHint(source);
if (Build.VERSION.SDK_INT >= 31) {
builder.setAutoEnterEnabled(canEnter()).setSeamlessResizeEnabled(true);
}
return builder.build();
}
static Rational aspectRatio(int width, int height) {
if (width <= 0 || height <= 0) return new Rational(16, 9);
// Android's standard PiP accepts only 1/2.39 through 2.39. Keep
// ultrawide and portrait desktops valid, with letterboxing in the view.
if ((long) width * 100 > (long) height * 239) return new Rational(239, 100);
if ((long) height * 100 > (long) width * 239) return new Rational(100, 239);
return new Rational(width, height);
}
}
@@ -0,0 +1,109 @@
package cn.crossdesk.mobile;
import android.app.Notification;
import android.app.NotificationChannel;
import android.app.NotificationManager;
import android.app.PendingIntent;
import android.app.Service;
import android.content.Context;
import android.content.Intent;
import android.content.pm.ServiceInfo;
import android.os.Build;
import android.os.IBinder;
import android.os.PowerManager;
import android.util.Log;
/** Foreground fallback for a user-started RTC session when native PiP is unavailable. */
public final class RemoteSessionService extends Service {
private static final String CHANNEL = "remote_connection";
private static final String STOP = "cn.crossdesk.mobile.STOP_REMOTE_SESSION";
private static final int NOTIFICATION = 41;
private static BackgroundSession current;
private static RemoteSessionService running;
private BackgroundSession owned;
private PowerManager.WakeLock wakeLock;
static BackgroundSession current() { return current; }
static boolean retain(Context context, BackgroundSession value) {
if (value.ended || value.session.isClosed()) return false;
if (current == value) return true;
if (current != null) return false;
current = value;
try {
context.startForegroundService(new Intent(context, RemoteSessionService.class).putExtra("session", value.token));
return true;
} catch (RuntimeException error) {
current = null;
Log.w("CrossDesk", "Background session service could not start", error);
return false;
}
}
static void release(Context context, BackgroundSession value) {
if (current != value) return;
current = null;
if (running != null) running.releaseWakeLock();
context.stopService(new Intent(context, RemoteSessionService.class));
}
static void refresh(BackgroundSession value) {
if (current == value && running != null && running.owned == value) running.update();
}
@Override public IBinder onBind(Intent intent) { return null; }
@Override public void onCreate() {
super.onCreate(); running = this;
getSystemService(NotificationManager.class).createNotificationChannel(
new NotificationChannel(CHANNEL, "远程连接", NotificationManager.IMPORTANCE_LOW));
wakeLock = getSystemService(PowerManager.class).newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "CrossDesk:RemoteSession");
wakeLock.setReferenceCounted(false);
}
@Override public int onStartCommand(Intent intent, int flags, int startId) {
BackgroundSession value = current;
if (value == null || value.ended || value.session.isClosed()) { stopSelf(startId); return START_NOT_STICKY; }
if (intent == null || !value.token.equals(intent.getStringExtra("session"))) return START_NOT_STICKY;
owned = value;
if (STOP.equals(intent.getAction())) { value.finish("已从通知断开远程连接"); return START_NOT_STICKY; }
try {
Notification notification = notification();
if (Build.VERSION.SDK_INT >= 34) startForeground(NOTIFICATION, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_SPECIAL_USE);
else startForeground(NOTIFICATION, notification);
update();
} catch (RuntimeException error) {
Log.w("CrossDesk", "Background session foreground promotion failed", error);
value.finish("后台保活未能启动,请保持应用在前台后重新连接");
}
return START_NOT_STICKY;
}
private Notification notification() {
PendingIntent open = PendingIntent.getActivity(this, 0,
new Intent(this, MainActivity.class).addFlags(Intent.FLAG_ACTIVITY_SINGLE_TOP | Intent.FLAG_ACTIVITY_CLEAR_TOP),
PendingIntent.FLAG_UPDATE_CURRENT | PendingIntent.FLAG_IMMUTABLE);
Intent stopIntent = new Intent(this, RemoteSessionService.class).setAction(STOP)
.setData(android.net.Uri.parse("crossdesk-session:" + owned.token)).putExtra("session", owned.token);
PendingIntent stop = PendingIntent.getService(this, 1, stopIntent, PendingIntent.FLAG_UPDATE_CURRENT | PendingIntent.FLAG_IMMUTABLE);
return new Notification.Builder(this, CHANNEL).setSmallIcon(R.drawable.ic_remote_session)
.setContentTitle("CrossDesk 远程连接")
.setContentText(owned.background ? "后台保持连接,点按返回远程桌面" : "已启用后台保活,切换应用可继续保持连接")
.setContentIntent(open).setOngoing(true).setOnlyAlertOnce(true).setShowWhen(false)
.setCategory(Notification.CATEGORY_SERVICE).setVisibility(Notification.VISIBILITY_PRIVATE)
.addAction(new Notification.Action.Builder(null, "返回", open).build())
.addAction(new Notification.Action.Builder(null, "断开", stop).build()).build();
}
@android.annotation.SuppressLint("WakelockTimeout") // Released on foreground return, disconnect, task removal and service destruction.
private void update() {
if (owned == null || owned.ended) return;
if (owned.background) { if (!wakeLock.isHeld()) wakeLock.acquire(); }
else releaseWakeLock();
getSystemService(NotificationManager.class).notify(NOTIFICATION, notification());
}
private void releaseWakeLock() { if (wakeLock != null && wakeLock.isHeld()) wakeLock.release(); }
@Override public void onTaskRemoved(Intent rootIntent) {
if (current == owned && owned != null) owned.finish("远程任务已关闭");
stopSelf();
}
@Override public void onDestroy() {
if (running == this) running = null;
releaseWakeLock(); stopForeground(STOP_FOREGROUND_REMOVE);
BackgroundSession value = owned;
if (current == value && value != null) { current = null; value.finish("后台连接服务已停止,请重新连接"); }
super.onDestroy();
}
}
@@ -27,6 +27,8 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
private final GestureDetector gestures;
private final ScaleGestureDetector scales;
private final RemoteViewport viewport = new RemoteViewport();
private final RemoteViewport pictureInPictureViewport = new RemoteViewport();
private boolean pictureInPicture;
private int frameWidth = 16, frameHeight = 9;
private float x = .5f, y = .5f, scrollY;
private boolean dragging, twoFinger, relative, viewportGesture, viewportPanActive;
@@ -34,6 +36,13 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
private long touchRevision;
private boolean suppressPointerGesture;
void setRelative(boolean value) { releaseMouse(); relative=value; }
void setPictureInPicture(boolean value) {
if(pictureInPicture==value)return;
releaseMouse();pictureInPicture=value;cursor.setVisibility(value?GONE:VISIBLE);fit();
}
Rect pictureInPictureBounds() {
Rect bounds=new Rect();video.getGlobalVisibleRect(bounds);return bounds;
}
void releaseMouse() { mouseInput.releaseAll(); dragging=false; suppressPointerGesture=true; }
private boolean suppressPointerGesture() {
return suppressPointerGesture || mouseInput.hasVirtualButton() || touchRevision != mouseInput.revision();
@@ -117,15 +126,17 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
}
private void fit() {
if (getWidth() == 0 || getHeight() == 0) return;
viewport.fit(getWidth(), getHeight(), frameWidth, frameHeight);
video.setLayoutParams(new LayoutParams(viewport.baseWidth(),viewport.baseHeight(),android.view.Gravity.CENTER));
RemoteViewport active=pictureInPicture?pictureInPictureViewport:viewport;
active.fit(getWidth(), getHeight(), frameWidth, frameHeight);
video.setLayoutParams(new LayoutParams(active.baseWidth(),active.baseHeight(),android.view.Gravity.CENTER));
applyViewport();
}
private void applyViewport() {
// Transform the existing Surface without enlarging its decoder buffers.
video.setScaleX(viewport.scale()); video.setScaleY(viewport.scale());
video.setTranslationX(viewport.offsetX()); video.setTranslationY(viewport.offsetY());
mouseInput.viewportWidth = viewport.width(); mouseInput.viewportHeight = viewport.height();
RemoteViewport active=pictureInPicture?pictureInPictureViewport:viewport;
video.setScaleX(active.scale()); video.setScaleY(active.scale());
video.setTranslationX(active.offsetX()); video.setTranslationY(active.offsetY());
mouseInput.viewportWidth = active.width(); mouseInput.viewportHeight = active.height();
cursor.invalidate();
}
private void sendMouse(float x, float y, int flag, int wheel) {
@@ -23,7 +23,7 @@ final class SessionControls extends FrameLayout {
private LinearLayout menu,keyboard;
private VirtualMouseBar mouseBar;
private FrameLayout mouseHost;
private boolean mouseVisible;
private boolean mouseVisible,pictureInPicture;
private ScrollView keyboardKeys;
private ScrollView menuScroll;
private int keyboardPreferredHeight;
@@ -63,10 +63,15 @@ final class SessionControls extends FrameLayout {
bar.setVisibility(GONE);
}
void cancelMouseInput(){if(mouseBar!=null)mouseBar.cancelInput();}
void setPictureInPicture(boolean value){
pictureInPicture=value;
if(value){cancelMouseInput();closeMenu();closeKeyboard();modifiers.clear();}
setVisibility(value?GONE:VISIBLE);positionMouse();
}
private void toggleMouse(){cancelMouseInput();mouseVisible=!mouseVisible;closeMenu();positionMouse();}
private void positionMouse(){
if(mouseBar==null)return;
boolean visible=mouseVisible&&menu==null&&waiting.getVisibility()==GONE;
boolean visible=!pictureInPicture&&mouseVisible&&menu==null&&waiting.getVisibility()==GONE;
if(!visible){if(mouseBar.getVisibility()==VISIBLE)cancelMouseInput();mouseBar.setVisibility(GONE);return;}
int[] location=new int[2],hostLocation=new int[2];getLocationOnScreen(location);mouseHost.getLocationOnScreen(hostLocation);
float width=ui.dp(140),height=ui.dp(48),gap=ui.dp(6),margin=ui.dp(8);
@@ -0,0 +1,4 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">
<path android:fillColor="#FFFFFFFF" android:pathData="M3,3h18v14h-8v3h4v2H7v-2h4v-3H3zM5,5v10h14V5z" />
</vector>
+5 -1
View File
@@ -477,6 +477,10 @@ extern "C" JNIEXPORT void JNICALL JNI_METHOD(nClipboard)(
s->Send(kClipboardStream, bytes);
}
extern "C" JNIEXPORT void JNICALL JNI_METHOD(nSurface)(JNIEnv* env, jclass, jlong handle, jobject surface) {
if (auto* s = From(handle)) s->Surface(env, surface);
if (auto* s = From(handle)) {
s->Surface(env, surface);
if (surface && s->controller && !s->stopping)
RequestVideoKeyFrame(s->controller, MakeDisplayStreamId(s->display.load()).c_str());
}
}
extern "C" JNIEXPORT void JNICALL JNI_METHOD(nDestroy)(JNIEnv*, jclass, jlong handle) { delete From(handle); }
+4
View File
@@ -22,6 +22,10 @@
<string>$(CURRENT_PROJECT_VERSION)</string>
<key>LSRequiresIPhoneOS</key>
<true/>
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
<key>NSLocalNetworkUsageDescription</key>
<string>CrossDesk 使用局域网发现并建立原生 P2P 远程控制连接。</string>
<key>UIApplicationSupportsIndirectInputEvents</key>
@@ -239,6 +239,16 @@ private final class RemoteAudioPlayer {
}
}
func endSession() {
queue.async {
self.enabled = false
self.player.stop()
self.engine.stop()
self.queuedBuffers = 0
try? AVAudioSession.sharedInstance().setActive(false, options: [.notifyOthersOnDeactivation])
}
}
func enqueue(_ data: Data) {
guard !data.isEmpty, data.count.isMultiple(of: MemoryLayout<Int16>.size) else { return }
queue.async {
@@ -339,6 +349,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
@Published private(set) var remotePlatform: RemoteHostPlatform = .unknown
let bridge = CrossDeskRTCBridge()
let pictureInPicture = RemotePictureInPicture()
let announcements = AnnouncementInbox()
let appUpdates = AppUpdateChecker()
private var appIsActive = false
@@ -392,6 +403,22 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
override init() {
super.init()
bridge.delegate = self
pictureInPicture.requestFrame = { [weak self] in
guard let self, self.isConnected else { return }
self.bridge.requestKeyFrame()
}
pictureInPicture.restoreInterface = { [weak self] in
guard let self, self.isConnected else { return false }
self.sessionVisible = true
return true
}
pictureInPicture.playbackChanged = { [weak self] playing in
guard let self, self.isConnected else { return }
self.audioPlayer.setEnabled(playing && self.audioEnabled)
}
pictureInPicture.failed = { [weak self] in
self?.controlStatus = "画中画未能开启,请保持应用在前台后重试。"
}
announcements.send = { [weak bridge] data, requestID in
bridge?.sendAnnouncementRequest(data, requestID: requestID)
}
@@ -408,6 +435,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
func applicationDidBecomeActive() {
appIsActive = true
pictureInPicture.didBecomeActive()
appUpdates.setEnabled(hasNetworkConsent)
privacyNotice.didBecomeActive(hasNetworkConsent: hasNetworkConsent)
refreshRecentConnectionPresenceAfterForeground()
@@ -656,6 +684,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
}
private func resetConnection() {
pictureInPicture.endSession()
clipboardStatus = ""
controlStatus = ""
transferStatus = ""
@@ -682,7 +711,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
displaySizes = []
selectedDisplay = 0
connectionStatus = "未连接"
audioPlayer.setEnabled(false)
audioPlayer.endSession()
AppOrientation.update(to: .portrait)
appUpdates.checkNow()
}
@@ -1124,6 +1153,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
case 1:
connectionStatus = "已连接"
sessionVisible = true
pictureInPicture.beginSession()
if videoSettings.requestID == 0 { updateVideoSettings() }
checkRemoteVersionIfNeeded()
recordSuccessfulConnectionIfNeeded()
@@ -1226,7 +1256,16 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
width: Int,
height: Int,
captureUptime: TimeInterval) {
guard hasNetworkConsent, isConnected, !videoWasBackgrounded else { return }
guard hasNetworkConsent, isConnected else { return }
if videoWasBackgrounded {
// Feed PiP directly, without publishing background SwiftUI updates.
if pictureInPicture.display(pixelBuffer) {
videoFrameID &+= 1
networkStatistics.recordSubmittedFrame(id: videoFrameID, captureUptime: captureUptime,
at: ProcessInfo.processInfo.systemUptime)
}
return
}
cancelVideoRecovery()
// Buffer and encoded dimensions must be one observable value. Adaptive
// resolution changes must never expose a new frame with the previous
@@ -60,7 +60,8 @@ struct RemoteSessionView: View {
pixelBuffer: session.pixelBuffer,
frameID: session.currentVideoFrameID,
captureUptime: session.frameCaptureUptime,
onFrameSubmitted: session.recordVideoFrameSubmission
onFrameSubmitted: session.recordVideoFrameSubmission,
pictureInPicture: session.pictureInPicture
)
if let cursorPosition,
@@ -1,4 +1,5 @@
import AVFoundation
import AVKit
import CoreMedia
import CoreVideo
import SwiftUI
@@ -15,16 +16,158 @@ struct NativeVideoView: UIViewRepresentable {
let frameID: UInt64
let captureUptime: TimeInterval
let onFrameSubmitted: (UInt64, TimeInterval) -> Void
let pictureInPicture: RemotePictureInPicture
func makeUIView(context: Context) -> SampleBufferVideoView {
SampleBufferVideoView(frame: .zero)
let view = SampleBufferVideoView(frame: .zero)
pictureInPicture.attach(view)
return view
}
func updateUIView(_ uiView: SampleBufferVideoView, context: Context) {
if uiView.display(pixelBuffer, isActive: scenePhase == .active) {
// Background frames go straight from the RTC delegate to the display
// layer: SwiftUI does not promise view updates while backgrounded.
guard scenePhase != .background, !pictureInPicture.isPaused else { return }
if uiView.display(pixelBuffer, isActive: true) {
onFrameSubmitted(frameID, captureUptime)
}
}
static func dismantleUIView(_ uiView: SampleBufferVideoView, coordinator: ()) {
_ = uiView.display(nil, isActive: false)
}
}
/// Uses actual live remote video for background playback, including when muted.
/// Closing/pausing PiP removes that background execution opportunity; it does
/// not fabricate silent audio or repeatedly request background task time.
final class RemotePictureInPicture: NSObject, AVPictureInPictureControllerDelegate,
AVPictureInPictureSampleBufferPlaybackDelegate {
private var controller: AVPictureInPictureController?
private var videoView: SampleBufferVideoView?
private var sessionActive = false
private var starting = false
private var paused = false
var requestFrame: (() -> Void)?
var restoreInterface: (() -> Bool)?
var playbackChanged: ((Bool) -> Void)?
var failed: (() -> Void)?
var isRendering: Bool {
sessionActive && !paused && (starting || controller?.isPictureInPictureActive == true)
}
var isPaused: Bool { paused && controller?.isPictureInPictureActive == true }
func attach(_ view: SampleBufferVideoView) {
guard videoView !== view else { return }
controller?.stopPictureInPicture()
controller?.delegate = nil
videoView = view
guard AVPictureInPictureController.isPictureInPictureSupported() else { return }
let source = AVPictureInPictureController.ContentSource(
sampleBufferDisplayLayer: view.videoLayer, playbackDelegate: self)
let controller = AVPictureInPictureController(contentSource: source)
controller.delegate = self
controller.requiresLinearPlayback = true
controller.canStartPictureInPictureAutomaticallyFromInline = sessionActive
self.controller = controller
}
func beginSession() {
guard !sessionActive else { return }
sessionActive = true
paused = false
// PiP needs an active playback session even when the remote is muted.
// There is no generated audio: only received desktop audio is played.
do {
let audio = AVAudioSession.sharedInstance()
try audio.setCategory(.playback, mode: .moviePlayback, options: [.mixWithOthers])
try audio.setActive(true)
} catch {
NSLog("CrossDesk could not activate Picture in Picture audio session: %@", error.localizedDescription)
}
controller?.canStartPictureInPictureAutomaticallyFromInline = true
controller?.invalidatePlaybackState()
}
func endSession() {
sessionActive = false
starting = false
paused = false
controller?.canStartPictureInPictureAutomaticallyFromInline = false
controller?.stopPictureInPicture()
controller?.invalidatePlaybackState()
_ = videoView?.display(nil, isActive: false)
}
func didBecomeActive() {
paused = false
if controller?.isPictureInPictureActive == true { controller?.stopPictureInPicture() }
controller?.invalidatePlaybackState()
playbackChanged?(true)
}
@discardableResult
func display(_ buffer: CVPixelBuffer) -> Bool {
guard isRendering else { return false }
return videoView?.display(buffer, isActive: true) ?? false
}
func pictureInPictureControllerWillStartPictureInPicture(_ controller: AVPictureInPictureController) {
guard self.controller === controller, sessionActive else { return }
starting = true
requestFrame?()
}
func pictureInPictureControllerDidStartPictureInPicture(_ controller: AVPictureInPictureController) {
guard self.controller === controller else { return }
starting = false
if !sessionActive { controller.stopPictureInPicture() }
else { requestFrame?() }
}
func pictureInPictureController(_ controller: AVPictureInPictureController,
failedToStartPictureInPictureWithError error: Error) {
guard self.controller === controller else { return }
starting = false
NSLog("CrossDesk Picture in Picture could not start: %@", error.localizedDescription)
failed?()
}
func pictureInPictureControllerDidStopPictureInPicture(_ controller: AVPictureInPictureController) {
guard self.controller === controller else { return }
starting = false
paused = false
playbackChanged?(true)
}
func pictureInPictureController(_ controller: AVPictureInPictureController,
restoreUserInterfaceForPictureInPictureStopWithCompletionHandler completion: @escaping (Bool) -> Void) {
completion(sessionActive && (restoreInterface?() ?? false))
}
func pictureInPictureController(_ controller: AVPictureInPictureController, setPlaying playing: Bool) {
paused = !playing
playbackChanged?(playing)
if playing { requestFrame?() }
controller.invalidatePlaybackState()
}
func pictureInPictureControllerTimeRangeForPlayback(_ controller: AVPictureInPictureController) -> CMTimeRange {
sessionActive ? CMTimeRange(start: .zero, duration: .positiveInfinity) : .invalid
}
func pictureInPictureControllerIsPlaybackPaused(_ controller: AVPictureInPictureController) -> Bool {
!sessionActive || paused
}
func pictureInPictureController(_ controller: AVPictureInPictureController,
didTransitionToRenderSize newRenderSize: CMVideoDimensions) { }
func pictureInPictureController(_ controller: AVPictureInPictureController,
skipByInterval skipInterval: CMTime, completion: @escaping () -> Void) {
completion() // Live desktop video has no seekable history.
}
}
final class SampleBufferVideoView: UIView {
@@ -32,7 +175,7 @@ final class SampleBufferVideoView: UIView {
AVSampleBufferDisplayLayer.self
}
private var videoLayer: AVSampleBufferDisplayLayer {
var videoLayer: AVSampleBufferDisplayLayer {
layer as! AVSampleBufferDisplayLayer
}
@@ -60,6 +203,14 @@ final class SampleBufferVideoView: UIView {
// calculation inside AVSampleBufferDisplayLayer. That rounding becomes
// visibly amplified when the remote desktop is zoomed up to 10x.
videoLayer.videoGravity = .resize
var timebase: CMTimebase?
if CMTimebaseCreateWithSourceClock(allocator: kCFAllocatorDefault,
sourceClock: CMClockGetHostTimeClock(),
timebaseOut: &timebase) == noErr, let timebase {
CMTimebaseSetTime(timebase, time: CMClockGetTime(CMClockGetHostTimeClock()))
CMTimebaseSetRate(timebase, rate: 1)
videoLayer.controlTimebase = timebase
}
}
/// True only when a new buffer was submitted; no presentation timestamp is
@@ -123,7 +274,7 @@ final class SampleBufferVideoView: UIView {
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: .invalid,
presentationTimeStamp: CMClockGetTime(CMClockGetHostTimeClock()),
decodeTimeStamp: .invalid
)
var sampleBuffer: CMSampleBuffer?
+28
View File
@@ -4,6 +4,34 @@ This target is a native MiniRTC controller. It uses the same WebSocket
signaling, libnice ICE, SRTP/RTP and data-stream protocol as the desktop app.
There is no `WKWebView` or browser runtime.
## Background connection and Picture in Picture
An established remote desktop session supports native Picture in Picture (PiP).
After the first video frame appears, switching to another app automatically
starts PiP when the device and system settings allow it. The PiP window shows
the live remote desktop; restoring the app returns to the existing session and
requests a fresh key frame. Remote mouse buttons are released when the app loses focus.
The app declares the `audio` background mode required by AVKit for PiP and
activates a playback audio session, including for muted remote video. Background
frames are submitted directly from the RTC delegate to the sample-buffer display
layer, without relying on SwiftUI background redraws. Only real remote audio is
played; no synthetic silent audio or recurring background-task assertions are
used. Disconnecting or withdrawing consent stops PiP and releases playback.
Background execution depends on active PiP or actual remote audio playback.
Closing or pausing PiP, disabling automatic PiP in system settings, locking the
device, losing the network, or force-quitting can suspend or interrupt the
session. iOS does not provide an unrestricted background socket keepalive.
The connection is not deliberately closed on background entry, but continuous
connectivity cannot be promised outside the supported playback modes.
On a physical iPhone/iPad, verify automatic PiP when switching apps with remote
audio both enabled and muted, continuously changing remote frames while another
app is in front, restoration, pause/resume, close, lock/unlock, and disconnection while PiP
is active. Also verify the system setting that disables automatic PiP. A build
or simulator check alone does not establish long-duration background behavior.
## Requirements
- Xcode 16 or newer