mirror of
https://github.com/kunkundi/crossdesk.git
synced 2026-10-19 18:33:00 +08:00
[feat] keep mobile sessions connected in background
This commit is contained in:
dijunkun
+21
-3
@@ -4,7 +4,7 @@
|
||||
|
||||
## 当前功能
|
||||
|
||||
- 官方或自托管 WSS 信令服务器；授权后进入前台即自动注册或登录本机身份，首页实时显示连接、重连及失败状态。发起桌面连接时复用该身份；切换服务器会重新登录，撤回授权或进入后台会关闭连接。
|
||||
- 官方或自托管 WSS 信令服务器；授权后进入前台即自动注册或登录本机身份，首页实时显示连接、重连及失败状态。发起桌面连接时复用该身份；切换服务器会重新登录，撤回授权会关闭连接。已有远程会话优先通过系统画中画保持连接，画中画不可用时由带连接通知的前台服务兜底；首页空闲信令在后台关闭。
|
||||
- 优先使用 MediaCodec 硬件解码 H.264 / AV1，硬件缓冲区直接提交 Surface；设备无对应硬件、分辨率超限或运行失败时，按原编码格式回退到 OpenH264 / dav1d。软件输出由独立显示线程将 NV12 转 RGBA；只保留最新待显示帧。MiniRTC 原有的通用压缩帧解码队列保持不变。
|
||||
- 48 kHz 单声道 PCM 播放，支持静音，音频队列有容量限制。
|
||||
- 相对鼠标（触控板）和绝对位置模式；单指点击、绝对模式按住拖动、长按右键，以及外接键盘、鼠标。画面手势与 iOS 一致：双指捏合以触点为中心缩放（1–10 倍），放大后双指拖动画面，原始比例下双指上下滑动控制滚轮。
|
||||
@@ -14,7 +14,7 @@
|
||||
- 可拖动的悬浮控制球、三列控制菜单和浮动 ASCII / 电脑键盘、常用快捷键；通过剪贴板发送中文等文本，再手动执行远端粘贴。
|
||||
- 悬浮菜单中的“虚拟鼠标”展开 140 × 48 dp 横向左键／滚轮／右键条，跟随悬浮球移动，可随时收起，不压缩远程画面。按住左键或右键后，同一根手指拖动即可拖拽，滑出按键范围仍继续，松开结束；也支持另一指在画面上移动光标。采用无文字的扁平分体按键，中间滚轮外观为 18 × 30 dp，三键均保留 44 × 44 dp 触控范围，通过留白分隔。无阴影、凹凸或按压位移，按压仅变色。等色横条随手指上下循环移动，松手停止；上下滑动可超出按键范围，在画面放大后仍控制远端滚动。切换模式、显示器、进入后台或取消触摸会释放按键。状态菜单按内容高度完整展开，只有安全区域空间不足时才滚动。
|
||||
- 悬浮控制栏支持选择并发送文件、接收电脑端文件、查看确认进度及保存收到的文件；Ctrl+Alt+Del 使用远端 Windows 系统服务，未启用服务时显示提示。
|
||||
- 设备身份与选择保存的连接密码使用 Android Keystore / AES-GCM 加密；首次联网需同意，后台切换会结束会话。
|
||||
- 设备身份与选择保存的连接密码使用 Android Keystore / AES-GCM 加密；首次联网需同意。远程会话在切换应用时自动使用系统画中画，不可用时自动使用通知保活。
|
||||
- 与 iOS 一致的首页卡片、顶部状态/公告/设置入口、密码弹层及分组设置页；最近连接按服务器隔离，长按可删除记录及密码。
|
||||
- 平板首页支持横竖屏和分屏，按当前可用宽度自动适配：900 dp 起连接与最近连接并排，较窄窗口纵向排列；最近连接随宽度显示 1–4 列，手机常规宽度仍为两列。首页最大宽度为 1200 dp，设置、公告及阅读页最大宽度为 720 dp，宽窗口中的密码和隐私弹窗居中限宽。旋转或调整分屏直接重排现有页面，保留输入、设置编辑和当前会话；手机全屏首页保持竖屏。
|
||||
- 页面转场沿用 iOS 的交互方式：设置、隐私、关于及公告页面进入时从右滑入，返回时反向滑出，背景页面轻微跟随；进入和退出远程会话使用 0.2 秒淡入淡出。遵循 Android 系统动画开关及速度设置。
|
||||
@@ -155,7 +155,21 @@ gh secret set ANDROID_KEY_PASSWORD --repo kunkundi/crossdesk
|
||||
|
||||
“发送文件”调用系统文件选择器，后台读取所选文件后使用桌面端相同的 `file` / `file_feedback` 协议传输，支持空文件和中文文件名。每次发送一个文件，以 64 KiB 分块、最多 1 MiB 未确认数据限制发送队列；进度来自接收端确认，30 秒没有有效进展则报告失败。接收时校验大小并追踪完整字节范围，允许跨线程回调乱序，未完成的文件不会提供保存入口。同名文件分开存储，不覆盖已有文件；可通过“保存收到的文件”导出最近收到的文件，连续接收多个文件时应逐个保存。
|
||||
|
||||
选取或保存文件期间保留会话，取消选择不会断开；文件选择器停留超过 5 分钟且应用仍不在前台时结束会话。其他进入后台的操作仍结束会话。发送副本和收到的文件暂存于应用私有缓存，结束会话后清除，需要保留的接收文件应在断开前导出；不申请全盘存储权限。
|
||||
选取或保存文件期间保留会话并暂时禁用自动画中画，取消选择不会断开；停留超过 5 分钟且应用仍不在前台时结束会话。发送副本和收到的文件暂存于应用私有缓存，结束会话后清除，需要保留的接收文件应在断开前导出；不申请全盘存储权限。
|
||||
|
||||
## 后台连接与画中画
|
||||
|
||||
与 iOS 一样，通过系统画中画显示真实远程视频并维持会话。连接成功且已显示远程画面后，Android 12 及以上启用系统自动画中画；Android 8–11 在离开应用时尝试进入。设备需支持画中画，系统需允许 CrossDesk 使用该功能。未连接、尚无画面或正在打开系统文件选择器时不自动进入。
|
||||
|
||||
Android 版本满足要求不代表固件提供画中画，部分低内存模式设备会禁用该功能。以系统的 `FEATURE_PICTURE_IN_PICTURE` 检测为准，单独允许画中画权限不能补齐系统能力。无系统画中画或权限被禁止时，连接建立后自动启动前台服务兜底；画中画进入失败也尝试使用该兜底。控制菜单不提供“画中画”或“后台保活”按钮，两者均在切换应用时自动处理。
|
||||
|
||||
画中画仅显示完整远程画面，隐藏本地光标、悬浮球、鼠标条和键盘，并释放远程按键。使用原有视频 Surface 和 RTC 连接，进入小窗时不会关闭音视频；通过系统小窗返回全屏继续操作。画面比例根据远端尺寸更新，超宽或超长画面在系统允许的比例范围内完整显示并留黑边。
|
||||
|
||||
正常画中画不启动兜底服务，可见的小窗使 Activity 保持在 started 状态，进入时仅暂停交互。关闭小窗会结束会话，不自动转为隐藏的后台连接。
|
||||
|
||||
兜底模式通过前台服务保留原 RTC/信令连接，并显示带“返回”和“断开”操作的常驻通知。后台停止本地画面渲染及音频播放，不创建悬浮窗；视频传输和解码不做协议层暂停。返回应用后重新绑定 Surface 并请求关键帧，恢复原静音状态、显示器和画面设置，不重新登录或连接远端。会话状态独立于 Activity，页面被系统回收后可恢复，整个进程被系统终止则不能恢复原连接。
|
||||
|
||||
服务仅用于用户主动建立且已成功连接的远程会话，不用于首页待机。Android 14 及以上使用 `specialUse` 前台服务类型并声明远程控制连接用途；Android 13 及以上在兜底首次启用时请求通知权限，拒绝后系统仅在活动应用管理中显示服务。CPU 部分唤醒锁仅在兜底会话后台期间持有，返回前台、断开、移除任务或服务销毁时释放。通知断开、远端断开、移除任务、撤回授权和强制停止均结束会话及服务，不自动重连。系统文件选择器仍有上述五分钟限制；系统强杀或网络中断仍可能结束连接。
|
||||
|
||||
首页设置里的“画面偏好”保存在本机，作为下一次连接的默认偏好；会话内画面设置与 iOS 一样只影响当前连接。初始质量按屏幕长边选择低（不超过 1280 像素）或中，采集帧率为 30 fps，电脑端报告支持后自动下发；旧版电脑未报告支持时禁用控件。网络页的 FPS 统计实际提交给 Android Surface 的帧，画面延时使用 MiniRTC 校准后的采集时间至提交时刻的估计值，未完成校准时显示“—”；RTT 使用 0.25 权重平滑，3 秒未更新的数据会过期。切换屏幕清除旧画面的帧率、分辨率和画面延时，保留网络统计。
|
||||
|
||||
@@ -210,6 +224,10 @@ UI 验证统一由人工完成，不维护页面导航、布局、动画、悬
|
||||
|
||||
控制栏联调需在真实远程会话中验证文件选择及取消、收发文件内容一致性、接收后导出、传输中断线；Windows 远端分别检查系统服务开启与关闭时的 Ctrl+Alt+Del 行为。
|
||||
|
||||
画中画需在真实会话中检查切换应用自动进入、其他应用前台时画面持续更新、返回全屏、关闭小窗、静音、超宽及竖屏画面、文件选择器取消、系统禁用画中画和锁屏。
|
||||
|
||||
前台服务兜底需在无原生画中画的实机上验证后台往返、Activity 重建时复用会话、通知断开、旧通知隔离、远端断开时释放唤醒锁，以及实际网络和画面恢复。设备测试前后应备份、核对并恢复服务器等应用设置，避免中断的测试留下本地测试地址。
|
||||
|
||||
## MediaCodec 编解码
|
||||
|
||||
实现参照 MiniRTC 的 NVCodec 分层：`src/media/mediacodec/` 负责硬件后端、能力查询和 JNI/Surface 资源；`src/media/video/encode/mediacodec/` 与 `src/media/video/decode/mediacodec/` 分别接入编码器、解码器接口，并负责同编码格式的软件回退。Android 工程直接编译这些源码。通用 API 仅需启用 `hardware_acceleration` 和 `native_video_output`。JavaVM 由 Android JNI 适配层在 `JNI_OnLoad` 中设置到后端私有实现；该桥接不安装为 SDK 头文件，也不导出为动态库符号。当前能力查询和 SurfaceTexture 创建依赖 JNI，缺少 JavaVM 时仍可使用软件编解码。独立输出线程在画面停止更新后仍会取出最后一帧，空闲时休眠。
|
||||
|
||||
@@ -2,12 +2,23 @@
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
|
||||
<uses-permission android:name="android.permission.INTERNET" />
|
||||
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_SPECIAL_USE" />
|
||||
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
|
||||
<uses-permission android:name="android.permission.WAKE_LOCK" />
|
||||
<application android:theme="@style/AppTheme" android:label="CrossDesk"
|
||||
android:icon="@mipmap/ic_launcher" android:allowBackup="false"
|
||||
android:fullBackupContent="false"
|
||||
android:dataExtractionRules="@xml/data_extraction_rules"
|
||||
android:supportsRtl="true" android:usesCleartextTraffic="false">
|
||||
<service android:name=".RemoteSessionService" android:exported="false"
|
||||
android:foregroundServiceType="specialUse">
|
||||
<property android:name="android.app.PROPERTY_SPECIAL_USE_FGS_SUBTYPE"
|
||||
android:value="Maintain a user-started remote desktop RTC/control session when picture-in-picture is unavailable. Background rendering and audio are suspended; the user can return or disconnect from the notification." />
|
||||
</service>
|
||||
<activity android:name=".MainActivity" android:exported="true"
|
||||
android:launchMode="singleTop"
|
||||
android:supportsPictureInPicture="true"
|
||||
android:configChanges="orientation|screenSize|smallestScreenSize|screenLayout|keyboard|keyboardHidden"
|
||||
android:resizeableActivity="true"
|
||||
android:windowSoftInputMode="adjustResize">
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
package cn.crossdesk.mobile;
|
||||
|
||||
import android.content.Context;
|
||||
import org.json.JSONArray;
|
||||
import org.json.JSONObject;
|
||||
import java.io.File;
|
||||
import java.util.UUID;
|
||||
|
||||
/** Main-thread session state; never retains a stopped Activity or its Views. */
|
||||
final class BackgroundSession implements NativeSession.Listener {
|
||||
final Context context;
|
||||
final NativeSession session;
|
||||
final String token = UUID.randomUUID().toString();
|
||||
final String host, remote;
|
||||
final int port;
|
||||
NativeSession.Listener view;
|
||||
String identity = "", platform = "", clipboard = "", fileStatus = "", endReason = "";
|
||||
JSONArray displays = new JSONArray();
|
||||
RemoteVideoSettings videoSettings;
|
||||
int signalState, selectedDisplay, width, height;
|
||||
boolean muted, remoteServiceAvailable, background, ended;
|
||||
File receivedFile;
|
||||
|
||||
BackgroundSession(Context context, NativeSession session, String host, int port, String remote) {
|
||||
this.context = context.getApplicationContext();
|
||||
this.session = session; this.host = host; this.port = port; this.remote = remote;
|
||||
}
|
||||
void attach(NativeSession.Listener listener) { view = listener; }
|
||||
void detach(NativeSession.Listener listener) { if (view == listener) view = null; }
|
||||
void setBackground(boolean value) {
|
||||
if (background == value || ended) return;
|
||||
background = value;
|
||||
// With no visible window, release rendering buffers and stop remote
|
||||
// audio. The RTC/control connection and file transfers remain alive.
|
||||
if (value) session.surface(null);
|
||||
session.control(2, value || muted ? 0 : 1);
|
||||
RemoteSessionService.refresh(this);
|
||||
}
|
||||
void finish(String reason) {
|
||||
if (ended) return;
|
||||
ended = true; endReason = reason; clipboard = "";
|
||||
session.close();
|
||||
RemoteSessionService.release(context, this);
|
||||
NativeSession.Listener listener = view; view = null;
|
||||
if (listener != null) listener.ended(reason);
|
||||
}
|
||||
@Override public void ended(String reason) { finish(reason); }
|
||||
@Override public void passwordRejected() { finish("远端访问密码已失效，请重新连接"); }
|
||||
@Override public void connected() { /* Already connected before the service takes ownership. */ }
|
||||
@Override public void status(String message) { if (view != null) view.status(message); }
|
||||
@Override public void signaling(int state, String id) {
|
||||
signalState = state; identity = id;
|
||||
if (view != null) view.signaling(state, id);
|
||||
}
|
||||
@Override public void data(JSONObject message) {
|
||||
if (view != null) { view.data(message); return; }
|
||||
if (message.optInt("type", -1) == 12 && videoSettings != null) videoSettings.receive(message);
|
||||
if (message.optInt("type", -1) == 5) {
|
||||
JSONObject service = message.optJSONObject("service_status");
|
||||
remoteServiceAvailable = service != null && service.optBoolean("available", false);
|
||||
}
|
||||
JSONObject info = message.optJSONObject("host_info");
|
||||
if (info != null) {
|
||||
platform = info.optString("platform", "");
|
||||
JSONArray values = info.optJSONArray("displays"); if (values != null) displays = values;
|
||||
if (videoSettings != null) videoSettings.setSupported(info.optBoolean("supports_video_settings", false));
|
||||
}
|
||||
}
|
||||
@Override public void videoSize(int w, int h) {
|
||||
width = w; height = h;
|
||||
if (view != null) view.videoSize(w, h);
|
||||
}
|
||||
@Override public void clipboard(String text) { clipboard = text; if (view != null) view.clipboard(text); }
|
||||
@Override public void statistics(RemoteNetworkStatistics.Snapshot value) { if (view != null) view.statistics(value); }
|
||||
@Override public void fileTransfer(String name, double progress, boolean sending, File received) {
|
||||
if (received != null) receivedFile = received;
|
||||
fileStatus = progress < 0 ? name + " 传输失败" : (sending ? "发送 " : "接收 ") + name
|
||||
+ (progress >= 1 ? " · 已完成" : " · " + (int)(progress * 100) + "%");
|
||||
if (view != null) view.fileTransfer(name, progress, sending, received);
|
||||
}
|
||||
@Override public void controlFeedback(String text) { if (view != null) view.controlFeedback(text); }
|
||||
@Override public void presence(JSONObject message) { if (view != null) view.presence(message); }
|
||||
@Override public void announcement(JSONObject message) { if (view != null) view.announcement(message); }
|
||||
@Override public void announcementFailed(String id) { if (view != null) view.announcementFailed(id); }
|
||||
}
|
||||
@@ -66,10 +66,17 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
private String signalingServer="", localIdentity="", signalError="";
|
||||
private int signalState=-1;
|
||||
private boolean foreground;
|
||||
private boolean pictureInPictureUi;
|
||||
private boolean enteredPictureInPicture;
|
||||
private BackgroundSession backgroundSession;
|
||||
private int videoWidth,videoHeight;
|
||||
private final RemotePictureInPicture pictureInPicture=new RemotePictureInPicture();
|
||||
private RemoteVideoView video;
|
||||
private SessionControls controls;
|
||||
private Dialog passwordDialog, progressDialog, privacyDialog;
|
||||
private AlertDialog disconnectDialog;
|
||||
private AlertDialog connectionDialog;
|
||||
private String pendingConnectionMessage="";
|
||||
private AlertDialog remoteUpdateDialog;
|
||||
private RemoteVersionCheck remoteVersionCheck;
|
||||
private String host, connectedRemote="", hostPlatform="", remoteClipboard="", pendingPassword="", page="home";
|
||||
@@ -217,7 +224,15 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
});renderRecentConnections();
|
||||
}
|
||||
ensureSignaling();
|
||||
if(!message.isEmpty())new AlertDialog.Builder(this).setTitle("连接提示").setMessage(message).setPositiveButton("确定",null).show();
|
||||
if(!message.isEmpty())pendingConnectionMessage=message;
|
||||
showConnectionMessage();
|
||||
}
|
||||
private void showConnectionMessage(){
|
||||
if(pendingConnectionMessage.isEmpty()||!foreground||isFinishing()||isDestroyed())return;
|
||||
if(connectionDialog!=null)connectionDialog.dismiss();
|
||||
connectionDialog=new AlertDialog.Builder(this).setTitle("连接提示").setMessage(pendingConnectionMessage).setPositiveButton("确定",null).create();
|
||||
connectionDialog.setOnDismissListener(dialog->connectionDialog=null);
|
||||
pendingConnectionMessage="";connectionDialog.show();
|
||||
}
|
||||
private String id(){return remote.getText().toString().replace(" ","");}
|
||||
static String formatId(String value){String digits=value.replaceAll("[^0-9]","");if(digits.length()>9)digits=digits.substring(0,9);return digits.replaceAll("(.{3})(?=.)","$1 ");}
|
||||
@@ -283,6 +298,8 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
window.setGravity(width>=600?Gravity.CENTER:Gravity.BOTTOM);
|
||||
}
|
||||
private void begin(String id,String secret){
|
||||
videoWidth=videoHeight=0;enteredPictureInPicture=false;
|
||||
pictureInPicture.disconnect();updatePictureInPicture();
|
||||
remoteServiceAvailable=false;receivedFile=null;
|
||||
resetRemoteVersionCheck();
|
||||
resetVideoSettings();
|
||||
@@ -307,17 +324,26 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
public void mute(){muted=!muted;if(session!=null)session.control(2,muted?0:1);controls.setMuted(muted);}
|
||||
public void mouse(){boolean relative=!preferences.getBoolean("relativeMouse",true);preferences.edit().putBoolean("relativeMouse",relative).apply();video.setRelative(relative);controls.setRelative(relative);}
|
||||
public void videoSettings(int field,int value){RemoteVideoSettings.Values old=MainActivity.this.videoSettings.selection;updateVideoSettings(new RemoteVideoSettings.Values(field==0?value:old.quality,field==1?value:old.frameRate,field==2?value:old.preference));}
|
||||
});controls.setRelative(preferences.getBoolean("relativeMouse",true));controls.videoSettings(videoSettings);root.addView(controls,new LinearLayout.LayoutParams(-1,-1));getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
|
||||
});controls.setRelative(preferences.getBoolean("relativeMouse",true));controls.setMuted(muted);controls.videoSettings(videoSettings);root.addView(controls,new LinearLayout.LayoutParams(-1,-1));getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
|
||||
controls.attachMouseBar(canvas,new VirtualMouseBar(ui,video.mouseInput));
|
||||
video.addOnLayoutChangeListener((view,left,top,right,bottom,oldLeft,oldTop,oldRight,oldBottom)->{
|
||||
if(!pictureInPictureUi&&!isInPictureInPictureMode())updatePictureInPicture();
|
||||
});
|
||||
pictureInPicture.connected();updatePictureInPicture();
|
||||
}
|
||||
private void confirmDisconnect(){disconnectDialog=new AlertDialog.Builder(this).setTitle("断开远程连接？").setMessage("断开后将返回首页").setNegativeButton("取消",null).setPositiveButton("断开连接",(d,w)->end("")).show();}
|
||||
private void end(String reason){
|
||||
if(backgroundSession!=null){BackgroundSession retained=backgroundSession;backgroundSession=null;retained.detach(this);retained.finish(reason);}
|
||||
boolean closeWindow=isInPictureInPictureMode();
|
||||
pictureInPicture.disconnect();updatePictureInPicture();
|
||||
if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();
|
||||
mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;receivedFile=null;remoteServiceAvailable=false;
|
||||
resetRemoteVersionCheck();
|
||||
resetVideoSettings();
|
||||
previewCapture=null;previewCopying=false;
|
||||
if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}if(session!=null){releaseKeys();session.close();session=null;}stopSignaling();pendingPassword="";remoteClipboard="";hostPlatform="";displays=new JSONArray();ready=false;video=null;controls=null;getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);home(reason);
|
||||
if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}if(session!=null){releaseKeys();session.close();session=null;}stopSignaling();pendingPassword="";remoteClipboard="";hostPlatform="";displays=new JSONArray();ready=false;video=null;controls=null;getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
|
||||
if(closeWindow){finish();return;}
|
||||
home(reason);
|
||||
appUpdates.checkNow();
|
||||
}
|
||||
private void settings(){
|
||||
@@ -409,7 +435,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
@Override public void announcementFailed(String requestId){announcementInbox.failed(requestId);}
|
||||
private void privacyPage(){
|
||||
LinearLayout content=formPage("privacy","隐私与授权",this::settings);LinearLayout card=section(content,"");boolean consent=preferences.getBoolean("networkConsent",false);link(card,"授权状态",consent?"已授权":"未授权",()->{});ui.divider(card);link(card,"隐私政策","",this::privacyPolicyPage);
|
||||
footer(content,"设备身份及选择保存的访问密码使用 Android Keystore 加密。连接记录仅保存在本机，不纳入设备备份。剪贴板内容在结束会话时清除。离开应用会结束当前会话。");
|
||||
footer(content,"设备身份及选择保存的访问密码使用 Android Keystore 加密。连接记录仅保存在本机，不纳入设备备份。剪贴板内容在结束会话时清除。离开应用后通过画中画或连接通知保持会话，可在应用或通知中断开。");
|
||||
if(consent)content.addView(ui.action("撤回联网授权",RED,()->new AlertDialog.Builder(this).setTitle("撤回联网授权？").setMessage("撤回后将停止远程连接、关闭画面预览保存并清除已有预览。连接记录和密码会保留，重新连接前需再次授权。").setNegativeButton("取消",null).setPositiveButton("撤回",(d,w)->{preferences.edit().putBoolean("networkConsent",false).apply();appUpdates.setEnabled(false);previews.setEnabled(false);stopSignaling();announcementInbox.reset();privacyPage();}).show()));else content.addView(ui.primary("阅读并授权",()->consent(this::privacyPage)));
|
||||
}
|
||||
private void privacyPolicyPage(){
|
||||
@@ -460,14 +486,16 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
private void openDocument(Intent intent,int request){
|
||||
if(documentSession!=null)return;
|
||||
documentSession=session;releaseKeys();
|
||||
pictureInPicture.choosingDocument(true);updatePictureInPicture();
|
||||
try{startActivityForResult(intent,request);mainHandler.postDelayed(documentTimeout,5*60*1000);}
|
||||
catch(android.content.ActivityNotFoundException error){documentSession=null;documentToSave=null;controlFeedback("此设备没有可用的文件选择器");}
|
||||
catch(android.content.ActivityNotFoundException error){documentSession=null;documentToSave=null;pictureInPicture.choosingDocument(false);updatePictureInPicture();controlFeedback("此设备没有可用的文件选择器");}
|
||||
}
|
||||
@Override protected void onActivityResult(int request,int result,Intent data){
|
||||
super.onActivityResult(request,result,data);
|
||||
if(request!=PICK_FILE&&request!=SAVE_FILE)return;
|
||||
NativeSession source=documentSession;java.io.File saved=documentToSave;
|
||||
documentSession=null;documentToSave=null;mainHandler.removeCallbacks(documentTimeout);
|
||||
pictureInPicture.choosingDocument(false);updatePictureInPicture();
|
||||
if(source==null||source!=session||!ready||result!=RESULT_OK||data==null||data.getData()==null)return;
|
||||
if(request==PICK_FILE)source.sendFile(data.getData());else if(saved!=null)source.saveFile(saved,data.getData());
|
||||
}
|
||||
@@ -515,6 +543,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
renderSignaling();signaling.start();
|
||||
}
|
||||
private void stopSignaling(){
|
||||
if(backgroundSession!=null&&backgroundSession.session==signaling){BackgroundSession retained=backgroundSession;backgroundSession=null;retained.detach(this);retained.finish("");}
|
||||
mainHandler.removeCallbacks(presenceTick);recentPresence.setConnected(false,android.os.SystemClock.elapsedRealtime());
|
||||
announcementInbox.setConnected(false);
|
||||
if(signaling!=null)signaling.close();
|
||||
@@ -550,6 +579,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
// The first success consumes pendingPassword; a duplicate must not save it again.
|
||||
if(session==null||ready)return;ready=true;if(progressDialog!=null){progressDialog.dismiss();progressDialog=null;}
|
||||
try{history.save(serverKey(),connectedRemote,connectedRemote,hostPlatform,rememberPassword,pendingPassword);}catch(Exception error){toast("密码保存失败，本次连接仍可使用");}pendingPassword="";sessionScreen();
|
||||
if(!canUsePictureInPicture())prepareBackgroundSession(true);
|
||||
if(videoSettings.supported&&videoSettings.requestId==0)updateVideoSettings(videoSettings.selection);
|
||||
if(foreground&&preferences.getBoolean("networkConsent",false))remoteVersionCheck.connected();
|
||||
}
|
||||
@@ -566,7 +596,7 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
}
|
||||
private void cancelVideoSettingsTimeout(){if(videoSettingsTimeout!=null)mainHandler.removeCallbacks(videoSettingsTimeout);videoSettingsTimeout=null;}
|
||||
private void showRemoteUpdate(){
|
||||
if(!ready||session==null||!foreground||isFinishing()||!preferences.getBoolean("networkConsent",false))return;
|
||||
if(!ready||session==null||!foreground||pictureInPictureUi||isInPictureInPictureMode()||isFinishing()||!preferences.getBoolean("networkConsent",false))return;
|
||||
remoteUpdateDialog=new AlertDialog.Builder(this).setTitle("升级提示").setMessage(RemoteVersionCheck.NOTICE).setPositiveButton("知道了",null).create();
|
||||
remoteUpdateDialog.setOnDismissListener(dialog->remoteUpdateDialog=null);
|
||||
remoteUpdateDialog.setCanceledOnTouchOutside(false);
|
||||
@@ -601,11 +631,11 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
if(foreground&&preferences.getBoolean("networkConsent",false))remoteVersionCheck.hostInfo(info);
|
||||
}
|
||||
}
|
||||
@Override public void videoSize(int width,int height){if(video!=null)video.videoSize(width,height);if(controls!=null)controls.videoSize(width,height);}
|
||||
@Override public void videoSize(int width,int height){videoWidth=width;videoHeight=height;if(video!=null)video.videoSize(width,height);if(controls!=null)controls.videoSize(width,height);pictureInPicture.videoSize(width,height);updatePictureInPicture();}
|
||||
@Override public void clipboard(String value){remoteClipboard=value;}
|
||||
@Override public void statistics(RemoteNetworkStatistics.Snapshot value){if(controls!=null)controls.statistics(value);if(value.fps>0)capturePreviewIfNeeded();}
|
||||
@Override public boolean dispatchKeyEvent(KeyEvent event){
|
||||
if(ready&&session!=null&&hasWindowFocus()&&!(getCurrentFocus() instanceof EditText)&&event.getKeyCode()!=KeyEvent.KEYCODE_BACK){int code=KeyMap.windowsCode(event.getKeyCode());if(code!=0){boolean down=event.getAction()==KeyEvent.ACTION_DOWN;if(down)pressed.add(code);else pressed.remove(code);session.key(code,down);return true;}}return super.dispatchKeyEvent(event);
|
||||
if(ready&&session!=null&&!pictureInPictureUi&&hasWindowFocus()&&!(getCurrentFocus() instanceof EditText)&&event.getKeyCode()!=KeyEvent.KEYCODE_BACK){int code=KeyMap.windowsCode(event.getKeyCode());if(code!=0){boolean down=event.getAction()==KeyEvent.ACTION_DOWN;if(down)pressed.add(code);else pressed.remove(code);session.key(code,down);return true;}}return super.dispatchKeyEvent(event);
|
||||
}
|
||||
private void releaseKeys(){if(session!=null)for(int code:pressed)session.key(code,false);pressed.clear();}
|
||||
private void navigateBack(){if(backAction==null)return;navigatingBack=true;try{backAction.run();}finally{navigatingBack=false;}}
|
||||
@@ -618,14 +648,124 @@ public final class MainActivity extends Activity implements NativeSession.Listen
|
||||
if(!page.equals("session"))updateHomeOrientation();
|
||||
sizePasswordDialog();
|
||||
if(privacyDialog instanceof PrivacyConsentDialog)((PrivacyConsentDialog)privacyDialog).updateSize();
|
||||
if(controls!=null)controls.post(controls::constrainPanels);
|
||||
if(controls!=null&&!pictureInPictureUi)controls.post(controls::constrainPanels);
|
||||
if(video!=null)video.post(this::updatePictureInPicture);
|
||||
}
|
||||
private boolean supportsPictureInPicture(){return getPackageManager().hasSystemFeature(android.content.pm.PackageManager.FEATURE_PICTURE_IN_PICTURE);}
|
||||
private boolean canUsePictureInPicture(){
|
||||
if(!supportsPictureInPicture())return false;
|
||||
android.app.AppOpsManager ops=getSystemService(android.app.AppOpsManager.class);
|
||||
int mode=ops.checkOpNoThrow(android.app.AppOpsManager.OPSTR_PICTURE_IN_PICTURE,android.os.Process.myUid(),getPackageName());
|
||||
return mode==android.app.AppOpsManager.MODE_ALLOWED||mode==android.app.AppOpsManager.MODE_DEFAULT;
|
||||
}
|
||||
private void captureBackgroundSession(){
|
||||
BackgroundSession retained=backgroundSession;if(retained==null)return;
|
||||
retained.identity=localIdentity;retained.signalState=signalState;retained.platform=hostPlatform;retained.displays=displays;
|
||||
retained.muted=muted;retained.selectedDisplay=selectedDisplay;retained.remoteServiceAvailable=remoteServiceAvailable;
|
||||
retained.clipboard=remoteClipboard;retained.videoSettings=videoSettings;retained.receivedFile=receivedFile;
|
||||
retained.width=videoWidth;retained.height=videoHeight;
|
||||
}
|
||||
private boolean prepareBackgroundSession(boolean inform){
|
||||
if(!ready||session==null||session.isClosed()||isFinishing()||!hasNetworkConsent())return false;
|
||||
if(backgroundSession!=null&&!backgroundSession.ended)return RemoteSessionService.current()==backgroundSession;
|
||||
BackgroundSession retained=new BackgroundSession(this,session,host,port,connectedRemote);
|
||||
backgroundSession=retained;captureBackgroundSession();retained.attach(this);session.setListener(retained);
|
||||
if(!RemoteSessionService.retain(this,retained)){
|
||||
session.setListener(this);retained.detach(this);backgroundSession=null;
|
||||
if(inform)toast("后台保活未能启动，请保持应用在前台");return false;
|
||||
}
|
||||
if(inform){
|
||||
toast("已启用后台保活，可从连接通知返回或断开");
|
||||
requestBackgroundNotifications();
|
||||
}
|
||||
return true;
|
||||
}
|
||||
private void requestBackgroundNotifications(){
|
||||
if(android.os.Build.VERSION.SDK_INT>=33&&checkSelfPermission(android.Manifest.permission.POST_NOTIFICATIONS)!=android.content.pm.PackageManager.PERMISSION_GRANTED&&!preferences.getBoolean("notificationPermissionAsked",false)){
|
||||
preferences.edit().putBoolean("notificationPermissionAsked",true).apply();
|
||||
requestPermissions(new String[]{android.Manifest.permission.POST_NOTIFICATIONS},43);
|
||||
}
|
||||
}
|
||||
private boolean restoreBackgroundSession(){
|
||||
BackgroundSession retained=backgroundSession!=null?backgroundSession:RemoteSessionService.current();
|
||||
if(retained==null)return false;
|
||||
if(retained.ended||retained.session.isClosed()||!hasNetworkConsent()||!host.equals(retained.host)||port!=retained.port){
|
||||
retained.detach(this);retained.finish("后台连接已结束，请重新连接");backgroundSession=null;
|
||||
if(session==retained.session)end(retained.endReason);return false;
|
||||
}
|
||||
boolean rebuild=session!=retained.session||video==null||!page.equals("session");
|
||||
backgroundSession=retained;session=signaling=retained.session;signalingServer=serverKey();
|
||||
connectedRemote=retained.remote;localIdentity=retained.identity;signalState=retained.signalState;signalError="";
|
||||
hostPlatform=retained.platform;displays=retained.displays;muted=retained.muted;selectedDisplay=retained.selectedDisplay;
|
||||
remoteServiceAvailable=retained.remoteServiceAvailable;remoteClipboard=retained.clipboard;receivedFile=retained.receivedFile;
|
||||
videoSettings=retained.videoSettings;ready=true;retained.attach(this);
|
||||
if(rebuild)sessionScreen();
|
||||
if(retained.width>0&&retained.height>0)videoSize(retained.width,retained.height);
|
||||
if(controls!=null){controls.setMuted(muted);controls.videoSettings(videoSettings);controls.fileStatus(retained.fileStatus,receivedFile!=null);}
|
||||
retained.setBackground(false);signaling(signalState,localIdentity);
|
||||
return true;
|
||||
}
|
||||
private android.app.PictureInPictureParams pictureInPictureParams(){
|
||||
return pictureInPicture.parameters(video==null||pictureInPictureUi||isInPictureInPictureMode()?null:video.pictureInPictureBounds());
|
||||
}
|
||||
private void updatePictureInPicture(){
|
||||
if(!supportsPictureInPicture()||isDestroyed())return;
|
||||
try{setPictureInPictureParams(pictureInPictureParams());}
|
||||
catch(IllegalArgumentException|IllegalStateException error){android.util.Log.w("CrossDesk","Picture in Picture parameters unavailable",error);}
|
||||
}
|
||||
private void enterRemotePictureInPicture(){
|
||||
if(isInPictureInPictureMode())return;
|
||||
if(!canUsePictureInPicture()){
|
||||
prepareBackgroundSession(false);return;
|
||||
}
|
||||
if(!pictureInPicture.canEnter()||session==null||session.isClosed()||isFinishing())return;
|
||||
android.app.PictureInPictureParams params=pictureInPictureParams();
|
||||
setPictureInPictureUi(true);
|
||||
boolean entered=false;
|
||||
try{entered=enterPictureInPictureMode(params);}
|
||||
catch(IllegalArgumentException|IllegalStateException error){android.util.Log.w("CrossDesk","Picture in Picture could not start",error);}
|
||||
if(!entered){setPictureInPictureUi(false);prepareBackgroundSession(false);}
|
||||
}
|
||||
private void setPictureInPictureUi(boolean value){
|
||||
pictureInPictureUi=value;
|
||||
if(value){releaseKeys();hideKeyboard();pageHost.finish();if(disconnectDialog!=null)disconnectDialog.dismiss();if(remoteUpdateDialog!=null)remoteUpdateDialog.dismiss();}
|
||||
if(controls!=null)controls.setPictureInPicture(value);
|
||||
if(video!=null)video.setPictureInPicture(value);
|
||||
}
|
||||
@Override protected void onUserLeaveHint(){
|
||||
super.onUserLeaveHint();
|
||||
if(documentSession!=null)return;
|
||||
if(!canUsePictureInPicture()||!pictureInPicture.canEnter())prepareBackgroundSession(false);
|
||||
else if(android.os.Build.VERSION.SDK_INT<31)enterRemotePictureInPicture();
|
||||
}
|
||||
@Override public void onPictureInPictureModeChanged(boolean active,Configuration config){
|
||||
super.onPictureInPictureModeChanged(active,config);if(active)enteredPictureInPicture=true;setPictureInPictureUi(active);
|
||||
if(!active&&video!=null)video.post(this::updatePictureInPicture);
|
||||
}
|
||||
@android.annotation.TargetApi(31)
|
||||
@Override public void onPictureInPictureUiStateChanged(android.app.PictureInPictureUiState state){
|
||||
super.onPictureInPictureUiStateChanged(state);
|
||||
if(android.os.Build.VERSION.SDK_INT>=35&&state.isTransitioningToPip())setPictureInPictureUi(true);
|
||||
}
|
||||
@Override protected void onResume(){super.onResume();if(!isInPictureInPictureMode())enteredPictureInPicture=false;setPictureInPictureUi(isInPictureInPictureMode());updatePictureInPicture();if(ready&&!canUsePictureInPicture())prepareBackgroundSession(true);if(backgroundSession!=null)requestBackgroundNotifications();}
|
||||
// onStart marks a new visit; focus/resume callbacks must not reopen a refused notice.
|
||||
@Override protected void onPause(){if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();super.onPause();}
|
||||
@Override protected void onStart(){super.onStart();foreground=true;appUpdates.setEnabled(preferences.getBoolean("networkConsent",false));if(!preferences.getBoolean("networkConsent",false))consent(this::renderSignaling);ensureSignaling();}
|
||||
@Override protected void onPause(){releaseKeys();if(controls!=null)controls.cancelMouseInput();if(video!=null)video.releaseMouse();super.onPause();}
|
||||
@Override protected void onStart(){super.onStart();foreground=true;restoreBackgroundSession();appUpdates.setEnabled(preferences.getBoolean("networkConsent",false));if(!preferences.getBoolean("networkConsent",false))consent(this::renderSignaling);ensureSignaling();showConnectionMessage();}
|
||||
@Override protected void onStop(){foreground=false;appUpdates.setEnabled(false);if(privacyDialog!=null)privacyDialog.dismiss();pageHost.finish();
|
||||
// The system document picker is part of this session's explicit file action.
|
||||
if(documentSession==null||documentSession!=session||isFinishing()){if(session!=null)end(isFinishing()?"":"应用已进入后台，会话已结束");stopSignaling();}
|
||||
if(connectionDialog!=null)connectionDialog.dismiss();
|
||||
// Closing native PiP is an explicit end, not a request for invisible
|
||||
// fallback. An unavailable/failed PiP can hand its live session off.
|
||||
if(!isFinishing()&&!enteredPictureInPicture&&ready&&(backgroundSession!=null||documentSession==null)&&prepareBackgroundSession(false)){
|
||||
captureBackgroundSession();backgroundSession.detach(this);backgroundSession.setBackground(true);
|
||||
mainHandler.removeCallbacks(presenceTick);recentPresence.setConnected(false,android.os.SystemClock.elapsedRealtime());announcementInbox.setConnected(false);
|
||||
cancelVideoSettingsTimeout();if(videoSettings.pending)videoSettings.expire(videoSettings.requestId);
|
||||
resetRemoteVersionCheck();if(disconnectDialog!=null)disconnectDialog.dismiss();
|
||||
}else if(documentSession==null||documentSession!=session||isFinishing()){if(session!=null)end(isFinishing()?"":"远程窗口已关闭，或后台保活未能启动，会话已结束");stopSignaling();}
|
||||
super.onStop();}
|
||||
@Override protected void onDestroy(){mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;appUpdates.setEnabled(false);pageHost.finish();resetRemoteVersionCheck();cancelVideoSettingsTimeout();previews.removeListener(previewChanged);if(privacyDialog!=null)privacyDialog.dismiss();if(previewDialog!=null)previewDialog.dismiss();announcementInbox.close();if(disconnectDialog!=null)disconnectDialog.dismiss();if(passwordDialog!=null)passwordDialog.dismiss();if(progressDialog!=null)progressDialog.dismiss();if(session!=null)session.close();stopSignaling();super.onDestroy();}
|
||||
@Override protected void onDestroy(){mainHandler.removeCallbacks(documentTimeout);documentSession=null;documentToSave=null;appUpdates.setEnabled(false);pageHost.finish();resetRemoteVersionCheck();cancelVideoSettingsTimeout();previews.removeListener(previewChanged);if(privacyDialog!=null)privacyDialog.dismiss();if(previewDialog!=null)previewDialog.dismiss();announcementInbox.close();if(disconnectDialog!=null)disconnectDialog.dismiss();if(passwordDialog!=null)passwordDialog.dismiss();if(progressDialog!=null)progressDialog.dismiss();
|
||||
if(connectionDialog!=null)connectionDialog.dismiss();
|
||||
boolean retained=!isFinishing()&&backgroundSession!=null&&!backgroundSession.ended&&RemoteSessionService.current()==backgroundSession;
|
||||
if(backgroundSession!=null)backgroundSession.detach(this);
|
||||
if(!retained){if(session!=null)session.close();stopSignaling();}else{mainHandler.removeCallbacks(presenceTick);session=signaling=null;}
|
||||
super.onDestroy();}
|
||||
}
|
||||
|
||||
@@ -40,7 +40,7 @@ final class NativeSession implements AutoCloseable {
|
||||
private static final ExecutorService RTC = Executors.newSingleThreadExecutor(r -> new Thread(r, "CrossDesk RTC"));
|
||||
private final Handler main = new Handler(Looper.getMainLooper());
|
||||
private final Context context;
|
||||
private final Listener listener;
|
||||
private volatile Listener listener;
|
||||
private final SecretStore secrets;
|
||||
private final String host, scope;
|
||||
private volatile String remote;
|
||||
@@ -108,6 +108,9 @@ final class NativeSession implements AutoCloseable {
|
||||
});
|
||||
}
|
||||
boolean isClosed() { return closed; }
|
||||
// UI callbacks run on the main thread. A service-owned state can replace
|
||||
// the Activity listener without recreating either native peer.
|
||||
void setListener(Listener value) { listener = java.util.Objects.requireNonNull(value); }
|
||||
private void publishSignaling() {
|
||||
int state = signalState;
|
||||
// Only expose the public device ID, never the identity's @password suffix.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
package cn.crossdesk.mobile;
|
||||
|
||||
import android.app.PictureInPictureParams;
|
||||
import android.graphics.Rect;
|
||||
import android.os.Build;
|
||||
import android.util.Rational;
|
||||
|
||||
/** Builds platform PiP parameters only for a live, visible desktop video. */
|
||||
final class RemotePictureInPicture {
|
||||
private boolean connected, choosingDocument;
|
||||
private int width, height;
|
||||
|
||||
void connected() { connected = true; }
|
||||
void videoSize(int width, int height) { this.width = width; this.height = height; }
|
||||
void choosingDocument(boolean value) { choosingDocument = value; }
|
||||
void clearVideo() { width = height = 0; }
|
||||
void disconnect() { connected = choosingDocument = false; clearVideo(); }
|
||||
|
||||
boolean canEnter() { return connected && !choosingDocument && width > 0 && height > 0; }
|
||||
|
||||
PictureInPictureParams parameters(Rect source) {
|
||||
PictureInPictureParams.Builder builder = new PictureInPictureParams.Builder()
|
||||
.setAspectRatio(aspectRatio(width, height));
|
||||
if (source != null && !source.isEmpty()) builder.setSourceRectHint(source);
|
||||
if (Build.VERSION.SDK_INT >= 31) {
|
||||
builder.setAutoEnterEnabled(canEnter()).setSeamlessResizeEnabled(true);
|
||||
}
|
||||
return builder.build();
|
||||
}
|
||||
|
||||
static Rational aspectRatio(int width, int height) {
|
||||
if (width <= 0 || height <= 0) return new Rational(16, 9);
|
||||
// Android's standard PiP accepts only 1/2.39 through 2.39. Keep
|
||||
// ultrawide and portrait desktops valid, with letterboxing in the view.
|
||||
if ((long) width * 100 > (long) height * 239) return new Rational(239, 100);
|
||||
if ((long) height * 100 > (long) width * 239) return new Rational(100, 239);
|
||||
return new Rational(width, height);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
package cn.crossdesk.mobile;
|
||||
|
||||
import android.app.Notification;
|
||||
import android.app.NotificationChannel;
|
||||
import android.app.NotificationManager;
|
||||
import android.app.PendingIntent;
|
||||
import android.app.Service;
|
||||
import android.content.Context;
|
||||
import android.content.Intent;
|
||||
import android.content.pm.ServiceInfo;
|
||||
import android.os.Build;
|
||||
import android.os.IBinder;
|
||||
import android.os.PowerManager;
|
||||
import android.util.Log;
|
||||
|
||||
/** Foreground fallback for a user-started RTC session when native PiP is unavailable. */
|
||||
public final class RemoteSessionService extends Service {
|
||||
private static final String CHANNEL = "remote_connection";
|
||||
private static final String STOP = "cn.crossdesk.mobile.STOP_REMOTE_SESSION";
|
||||
private static final int NOTIFICATION = 41;
|
||||
private static BackgroundSession current;
|
||||
private static RemoteSessionService running;
|
||||
private BackgroundSession owned;
|
||||
private PowerManager.WakeLock wakeLock;
|
||||
|
||||
static BackgroundSession current() { return current; }
|
||||
static boolean retain(Context context, BackgroundSession value) {
|
||||
if (value.ended || value.session.isClosed()) return false;
|
||||
if (current == value) return true;
|
||||
if (current != null) return false;
|
||||
current = value;
|
||||
try {
|
||||
context.startForegroundService(new Intent(context, RemoteSessionService.class).putExtra("session", value.token));
|
||||
return true;
|
||||
} catch (RuntimeException error) {
|
||||
current = null;
|
||||
Log.w("CrossDesk", "Background session service could not start", error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
static void release(Context context, BackgroundSession value) {
|
||||
if (current != value) return;
|
||||
current = null;
|
||||
if (running != null) running.releaseWakeLock();
|
||||
context.stopService(new Intent(context, RemoteSessionService.class));
|
||||
}
|
||||
static void refresh(BackgroundSession value) {
|
||||
if (current == value && running != null && running.owned == value) running.update();
|
||||
}
|
||||
@Override public IBinder onBind(Intent intent) { return null; }
|
||||
@Override public void onCreate() {
|
||||
super.onCreate(); running = this;
|
||||
getSystemService(NotificationManager.class).createNotificationChannel(
|
||||
new NotificationChannel(CHANNEL, "远程连接", NotificationManager.IMPORTANCE_LOW));
|
||||
wakeLock = getSystemService(PowerManager.class).newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "CrossDesk:RemoteSession");
|
||||
wakeLock.setReferenceCounted(false);
|
||||
}
|
||||
@Override public int onStartCommand(Intent intent, int flags, int startId) {
|
||||
BackgroundSession value = current;
|
||||
if (value == null || value.ended || value.session.isClosed()) { stopSelf(startId); return START_NOT_STICKY; }
|
||||
if (intent == null || !value.token.equals(intent.getStringExtra("session"))) return START_NOT_STICKY;
|
||||
owned = value;
|
||||
if (STOP.equals(intent.getAction())) { value.finish("已从通知断开远程连接"); return START_NOT_STICKY; }
|
||||
try {
|
||||
Notification notification = notification();
|
||||
if (Build.VERSION.SDK_INT >= 34) startForeground(NOTIFICATION, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_SPECIAL_USE);
|
||||
else startForeground(NOTIFICATION, notification);
|
||||
update();
|
||||
} catch (RuntimeException error) {
|
||||
Log.w("CrossDesk", "Background session foreground promotion failed", error);
|
||||
value.finish("后台保活未能启动，请保持应用在前台后重新连接");
|
||||
}
|
||||
return START_NOT_STICKY;
|
||||
}
|
||||
private Notification notification() {
|
||||
PendingIntent open = PendingIntent.getActivity(this, 0,
|
||||
new Intent(this, MainActivity.class).addFlags(Intent.FLAG_ACTIVITY_SINGLE_TOP | Intent.FLAG_ACTIVITY_CLEAR_TOP),
|
||||
PendingIntent.FLAG_UPDATE_CURRENT | PendingIntent.FLAG_IMMUTABLE);
|
||||
Intent stopIntent = new Intent(this, RemoteSessionService.class).setAction(STOP)
|
||||
.setData(android.net.Uri.parse("crossdesk-session:" + owned.token)).putExtra("session", owned.token);
|
||||
PendingIntent stop = PendingIntent.getService(this, 1, stopIntent, PendingIntent.FLAG_UPDATE_CURRENT | PendingIntent.FLAG_IMMUTABLE);
|
||||
return new Notification.Builder(this, CHANNEL).setSmallIcon(R.drawable.ic_remote_session)
|
||||
.setContentTitle("CrossDesk 远程连接")
|
||||
.setContentText(owned.background ? "后台保持连接，点按返回远程桌面" : "已启用后台保活，切换应用可继续保持连接")
|
||||
.setContentIntent(open).setOngoing(true).setOnlyAlertOnce(true).setShowWhen(false)
|
||||
.setCategory(Notification.CATEGORY_SERVICE).setVisibility(Notification.VISIBILITY_PRIVATE)
|
||||
.addAction(new Notification.Action.Builder(null, "返回", open).build())
|
||||
.addAction(new Notification.Action.Builder(null, "断开", stop).build()).build();
|
||||
}
|
||||
@android.annotation.SuppressLint("WakelockTimeout") // Released on foreground return, disconnect, task removal and service destruction.
|
||||
private void update() {
|
||||
if (owned == null || owned.ended) return;
|
||||
if (owned.background) { if (!wakeLock.isHeld()) wakeLock.acquire(); }
|
||||
else releaseWakeLock();
|
||||
getSystemService(NotificationManager.class).notify(NOTIFICATION, notification());
|
||||
}
|
||||
private void releaseWakeLock() { if (wakeLock != null && wakeLock.isHeld()) wakeLock.release(); }
|
||||
@Override public void onTaskRemoved(Intent rootIntent) {
|
||||
if (current == owned && owned != null) owned.finish("远程任务已关闭");
|
||||
stopSelf();
|
||||
}
|
||||
@Override public void onDestroy() {
|
||||
if (running == this) running = null;
|
||||
releaseWakeLock(); stopForeground(STOP_FOREGROUND_REMOVE);
|
||||
BackgroundSession value = owned;
|
||||
if (current == value && value != null) { current = null; value.finish("后台连接服务已停止，请重新连接"); }
|
||||
super.onDestroy();
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,8 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
|
||||
private final GestureDetector gestures;
|
||||
private final ScaleGestureDetector scales;
|
||||
private final RemoteViewport viewport = new RemoteViewport();
|
||||
private final RemoteViewport pictureInPictureViewport = new RemoteViewport();
|
||||
private boolean pictureInPicture;
|
||||
private int frameWidth = 16, frameHeight = 9;
|
||||
private float x = .5f, y = .5f, scrollY;
|
||||
private boolean dragging, twoFinger, relative, viewportGesture, viewportPanActive;
|
||||
@@ -34,6 +36,13 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
|
||||
private long touchRevision;
|
||||
private boolean suppressPointerGesture;
|
||||
void setRelative(boolean value) { releaseMouse(); relative=value; }
|
||||
void setPictureInPicture(boolean value) {
|
||||
if(pictureInPicture==value)return;
|
||||
releaseMouse();pictureInPicture=value;cursor.setVisibility(value?GONE:VISIBLE);fit();
|
||||
}
|
||||
Rect pictureInPictureBounds() {
|
||||
Rect bounds=new Rect();video.getGlobalVisibleRect(bounds);return bounds;
|
||||
}
|
||||
void releaseMouse() { mouseInput.releaseAll(); dragging=false; suppressPointerGesture=true; }
|
||||
private boolean suppressPointerGesture() {
|
||||
return suppressPointerGesture || mouseInput.hasVirtualButton() || touchRevision != mouseInput.revision();
|
||||
@@ -117,15 +126,17 @@ final class RemoteVideoView extends FrameLayout implements SurfaceHolder.Callbac
|
||||
}
|
||||
private void fit() {
|
||||
if (getWidth() == 0 || getHeight() == 0) return;
|
||||
viewport.fit(getWidth(), getHeight(), frameWidth, frameHeight);
|
||||
video.setLayoutParams(new LayoutParams(viewport.baseWidth(),viewport.baseHeight(),android.view.Gravity.CENTER));
|
||||
RemoteViewport active=pictureInPicture?pictureInPictureViewport:viewport;
|
||||
active.fit(getWidth(), getHeight(), frameWidth, frameHeight);
|
||||
video.setLayoutParams(new LayoutParams(active.baseWidth(),active.baseHeight(),android.view.Gravity.CENTER));
|
||||
applyViewport();
|
||||
}
|
||||
private void applyViewport() {
|
||||
// Transform the existing Surface without enlarging its decoder buffers.
|
||||
video.setScaleX(viewport.scale()); video.setScaleY(viewport.scale());
|
||||
video.setTranslationX(viewport.offsetX()); video.setTranslationY(viewport.offsetY());
|
||||
mouseInput.viewportWidth = viewport.width(); mouseInput.viewportHeight = viewport.height();
|
||||
RemoteViewport active=pictureInPicture?pictureInPictureViewport:viewport;
|
||||
video.setScaleX(active.scale()); video.setScaleY(active.scale());
|
||||
video.setTranslationX(active.offsetX()); video.setTranslationY(active.offsetY());
|
||||
mouseInput.viewportWidth = active.width(); mouseInput.viewportHeight = active.height();
|
||||
cursor.invalidate();
|
||||
}
|
||||
private void sendMouse(float x, float y, int flag, int wheel) {
|
||||
|
||||
@@ -23,7 +23,7 @@ final class SessionControls extends FrameLayout {
|
||||
private LinearLayout menu,keyboard;
|
||||
private VirtualMouseBar mouseBar;
|
||||
private FrameLayout mouseHost;
|
||||
private boolean mouseVisible;
|
||||
private boolean mouseVisible,pictureInPicture;
|
||||
private ScrollView keyboardKeys;
|
||||
private ScrollView menuScroll;
|
||||
private int keyboardPreferredHeight;
|
||||
@@ -63,10 +63,15 @@ final class SessionControls extends FrameLayout {
|
||||
bar.setVisibility(GONE);
|
||||
}
|
||||
void cancelMouseInput(){if(mouseBar!=null)mouseBar.cancelInput();}
|
||||
void setPictureInPicture(boolean value){
|
||||
pictureInPicture=value;
|
||||
if(value){cancelMouseInput();closeMenu();closeKeyboard();modifiers.clear();}
|
||||
setVisibility(value?GONE:VISIBLE);positionMouse();
|
||||
}
|
||||
private void toggleMouse(){cancelMouseInput();mouseVisible=!mouseVisible;closeMenu();positionMouse();}
|
||||
private void positionMouse(){
|
||||
if(mouseBar==null)return;
|
||||
boolean visible=mouseVisible&&menu==null&&waiting.getVisibility()==GONE;
|
||||
boolean visible=!pictureInPicture&&mouseVisible&&menu==null&&waiting.getVisibility()==GONE;
|
||||
if(!visible){if(mouseBar.getVisibility()==VISIBLE)cancelMouseInput();mouseBar.setVisibility(GONE);return;}
|
||||
int[] location=new int[2],hostLocation=new int[2];getLocationOnScreen(location);mouseHost.getLocationOnScreen(hostLocation);
|
||||
float width=ui.dp(140),height=ui.dp(48),gap=ui.dp(6),margin=ui.dp(8);
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
<vector xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">
|
||||
<path android:fillColor="#FFFFFFFF" android:pathData="M3,3h18v14h-8v3h4v2H7v-2h4v-3H3zM5,5v10h14V5z" />
|
||||
</vector>
|
||||
@@ -477,6 +477,10 @@ extern "C" JNIEXPORT void JNICALL JNI_METHOD(nClipboard)(
|
||||
s->Send(kClipboardStream, bytes);
|
||||
}
|
||||
extern "C" JNIEXPORT void JNICALL JNI_METHOD(nSurface)(JNIEnv* env, jclass, jlong handle, jobject surface) {
|
||||
if (auto* s = From(handle)) s->Surface(env, surface);
|
||||
if (auto* s = From(handle)) {
|
||||
s->Surface(env, surface);
|
||||
if (surface && s->controller && !s->stopping)
|
||||
RequestVideoKeyFrame(s->controller, MakeDisplayStreamId(s->display.load()).c_str());
|
||||
}
|
||||
}
|
||||
extern "C" JNIEXPORT void JNICALL JNI_METHOD(nDestroy)(JNIEnv*, jclass, jlong handle) { delete From(handle); }
|
||||
|
||||
@@ -22,6 +22,10 @@
|
||||
<string>$(CURRENT_PROJECT_VERSION)</string>
|
||||
<key>LSRequiresIPhoneOS</key>
|
||||
<true/>
|
||||
<key>UIBackgroundModes</key>
|
||||
<array>
|
||||
<string>audio</string>
|
||||
</array>
|
||||
<key>NSLocalNetworkUsageDescription</key>
|
||||
<string>CrossDesk 使用局域网发现并建立原生 P2P 远程控制连接。</string>
|
||||
<key>UIApplicationSupportsIndirectInputEvents</key>
|
||||
|
||||
@@ -239,6 +239,16 @@ private final class RemoteAudioPlayer {
|
||||
}
|
||||
}
|
||||
|
||||
func endSession() {
|
||||
queue.async {
|
||||
self.enabled = false
|
||||
self.player.stop()
|
||||
self.engine.stop()
|
||||
self.queuedBuffers = 0
|
||||
try? AVAudioSession.sharedInstance().setActive(false, options: [.notifyOthersOnDeactivation])
|
||||
}
|
||||
}
|
||||
|
||||
func enqueue(_ data: Data) {
|
||||
guard !data.isEmpty, data.count.isMultiple(of: MemoryLayout<Int16>.size) else { return }
|
||||
queue.async {
|
||||
@@ -339,6 +349,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
@Published private(set) var remotePlatform: RemoteHostPlatform = .unknown
|
||||
|
||||
let bridge = CrossDeskRTCBridge()
|
||||
let pictureInPicture = RemotePictureInPicture()
|
||||
let announcements = AnnouncementInbox()
|
||||
let appUpdates = AppUpdateChecker()
|
||||
private var appIsActive = false
|
||||
@@ -392,6 +403,22 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
override init() {
|
||||
super.init()
|
||||
bridge.delegate = self
|
||||
pictureInPicture.requestFrame = { [weak self] in
|
||||
guard let self, self.isConnected else { return }
|
||||
self.bridge.requestKeyFrame()
|
||||
}
|
||||
pictureInPicture.restoreInterface = { [weak self] in
|
||||
guard let self, self.isConnected else { return false }
|
||||
self.sessionVisible = true
|
||||
return true
|
||||
}
|
||||
pictureInPicture.playbackChanged = { [weak self] playing in
|
||||
guard let self, self.isConnected else { return }
|
||||
self.audioPlayer.setEnabled(playing && self.audioEnabled)
|
||||
}
|
||||
pictureInPicture.failed = { [weak self] in
|
||||
self?.controlStatus = "画中画未能开启，请保持应用在前台后重试。"
|
||||
}
|
||||
announcements.send = { [weak bridge] data, requestID in
|
||||
bridge?.sendAnnouncementRequest(data, requestID: requestID)
|
||||
}
|
||||
@@ -408,6 +435,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
|
||||
func applicationDidBecomeActive() {
|
||||
appIsActive = true
|
||||
pictureInPicture.didBecomeActive()
|
||||
appUpdates.setEnabled(hasNetworkConsent)
|
||||
privacyNotice.didBecomeActive(hasNetworkConsent: hasNetworkConsent)
|
||||
refreshRecentConnectionPresenceAfterForeground()
|
||||
@@ -656,6 +684,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
}
|
||||
|
||||
private func resetConnection() {
|
||||
pictureInPicture.endSession()
|
||||
clipboardStatus = ""
|
||||
controlStatus = ""
|
||||
transferStatus = ""
|
||||
@@ -682,7 +711,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
displaySizes = []
|
||||
selectedDisplay = 0
|
||||
connectionStatus = "未连接"
|
||||
audioPlayer.setEnabled(false)
|
||||
audioPlayer.endSession()
|
||||
AppOrientation.update(to: .portrait)
|
||||
appUpdates.checkNow()
|
||||
}
|
||||
@@ -1124,6 +1153,7 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
case 1:
|
||||
connectionStatus = "已连接"
|
||||
sessionVisible = true
|
||||
pictureInPicture.beginSession()
|
||||
if videoSettings.requestID == 0 { updateVideoSettings() }
|
||||
checkRemoteVersionIfNeeded()
|
||||
recordSuccessfulConnectionIfNeeded()
|
||||
@@ -1226,7 +1256,16 @@ final class RemoteSessionModel: NSObject, ObservableObject, CrossDeskRTCBridgeDe
|
||||
width: Int,
|
||||
height: Int,
|
||||
captureUptime: TimeInterval) {
|
||||
guard hasNetworkConsent, isConnected, !videoWasBackgrounded else { return }
|
||||
guard hasNetworkConsent, isConnected else { return }
|
||||
if videoWasBackgrounded {
|
||||
// Feed PiP directly, without publishing background SwiftUI updates.
|
||||
if pictureInPicture.display(pixelBuffer) {
|
||||
videoFrameID &+= 1
|
||||
networkStatistics.recordSubmittedFrame(id: videoFrameID, captureUptime: captureUptime,
|
||||
at: ProcessInfo.processInfo.systemUptime)
|
||||
}
|
||||
return
|
||||
}
|
||||
cancelVideoRecovery()
|
||||
// Buffer and encoded dimensions must be one observable value. Adaptive
|
||||
// resolution changes must never expose a new frame with the previous
|
||||
|
||||
@@ -60,7 +60,8 @@ struct RemoteSessionView: View {
|
||||
pixelBuffer: session.pixelBuffer,
|
||||
frameID: session.currentVideoFrameID,
|
||||
captureUptime: session.frameCaptureUptime,
|
||||
onFrameSubmitted: session.recordVideoFrameSubmission
|
||||
onFrameSubmitted: session.recordVideoFrameSubmission,
|
||||
pictureInPicture: session.pictureInPicture
|
||||
)
|
||||
|
||||
if let cursorPosition,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import AVFoundation
|
||||
import AVKit
|
||||
import CoreMedia
|
||||
import CoreVideo
|
||||
import SwiftUI
|
||||
@@ -15,16 +16,158 @@ struct NativeVideoView: UIViewRepresentable {
|
||||
let frameID: UInt64
|
||||
let captureUptime: TimeInterval
|
||||
let onFrameSubmitted: (UInt64, TimeInterval) -> Void
|
||||
let pictureInPicture: RemotePictureInPicture
|
||||
|
||||
func makeUIView(context: Context) -> SampleBufferVideoView {
|
||||
SampleBufferVideoView(frame: .zero)
|
||||
let view = SampleBufferVideoView(frame: .zero)
|
||||
pictureInPicture.attach(view)
|
||||
return view
|
||||
}
|
||||
|
||||
func updateUIView(_ uiView: SampleBufferVideoView, context: Context) {
|
||||
if uiView.display(pixelBuffer, isActive: scenePhase == .active) {
|
||||
// Background frames go straight from the RTC delegate to the display
|
||||
// layer: SwiftUI does not promise view updates while backgrounded.
|
||||
guard scenePhase != .background, !pictureInPicture.isPaused else { return }
|
||||
if uiView.display(pixelBuffer, isActive: true) {
|
||||
onFrameSubmitted(frameID, captureUptime)
|
||||
}
|
||||
}
|
||||
|
||||
static func dismantleUIView(_ uiView: SampleBufferVideoView, coordinator: ()) {
|
||||
_ = uiView.display(nil, isActive: false)
|
||||
}
|
||||
}
|
||||
|
||||
/// Uses actual live remote video for background playback, including when muted.
|
||||
/// Closing/pausing PiP removes that background execution opportunity; it does
|
||||
/// not fabricate silent audio or repeatedly request background task time.
|
||||
final class RemotePictureInPicture: NSObject, AVPictureInPictureControllerDelegate,
|
||||
AVPictureInPictureSampleBufferPlaybackDelegate {
|
||||
private var controller: AVPictureInPictureController?
|
||||
private var videoView: SampleBufferVideoView?
|
||||
private var sessionActive = false
|
||||
private var starting = false
|
||||
private var paused = false
|
||||
var requestFrame: (() -> Void)?
|
||||
var restoreInterface: (() -> Bool)?
|
||||
var playbackChanged: ((Bool) -> Void)?
|
||||
var failed: (() -> Void)?
|
||||
|
||||
var isRendering: Bool {
|
||||
sessionActive && !paused && (starting || controller?.isPictureInPictureActive == true)
|
||||
}
|
||||
var isPaused: Bool { paused && controller?.isPictureInPictureActive == true }
|
||||
|
||||
func attach(_ view: SampleBufferVideoView) {
|
||||
guard videoView !== view else { return }
|
||||
controller?.stopPictureInPicture()
|
||||
controller?.delegate = nil
|
||||
videoView = view
|
||||
guard AVPictureInPictureController.isPictureInPictureSupported() else { return }
|
||||
let source = AVPictureInPictureController.ContentSource(
|
||||
sampleBufferDisplayLayer: view.videoLayer, playbackDelegate: self)
|
||||
let controller = AVPictureInPictureController(contentSource: source)
|
||||
controller.delegate = self
|
||||
controller.requiresLinearPlayback = true
|
||||
controller.canStartPictureInPictureAutomaticallyFromInline = sessionActive
|
||||
self.controller = controller
|
||||
}
|
||||
|
||||
func beginSession() {
|
||||
guard !sessionActive else { return }
|
||||
sessionActive = true
|
||||
paused = false
|
||||
// PiP needs an active playback session even when the remote is muted.
|
||||
// There is no generated audio: only received desktop audio is played.
|
||||
do {
|
||||
let audio = AVAudioSession.sharedInstance()
|
||||
try audio.setCategory(.playback, mode: .moviePlayback, options: [.mixWithOthers])
|
||||
try audio.setActive(true)
|
||||
} catch {
|
||||
NSLog("CrossDesk could not activate Picture in Picture audio session: %@", error.localizedDescription)
|
||||
}
|
||||
controller?.canStartPictureInPictureAutomaticallyFromInline = true
|
||||
controller?.invalidatePlaybackState()
|
||||
}
|
||||
|
||||
func endSession() {
|
||||
sessionActive = false
|
||||
starting = false
|
||||
paused = false
|
||||
controller?.canStartPictureInPictureAutomaticallyFromInline = false
|
||||
controller?.stopPictureInPicture()
|
||||
controller?.invalidatePlaybackState()
|
||||
_ = videoView?.display(nil, isActive: false)
|
||||
}
|
||||
|
||||
func didBecomeActive() {
|
||||
paused = false
|
||||
if controller?.isPictureInPictureActive == true { controller?.stopPictureInPicture() }
|
||||
controller?.invalidatePlaybackState()
|
||||
playbackChanged?(true)
|
||||
}
|
||||
|
||||
@discardableResult
|
||||
func display(_ buffer: CVPixelBuffer) -> Bool {
|
||||
guard isRendering else { return false }
|
||||
return videoView?.display(buffer, isActive: true) ?? false
|
||||
}
|
||||
|
||||
func pictureInPictureControllerWillStartPictureInPicture(_ controller: AVPictureInPictureController) {
|
||||
guard self.controller === controller, sessionActive else { return }
|
||||
starting = true
|
||||
requestFrame?()
|
||||
}
|
||||
|
||||
func pictureInPictureControllerDidStartPictureInPicture(_ controller: AVPictureInPictureController) {
|
||||
guard self.controller === controller else { return }
|
||||
starting = false
|
||||
if !sessionActive { controller.stopPictureInPicture() }
|
||||
else { requestFrame?() }
|
||||
}
|
||||
|
||||
func pictureInPictureController(_ controller: AVPictureInPictureController,
|
||||
failedToStartPictureInPictureWithError error: Error) {
|
||||
guard self.controller === controller else { return }
|
||||
starting = false
|
||||
NSLog("CrossDesk Picture in Picture could not start: %@", error.localizedDescription)
|
||||
failed?()
|
||||
}
|
||||
|
||||
func pictureInPictureControllerDidStopPictureInPicture(_ controller: AVPictureInPictureController) {
|
||||
guard self.controller === controller else { return }
|
||||
starting = false
|
||||
paused = false
|
||||
playbackChanged?(true)
|
||||
}
|
||||
|
||||
func pictureInPictureController(_ controller: AVPictureInPictureController,
|
||||
restoreUserInterfaceForPictureInPictureStopWithCompletionHandler completion: @escaping (Bool) -> Void) {
|
||||
completion(sessionActive && (restoreInterface?() ?? false))
|
||||
}
|
||||
|
||||
func pictureInPictureController(_ controller: AVPictureInPictureController, setPlaying playing: Bool) {
|
||||
paused = !playing
|
||||
playbackChanged?(playing)
|
||||
if playing { requestFrame?() }
|
||||
controller.invalidatePlaybackState()
|
||||
}
|
||||
|
||||
func pictureInPictureControllerTimeRangeForPlayback(_ controller: AVPictureInPictureController) -> CMTimeRange {
|
||||
sessionActive ? CMTimeRange(start: .zero, duration: .positiveInfinity) : .invalid
|
||||
}
|
||||
|
||||
func pictureInPictureControllerIsPlaybackPaused(_ controller: AVPictureInPictureController) -> Bool {
|
||||
!sessionActive || paused
|
||||
}
|
||||
|
||||
func pictureInPictureController(_ controller: AVPictureInPictureController,
|
||||
didTransitionToRenderSize newRenderSize: CMVideoDimensions) { }
|
||||
|
||||
func pictureInPictureController(_ controller: AVPictureInPictureController,
|
||||
skipByInterval skipInterval: CMTime, completion: @escaping () -> Void) {
|
||||
completion() // Live desktop video has no seekable history.
|
||||
}
|
||||
}
|
||||
|
||||
final class SampleBufferVideoView: UIView {
|
||||
@@ -32,7 +175,7 @@ final class SampleBufferVideoView: UIView {
|
||||
AVSampleBufferDisplayLayer.self
|
||||
}
|
||||
|
||||
private var videoLayer: AVSampleBufferDisplayLayer {
|
||||
var videoLayer: AVSampleBufferDisplayLayer {
|
||||
layer as! AVSampleBufferDisplayLayer
|
||||
}
|
||||
|
||||
@@ -60,6 +203,14 @@ final class SampleBufferVideoView: UIView {
|
||||
// calculation inside AVSampleBufferDisplayLayer. That rounding becomes
|
||||
// visibly amplified when the remote desktop is zoomed up to 10x.
|
||||
videoLayer.videoGravity = .resize
|
||||
var timebase: CMTimebase?
|
||||
if CMTimebaseCreateWithSourceClock(allocator: kCFAllocatorDefault,
|
||||
sourceClock: CMClockGetHostTimeClock(),
|
||||
timebaseOut: &timebase) == noErr, let timebase {
|
||||
CMTimebaseSetTime(timebase, time: CMClockGetTime(CMClockGetHostTimeClock()))
|
||||
CMTimebaseSetRate(timebase, rate: 1)
|
||||
videoLayer.controlTimebase = timebase
|
||||
}
|
||||
}
|
||||
|
||||
/// True only when a new buffer was submitted; no presentation timestamp is
|
||||
@@ -123,7 +274,7 @@ final class SampleBufferVideoView: UIView {
|
||||
|
||||
var timing = CMSampleTimingInfo(
|
||||
duration: .invalid,
|
||||
presentationTimeStamp: .invalid,
|
||||
presentationTimeStamp: CMClockGetTime(CMClockGetHostTimeClock()),
|
||||
decodeTimeStamp: .invalid
|
||||
)
|
||||
var sampleBuffer: CMSampleBuffer?
|
||||
|
||||
@@ -4,6 +4,34 @@ This target is a native MiniRTC controller. It uses the same WebSocket
|
||||
signaling, libnice ICE, SRTP/RTP and data-stream protocol as the desktop app.
|
||||
There is no `WKWebView` or browser runtime.
|
||||
|
||||
## Background connection and Picture in Picture
|
||||
|
||||
An established remote desktop session supports native Picture in Picture (PiP).
|
||||
After the first video frame appears, switching to another app automatically
|
||||
starts PiP when the device and system settings allow it. The PiP window shows
|
||||
the live remote desktop; restoring the app returns to the existing session and
|
||||
requests a fresh key frame. Remote mouse buttons are released when the app loses focus.
|
||||
|
||||
The app declares the `audio` background mode required by AVKit for PiP and
|
||||
activates a playback audio session, including for muted remote video. Background
|
||||
frames are submitted directly from the RTC delegate to the sample-buffer display
|
||||
layer, without relying on SwiftUI background redraws. Only real remote audio is
|
||||
played; no synthetic silent audio or recurring background-task assertions are
|
||||
used. Disconnecting or withdrawing consent stops PiP and releases playback.
|
||||
|
||||
Background execution depends on active PiP or actual remote audio playback.
|
||||
Closing or pausing PiP, disabling automatic PiP in system settings, locking the
|
||||
device, losing the network, or force-quitting can suspend or interrupt the
|
||||
session. iOS does not provide an unrestricted background socket keepalive.
|
||||
The connection is not deliberately closed on background entry, but continuous
|
||||
connectivity cannot be promised outside the supported playback modes.
|
||||
|
||||
On a physical iPhone/iPad, verify automatic PiP when switching apps with remote
|
||||
audio both enabled and muted, continuously changing remote frames while another
|
||||
app is in front, restoration, pause/resume, close, lock/unlock, and disconnection while PiP
|
||||
is active. Also verify the system setting that disables automatic PiP. A build
|
||||
or simulator check alone does not establish long-duration background behavior.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Xcode 16 or newer
|
||||
|
||||
Reference in New Issue
Block a user