diff --git a/docs/SellySDK_直播推拉流接入文档_Android.md b/docs/SellySDK_直播推拉流接入文档_Android.md index bd59e63..4bc7a99 100644 --- a/docs/SellySDK_直播推拉流接入文档_Android.md +++ b/docs/SellySDK_直播推拉流接入文档_Android.md @@ -582,7 +582,9 @@ player.play() - `RTMP` 播放支持 `SurfaceView`、`TextureView`、`SurfaceTexture` - `RTC/WHEP` 播放支持 `SurfaceViewRenderer`、`TextureView`,以及高级场景下的 `SurfaceTexture` -- 当前版本建议在 **开始播放前** 选定渲染后端;当前 Demo 在首页设置中统一选择,进入页面后不再暴露热切换 +- `RTMP/VOD` 的 `TextureView / SurfaceTexture` 默认走 **direct output**,优先保证首帧和低延迟 +- 当前版本建议在 **开始播放前** 选定渲染后端;运行中如需变更目标,请走 `clearRenderTarget()` + `setRenderView(...)` / `setRenderSurfaceTexture(...)` 的显式重绑流程 +- Flutter 场景优先使用 `setRenderSurfaceTexture(...)`,配合 Flutter `Texture` widget 使用;如 UI 层级正确性优先,不建议继续依赖 `Hybrid Composition + SurfaceView` 控制接口: @@ -599,6 +601,29 @@ player.play() - `clearRenderTarget()`:解绑当前渲染面,播放会话可继续存活 - `seekBy(deltaMs)`:播放进度跳转(仅在流支持快进/回放时有效) +### 7.3.2 Flutter / SurfaceTexture 接入建议 + +如果业务侧需要把视频放到 Flutter UI 层下面,并正常叠加按钮、封面、弹层、动画,推荐使用: + +- Flutter 侧创建 `TextureRegistry.SurfaceTextureEntry` +- Android 插件层取出 `SurfaceTexture` +- 调用 `setRenderSurfaceTexture(surfaceTexture, width, height)` +- Flutter 页面使用 `Texture(textureId)` 显示视频 + +示意: + +```kotlin +player.setRenderSurfaceTexture(surfaceTexture, width, height) +player.prepareToPlay() +player.play() +``` + +说明: + +- `SurfaceTexture` 生命周期由调用方负责 +- 销毁前建议先调用 `clearRenderTarget()` 或直接 `release()` +- 如果页面重建、Texture 重新申请或 Flutter 侧切换 textureId,需要重新绑定新的 `SurfaceTexture` + ### 7.4 播放回调 ```kotlin @@ -622,6 +647,60 @@ player.delegate = object : SellyLiveVideoPlayerDelegate { - `Reconnecting` - `Failed` +首帧语义说明: + +- 默认 `DIRECT` 模式下,`onFirstVideoFrameRendered()` 对应 decoder 首帧可用时机 +- 对 `TextureView / SurfaceTexture` 且启用了 playback processing 的场景,SDK 会等待目标渲染面确认首帧已真正呈现后,再触发 `onFirstVideoFrameRendered()` +- `onFirstAudioFrameRendered()` 仍表示音频首帧可播放时机;在 texture-backed processing 场景中,音频与视频首帧不一定完全同一时刻 + +### 7.4.1 播放侧帧回调与二次处理 + +播放器支持一组独立于采集/推流链路的播放侧高级能力: + +- `setPlaybackFrameObserver(observer)`:播放侧只读帧回调 +- `setPlaybackVideoProcessor(processor)`:播放侧可写纹理处理 + +当前能力边界: + +- 当前仅支持 **texture-backed** 播放目标:`TextureView` / `SurfaceTexture` +- 当前仅支持 `preferredFormat = TEXTURE_2D` +- 当前仅支持 `stage = RENDER_PRE_DISPLAY` +- 当前默认渲染模式为 `DIRECT` +- 只有设置了有效的 observer / processor,才会切到 `PROCESSING` +- 如果当前 render target 已经绑定,新增或移除 observer / processor 后,需要 **重绑一次 texture render target** 才会生效 +- `RTC/WHEP` 播放当前不支持这套 playback processing;当前主要用于 `RTMP/VOD` 播放链 + +只读 observer 示例: + +```kotlin +player.setPlaybackFrameObserver(object : PlaybackFrameObserver { + override val config = PlaybackFrameObserverConfig( + preferredFormat = VideoProcessFormat.TEXTURE_2D, + stage = VideoStage.RENDER_PRE_DISPLAY + ) + + override fun onTextureFrame(frame: VideoTextureFrame) { + // 读取播放侧纹理帧信息 + } +}) +``` + +可写 processor 示例: + +```kotlin +player.setPlaybackVideoProcessor(object : PlaybackVideoProcessor { + override val config = PlaybackVideoProcessorConfig( + preferredFormat = VideoProcessFormat.TEXTURE_2D, + mode = VideoProcessMode.READ_WRITE, + stage = VideoStage.RENDER_PRE_DISPLAY + ) + + override fun processTexture(input: VideoTextureFrame, outputTextureId: Int) { + // 将后处理结果写入 outputTextureId + } +}) +``` + ### 7.5 播放 API 速览(含 Demo 未覆盖) 创建与渲染: @@ -634,6 +713,8 @@ player.delegate = object : SellyLiveVideoPlayerDelegate { - `setRenderSurfaceTexture(surfaceTexture, width, height)`:绑定 `SurfaceTexture`(调用方负责 SurfaceTexture 生命周期) - `clearRenderTarget()`:解绑当前渲染面 - `getRenderView()`:获取当前渲染 View +- `setPlaybackFrameObserver(observer)`:设置播放侧只读 observer(texture 路径) +- `setPlaybackVideoProcessor(processor)`:设置播放侧 processor(texture 路径) 播放控制: @@ -655,6 +736,12 @@ player.delegate = object : SellyLiveVideoPlayerDelegate { - `setRenderView(surfaceView)` / `setRenderView(textureView)`:手动绑定现有 View - `setRenderSurfaceTexture(surfaceTexture, width, height)`:高级场景使用 `SurfaceTexture`(调用方负责 SurfaceTexture 生命周期) - `clearRenderTarget()`:解绑当前渲染面但不一定立即销毁播放实例 +- `setPlaybackFrameObserver(observer)` / `setPlaybackVideoProcessor(processor)`:点播同样支持 texture-backed playback processing + +补充说明: + +- 点播在重绑 `TextureView / SurfaceTexture` 后,会自动复用最近一次视频宽高信息,保持正确显示比例 +- 如在已有 texture 目标上新增或移除 observer / processor,也需要重绑一次 texture render target 才会应用新的渲染模式 因此 Demo 中点播页的 `SurfaceView / TextureView` 选择,也与直播播放页保持一致,均在首页设置中统一生效。 @@ -681,6 +768,10 @@ player.delegate = object : SellyLiveVideoPlayerDelegate { - 推流前先完成采集预览 - `SurfaceView / TextureView` backend 建议在开始推流或播放前选定 +- Flutter 场景优先使用 `setRenderSurfaceTexture(...)`,不要把 `Hybrid Composition + SurfaceView` 当成默认方案 +- 普通播放默认保持 `DIRECT`;只有确实需要播放侧帧观察或纹理后处理时,再启用 playback processing +- playback processing 当前仅建议用于 `TextureView / SurfaceTexture + TEXTURE_2D + RENDER_PRE_DISPLAY` +- 变更 texture 路径的 observer / processor 后,显式重绑一次 render target - `RTC/WHIP` 的美颜、滤镜、水印、观测优先使用 `TEXTURE_2D` - `I420 / RGBA` 仅在算法必须访问 CPU 像素时再使用 - 完整重写输出的 GPU 处理器设置 `fullRewrite = true`;叠加类处理保留默认值 @@ -723,13 +814,13 @@ SDK 不解析 URL 中的鉴权信息,所有鉴权均通过 `token` 属性完 - 普通原生 Android 页面,优先使用默认 `SurfaceView`,性能最优 - 需要与按钮、封面、弹层等普通 View 正常混排时,优先使用 `TextureView` -- Flutter 场景通过 `setRenderSurfaceTexture()` 接入,走 `TextureView` 同一套渲染管线 +- Flutter 场景通过 `setRenderSurfaceTexture()` 接入,配合 Flutter `Texture` widget 使用 - 当前版本建议在开始推流/播放前选定 backend;当前 Demo 在首页设置中统一选择,进入页面后不支持切换 ### Q5.1:`TextureView` 模式下,VOD/RTMP 播放的 `BufferQueueProducer timeout` 日志是什么? **A:** -SDK 内部使用 GL Bridge 将 MediaCodec 硬解输出通过 OpenGL 中转渲染到 TextureView,大幅减少此类日志。如在极端场景下仍偶现,属于 Android 系统 BufferQueue 机制限制,不影响播放功能。`SurfaceView` 路径不存在此问题。 +当前 `RTMP/VOD` 的 `TextureView / SurfaceTexture` 默认走 direct output,以缩短首帧和减少黑屏。极端机型或系统版本下仍可能偶现 `BufferQueueProducer timeout` / `BufferQueue has been abandoned` 之类系统日志;如果不伴随黑屏、花屏、卡死,通常可视为 Android BufferQueue 机制噪声。开启 playback processing 时,texture 路径内部会启用额外的处理中转链,日志形态也可能与 direct 模式不同。 ### Q5.2:`attach` 和 `set` 两套 API 的区别? **A:** diff --git a/example/libs/sellycloudsdk-1.0.1.aar b/example/libs/sellycloudsdk-1.0.1.aar index a99eebd..dcd208d 100644 Binary files a/example/libs/sellycloudsdk-1.0.1.aar and b/example/libs/sellycloudsdk-1.0.1.aar differ diff --git a/example/src/main/java/com/demo/SellyCloudSDK/FeatureHubActivity.kt b/example/src/main/java/com/demo/SellyCloudSDK/FeatureHubActivity.kt index f85f210..a5650f8 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/FeatureHubActivity.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/FeatureHubActivity.kt @@ -171,7 +171,7 @@ class FeatureHubActivity : AppCompatActivity() { if (dy <= 0) return val lastVisible = layoutManager.findLastVisibleItemPosition() if (lastVisible >= aliveAdapter.itemCount - 2) { - appendNextPage() + recyclerView.post { appendNextPage() } } } }) @@ -243,12 +243,14 @@ class FeatureHubActivity : AppCompatActivity() { } val url = item.url?.trim().orEmpty() + val xorKey = item.xorKey.orEmpty() val intent = if (url.isNotEmpty()) { LivePlayActivity.createIntent( this, resolvePlayModeFromUrl(url), url, - autoStart = true + autoStart = true, + xorKeyHex = xorKey ) } else { val liveMode = resolvePlayMode(item.playProtocol) @@ -263,7 +265,8 @@ class FeatureHubActivity : AppCompatActivity() { params.vhost, params.appName, params.streamName, - autoStart = true + autoStart = true, + xorKeyHex = xorKey ) }.apply { item.previewImage?.let { putExtra(LivePlayActivity.EXTRA_PREVIEW_IMAGE_URL, it) } diff --git a/example/src/main/java/com/demo/SellyCloudSDK/live/LivePlayActivity.kt b/example/src/main/java/com/demo/SellyCloudSDK/live/LivePlayActivity.kt index d1b966b..3423841 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/live/LivePlayActivity.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/live/LivePlayActivity.kt @@ -45,6 +45,9 @@ import com.demo.SellyCloudSDK.live.env.normalizedAppName import com.demo.SellyCloudSDK.live.env.normalizedVhost import com.demo.SellyCloudSDK.live.env.toLiveMode import com.demo.SellyCloudSDK.live.util.GalleryImageSaver +import com.demo.SellyCloudSDK.playback.PlaybackProcessingPreset +import com.demo.SellyCloudSDK.playback.PlaybackTextureObserverDemo +import com.demo.SellyCloudSDK.playback.PlaybackTexturePatchProcessor import com.sellycloud.sellycloudsdk.SellyLatencyChasingUpdate import com.sellycloud.sellycloudsdk.SellyLiveMode import com.sellycloud.sellycloudsdk.SellyLiveVideoPlayer @@ -89,6 +92,9 @@ class LivePlayActivity : AppCompatActivity() { private var lastLatencyChasingUpdate: SellyLatencyChasingUpdate? = null private var hasReleasedPlayer: Boolean = false private var logEnabled: Boolean = true + private var processingPreset: PlaybackProcessingPreset = PlaybackProcessingPreset.DIRECT + private var renderTargetRebindCount: Int = 0 + private var lastRenderTargetRebindCostMs: Long? = null private val logLines: ArrayDeque = ArrayDeque() private val logTimeFormat = SimpleDateFormat("HH:mm:ss.SSS", Locale.getDefault()) @@ -96,6 +102,14 @@ class LivePlayActivity : AppCompatActivity() { private var logSummaryView: TextView? = null private var logContentView: TextView? = null private var logFloatingButton: View? = null + private var toolsFloatingButton: View? = null + + private val playbackObserverDemo by lazy(LazyThreadSafetyMode.NONE) { + PlaybackTextureObserverDemo(::logEvent) + } + private val playbackPatchProcessor by lazy(LazyThreadSafetyMode.NONE) { + PlaybackTexturePatchProcessor(::logEvent) + } private val storagePermissionLauncher = registerForActivityResult( ActivityResultContracts.RequestPermission() @@ -111,6 +125,7 @@ class LivePlayActivity : AppCompatActivity() { setContentView(binding.root) supportActionBar?.hide() addLogFloatingButton() + addToolsFloatingButton() envStore = LiveEnvSettingsStore(this) useTextureView = AvDemoSettingsStore(this).read().renderBackendPreference.isTextureView() @@ -221,8 +236,9 @@ class LivePlayActivity : AppCompatActivity() { binding.actionScreenshot.setOnClickListener { captureCurrentFrame() } binding.actionPip.setOnClickListener { enterPipMode() } - val backend = if (useTextureView) RenderBackend.TEXTURE_VIEW else RenderBackend.SURFACE_VIEW + val backend = currentRenderBackend() playerClient.attachRenderView(binding.renderContainer, backend) + logEvent("渲染目标已绑定: backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") if (args.autoStart) { lifecycleScope.launch { @@ -429,6 +445,7 @@ class LivePlayActivity : AppCompatActivity() { binding.controlBar.visibility = controlsVisibility binding.btnClose.visibility = controlsVisibility logFloatingButton?.visibility = controlsVisibility + toolsFloatingButton?.visibility = controlsVisibility if (isInPip) { binding.ivPreview.visibility = View.GONE } else { @@ -547,6 +564,99 @@ class LivePlayActivity : AppCompatActivity() { logFloatingButton = button } + private fun addToolsFloatingButton() { + val sizePx = dpToPx(44) + val marginEndPx = dpToPx(72) + val controlBarHeight = resources.getDimensionPixelSize(R.dimen.av_control_bar_height) + val marginBottomPx = controlBarHeight + dpToPx(16) + val bgDrawable = GradientDrawable(GradientDrawable.Orientation.TOP_BOTTOM, intArrayOf( + Color.parseColor("#B33B0764"), + Color.parseColor("#803B0764") + )).apply { + shape = GradientDrawable.OVAL + setStroke(dpToPx(1), Color.parseColor("#55FFFFFF")) + } + val button = AppCompatTextView(this).apply { + text = "测" + setTextColor(Color.parseColor("#F8FAFC")) + textSize = 11f + gravity = Gravity.CENTER + background = bgDrawable + elevation = dpToPx(4).toFloat() + setShadowLayer(2f, 0f, 1f, Color.parseColor("#66000000")) + isClickable = true + isFocusable = true + contentDescription = "播放处理与回归工具" + setOnClickListener { showPlaybackToolsDialog() } + } + val params = FrameLayout.LayoutParams(sizePx, sizePx).apply { + gravity = Gravity.END or Gravity.BOTTOM + marginEnd = marginEndPx + bottomMargin = marginBottomPx + } + addContentView(button, params) + toolsFloatingButton = button + } + + private fun showPlaybackToolsDialog() { + val container = LinearLayout(this).apply { + orientation = LinearLayout.VERTICAL + setPadding(dpToPx(20), dpToPx(16), dpToPx(20), dpToPx(8)) + } + val summary = TextView(this).apply { + text = "当前后端: ${currentRenderBackendLabel()}\n" + + "当前协议: ${args.liveMode.name}\n" + + "当前模式: ${processingPreset.label}\n" + + "说明: processing 仅支持 RTMP + TextureView。" + setTextColor(Color.parseColor("#E5E7EB")) + textSize = 13f + } + container.addView(summary) + container.addView(spaceView(dpToPx(12))) + container.addView(createToolActionButton("切换 DIRECT 直出") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.DIRECT, trigger = "工具面板") + }) + container.addView(createToolActionButton("切换 PROCESSING Observer") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.OBSERVER, trigger = "工具面板") + }) + container.addView(createToolActionButton("切换 PROCESSING 红块 Processor") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.PROCESSOR, trigger = "工具面板") + }) + container.addView(createToolActionButton("仅重绑当前目标") { + rebindRenderTarget("手动回归") + }) + + AlertDialog.Builder(this) + .setTitle("播放处理 / 目标重绑") + .setView(container) + .setNegativeButton("关闭", null) + .show() + } + + private fun createToolActionButton(label: String, onClick: () -> Unit): View { + return AppCompatTextView(this).apply { + text = label + gravity = Gravity.CENTER + textSize = 14f + setTextColor(Color.parseColor("#F8FAFC")) + background = GradientDrawable().apply { + cornerRadius = dpToPx(10).toFloat() + setColor(Color.parseColor("#334155")) + setStroke(dpToPx(1), Color.parseColor("#475569")) + } + setPadding(dpToPx(12), dpToPx(12), dpToPx(12), dpToPx(12)) + isClickable = true + isFocusable = true + setOnClickListener { onClick() } + layoutParams = LinearLayout.LayoutParams( + ViewGroup.LayoutParams.MATCH_PARENT, + ViewGroup.LayoutParams.WRAP_CONTENT + ).apply { + bottomMargin = dpToPx(10) + } + } + } + private fun showLogDialog() { if (logDialog?.isShowing == true) { refreshLogDialogContent() @@ -680,10 +790,20 @@ class LivePlayActivity : AppCompatActivity() { builder.append("streamName: ").append(params.streamName).append('\n') } builder.append("当前状态: ").append(formatState(currentState)).append('\n') + builder.append("渲染后端: ").append(currentRenderBackendLabel()).append('\n') + builder.append("播放处理: ").append(processingPreset.label).append('\n') builder.append("是否播放中: ").append(if (isPlaying) "是" else "否").append('\n') builder.append("是否静音: ").append(if (isMuted) "是" else "否").append('\n') builder.append("首帧视频耗时(ms): ").append(firstVideoFrameCostMs ?: "未统计").append('\n') builder.append("首帧音频耗时(ms): ").append(firstAudioFrameCostMs ?: "未统计").append('\n') + builder.append("目标重绑次数: ").append(renderTargetRebindCount).append('\n') + builder.append("最近重绑耗时(ms): ").append(lastRenderTargetRebindCostMs ?: "未统计").append('\n') + val processingDetail = when (processingPreset) { + PlaybackProcessingPreset.DIRECT -> "processing: 关闭" + PlaybackProcessingPreset.OBSERVER -> playbackObserverDemo.summary() + PlaybackProcessingPreset.PROCESSOR -> playbackPatchProcessor.summary() + } + builder.append(processingDetail).append('\n') val attemptElapsed = playAttemptStartElapsedMs?.let { SystemClock.elapsedRealtime() - it } if (attemptElapsed == null) { builder.append("本次播放已耗时(ms): 未开始").append('\n') @@ -752,7 +872,81 @@ class LivePlayActivity : AppCompatActivity() { isLatencyChasingActive = false lastLatencyChasingSpeed = null lastLatencyChasingUpdate = null - logEvent("播放尝试开始") + logEvent("播放尝试开始: backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") + } + + private fun applyPlaybackProcessingPreset(preset: PlaybackProcessingPreset, trigger: String) { + if (preset == processingPreset) { + logEvent("播放处理保持不变: ${preset.label}, trigger=$trigger") + Toast.makeText(this, "当前已是 ${preset.label}", Toast.LENGTH_SHORT).show() + return + } + if (args.liveMode != SellyLiveMode.RTMP && preset != PlaybackProcessingPreset.DIRECT) { + logEvent("播放处理切换被拒绝: liveMode=${args.liveMode.name} 当前仅支持 RTMP") + Toast.makeText(this, "当前 demo 仅支持 RTMP 播放 processing", Toast.LENGTH_SHORT).show() + return + } + if (!useTextureView && preset != PlaybackProcessingPreset.DIRECT) { + logEvent("播放处理切换被拒绝: backend=${currentRenderBackendLabel()} 不支持 ${preset.label}") + Toast.makeText(this, "播放 processing 仅支持 TextureView 后端", Toast.LENGTH_SHORT).show() + return + } + + processingPreset = preset + configurePlaybackProcessing() + logEvent("播放处理切换: mode=${preset.label}, trigger=$trigger") + rebindRenderTarget("processing_${preset.name.lowercase(Locale.US)}") + Toast.makeText(this, "已切到 ${preset.label}", Toast.LENGTH_SHORT).show() + } + + private fun configurePlaybackProcessing() { + when (processingPreset) { + PlaybackProcessingPreset.DIRECT -> { + playerClient.setPlaybackFrameObserver(null) + playerClient.setPlaybackVideoProcessor(null) + } + PlaybackProcessingPreset.OBSERVER -> { + playerClient.setPlaybackVideoProcessor(null) + playerClient.setPlaybackFrameObserver(playbackObserverDemo) + } + PlaybackProcessingPreset.PROCESSOR -> { + playerClient.setPlaybackFrameObserver(null) + playerClient.setPlaybackVideoProcessor(playbackPatchProcessor) + } + } + } + + private fun rebindRenderTarget(reason: String) { + if (hasReleasedPlayer) return + val shouldResumePlayback = currentState == SellyPlayerState.Playing || + currentState == SellyPlayerState.Connecting || + currentState == SellyPlayerState.Reconnecting + val startedAtMs = SystemClock.elapsedRealtime() + val backend = currentRenderBackend() + logEvent("目标重绑开始: reason=$reason, backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") + playerClient.clearRenderTarget() + playerClient.attachRenderView(binding.renderContainer, backend) + if (shouldResumePlayback) { + logEvent("目标重绑后恢复播放: previousState=${formatState(currentState)}") + startPlayAttempt() + resetPreviewForPlayback() + playerClient.prepareToPlay() + playerClient.play() + } else if (currentState == SellyPlayerState.Paused) { + logEvent("目标重绑完成: 当前处于暂停态,变更将在下次播放时生效") + } + val costMs = SystemClock.elapsedRealtime() - startedAtMs + renderTargetRebindCount += 1 + lastRenderTargetRebindCostMs = costMs + logEvent("目标重绑完成: count=$renderTargetRebindCount, cost=${costMs}ms") + } + + private fun currentRenderBackend(): RenderBackend { + return if (useTextureView) RenderBackend.TEXTURE_VIEW else RenderBackend.SURFACE_VIEW + } + + private fun currentRenderBackendLabel(): String { + return if (useTextureView) "TextureView" else "SurfaceView" } private fun formatLatencyChasingSpeed(speed: Float): String { @@ -840,7 +1034,8 @@ class LivePlayActivity : AppCompatActivity() { vhost: String, appName: String, streamName: String, - autoStart: Boolean = true + autoStart: Boolean = true, + xorKeyHex: String = "" ): Intent { return Intent(context, LivePlayActivity::class.java) .putExtra(EXTRA_PLAY_PROTOCOL, liveMode.name) @@ -848,6 +1043,7 @@ class LivePlayActivity : AppCompatActivity() { .putExtra(EXTRA_PLAY_APP_NAME, appName) .putExtra(EXTRA_PLAY_STREAM_NAME, streamName) .putExtra(EXTRA_AUTO_START, autoStart) + .putExtra(EXTRA_XOR_KEY_HEX, xorKeyHex) } fun closePipIfAny(): Boolean { @@ -908,7 +1104,8 @@ class LivePlayActivity : AppCompatActivity() { val input = intent.getStringExtra(EXTRA_STREAM_ID_OR_URL).orEmpty() .ifBlank { playParams?.streamName ?: env.defaultStreamId } val autoStart = intent.getBooleanExtra(EXTRA_AUTO_START, true) - val xorKeyHex = intent.getStringExtra(EXTRA_XOR_KEY_HEX).orEmpty().trim() + val rawXorKey = intent.getStringExtra(EXTRA_XOR_KEY_HEX).orEmpty().trim() + val xorKeyHex = sanitizeXorKeyHex(rawXorKey) val mode = resolveLiveMode(rawProtocol, input, env) return Args( liveMode = mode, @@ -920,6 +1117,22 @@ class LivePlayActivity : AppCompatActivity() { ) } + private val HEX_REGEX = Regex("^[0-9a-fA-F]+$") + + /** + * Validate and normalize the XOR key. Returns empty string if invalid + * to prevent native crash from malformed keys. + */ + private fun sanitizeXorKeyHex(raw: String): String { + if (raw.isBlank()) return "" + val hex = if (raw.startsWith("0x", ignoreCase = true)) raw.substring(2) else raw + if (hex.isEmpty() || hex.length % 2 != 0 || !HEX_REGEX.matches(hex)) { + android.util.Log.w("LivePlayActivity", "Invalid xorKeyHex '$raw', ignoring to prevent crash") + return "" + } + return hex + } + private fun resolveLiveMode(raw: String?, input: String, env: LiveEnvSettings): SellyLiveMode { val normalized = raw?.trim()?.uppercase() val modeFromExtra = when (normalized) { diff --git a/example/src/main/java/com/demo/SellyCloudSDK/live/LivePushActivity.kt b/example/src/main/java/com/demo/SellyCloudSDK/live/LivePushActivity.kt index 19e6993..4d30ea1 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/live/LivePushActivity.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/live/LivePushActivity.kt @@ -37,7 +37,10 @@ import com.demo.SellyCloudSDK.live.auth.LiveAuthHelper import com.demo.SellyCloudSDK.live.auth.LiveTokenSigner import com.demo.SellyCloudSDK.live.env.LiveEnvSettingsStore import com.demo.SellyCloudSDK.live.env.applyToSdkRuntimeConfig +import com.demo.SellyCloudSDK.live.env.normalizedAppName +import com.demo.SellyCloudSDK.live.env.normalizedVhost import com.demo.SellyCloudSDK.live.env.toLiveMode +import com.demo.SellyCloudSDK.live.square.StreamXorRepository import com.demo.SellyCloudSDK.live.util.GalleryImageSaver import com.sellycloud.sellycloudsdk.CpuUsage import com.sellycloud.sellycloudsdk.Disposable @@ -85,6 +88,7 @@ class LivePushActivity : AppCompatActivity() { private var useTextureView: Boolean = false private var isPublishing: Boolean = false + private var hasReportedXor: Boolean = false private var isStatsCollapsed: Boolean = false private var latestStats: SellyLivePusherStats? = null private var isMuted: Boolean = false @@ -361,11 +365,109 @@ class LivePushActivity : AppCompatActivity() { updateLayoutForOrientationAndState() updateStatusPanel() updateStatsFromStats(latestStats) + if (state == SellyLiveStatus.Publishing && !hasReportedXor) { + reportXorKeyToServer() + } if (state == SellyLiveStatus.Stopped) { + if (hasReportedXor) { + clearXorKeyOnServer() + } + hasReportedXor = false navigateHomeAfterStop() } } + /** + * Report the current XOR key state to the server after push starts. + * Always reports — empty xorKey means "disable & clear cached key". + */ + private fun reportXorKeyToServer() { + val settings = settingsStore.read() + val env = envStore.read() + val xorKey = settings.xorKeyHex + + // Resolve actual stream/app/vhost from the push target + val streamTarget = if (settings.useUrlMode) { + parseRtmpUrl(settings.pushUrl) + } else { + StreamTarget( + stream = settings.streamId, + app = env.normalizedAppName(), + vhost = env.normalizedVhost() + ) + } + if (streamTarget == null || streamTarget.stream.isBlank()) { + debugLog("XOR report skipped: cannot resolve stream target") + return + } + + lastReportedStreamTarget = streamTarget + lifecycleScope.launch { + val ok = StreamXorRepository.reportXorKey( + stream = streamTarget.stream, + app = streamTarget.app, + vhost = streamTarget.vhost, + xorKey = xorKey + ) + if (ok) { + hasReportedXor = true + } + debugLog("XOR report: stream=${streamTarget.stream}, app=${streamTarget.app}, key=$xorKey -> $ok") + } + } + + /** + * Best-effort clear of the XOR key on stop. + */ + private fun clearXorKeyOnServer() { + val target = lastReportedStreamTarget ?: return + lifecycleScope.launch { + val ok = StreamXorRepository.reportXorKey( + stream = target.stream, + app = target.app, + vhost = target.vhost, + xorKey = "" + ) + debugLog("XOR clear: stream=${target.stream} -> $ok") + } + } + + private var lastReportedStreamTarget: StreamTarget? = null + + private data class StreamTarget(val stream: String, val app: String, val vhost: String?) + + /** + * Parse vhost/app/stream from an RTMP URL like: + * rtmp://host:port/app/stream?token=...&vhost=xxx + */ + private fun parseRtmpUrl(url: String): StreamTarget? { + try { + val trimmed = url.trim() + if (!trimmed.lowercase().startsWith("rtmp://")) return null + // Strip scheme + val withoutScheme = trimmed.substringAfter("://") + // Extract host (authority) before first / + val host = withoutScheme.substringBefore("/").substringBefore(":").takeIf { it.isNotBlank() } + val pathAndQuery = withoutScheme.substringAfter("/", "") + if (pathAndQuery.isBlank()) return null + val pathPart = pathAndQuery.substringBefore("?") + val segments = pathPart.split("/").filter { it.isNotBlank() } + if (segments.size < 2) return null + val app = segments[0] + val stream = segments[1] + // Extract vhost from query params; fallback to URL host + val query = pathAndQuery.substringAfter("?", "") + val queryVhost = query.split("&") + .firstOrNull { it.startsWith("vhost=") } + ?.substringAfter("vhost=") + ?.takeIf { it.isNotBlank() } + val vhost = queryVhost ?: host + return StreamTarget(stream = stream, app = app, vhost = vhost) + } catch (_: Exception) { + return null + } + } + private fun navigateHomeAfterStop() { if (hasNavigatedHome || isFinishing || isDestroyed) return hasNavigatedHome = true @@ -613,11 +715,8 @@ class LivePushActivity : AppCompatActivity() { private fun toggleFrameInterceptor() { frameInterceptorMode = when (frameInterceptorMode) { FrameInterceptorMode.OFF -> FrameInterceptorMode.OBSERVE - FrameInterceptorMode.OBSERVE -> FrameInterceptorMode.CPU_EMPTY - FrameInterceptorMode.CPU_EMPTY -> FrameInterceptorMode.CPU_SINGLE - FrameInterceptorMode.CPU_SINGLE -> FrameInterceptorMode.CPU_DOUBLE - FrameInterceptorMode.CPU_DOUBLE -> FrameInterceptorMode.OFF - FrameInterceptorMode.MODIFY -> FrameInterceptorMode.OBSERVE + FrameInterceptorMode.OBSERVE -> FrameInterceptorMode.OFF + else -> FrameInterceptorMode.OBSERVE } resetFrameCallbackWindow(if (frameInterceptorMode == FrameInterceptorMode.OFF) "off" else frameInterceptorMode.label) applyFrameInterceptorState() @@ -677,46 +776,12 @@ class LivePushActivity : AppCompatActivity() { }) } - FrameInterceptorMode.CPU_EMPTY -> { - addFrameObserver(activePusher, object : VideoFrameObserver { - override val config: VideoFrameObserverConfig = VideoFrameObserverConfig( - preferredFormat = VideoProcessFormat.I420 - ) - }) - } - - FrameInterceptorMode.CPU_SINGLE -> { - addFrameObserver(activePusher, object : VideoFrameObserver { - override val config: VideoFrameObserverConfig = VideoFrameObserverConfig( - preferredFormat = VideoProcessFormat.I420 - ) - - override fun onFrame(frame: SellyVideoFrame) { - recordCpuObserverFrame(mode = frameInterceptorMode, frame = frame) - } - }) - } - + FrameInterceptorMode.CPU_EMPTY, + FrameInterceptorMode.CPU_SINGLE, FrameInterceptorMode.CPU_DOUBLE -> { - addFrameObserver(activePusher, object : VideoFrameObserver { - override val config: VideoFrameObserverConfig = VideoFrameObserverConfig( - preferredFormat = VideoProcessFormat.I420 - ) - - override fun onFrame(frame: SellyVideoFrame) { - recordCpuObserverFrame(mode = frameInterceptorMode, frame = frame) - } - }) - addFrameObserver(activePusher, object : VideoFrameObserver { - override val config: VideoFrameObserverConfig = VideoFrameObserverConfig( - preferredFormat = VideoProcessFormat.I420 - ) - - override fun onFrame(frame: SellyVideoFrame) { - // Intentionally empty. This verifies that multiple CPU observers - // sharing the same format do not force duplicate conversion work. - } - }) + // CPU observer modes removed from default toggle. + // SDK still supports I420/RGBA observers; these were disabled in Demo + // because the current pure-Kotlin color conversion is too slow for production. } FrameInterceptorMode.MODIFY -> { @@ -1389,8 +1454,13 @@ class LivePushActivity : AppCompatActivity() { } val editNs = System.nanoTime() - editStartNs + // Return a new SellyVideoFrame wrapping the same (modified) buffer. + // SDK checks identity (currentCpuFrame !== inputCpuFrame) to decide whether to upload; + // returning the same object would skip the upload even though the buffer was modified. + frame.buffer.retain() + val modifiedFrame = SellyVideoFrame(frame.buffer, frame.rotation, frame.timestampNs) return ModifiedFrameTrace( - frame = frame, + frame = modifiedFrame, bufferKind = bufferKind, patchLabel = patchLabel, editNs = editNs diff --git a/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveListRepository.kt b/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveListRepository.kt index 49f25d5..f130edf 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveListRepository.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveListRepository.kt @@ -28,7 +28,8 @@ data class AliveStreamItem( val previewImage: String?, val durationSeconds: Long?, val playProtocol: String?, - val streamPk: String? + val streamPk: String?, + val xorKey: String? = null ) val AliveStreamItem.isPkStream: Boolean @@ -101,6 +102,8 @@ private fun JSONObject.toAliveItem(): AliveStreamItem { val streamPk = optString("stream_pk") .ifBlank { optString("streamPk") } .takeIf { it.isNotBlank() } + val xorKey = optString("xor_key") + .takeIf { it.isNotBlank() } return AliveStreamItem( vhost = vhost, @@ -110,6 +113,7 @@ private fun JSONObject.toAliveItem(): AliveStreamItem { previewImage = previewImage, durationSeconds = durationSeconds, playProtocol = playProtocol, - streamPk = streamPk + streamPk = streamPk, + xorKey = xorKey ) } diff --git a/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveStreamAdapter.kt b/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveStreamAdapter.kt index e1c591f..c9d04fa 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveStreamAdapter.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/live/square/AliveStreamAdapter.kt @@ -53,6 +53,9 @@ class AliveStreamAdapter( binding.tvStreamName.text = title binding.tvPkBadge.visibility = if (item.isPkStream) View.VISIBLE else View.GONE + val hasXor = !item.xorKey.isNullOrBlank() + binding.tvXorBadge.visibility = if (hasXor) View.VISIBLE else View.GONE + if (hasXor) binding.tvXorBadge.text = "\uD83D\uDD12" val protocol = item.playProtocol ?.trim() diff --git a/example/src/main/java/com/demo/SellyCloudSDK/live/square/StreamXorRepository.kt b/example/src/main/java/com/demo/SellyCloudSDK/live/square/StreamXorRepository.kt new file mode 100644 index 0000000..e553e8d --- /dev/null +++ b/example/src/main/java/com/demo/SellyCloudSDK/live/square/StreamXorRepository.kt @@ -0,0 +1,57 @@ +package com.demo.SellyCloudSDK.live.square + +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.withContext +import okhttp3.MediaType.Companion.toMediaType +import okhttp3.OkHttpClient +import okhttp3.Request +import okhttp3.RequestBody.Companion.toRequestBody +import org.json.JSONObject + +private const val STREAM_XOR_URL = "http://rtmp.sellycloud.io:8089/live/sdk/demo/stream-xor" +private val JSON_MEDIA_TYPE = "application/json; charset=utf-8".toMediaType() + +/** + * Reports the XOR encryption state for a push stream. + * + * - `xorKey` non-empty: tells the server this stream uses XOR encryption with this key. + * - `xorKey` empty/null: clears the cached key on the server. + * + * The server caches the key in memory. When `GET /live/sdk/alive-list` returns, matching + * streams will include the `xor_key` field so viewers can auto-decrypt. + */ +object StreamXorRepository { + private val client = OkHttpClient() + + /** + * @param stream Stream name (required). + * @param app Application name (required, e.g. "live"). + * @param vhost Virtual host (optional). + * @param xorKey XOR hex key. Empty string or null means "disable & clear". + * @return `true` if the server accepted the request. + */ + suspend fun reportXorKey( + stream: String, + app: String, + vhost: String? = null, + xorKey: String? + ): Boolean = withContext(Dispatchers.IO) { + try { + val body = JSONObject().apply { + put("stream", stream) + put("app", app) + if (!vhost.isNullOrBlank()) put("vhost", vhost) + put("xor_key", xorKey.orEmpty()) + } + val request = Request.Builder() + .url(STREAM_XOR_URL) + .post(body.toString().toRequestBody(JSON_MEDIA_TYPE)) + .build() + client.newCall(request).execute().use { response -> + response.isSuccessful + } + } catch (_: Exception) { + false + } + } +} diff --git a/example/src/main/java/com/demo/SellyCloudSDK/playback/PlaybackProcessingDebug.kt b/example/src/main/java/com/demo/SellyCloudSDK/playback/PlaybackProcessingDebug.kt new file mode 100644 index 0000000..7f54ed6 --- /dev/null +++ b/example/src/main/java/com/demo/SellyCloudSDK/playback/PlaybackProcessingDebug.kt @@ -0,0 +1,192 @@ +package com.demo.SellyCloudSDK.playback + +import android.opengl.GLES20 +import android.os.SystemClock +import com.sellycloud.sellycloudsdk.PlaybackFrameObserver +import com.sellycloud.sellycloudsdk.PlaybackFrameObserverConfig +import com.sellycloud.sellycloudsdk.PlaybackVideoProcessor +import com.sellycloud.sellycloudsdk.PlaybackVideoProcessorConfig +import com.sellycloud.sellycloudsdk.VideoProcessFormat +import com.sellycloud.sellycloudsdk.VideoProcessMode +import com.sellycloud.sellycloudsdk.VideoStage +import com.sellycloud.sellycloudsdk.VideoTextureFrame +import java.util.Locale +import java.util.concurrent.atomic.AtomicInteger +import java.util.concurrent.atomic.AtomicLong + +enum class PlaybackProcessingPreset(val label: String) { + DIRECT("DIRECT"), + OBSERVER("PROCESSING_OBSERVER"), + PROCESSOR("PROCESSING_PROCESSOR") +} + +class PlaybackTextureObserverDemo( + private val log: (String) -> Unit +) : PlaybackFrameObserver { + + override val config: PlaybackFrameObserverConfig = PlaybackFrameObserverConfig( + preferredFormat = VideoProcessFormat.TEXTURE_2D, + stage = VideoStage.RENDER_PRE_DISPLAY + ) + + @Volatile + private var lastFrameWidth: Int = 0 + @Volatile + private var lastFrameHeight: Int = 0 + @Volatile + private var lastFrameRotation: Int = 0 + @Volatile + private var lastFps: Float = 0f + + private val frameCounter = AtomicInteger(0) + private val windowStartMs = AtomicLong(0L) + + override fun onGlContextCreated() { + log("processing observer: GL context created") + } + + override fun onGlContextDestroyed() { + log("processing observer: GL context destroyed") + } + + override fun onTextureFrame(frame: VideoTextureFrame) { + lastFrameWidth = frame.width + lastFrameHeight = frame.height + lastFrameRotation = frame.rotation + + val nowMs = SystemClock.elapsedRealtime() + val startedAtMs = windowStartMs.updateAndGet { existing -> if (existing == 0L) nowMs else existing } + val count = frameCounter.incrementAndGet() + val elapsedMs = nowMs - startedAtMs + if (elapsedMs < 1_000L) return + + lastFps = count * 1000f / elapsedMs + frameCounter.set(0) + windowStartMs.set(nowMs) + log( + "processing observer: texture fps=${String.format(Locale.US, "%.1f", lastFps)}, " + + "size=${frame.width}x${frame.height}, rotation=${frame.rotation}" + ) + } + + fun summary(): String { + if (lastFrameWidth <= 0 || lastFrameHeight <= 0) return "observer: 等待纹理帧" + return "observer: ${String.format(Locale.US, "%.1f", lastFps)}fps, ${lastFrameWidth}x${lastFrameHeight}, rot=$lastFrameRotation" + } +} + +class PlaybackTexturePatchProcessor( + private val log: (String) -> Unit +) : PlaybackVideoProcessor { + + override val config: PlaybackVideoProcessorConfig = PlaybackVideoProcessorConfig( + preferredFormat = VideoProcessFormat.TEXTURE_2D, + mode = VideoProcessMode.READ_WRITE, + stage = VideoStage.RENDER_PRE_DISPLAY, + fullRewrite = false + ) + + @Volatile + private var lastFrameWidth: Int = 0 + @Volatile + private var lastFrameHeight: Int = 0 + @Volatile + private var lastFrameRotation: Int = 0 + @Volatile + private var lastPatchFps: Float = 0f + + private val patchCounter = AtomicInteger(0) + private val windowStartMs = AtomicLong(0L) + private var framebuffer = 0 + + override fun onGlContextCreated() { + log("processing processor: GL context created") + } + + override fun onGlContextDestroyed() { + if (framebuffer != 0) { + GLES20.glDeleteFramebuffers(1, intArrayOf(framebuffer), 0) + framebuffer = 0 + } + log("processing processor: GL context destroyed") + } + + override fun processTexture(input: VideoTextureFrame, outputTextureId: Int) { + if (outputTextureId <= 0) return + lastFrameWidth = input.width + lastFrameHeight = input.height + lastFrameRotation = input.rotation + + val patchWidth = (input.width * 0.18f).toInt().coerceAtLeast(48) + val patchHeight = (input.height * 0.10f).toInt().coerceAtLeast(32) + ensureFramebuffer() + if (framebuffer == 0) return + + val previousFramebuffer = IntArray(1) + val previousViewport = IntArray(4) + val scissorWasEnabled = GLES20.glIsEnabled(GLES20.GL_SCISSOR_TEST) + val previousClearColor = FloatArray(4) + GLES20.glGetIntegerv(GLES20.GL_FRAMEBUFFER_BINDING, previousFramebuffer, 0) + GLES20.glGetIntegerv(GLES20.GL_VIEWPORT, previousViewport, 0) + GLES20.glGetFloatv(GLES20.GL_COLOR_CLEAR_VALUE, previousClearColor, 0) + + GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, framebuffer) + GLES20.glFramebufferTexture2D( + GLES20.GL_FRAMEBUFFER, + GLES20.GL_COLOR_ATTACHMENT0, + GLES20.GL_TEXTURE_2D, + outputTextureId, + 0 + ) + GLES20.glViewport(0, 0, input.width, input.height) + GLES20.glEnable(GLES20.GL_SCISSOR_TEST) + GLES20.glScissor(0, 0, patchWidth, patchHeight) + GLES20.glClearColor(0.98f, 0.20f, 0.24f, 1.0f) + GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT) + + GLES20.glFramebufferTexture2D( + GLES20.GL_FRAMEBUFFER, + GLES20.GL_COLOR_ATTACHMENT0, + GLES20.GL_TEXTURE_2D, + 0, + 0 + ) + GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, previousFramebuffer[0]) + GLES20.glViewport(previousViewport[0], previousViewport[1], previousViewport[2], previousViewport[3]) + GLES20.glClearColor( + previousClearColor[0], + previousClearColor[1], + previousClearColor[2], + previousClearColor[3] + ) + if (!scissorWasEnabled) { + GLES20.glDisable(GLES20.GL_SCISSOR_TEST) + } + + val nowMs = SystemClock.elapsedRealtime() + val startedAtMs = windowStartMs.updateAndGet { existing -> if (existing == 0L) nowMs else existing } + val count = patchCounter.incrementAndGet() + val elapsedMs = nowMs - startedAtMs + if (elapsedMs < 1_000L) return + + lastPatchFps = count * 1000f / elapsedMs + patchCounter.set(0) + windowStartMs.set(nowMs) + log( + "processing processor: red patch fps=${String.format(Locale.US, "%.1f", lastPatchFps)}, " + + "size=${input.width}x${input.height}, rotation=${input.rotation}" + ) + } + + fun summary(): String { + if (lastFrameWidth <= 0 || lastFrameHeight <= 0) return "processor: 等待纹理帧" + return "processor: ${String.format(Locale.US, "%.1f", lastPatchFps)}fps, ${lastFrameWidth}x${lastFrameHeight}, rot=$lastFrameRotation" + } + + private fun ensureFramebuffer() { + if (framebuffer != 0) return + val framebuffers = IntArray(1) + GLES20.glGenFramebuffers(1, framebuffers, 0) + framebuffer = framebuffers[0] + } +} diff --git a/example/src/main/java/com/demo/SellyCloudSDK/vod/VodPlayActivity.kt b/example/src/main/java/com/demo/SellyCloudSDK/vod/VodPlayActivity.kt index 94ba349..b8b8e60 100644 --- a/example/src/main/java/com/demo/SellyCloudSDK/vod/VodPlayActivity.kt +++ b/example/src/main/java/com/demo/SellyCloudSDK/vod/VodPlayActivity.kt @@ -32,6 +32,9 @@ import com.demo.SellyCloudSDK.R import com.demo.SellyCloudSDK.avdemo.AvDemoSettingsStore import com.demo.SellyCloudSDK.databinding.ActivityVodPlayBinding import com.demo.SellyCloudSDK.live.util.GalleryImageSaver +import com.demo.SellyCloudSDK.playback.PlaybackProcessingPreset +import com.demo.SellyCloudSDK.playback.PlaybackTextureObserverDemo +import com.demo.SellyCloudSDK.playback.PlaybackTexturePatchProcessor import com.sellycloud.sellycloudsdk.SellyCloudManager import com.sellycloud.sellycloudsdk.SellyLiveError import com.sellycloud.sellycloudsdk.SellyPlayerState @@ -70,6 +73,9 @@ class VodPlayActivity : AppCompatActivity() { private var firstAudioFrameElapsedMs: Long? = null private var firstAudioFrameCostMs: Long? = null private var bufferingActive = false + private var processingPreset: PlaybackProcessingPreset = PlaybackProcessingPreset.DIRECT + private var renderTargetRebindCount = 0 + private var lastRenderTargetRebindCostMs: Long? = null private var progressJob: Job? = null @@ -79,6 +85,14 @@ class VodPlayActivity : AppCompatActivity() { private var logSummaryView: TextView? = null private var logContentView: TextView? = null private var logFloatingButton: View? = null + private var toolsFloatingButton: View? = null + + private val playbackObserverDemo by lazy(LazyThreadSafetyMode.NONE) { + PlaybackTextureObserverDemo(::logEvent) + } + private val playbackPatchProcessor by lazy(LazyThreadSafetyMode.NONE) { + PlaybackTexturePatchProcessor(::logEvent) + } private val storagePermissionLauncher = registerForActivityResult( ActivityResultContracts.RequestPermission() @@ -95,6 +109,7 @@ class VodPlayActivity : AppCompatActivity() { supportActionBar?.hide() useTextureView = AvDemoSettingsStore(this).read().renderBackendPreference.isTextureView() addLogFloatingButton() + addToolsFloatingButton() binding.btnClose.setOnClickListener { finish() } binding.actionPlay.setOnClickListener { togglePlay() } @@ -253,8 +268,9 @@ class VodPlayActivity : AppCompatActivity() { client.setMuted(isMuted) } - val backend = if (useTextureView) RenderBackend.TEXTURE_VIEW else RenderBackend.SURFACE_VIEW + val backend = currentRenderBackend() renderView = vodPlayer.attachRenderView(binding.renderContainer, backend) + logEvent("渲染目标已绑定: backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") player = vodPlayer startPlayAttempt() vodPlayer.prepareAsync() @@ -427,6 +443,98 @@ class VodPlayActivity : AppCompatActivity() { logFloatingButton = button } + private fun addToolsFloatingButton() { + val sizePx = dpToPx(44) + val marginEndPx = dpToPx(72) + val controlBarHeight = resources.getDimensionPixelSize(R.dimen.av_control_bar_height) + val marginBottomPx = controlBarHeight + dpToPx(80) + val bgDrawable = GradientDrawable(GradientDrawable.Orientation.TOP_BOTTOM, intArrayOf( + Color.parseColor("#B33B0764"), + Color.parseColor("#803B0764") + )).apply { + shape = GradientDrawable.OVAL + setStroke(dpToPx(1), Color.parseColor("#55FFFFFF")) + } + val button = AppCompatTextView(this).apply { + text = "测" + setTextColor(Color.parseColor("#F8FAFC")) + textSize = 11f + gravity = Gravity.CENTER + background = bgDrawable + elevation = dpToPx(4).toFloat() + setShadowLayer(2f, 0f, 1f, Color.parseColor("#66000000")) + isClickable = true + isFocusable = true + contentDescription = "播放处理与回归工具" + setOnClickListener { showPlaybackToolsDialog() } + } + val params = FrameLayout.LayoutParams(sizePx, sizePx).apply { + gravity = Gravity.END or Gravity.BOTTOM + marginEnd = marginEndPx + bottomMargin = marginBottomPx + } + addContentView(button, params) + toolsFloatingButton = button + } + + private fun showPlaybackToolsDialog() { + val container = LinearLayout(this).apply { + orientation = LinearLayout.VERTICAL + setPadding(dpToPx(20), dpToPx(16), dpToPx(20), dpToPx(8)) + } + val summary = TextView(this).apply { + text = "当前后端: ${currentRenderBackendLabel()}\n" + + "当前模式: ${processingPreset.label}\n" + + "说明: processing 仅支持 TextureView。" + setTextColor(Color.parseColor("#E5E7EB")) + textSize = 13f + } + container.addView(summary) + container.addView(spaceView(dpToPx(12))) + container.addView(createToolActionButton("切换 DIRECT 直出") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.DIRECT, trigger = "工具面板") + }) + container.addView(createToolActionButton("切换 PROCESSING Observer") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.OBSERVER, trigger = "工具面板") + }) + container.addView(createToolActionButton("切换 PROCESSING 红块 Processor") { + applyPlaybackProcessingPreset(PlaybackProcessingPreset.PROCESSOR, trigger = "工具面板") + }) + container.addView(createToolActionButton("仅重绑当前目标") { + rebindRenderTarget("手动回归") + }) + + AlertDialog.Builder(this) + .setTitle("播放处理 / 目标重绑") + .setView(container) + .setNegativeButton("关闭", null) + .show() + } + + private fun createToolActionButton(label: String, onClick: () -> Unit): View { + return AppCompatTextView(this).apply { + text = label + gravity = Gravity.CENTER + textSize = 14f + setTextColor(Color.parseColor("#F8FAFC")) + background = GradientDrawable().apply { + cornerRadius = dpToPx(10).toFloat() + setColor(Color.parseColor("#334155")) + setStroke(dpToPx(1), Color.parseColor("#475569")) + } + setPadding(dpToPx(12), dpToPx(12), dpToPx(12), dpToPx(12)) + isClickable = true + isFocusable = true + setOnClickListener { onClick() } + layoutParams = LinearLayout.LayoutParams( + ViewGroup.LayoutParams.MATCH_PARENT, + ViewGroup.LayoutParams.WRAP_CONTENT + ).apply { + bottomMargin = dpToPx(10) + } + } + } + private fun showLogDialog() { if (logDialog?.isShowing == true) { refreshLogDialogContent() @@ -552,12 +660,22 @@ class VodPlayActivity : AppCompatActivity() { private fun buildLogSummary(): String { val builder = StringBuilder() builder.append("状态: ").append(formatState(currentState)).append('\n') + builder.append("渲染后端: ").append(currentRenderBackendLabel()).append('\n') + builder.append("播放处理: ").append(processingPreset.label).append('\n') builder.append("是否播放中: ").append(if (isPlaying) "是" else "否").append('\n') builder.append("是否静音: ").append(if (isMuted) "是" else "否").append('\n') builder.append("总时长: ").append(if (durationMs > 0) formatTime(durationMs) else "--").append('\n') builder.append("当前进度: ").append(formatTime(player?.getCurrentPositionMs() ?: 0L)).append('\n') builder.append("首帧视频耗时(ms): ").append(firstVideoFrameCostMs ?: "未统计").append('\n') builder.append("首帧音频耗时(ms): ").append(firstAudioFrameCostMs ?: "未统计").append('\n') + builder.append("目标重绑次数: ").append(renderTargetRebindCount).append('\n') + builder.append("最近重绑耗时(ms): ").append(lastRenderTargetRebindCostMs ?: "未统计").append('\n') + val processingDetail = when (processingPreset) { + PlaybackProcessingPreset.DIRECT -> "processing: 关闭" + PlaybackProcessingPreset.OBSERVER -> playbackObserverDemo.summary() + PlaybackProcessingPreset.PROCESSOR -> playbackPatchProcessor.summary() + } + builder.append(processingDetail).append('\n') val attemptElapsed = playAttemptStartElapsedMs?.let { SystemClock.elapsedRealtime() - it } if (attemptElapsed == null) { builder.append("本次播放已耗时(ms): 未开始").append('\n') @@ -624,7 +742,65 @@ class VodPlayActivity : AppCompatActivity() { firstAudioFrameElapsedMs = null firstAudioFrameCostMs = null bufferingActive = false - logEvent("播放尝试开始") + logEvent("播放尝试开始: backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") + } + + private fun applyPlaybackProcessingPreset(preset: PlaybackProcessingPreset, trigger: String) { + if (preset == processingPreset) { + logEvent("播放处理保持不变: ${preset.label}, trigger=$trigger") + Toast.makeText(this, "当前已是 ${preset.label}", Toast.LENGTH_SHORT).show() + return + } + if (!useTextureView && preset != PlaybackProcessingPreset.DIRECT) { + logEvent("播放处理切换被拒绝: backend=${currentRenderBackendLabel()} 不支持 ${preset.label}") + Toast.makeText(this, "播放 processing 仅支持 TextureView 后端", Toast.LENGTH_SHORT).show() + return + } + + processingPreset = preset + configurePlaybackProcessing() + logEvent("播放处理切换: mode=${preset.label}, trigger=$trigger") + rebindRenderTarget("processing_${preset.name.lowercase(Locale.US)}") + Toast.makeText(this, "已切到 ${preset.label}", Toast.LENGTH_SHORT).show() + } + + private fun configurePlaybackProcessing() { + val currentPlayer = player ?: return + when (processingPreset) { + PlaybackProcessingPreset.DIRECT -> { + currentPlayer.setPlaybackFrameObserver(null) + currentPlayer.setPlaybackVideoProcessor(null) + } + PlaybackProcessingPreset.OBSERVER -> { + currentPlayer.setPlaybackVideoProcessor(null) + currentPlayer.setPlaybackFrameObserver(playbackObserverDemo) + } + PlaybackProcessingPreset.PROCESSOR -> { + currentPlayer.setPlaybackFrameObserver(null) + currentPlayer.setPlaybackVideoProcessor(playbackPatchProcessor) + } + } + } + + private fun rebindRenderTarget(reason: String) { + val currentPlayer = player ?: return + val startedAtMs = SystemClock.elapsedRealtime() + val backend = currentRenderBackend() + logEvent("目标重绑开始: reason=$reason, backend=${currentRenderBackendLabel()}, processing=${processingPreset.label}") + currentPlayer.clearRenderTarget() + renderView = currentPlayer.attachRenderView(binding.renderContainer, backend) + val costMs = SystemClock.elapsedRealtime() - startedAtMs + renderTargetRebindCount += 1 + lastRenderTargetRebindCostMs = costMs + logEvent("目标重绑完成: count=$renderTargetRebindCount, cost=${costMs}ms") + } + + private fun currentRenderBackend(): RenderBackend { + return if (useTextureView) RenderBackend.TEXTURE_VIEW else RenderBackend.SURFACE_VIEW + } + + private fun currentRenderBackendLabel(): String { + return if (useTextureView) "TextureView" else "SurfaceView" } private fun formatState(state: SellyPlayerState): String { diff --git a/example/src/main/res/layout/item_live_square_card.xml b/example/src/main/res/layout/item_live_square_card.xml index e525bcc..392a35d 100644 --- a/example/src/main/res/layout/item_live_square_card.xml +++ b/example/src/main/res/layout/item_live_square_card.xml @@ -130,6 +130,14 @@ android:text="RTMP" android:textColor="@color/brand_primary_text_on" android:textSize="11sp" /> + +