Skip to content

Commit 3b21c83

Browse files
munet-rei[bot]clansty
munet-rei[bot]
and
clansty
authored
feat: Add batch PV conversion tool (#63)
* feat: Add batch PV conversion tool A new tool in the Tools page that bulk converts every PV (Promotional Video) under StreamingAssets/A###/MovieData in both directions: - USM/DAT -> MP4: writes a sibling .mp4 next to each source, original kept untouched - MP4 -> USM/DAT: writes to a temp file, validates non-empty, atomically swaps in the final .dat, then sends the source .mp4 to the recycle bin Implementation notes: - New endpoint VideoConvertToolController.BatchConvertPvTool, SSE with events Progress / FileError / Success / Cancelled / Error. - Progress payload is JSON (camelCase) with processed / total / fileProgress / fileName / failed. - All SSE frames go through a single-writer Channel<string> so that Xabe's synchronous OnProgress events from inside FFmpeg cannot interleave Response.WriteAsync calls. - Direct file system enumeration via StaticSettings.AssetsDirs rather than MovieDataMap, so files that share an ID across multiple asset dirs (or .mp4 + .dat siblings) are not silently skipped. - Cancellation is checked before destructive file ops; a partial output in temp is removed before throwing, so cancel cannot corrupt the source. - Sponsored feature, gated on IapManager.License == Active. - Final settings.ScanMovieData() in the finally block resynchronizes MovieDataMap with on-disk state. Frontend: src/views/Tools/BatchVideoConvertModal.tsx, a 3-step modal (Configure direction -> live Progress with overall + current-file bars and a collapsible per-file error list -> Done with summary). Uses fetchEventSource + AbortController, matching the existing single-file VideoConvertModal pattern. Surfaces 'no files' and 'needs sponsor' as friendly toasts instead of crash reports. Notes: - Locale.Designer.cs was updated by hand to expose the two new resource strings. - Front/src/client/apiGen.ts was NOT regenerated (requires a running backend on localhost:5181). The new endpoint is consumed directly via getUrl + fetchEventSource, matching the other SSE tool endpoints, so no client regen is strictly required, but a follow-up `pnpm genClient` is recommended after merging. * refactor: batch PV scans a user-picked folder instead of the game dir Per review feedback, batch PV conversion no longer scans the game's StreamingAssets/A###/MovieData. The user picks an arbitrary folder via the existing OpenFolderDialog endpoint and we convert every matching file in that folder in place. Backend (VideoConvertToolController.cs): - BatchConvertPvTool now takes folderPath in addition to direction. - Direct Directory.EnumerateFiles on the chosen folder, filtered by direction-derived source extensions. Removed the numeric-filename filter and EnumerateMoviePvs helper since arbitrary user folders contain arbitrarily named files. - Removed StaticSettings dependency: no AssetsDirs scan, no MovieDataMap mutation, no settings.ScanMovieData() call in finally. Constructor no longer takes StaticSettings. - Returns the new BatchConvertPvFolderNotFound error when the path is missing or does not exist. - All preserved: license gate, single-writer Channel SSE serialization, cancellation-before-delete, temp->validate->atomic-move, source MP4 to recycle bin, JSON camelCase payload. Frontend (BatchVideoConvertModal.tsx): - trigger() now opens the native folder picker via api.OpenFolderDialog (reusing the OobeController endpoint) before showing the modal. If the user cancels the picker, no modal is opened. - Configure step displays the selected path with a Change folder button that re-opens the picker. - start() includes folderPath as a query param when opening the SSE. - folderNotFound joins the friendly-error allowlist that surfaces as a toast instead of going through globalCapture. i18n: drop the 'same directory' wording from the direction hints, add selectFolder / changeFolder / selectedFolder / folderNotFound for zh, zh-TW, en. Backend Locale resx (and Designer.cs) gets the matching BatchConvertPvFolderNotFound entry. * feat: 合并 PV 转换入口 * chore: 界面修改 * fix: 使用结构化错误处理批量 PV 转换 --------- Co-authored-by: 玲 <286150633+munet-rei[bot]@users.noreply.github.com> Co-authored-by: Clansty <i@gao4.pw>
1 parent 151492f commit 3b21c83

12 files changed

Lines changed: 907 additions & 25 deletions

File tree

‎MaiChartManager/Controllers/Tools/VideoConvertToolController.cs‎

Lines changed: 277 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
1+
using System.Text.Json;
2+
using System.Threading.Channels;
13
using MaiChartManager.Utils;
24
using Microsoft.AspNetCore.Mvc;
5+
using Microsoft.VisualBasic.FileIO;
36

47
namespace MaiChartManager.Controllers.Tools;
58

@@ -83,4 +86,277 @@ await VideoConvert.ConvertVideoToUsm(
8386
await Response.Body.FlushAsync();
8487
}
8588
}
86-
}
89+
90+
public enum BatchConvertPvDirection
91+
{
92+
/// <summary>USM/DAT → MP4</summary>
93+
UsmToMp4,
94+
/// <summary>MP4 → USM/DAT</summary>
95+
Mp4ToUsm
96+
}
97+
98+
public enum BatchConvertPvEventType
99+
{
100+
/// <summary>整体 + 当前文件进度，data 为 JSON</summary>
101+
Progress,
102+
/// <summary>单文件失败，仍然继续处理后续文件</summary>
103+
FileError,
104+
/// <summary>全部完成，data 为 "processed/total|failedCount"</summary>
105+
Success,
106+
/// <summary>致命错误，停止</summary>
107+
Error,
108+
/// <summary>被取消，data 为 "processed/total"</summary>
109+
Cancelled
110+
}
111+
112+
private static readonly JsonSerializerOptions BatchJsonOptions = new()
113+
{
114+
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
115+
};
116+
117+
private record BatchProgressPayload(int Processed, int Total, int FileProgress, string FileName, int Failed);
118+
private record BatchErrorPayload(string Code, string Message);
119+
120+
private static class BatchConvertPvErrorCode
121+
{
122+
public const string NeedLicense = "NEED_LICENSE";
123+
public const string NoFiles = "NO_FILES";
124+
public const string FolderNotFound = "FOLDER_NOT_FOUND";
125+
public const string ConvertFailed = "CONVERT_FAILED";
126+
}
127+
128+
/// <summary>
129+
/// 批量转换用户选择的文件夹内所有 PV：USM/DAT ↔ MP4。
130+
/// 使用 SSE 实时推送整体进度（已处理/总数）+ 当前文件进度。
131+
/// 客户端断开连接时通过 RequestAborted 触发取消，循环在下一个文件之间退出。
132+
/// 所有 SSE 写入通过单写者 Channel 串行化，避免 Xabe 同步进度事件触发的 async-void 写入交错。
133+
/// </summary>
134+
[HttpPost]
135+
public async Task BatchConvertPvTool([FromQuery] string folderPath, [FromQuery] BatchConvertPvDirection direction)
136+
{
137+
Response.Headers.Append("Content-Type", "text/event-stream");
138+
139+
// PV 转换属于赞助功能
140+
if (IapManager.License != IapManager.LicenseStatus.Active)
141+
{
142+
await WriteBatchError(BatchConvertPvErrorCode.NeedLicense, Locale.BatchConvertPvNeedLicense);
143+
return;
144+
}
145+
146+
if (string.IsNullOrWhiteSpace(folderPath) || !Directory.Exists(folderPath))
147+
{
148+
await WriteBatchError(BatchConvertPvErrorCode.FolderNotFound, Locale.BatchConvertPvFolderNotFound);
149+
return;
150+
}
151+
152+
// 直接枚举用户选择的文件夹（不递归），按方向筛选源扩展名
153+
var sourceExtensions = direction == BatchConvertPvDirection.UsmToMp4
154+
? new HashSet<string>(StringComparer.OrdinalIgnoreCase) { ".dat", ".usm" }
155+
: new HashSet<string>(StringComparer.OrdinalIgnoreCase) { ".mp4" };
156+
157+
var files = Directory.EnumerateFiles(folderPath)
158+
.Where(f => sourceExtensions.Contains(Path.GetExtension(f)))
159+
.OrderBy(f => f, StringComparer.OrdinalIgnoreCase)
160+
.ToList();
161+
162+
if (files.Count == 0)
163+
{
164+
await WriteBatchError(BatchConvertPvErrorCode.NoFiles, Locale.BatchConvertPvNoFiles);
165+
return;
166+
}
167+
168+
var total = files.Count;
169+
var processed = 0;
170+
var failedCount = 0;
171+
var cancellationToken = HttpContext.RequestAborted;
172+
173+
// 单写者 Channel：所有 SSE 帧（不论来自循环还是 OnProgress）都进入这条队列
174+
var sseChannel = Channel.CreateUnbounded<string>(new UnboundedChannelOptions
175+
{
176+
SingleReader = true,
177+
SingleWriter = false,
178+
});
179+
180+
var writer = WriteSseFrames(sseChannel.Reader, cancellationToken);
181+
182+
try
183+
{
184+
foreach (var inputPath in files)
185+
{
186+
if (cancellationToken.IsCancellationRequested) break;
187+
188+
var fileName = Path.GetFileName(inputPath);
189+
await EnqueueProgress(sseChannel.Writer, processed, total, 0, fileName, failedCount);
190+
191+
try
192+
{
193+
var directory = Path.GetDirectoryName(inputPath)!;
194+
var nameWithoutExt = Path.GetFileNameWithoutExtension(inputPath);
195+
196+
if (direction == BatchConvertPvDirection.UsmToMp4)
197+
{
198+
var outputPath = Path.Combine(directory, nameWithoutExt + ".mp4");
199+
var snapshot = (Processed: processed, Failed: failedCount);
200+
await VideoConvert.ConvertUsmToMp4(
201+
inputPath,
202+
outputPath,
203+
percent => EnqueueProgressFireAndForget(sseChannel.Writer, snapshot.Processed, total, percent, fileName, snapshot.Failed));
204+
}
205+
else
206+
{
207+
// MP4 → USM（VP9）：先输出到临时文件，验证后再覆盖目标
208+
var finalPath = Path.Combine(directory, nameWithoutExt + ".dat");
209+
var tempPath = finalPath + ".tmp";
210+
var snapshot = (Processed: processed, Failed: failedCount);
211+
try
212+
{
213+
await VideoConvert.ConvertVideo(new VideoConvert.VideoConvertOptions
214+
{
215+
InputPath = inputPath,
216+
OutputPath = tempPath,
217+
NoScale = StaticSettings.Config.NoScale,
218+
UseH264 = false,
219+
UseYuv420p = StaticSettings.Config.Yuv420p,
220+
Padding = 0,
221+
TaskbarProgress = false,
222+
OnProgress = percent => EnqueueProgressFireAndForget(sseChannel.Writer, snapshot.Processed, total, percent, fileName, snapshot.Failed)
223+
});
224+
225+
if (!System.IO.File.Exists(tempPath) || new FileInfo(tempPath).Length == 0)
226+
{
227+
throw new Exception("Converted DAT is missing or empty");
228+
}
229+
230+
// 取消检查必须在覆盖目标前，避免取消时仍然损毁已有 DAT
231+
cancellationToken.ThrowIfCancellationRequested();
232+
233+
if (System.IO.File.Exists(finalPath))
234+
{
235+
System.IO.File.Delete(finalPath);
236+
}
237+
System.IO.File.Move(tempPath, finalPath);
238+
}
239+
catch
240+
{
241+
try { if (System.IO.File.Exists(tempPath)) System.IO.File.Delete(tempPath); }
242+
catch { /* ignored */ }
243+
throw;
244+
}
245+
}
246+
247+
processed++;
248+
await EnqueueProgress(sseChannel.Writer, processed, total, 100, fileName, failedCount);
249+
}
250+
catch (OperationCanceledException)
251+
{
252+
throw;
253+
}
254+
catch (Exception fileEx)
255+
{
256+
logger.LogError(fileEx, "Failed to convert PV file {File}", inputPath);
257+
failedCount++;
258+
processed++;
259+
await EnqueueEvent(sseChannel.Writer, BatchConvertPvEventType.FileError, $"{fileName}: {fileEx.Message}");
260+
await EnqueueProgress(sseChannel.Writer, processed, total, 100, fileName, failedCount);
261+
}
262+
}
263+
264+
if (cancellationToken.IsCancellationRequested)
265+
{
266+
await EnqueueEvent(sseChannel.Writer, BatchConvertPvEventType.Cancelled, $"{processed}/{total}");
267+
}
268+
else
269+
{
270+
await EnqueueEvent(sseChannel.Writer, BatchConvertPvEventType.Success, $"{processed}/{total}|{failedCount}");
271+
}
272+
}
273+
catch (OperationCanceledException)
274+
{
275+
await EnqueueEvent(sseChannel.Writer, BatchConvertPvEventType.Cancelled, $"{processed}/{total}");
276+
}
277+
catch (Exception ex)
278+
{
279+
logger.LogError(ex, "Batch PV conversion failed");
280+
SentrySdk.CaptureException(ex);
281+
try
282+
{
283+
await EnqueueError(sseChannel.Writer, BatchConvertPvErrorCode.ConvertFailed, string.Format(Locale.ConvertFailed, ex.Message));
284+
}
285+
catch
286+
{
287+
// 客户端可能已断开
288+
}
289+
}
290+
finally
291+
{
292+
sseChannel.Writer.TryComplete();
293+
try
294+
{
295+
await writer;
296+
}
297+
catch
298+
{
299+
// writer 自己负责吞掉客户端断开异常
300+
}
301+
}
302+
}
303+
304+
private static string SanitizeSseLine(string data) =>
305+
data.Replace("\r", " ").Replace("\n", " ");
306+
307+
private static string CreateBatchErrorData(string code, string message) =>
308+
JsonSerializer.Serialize(new BatchErrorPayload(code, message), BatchJsonOptions);
309+
310+
private async Task WriteBatchError(string code, string message)
311+
{
312+
await Response.WriteAsync($"event: {BatchConvertPvEventType.Error}\ndata: {SanitizeSseLine(CreateBatchErrorData(code, message))}\n\n");
313+
await Response.Body.FlushAsync();
314+
}
315+
316+
private static ValueTask EnqueueEvent(ChannelWriter<string> writer, BatchConvertPvEventType eventType, string data) =>
317+
writer.WriteAsync($"event: {eventType}\ndata: {SanitizeSseLine(data)}\n\n");
318+
319+
private static ValueTask EnqueueError(ChannelWriter<string> writer, string code, string message) =>
320+
EnqueueEvent(writer, BatchConvertPvEventType.Error, CreateBatchErrorData(code, message));
321+
322+
private static ValueTask EnqueueProgress(ChannelWriter<string> writer, int processed, int total, int fileProgress, string fileName, int failed)
323+
{
324+
var payload = JsonSerializer.Serialize(new BatchProgressPayload(processed, total, fileProgress, fileName, failed), BatchJsonOptions);
325+
return writer.WriteAsync($"event: {BatchConvertPvEventType.Progress}\ndata: {payload}\n\n");
326+
}
327+
328+
private static void EnqueueProgressFireAndForget(ChannelWriter<string> writer, int processed, int total, int fileProgress, string fileName, int failed)
329+
{
330+
var payload = JsonSerializer.Serialize(new BatchProgressPayload(processed, total, fileProgress, fileName, failed), BatchJsonOptions);
331+
// Channel 是无界的，TryWrite 同步入队，避免在 Xabe 的同步进度事件里 await
332+
writer.TryWrite($"event: {BatchConvertPvEventType.Progress}\ndata: {payload}\n\n");
333+
}
334+
335+
private async Task WriteSseFrames(ChannelReader<string> reader, CancellationToken cancellationToken)
336+
{
337+
try
338+
{
339+
await foreach (var frame in reader.ReadAllAsync(cancellationToken))
340+
{
341+
try
342+
{
343+
await Response.WriteAsync(frame, cancellationToken);
344+
await Response.Body.FlushAsync(cancellationToken);
345+
}
346+
catch (OperationCanceledException)
347+
{
348+
return;
349+
}
350+
catch (Exception ex)
351+
{
352+
logger.LogDebug(ex, "SSE frame write failed (client disconnected?)");
353+
return;
354+
}
355+
}
356+
}
357+
catch (OperationCanceledException)
358+
{
359+
// ignore
360+
}
361+
}
362+
}

‎MaiChartManager/Front/src/client/apiGen.ts‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,11 @@ export enum HardwareAccelerationStatus {
6060
Disabled = "Disabled",
6161
}
6262

63+
export enum BatchConvertPvDirection {
64+
UsmToMp4 = "UsmToMp4",
65+
Mp4ToUsm = "Mp4ToUsm",
66+
}
67+
6368
export enum AssetType {
6469
Music = "Music",
6570
Movie = "Movie",
@@ -2611,6 +2616,27 @@ export class Api<
26112616
...params,
26122617
}),
26132618

2619+
/**
2620+
* No description
2621+
*
2622+
* @tags VideoConvertTool
2623+
* @name BatchConvertPvTool
2624+
* @request POST:/MaiChartManagerServlet/BatchConvertPvToolApi
2625+
*/
2626+
BatchConvertPvTool: (
2627+
query?: {
2628+
folderPath?: string;
2629+
direction?: BatchConvertPvDirection;
2630+
},
2631+
params: RequestParams = {},
2632+
) =>
2633+
this.request<void, any>({
2634+
path: `/MaiChartManagerServlet/BatchConvertPvToolApi`,
2635+
method: "POST",
2636+
query: query,
2637+
...params,
2638+
}),
2639+
26142640
/**
26152641
* No description
26162642
*

‎MaiChartManager/Front/src/locales/en.yaml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -520,8 +520,40 @@ tools:
520520
videoConvertError: Video conversion error
521521
imageToAb: Image to AssetBundle
522522
imageToAbError: Image to AB conversion error
523+
pvConvert:
524+
label: Convert PV
525+
single: Single convert
526+
singleHint: Pick one PV file. DAT/USM to MP4, or another video format to DAT
527+
batch: Batch convert
528+
batchHint: Pick a folder for batch conversion, with DAT/USM and MP4 direction options
523529
videoOptions:
524530
processing: Still processing, please wait...
531+
batchPv:
532+
label: Batch Convert PVs
533+
title: Batch Convert PVs
534+
selectFolder: Pick folder
535+
changeFolder: Change folder
536+
selectedFolder: Selected folder
537+
folderNotFound: Selected folder does not exist
538+
direction: Conversion direction
539+
directionUsmToMp4: USM / DAT → MP4
540+
directionMp4ToUsm: MP4 → USM / DAT
541+
directionUsmToMp4Hint: Writes a sibling .mp4 next to each DAT/USM in the chosen folder; originals are kept
542+
directionMp4ToUsmHint: Writes a sibling .dat next to each MP4 in the chosen folder; source .mp4 files are kept
543+
start: Start
544+
cancel: Cancel
545+
cancelling: Cancelling...
546+
cancelHint: Cancel takes effect after the current file finishes
547+
close: Close
548+
overall: Overall
549+
currentFile: Current file
550+
currentFileProgress: Current file progress
551+
completedSummary: 'Completed: {success} / {total} succeeded ({failed} failed)'
552+
cancelledSummary: 'Cancelled: {completed} / {total} done'
553+
noFiles: No PV files in that folder match the selected direction
554+
needLicense: This feature requires sponsor activation
555+
error: Batch PV conversion error
556+
fileErrors: 'File errors encountered:'
525557
error:
526558
title: Error
527559
unknown: Unknown error occurred

0 commit comments

Comments
 (0)