Skip to content

Commit 1bacf90

Browse files
hueifenghueifeng
authored
Feature/magicodes ie io pre commit review (#619)
* feat(IO): 新增并实现 Magicodes.IE.IO 包(零 EPPlus 自写流式 xlsx writer/reader、模板导出、表格、低分配异步) * ci: IO 测试扩展为全平台矩阵(Linux/Windows/macOS × net8.0/net10.0)，修复 check 门禁 allowed-skips 遗漏 * fix: IO 测试项目缺失 GitHubActionsTestLogger 导致 CI --logger GitHubActions 报错 * ci: 重构 CI 矩阵，三合一 test step，macOS 真失败检测，matrix 笛卡尔积更优雅 * ci: IO 矩阵补全 net6.0(三平台) 和 net471(Windows) * fix: IO 测试 net471 兼容：添加 System.Memory，修复 File.ReadAllBytesAsync/WriteAllBytesAsync/Write/MemoryStream.Write/string.Replace API，NuGet 集成测试 #if NET 包裹 * fix: IO 测试 net471 兼容(续)：修复 IndexOf/EndsWith/Replace/StringComparison、Split/StringSplitOptions、await using(IAsyncDisposable)，补 System.Runtime.CompilerServices.Unsafe * fix: IO 测试 net471 补 System.Buffers(ArrayBufferWriter/IBufferWriter)，Guardrails 改用 byte[] + GetString(byte[],int,int) 规避 net471 无 Span 重载 * fix: net471 补 ArrayBufferWriter<T> polyfill(实现 System.Memory 提供的 IBufferWriter<T>)，移除无用的 System.Buffers 包引用 * fix: IO 测试跨框架稳定性 - net471 double/decimal 格式差异改数值容差；net471 异步写失败注入跳过(平台限制)；MiniExcel 临时文件锁加重试且 File.OpenRead 加 using 防句柄泄漏 * chore: trigger CI * fix(IO): XlsxReader 支持 leaveOpen；Xlsx.Read(Stream) 实际释放传入流(与文档一致)；新增对称的 Read(string)/ReadAsync(string) 路径重载并拥有释放 FileStream * feat(io): expose leaveOpen on Read/ReadAsync and WriteAsync(path) overload; fail fast on ZIP32 4GB limit - Xlsx.Read/ReadAsync 增加 leaveOpen 参数，默认释放流，可选保留\n- 新增 Xlsx.WriteAsync(path) 便利重载\n- ForwardOnlyZipWriter 在超过 ZIP32(4GB)上限时提前抛出清晰异常\n- Directory.Build.props NoWarn 加入 NU5119，使 TreatWarningsAsErrors 下打包成功\n- 补充 WriteAsync(path)/leaveOpen 保留与默认释放的单元测试 * fix(io): honor RowFilter on async IAsyncEnumerable path when AutoSst is enabled - RunAsync 的 AutoSst 探测改用已过滤的 preparedData(原为原始 data),避免 RowFilter 被静默丢弃导致全量导出 - 修正 WriteAsync(IAsyncEnumerable) XML 文档:去掉误导性的 concurrently 措辞 - 新增回归测试 WriteAsync_AsyncEnumerable_AppliesRowFilter_WhenAutoSstEnabled(异步流+AutoSst+RowFilter 叠加)
1 parent e7f337b commit 1bacf90

80 files changed

Lines changed: 17121 additions & 201 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/dotnetcore.yml‎

Lines changed: 198 additions & 124 deletions
Large diffs are not rendered by default.

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,3 +265,7 @@ __pycache__/
265265
/global.json
266266
!/global.json
267267
/.omo/
268+
269+
# 前端构建产物 (typescript/ 演示的 Vite 打包输出, 勿入库)
270+
index-*.js
271+
index-*.js.map

‎Directory.Build.props‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,10 @@
3030
<Project>
3131

3232
<PropertyGroup>
33-
<Version>2.8.5</Version>
33+
<Version>2.9.0</Version>
3434
<IncludeSymbols>true</IncludeSymbols>
3535
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
36-
<NoWarn>1701;1702;CS1591;CS1573;1591;NU1507</NoWarn>
36+
<NoWarn>1701;1702;CS1591;CS1573;1591;NU1507;NU5119</NoWarn>
3737
<CheckEolTargetFramework>false</CheckEolTargetFramework>
3838
<NuGetAuditMode>direct</NuGetAuditMode>
3939
</PropertyGroup>

‎Directory.Packages.props‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22
<PropertyGroup>
33
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
44
</PropertyGroup>
5-
65
<ItemGroup>
6+
<PackageVersion Include="DocumentFormat.OpenXml" Version="3.5.1" />
77
<PackageVersion Include="DynamicExpresso.Core" Version="2.3.3" />
8+
<PackageVersion Include="LightExcel" Version="2026.6.4.1" />
89
<PackageVersion Include="System.Linq.Dynamic.Core" Version="1.7.2" />
910
<PackageVersion Include="CsvHelper" Version="27.2.1" />
1011
<PackageVersion Include="HtmlToOpenXml.dll" Version="3.1.0" />
@@ -21,27 +22,34 @@
2122
<PackageVersion Include="xunit.runner.visualstudio" Version="2.4.5" />
2223
<PackageVersion Include="Shouldly" Version="4.0.3" />
2324
<PackageVersion Include="Codeuctivity.ImageSharpCompare" Version="2.0.76" />
25+
<PackageVersion Include="Microsoft.NETFramework.ReferenceAssemblies.net471" Version="1.0.3" />
2426
<PackageVersion Include="coverlet.collector" Version="3.1.2" />
2527
<PackageVersion Include="GitHubActionsTestLogger" Version="2.4.1" />
2628
<PackageVersion Include="System.ComponentModel.Annotations" Version="4.7.0" />
2729
<PackageVersion Include="System.Runtime.Loader" Version="4.3.0" />
30+
<!-- netstandard2.0 polyfills for Magicodes.IE.IO -->
31+
<PackageVersion Include="System.Memory" Version="4.5.5" />
32+
<PackageVersion Include="System.Runtime.CompilerServices.Unsafe" Version="6.0.0" />
33+
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" />
34+
<PackageVersion Include="PolySharp" Version="1.15.0" />
2835
<!-- Transitive CVE overrides (Stage 2 Step 6): see Excel.NPOI.csproj for context.
2936
Top-level PackageReference in the consumer csproj forces NuGet to honor these. -->
3037
<PackageVersion Include="System.Text.Encodings.Web" Version="8.0.0" />
3138
<PackageVersion Include="System.Security.Cryptography.Xml" Version="8.0.3" />
3239
<PackageVersion Include="NPOI" Version="2.8.0" />
3340
<PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
3441
<PackageVersion Include="Microsoft.SourceLink.GitHub" Version="8.0.0" />
35-
<PackageVersion Include="BenchmarkDotNet" Version="0.14.0" />
42+
<PackageVersion Include="BenchmarkDotNet" Version="0.15.0" />
3643
<PackageVersion Include="MiniExcel" Version="1.29.0" />
44+
45+
<PackageVersion Include="ClosedXML" Version="0.102.3" />
3746
<PackageVersion Include="FakeItEasy" Version="7.2.0" />
3847
<PackageVersion Include="MSTest.TestAdapter" Version="2.2.7" />
3948
<PackageVersion Include="MSTest.TestFramework" Version="2.2.7" />
4049
<PackageVersion Include="System.Resources.Extensions" Version="5.0.0" />
4150
<PackageVersion Include="System.Xml.XPath.XmlDocument" Version="4.3.0" />
4251
<PackageVersion Include="CS-Script" Version="4.6.5" />
4352
</ItemGroup>
44-
4553
<ItemGroup>
4654
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="3.1.2" />
4755
<PackageVersion Include="Microsoft.Extensions.DependencyModel" Version="3.1.6" />

‎RELEASE.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,138 @@
11
# Release Log
22

3+
## 2.9.0
4+
**Magicodes.IE.IO 首发** — 零 EPPlus 依赖、自写流式 xlsx writer/reader、模板导出、表格/样式、低分配异步性能工作。详见下方「新增」段。
5+
6+
### 新增 — Magicodes.IE.IO (独立新包,零 EPPlus)
7+
8+
阶段二全套交付:自写 xlsx writer,零外部依赖。
9+
10+
#### Stage 2.5 — Span / 向量化 XML 转义 / Utf8Formatter
11+
- 弃 char[] + UTF8Encoding 二次编码 → 直接 pooled byte[] + UTF-8 直写
12+
- XML 转义手写 ASCII escape byte 表(`&` / `<` / `>` / `"` / `'` 5 字符)
13+
- 数字直写 `Utf8Formatter.TryFormat(double/long)`,culture-invariant
14+
- 性能(100k × 4 string):79ms → 32ms(60% ↑)
15+
16+
#### Stage 2.1 — 易用性(属性驱动 + 值类型 + 多 sheet + 列宽 + 冻结)
17+
- 自动识别 `[Display(Name)]` / `[Description]` / `[DisplayFormat]` 做表头与格式
18+
- fluent 覆盖 attribute,3 层披露:fluent > attribute > 默认
19+
- 值类型(struct / record struct)支持,取消 `where T : class` 约束
20+
- 多 sheet:`Xlsx.SaveMultiSheetAsBytes(sheet1, sheet2, ...)`
21+
- 列宽:`WithWidth(30.5)`,支持连续相同宽度合并 `<col>`
22+
- 冻结首行:`WithFreezeHeader(true)`,注入 `<sheetViews><pane ySplit="1" state="frozen"/></sheetViews>`
23+
24+
#### Stage 2.2 — TypedRowPlan<T>(class 路径零 object 装箱)
25+
- 派生 `TypedRowPlan<T> : RowPlan` 暴露 `Func<T, CellValue>[]` typed getters
26+
- `XlsxWriter.WriteRows<T>(data, TypedRowPlan<T>)` 走 typed path
27+
- 属性类型分发(string/int/long/double/decimal/datetime/bool/enum 强类型编译 lambda)
28+
29+
#### Stage 2.3 — 压缩档位可选
30+
- `XlsxWriter(..., CompressionLevel compression)` 参数
31+
- `Xlsx.SaveAsBytes(data, compression: CompressionLevel.NoCompression)` facade
32+
- NoCompression 路径 100k string 38ms → 18ms(2x 加速,产物从 1MB → 14MB)
33+
34+
#### Stage 2.4 — IAsyncEnumerable + 真异步流
35+
- `Xlsx.SaveAsAsync(path, IAsyncEnumerable<T> data, CancellationToken)`
36+
- `Xlsx.SaveAsBytesAsync(IAsyncEnumerable<T> data, CancellationToken)`
37+
- `XlsxWriter.WriteRowsAsync<T>` 内部 `await foreach + ThrowIfCancellationRequested`
38+
- 大表 + 数据库查询场景:不一次性装内存,边查边写
39+
40+
#### Stage 2.6 — CSV / JSON 共用 profile
41+
> 注：当前 2.9.0 构建已移除 CSV/JSON 实现（`Csv.cs`/`Json.cs` 不在产物中），集中做 Excel。此 stage 保留为历史记录。
42+
- `Csv` / `Json` 静态门面,同一份 `ExportProfile<T>` 可输出 xlsx/csv/json
43+
- CSV RFC 4180:含 `,` / `"` / `\r` / `\n` 自动双引号包裹 + 内部 `"` 转 `""`
44+
- JSON 用 STJ(`System.Text.Json`,BCL 内置)
45+
46+
#### Stage 2.7 — BenchmarkDotNet 横向对比(正式报告)
47+
- BenchmarkDotNet v0.14.0,macOS Sequoia 15.6.1,Apple M4,ShortRun,Release/net8.0
48+
- 数据来源:`src/Magicodes.IE.Benchmarks/XlsxIO_Benchmarks.cs` (`--filter "*XlsxIO*"`)
49+
- 结果(Mean 毫秒):
50+
51+
| 库 | 1k string | 10k string | 10k number | 100k string |
52+
|---|---|---|---|---|
53+
| **Magicodes.IE.IO(自写)** | 0.70ms | 3.95ms | 5.79ms | 58.70ms |
54+
| MiniExcel | 3.84ms | 18.29ms | 13.65ms | 163.34ms |
55+
| ClosedXML | 11.93ms | 71.13ms | — | — |
56+
57+
**结论**：Magicodes.IE.IO 在该组数据中明显快于 MiniExcel 和 ClosedXML。
58+
59+
#### Stage 2.8 — IExporter<T> 抽象
60+
> 注：`IExporter<T>` / `XlsxExporter<T>` 已在当前重构中移除（`IExporter.cs`、`Engine/Exporters.cs` 删除），API 统一到 `Xlsx` 静态门面（见 P2-1）。
61+
- `IExporter<T>` 接口 + `XlsxExporter<T>`(已实现) + `CsvExporter<T>` / `JsonExporter<T>`(stage 三 暂未实现,集中做 Excel)
62+
63+
#### Stage 3.3 — `[ExporterHeader]` attribute
64+
- IE 老用户熟悉的 `[ExporterHeader(Name = ..., IsIgnore = ..., Format = ..., Width = ..., Index = ...)]`
65+
- 优先级:fluent cfg > `[ExporterHeader]` > `[Display]` > `[Description]` > 属性名
66+
- 不再需要切到 BCL `[Display]`
67+
68+
#### Stage 3.1 — AOT 警告抑制
69+
- `[UnconditionalSuppressMessage("Trimming", "IL2026")]` + `("AOT", "IL3050")`
70+
- `Expression.Compile` 在 Native AOT 下回退到 `PropertyInfo.GetValue` 路径
71+
- 跨 TFM(net6/8/10)统一,不引入 `RequiresDynamicCode` (net7+) 以保持兼容性
72+
73+
#### P2-1 — API 统一到 `Xlsx` 静态门面(对齐 MiniExcel 风格)
74+
- `XlsxQuery` / `TemplateExporter` 静态类合并到 `Xlsx` 静态类下
75+
- `Xlsx.SheetExport` 嵌套类提升为顶层 `SheetExport`
76+
- 用户 API 现在统一是 `Xlsx.{SaveAs, SaveAsBytes, SaveAsAsync, SaveMultiSheetAsBytes, Query, QueryAsync, ExportByTemplateAsync}` 一行 IDE 提示
77+
- 删除 `TemplateExporter.cs` 文件,功能内置到 `Xlsx.cs`
78+
- `XlsxReader.cs` 只剩 `XlsxReader` 和 `XlsxImporterProfile<T>`(底层 API)
79+
- 测试 46/46 全过(三档 TFM)
80+
81+
### 阶段 3 P3 — 性能 / 内存 / GC 完整对比
82+
BenchmarkDotNet v0.14.0,Apple M4,.NET 8.0.19,MediumJob(15 迭代 × 2 启动)。
83+
84+
| 场景 | 库 | Mean | Allocated | Gen2 | ThreadPool Items | Lock |
85+
|---|---|---|---|---|---|---|
86+
| | **Magicodes.IE.IO** | 4.73 ms | 2.80 MB | **8** | 0 | 0 |
87+
| | MiniExcel | 18.65 ms | 38.76 MB | 3312 | 0 | 0 |
88+
| | ClosedXML | 66.13 ms | 86.71 MB | 1666 | 0 | 0 |
89+
| 100k × 4 string | **Magicodes.IE.IO** | **54.01 ms** | 30.5 MB | **1222** | 0 | 0 |
90+
| | MiniExcel | 161.00 ms | 245.01 MB | 2666 | 0 | 0 |
91+
| 10k × 4 number | **Magicodes.IE.IO** | **5.90 ms** | **1.43 MB** | 109 | 0 | 0 |
92+
| | MiniExcel | 14.29 ms | 34.18 MB | 3437 | 0 | 0 |
93+
94+
**亮点**:
95+
96+
- 完全无锁,纯同步写
97+
98+
#### P3-1 RowPlan 缓存(同 T + profile 复用)
99+
- `ConcurrentDictionary<(Type, profile hash), TypedRowPlan<T>>`
100+
- 同一 profile 多次 `SaveAsBytes` 反射只走一次
101+
- 测试:`RowPlanCache_Same/DifferentProfile_ReusedAcrossCalls`
102+
103+
#### P3-2 样式系统真生效(ResolveColumnStyles)
104+
- 拆 `WriteStyles` 为 `BuildStylePool`(算 xfId 回写 ColumnMeta.StyleId)+ `EmitStylePoolXml`(Dispose 时写)
105+
- facade 在 `WriteHeader` 之前调 `ResolveColumnStyles`,让表头 cell 真带 `s="N"`
106+
- 测试 verify:`Style_BoldHeader_RealXfIdOnCell` / `Style_FontColor_WritesColor` / `Style_Wrap_WritesWrapText` / `Style_FontName_WritesCustomName`
107+
108+
#### P3-3 sharedStrings reader 兼容
109+
- `XlsxReader` 构造时加载 `xl/sharedStrings.xml`
110+
- `t="s"` cell 解析时按 `<v>` 下标查 sharedStrings 数组
111+
- 真 Office 写的 xlsx 现在可读
112+
- 测试:`Read_SharedStrings_RealXlsxFormat`(手工拼 t="s" 风格 xlsx)
113+
114+
#### P3-4 字体名 + 字体颜色真生效
115+
- `ColumnConfig.WithFontName("微软雅黑")` 写进 styles.xml `<font><name val="..."/></font>`
116+
- `fontSet` 元组增 `Name` 字段,去重时按 Bold/Size/Color/Name
117+
- 测试:`Style_FontName_WritesCustomName`
118+
119+
#### ThreadPool 调度优化(性能)
120+
- `Xlsx.WriteAsync<T>` 去掉 `await Task.Yield()`
121+
- 实测 `Completed Work Items` 从 1 → 0,Gen2 31 → 7.8(4 倍下)
122+
- 同步路径不再让出 ThreadPool
123+
124+
### 新增文件
125+
- `src/Magicodes.IE.IO/Magicodes.IE.IO.csproj`
126+
- `src/Magicodes.IE.IO/XlsxWriter.cs`(自写 writer 核心)
127+
- `src/Magicodes.IE.IO/Xlsx.cs`(静态门面)
128+
- `src/Magicodes.IE.IO/ExportProfile.cs`(frozen fluent profile)
129+
- `src/Magicodes.IE.IO/ExporterHeaderAttribute.cs`
130+
- `tests/Magicodes.IE.IO.Tests/` 34 个测试(三档 TFM 全过)
131+
- `src/Magicodes.IE.Benchmarks/XlsxIO_Benchmarks.cs`
132+
133+
### 依赖
134+
- 新增:`ClosedXML` 0.102.3(仅 Benchmarks 用,生产包零依赖)
135+
3136
## 2.8.5
4137
**2026.06.21**
5138

‎src/Magicodes.IE.Benchmarks/LabBenchmarks.cs‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using Magicodes.Benchmarks.Models;
55
using Magicodes.ExporterAndImporter.Excel;
66
using Magicodes.ExporterAndImporter.Excel.Utility;
7+
using Magicodes.IE.IO;
78
using MiniExcelLibs;
89
using System;
910
using System.Collections.Generic;
@@ -63,10 +64,10 @@ public async Task ExportExcelAsByteArrayAsyncTest()
6364
}
6465

6566
[Benchmark]
66-
public async Task ExportMiniExcelAsyncTest()
67+
public async Task ExportMioAsyncTest()
6768
{
6869
var memoryStream = new MemoryStream();
69-
await memoryStream.SaveAsAsync(_exportTestData);
70+
Xlsx.Write(memoryStream, _exportTestData);
7071
memoryStream.Seek(0, SeekOrigin.Begin);
7172
}
7273
}

‎src/Magicodes.IE.Benchmarks/Magicodes.Benchmarks.csproj‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,17 @@
1-
<Project Sdk="Microsoft.NET.Sdk">
1+
<Project Sdk="Microsoft.NET.Sdk">
22

33
<PropertyGroup>
44
<OutputType>Exe</OutputType>
5-
<TargetFramework>net8.0</TargetFramework>
5+
<TargetFrameworks>net8.0;net10.0</TargetFrameworks>
66
<IsPackable>false</IsPackable>
77
</PropertyGroup>
88

99
<ItemGroup>
1010
<PackageReference Include="BenchmarkDotNet" />
11+
<PackageReference Include="LightExcel" />
1112
<PackageReference Include="MiniExcel" />
13+
14+
<PackageReference Include="ClosedXML" />
1215
<PackageReference Include="Newtonsoft.Json" />
1316
</ItemGroup>
1417

@@ -17,6 +20,13 @@
1720
<ProjectReference Include="..\Magicodes.ExporterAndImporter.Csv\Magicodes.IE.Csv.csproj" />
1821
<ProjectReference Include="..\Magicodes.ExporterAndImporter.Excel\Magicodes.IE.Excel.csproj" />
1922
<ProjectReference Include="..\Magicodes.ExporterAndImporter.Pdf\Magicodes.IE.Pdf.csproj" />
23+
<ProjectReference Include="..\Magicodes.IE.IO\Magicodes.IE.IO.csproj" />
24+
<ProjectReference Include="..\EPPlus\EPPlus\Magicodes.IE.EPPlus.csproj" />
25+
<!-- 让 [XlsxExportable] 在 benchmarks 项目里真正触发源生成器。
26+
IO 项目的 XlsxGeneratedGettersRegistry 已用 InternalsVisibleTo("Magicodes.Benchmarks") 授权访问。 -->
27+
<ProjectReference Include="..\Magicodes.IE.IO.SourceGenerator\Magicodes.IE.IO.SourceGenerator.csproj"
28+
OutputItemType="Analyzer"
29+
ReferenceOutputAssembly="false" />
2030
</ItemGroup>
2131

2232
<ItemGroup>

0 commit comments

Comments
 (0)