Provides an implementation of the IngestionDocumentReader class for the Markdown files using MarkDig library.
From the command-line:
dotnet add package Microsoft.Extensions.DataIngestion.Markdig --prereleaseOr directly in the C# project file:
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DataIngestion.Markdig" Version="[CURRENTVERSION]" />
</ItemGroup>using Microsoft.Extensions.DataIngestion;
IngestionDocumentReader reader = new MarkdownReader();
using IngestionPipeline<string> pipeline = new(CreateChunker(), CreateWriter());
await foreach (IngestionResult result in pipeline.ProcessAsync(reader, directory, "*.md"))
{
Console.WriteLine($"Processed '{result.DocumentId}'. Succeeded: {result.Succeeded}");
}We welcome feedback and contributions in our GitHub repo.