Include image address in the produced chunk

[adamsitnik]

When we started Data Ingestion, I implemented a bunch of readers for most common parsing services like Azure Document Intelligence or LlamaParse and established a common API surface for both documents and the readers.

Back then, most models supported only text as an input for embedding generator. So what we did was providing AlternativeTextEnricher that generates alt text for all images and special-casing the images in all the chunkers by including their alt text in the input for embedding generators:

extensions/src/Libraries/Microsoft.Extensions.DataIngestion/Chunkers/ElementsChunker.cs

Lines 56 to 61 in d825ca2

	// Image exposes:
	// - Markdown:  which is not very useful for embedding.
	// - AlternativeText: usually a short description of the image, can be null or empty. It is usually less than 50 words.
	// - Text: result of OCR, can be longer, but also can be null or empty. It can be several hundred words.
	// We prefer AlternativeText over Text, as it is usually more relevant.
	IngestionDocumentImage image => image.AlternativeText ?? image.Text,

What I've missed was that in more advanced RAG scenario, the users may be interested in seeing the original image.

So for given image:

![Screenshot of a comment on a GitHub issue showing an image, added in the Markdown, of an Octocat smiling and raising a tentacle.](https://myoctocat.com/assets/images/base-octocat.svg)

We need to include entire markdown in chunks text. So the description is included in the embedding, but when LLM searchers for best match using vector search, it obtains the original text and is capable of showing the image to the end user.

We need to keep in mind that markdown is readonly:

extensions/src/Libraries/Microsoft.Extensions.DataIngestion.Abstractions/IngestionDocumentElement.cs

Line 21 in d825ca2

private protected string _markdown;

But GetMarkdown is not a property in order to avoid caching markdown that could change over time (by for example removing an element from a section):

extensions/src/Libraries/Microsoft.Extensions.DataIngestion.Abstractions/IngestionDocumentElement.cs

Lines 95 to 96 in d825ca2

	public override string GetMarkdown()
	=> string.Join(Environment.NewLine, Elements.Select(e => e.GetMarkdown()));

And all the creations of image element are quite simple in this repo. But we are going to provide more readers soon and they provide more info:

https://github.com/adamsitnik/dataingestion/blob/5e46702aac10c2d535989e88e10986880150fe8e/src/LlamaParse.DocumentReader/LlamaParseReader.cs#L189-L195

https://github.com/adamsitnik/dataingestion/blob/5e46702aac10c2d535989e88e10986880150fe8e/src/Azure.AI.DocumentIntelligence.DocumentReader/DocumentIntelligenceReader.cs#L185-L198

Right now I can see that we have at least these two possible solutions:

• Override GetMarkdown for IngestionDocumentImage. It may require re-parsing the image markdown (and we don't want to introduce any dependencies to the abstractions lib).
• Request the users to provide all necessary info when creating the image element and just make GetMarkdown use string.Format and return all the info without caching it

The two solutions mentioned above don't cover the case when parser (reader) has set image element Text property to a custom value (example: a result of OCR)

[adamsitnik]

Fixed with #7604