README.md

Intelligent PDF Summarizer - .NET

The purpose of this sample application is to demonstrate how Durable Functions can be leveraged to create intelligent applications, particularly in a document processing scenario. Order and durability are key here because the results from one activity are passed to the next. Also, calls to services like Cognitive Service or LLMs are costly and should not be repeated in the event of failures.

This sample integrates various Azure services, including the new Durable Task Scheduler for Azure Durable Functions, Azure Storage, Azure Cognitive Services, and Azure Open AI.

The application showcases how PDFs can be ingested and intelligently scanned to determine their content.

The application's workflow is as follows:

1. PDFs are uploaded to a blob storage input container.
2. A durable function is triggered upon blob upload.
3. The Durable Task Scheduler for Azure Durable Functions schedules an orchestration that:

• • Downloads the blob (PDF).
• • Utilizes the Azure Cognitive Service Form Recognizer endpoint to extract the text from the PDF.
• • Sends the extracted text to Azure Open AI to analyze and determine the content of the PDF.
• • Save the summary results from Azure Open AI to a new file and upload it to the output blob container.

Below, you will find the instructions to set up and run this app locally..

Prerequsites

• Create an active Azure subscription.
• Install the latest Azure Functions Core Tools to use the CLI
• Enroll in the Durable Task Scheduler Preview and create a scheduler / taskhub resource.
• Start and configure an Azurite storage emulator for local storage.

local.settings.json

You will need to configure a local.settings.json file at the root of the repo that looks similar to the below. Make sure to replace the placeholders with your specific values.

{
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated",
    "DURABLE_TASK_SERVICE_CONNECTION_STRING": "Endpoint=<DTS-SCHEDULER-ENDPOINT>;Authentication=DefaultAzure",
    "TASKHUB_NAME": "<TASKHUB-NAME>",
    "COGNITIVE_SERVICES_ENDPOINT": "<COGNITIVE-SERVICE-ENDPOINT>",
    "AZURE_OPENAI_ENDPOINT": "<AZURE-OPEN-AI-ENDPOINT>",
    "CHAT_MODEL_DEPLOYMENT_NAME": "<AZURE-OPEN-AI-MODEL>"
  }
}

Running the app locally

1. Start Azurite: Begin by starting Azurite, the local Azure Storage emulator.

2. Start the Function App: Start the function app to run the application locally.

func start --verbose

Inspect the code

This app leverages Durable Functions to orchestrate the application workflow. By using Durable Functions, there's no need for additional infrastructure like queues and state stores to manage task coordination and durability, which significantly reduces the complexity for developers.

Take a look at the code snippet below, the ProcessDocument defines the entire workflow, which consists of a series of steps (activities) that need to be scheduled in sequence. Coordination is key, as the output of one activity is passed as an input to the next. Additionally, Durable Functions handle durability and retries, which ensure that if a failure occurs, such as a transient error or an issue with a dependent service, the workflow can recover gracefully.

Deploy the app to Azure

Use the Azure Developer CLI (azd) to easily deploy the app.

1. In the root of the project, run the following command to provision and deploy the app:

   azd up

2. When prompted, provide:

   • A name for your Azure Developer CLI environment.
   • The Azure subscription you'd like to use.
   • The Azure location to use.

   Note: At the moment, the Durable Task Scheduler can only be deployed to West US2 and NorthCentral US. Unfortunately, the AZD CLI does not support North Central US and Azure Open AI does not support West US2. The best way to navigage discrepencies is to select West US2 as your AZD location and hardcode the openAi module location to a region like East US2.

Once the azd up command finishes, the app will have successfully provisioned and deployed.

Using the app

To use the app, simply upload a PDF to the Blob Storage input container. Once the PDF is transferred, it will be processed using document intelligence and Azure OpenAI. The resulting summary will be saved to a new file and uploaded to the output container.

Review Execution History in the Durable Task Scheduler Dashboard

Since the application leverages the Durable Task Scheduler backend storage provider for Durable Functions, all orchestration executions are persisted and can be viewed in the monitoring and management dashboard.

This dashboard enables you to monitor the status of ongoing and failed orchestrations, and troubleshoot any issues causing orchestrations to fail or become "stuck." It also displays the outputs from each activity execution. Additionally, you can use the timeline and sequence diagrams to gain insights into the orchestration execution, including the duration of each activity.