diff --git a/model-latency-benchmarking/caching-demo-prompts-for-benchmarking.jsonl b/model-latency-benchmarking/caching-demo-prompts-for-benchmarking.jsonl
new file mode 100644
index 000000000..7dd029372
--- /dev/null
+++ b/model-latency-benchmarking/caching-demo-prompts-for-benchmarking.jsonl
@@ -0,0 +1,4 @@
+{"text_prompt": "Using the reference document above, summarize the six pillars of the AWS Well-Architected Framework and explain how prompt caching reduces latency and input-token cost for workloads that reuse a large static context.", "expected_output_tokens": 200, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-haiku-4-5-20251001-v1:0", "region": "us-east-1", "inference_profile": "standard", "prompt_caching": true, "cache_ttl": "5m", "cached_context": "## AWS Well-Architected Framework Overview\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 2)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 2)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 2)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 2)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 2)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 2)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 2)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 2)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 2)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 2)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 3)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 3)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 3)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 3)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 3)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 3)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 3)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 3)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 3)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 3)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors."}
+{"text_prompt": "Using the reference document above, summarize the six pillars of the AWS Well-Architected Framework and explain how prompt caching reduces latency and input-token cost for workloads that reuse a large static context.", "expected_output_tokens": 200, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-haiku-4-5-20251001-v1:0", "region": "us-east-1", "inference_profile": "standard", "prompt_caching": false, "cache_ttl": "5m", "cached_context": "## AWS Well-Architected Framework Overview\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 2)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 2)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 2)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 2)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 2)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 2)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 2)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 2)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 2)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 2)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 3)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 3)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 3)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 3)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 3)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 3)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 3)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 3)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 3)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 3)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors."}
+{"text_prompt": "Using the reference document above, summarize the six pillars of the AWS Well-Architected Framework and explain how prompt caching reduces latency and input-token cost for workloads that reuse a large static context.", "expected_output_tokens": 200, "task_type": "Text-Generation", "model_id": "us.amazon.nova-pro-v1:0", "region": "us-east-1", "inference_profile": "standard", "prompt_caching": true, "cache_ttl": "5m", "cached_context": "## AWS Well-Architected Framework Overview\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 2)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 2)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 2)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 2)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 2)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 2)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 2)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 2)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 2)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 2)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 3)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 3)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 3)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 3)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 3)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 3)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 3)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 3)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 3)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 3)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors."}
+{"text_prompt": "Using the reference document above, summarize the six pillars of the AWS Well-Architected Framework and explain how prompt caching reduces latency and input-token cost for workloads that reuse a large static context.", "expected_output_tokens": 200, "task_type": "Text-Generation", "model_id": "us.amazon.nova-pro-v1:0", "region": "us-east-1", "inference_profile": "standard", "prompt_caching": false, "cache_ttl": "5m", "cached_context": "## AWS Well-Architected Framework Overview\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 2)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 2)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 2)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 2)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 2)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 2)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 2)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 2)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 2)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 2)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors.\n\n## AWS Well-Architected Framework Overview (Detailed Guidance, Part 3)\nThe AWS Well-Architected Framework helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. The framework is organized around six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar contains a set of design principles and best practices that teams can apply when designing and operating systems in the cloud. The framework is not a checklist to be completed once; it is a continuous mechanism for reviewing workloads, identifying risks, and prioritizing remediation. Architects use the framework during design reviews, before major launches, and at regular intervals after a workload reaches production so that decisions stay aligned with evolving business goals and customer needs.\n\n## Operational Excellence Pillar (Detailed Guidance, Part 3)\nThe operational excellence pillar focuses on running and monitoring systems to deliver business value and continually improving supporting processes and procedures. Key topics include organizing teams around business outcomes, implementing observability across the stack, automating responses to events, and performing frequent, small, reversible changes. Teams that practice operational excellence define runbooks for routine operations and playbooks for unexpected events. They instrument their applications with structured logs, metrics, and traces, and they review operational metrics in regular operational readiness reviews. Game days and failure injection exercises help teams validate that their procedures work under realistic conditions before a genuine incident occurs. The goal is to learn from every operational event and feed those lessons back into design and process.\n\n## Security Pillar (Detailed Guidance, Part 3)\nThe security pillar describes how to protect data, systems, and assets to take advantage of cloud technologies to improve security. Core practices include implementing a strong identity foundation, applying least privilege, enabling traceability, applying security at all layers, automating security best practices, protecting data in transit and at rest, keeping people away from raw data, and preparing for security events. Identity and access management is central: every principal should have only the permissions it needs, and those permissions should be reviewed regularly. Encryption keys should be managed with a dedicated key management service, and sensitive material should never be embedded in source code or configuration files. Detective controls such as centralized logging and continuous monitoring allow teams to detect anomalies and respond before they escalate into incidents.\n\n## Reliability Pillar (Detailed Guidance, Part 3)\nThe reliability pillar ensures a workload performs its intended function correctly and consistently when it is expected to. This includes the ability to operate and test the workload through its total lifecycle. Reliability practices cover foundations such as managing service quotas and network topology, workload architecture that isolates faults, change management that uses automation to deploy and roll back safely, and failure management that anticipates, withstands, and recovers from disruptions. Designing for reliability means assuming that components will fail and building systems that degrade gracefully. Techniques such as health checks, automatic failover across availability zones, backups with tested restores, and well-defined recovery time and recovery point objectives are foundational. Reliability is verified through testing, not assumed from design.\n\n## Performance Efficiency Pillar (Detailed Guidance, Part 3)\nThe performance efficiency pillar focuses on using computing resources efficiently to meet system requirements and maintaining that efficiency as demand changes and technologies evolve. It encourages teams to democratize advanced technologies, go global in minutes, use serverless architectures, experiment more often, and consider mechanical sympathy when selecting resources. Choosing the right instance families, storage classes, and database engines for a workload can dramatically change both performance and cost. Caching is a recurring theme: caching static content at the edge, caching database query results, and caching expensive computations all reduce latency and load. Teams should benchmark candidate configurations with representative traffic rather than relying on intuition, and they should re-evaluate choices as new instance types and managed services become available.\n\n## Cost Optimization Pillar (Detailed Guidance, Part 3)\nThe cost optimization pillar helps teams avoid unnecessary costs. It includes understanding spending over time, selecting the most cost-effective resources, matching supply with demand, and continuously analyzing and attributing expenditure. Practices include adopting a consumption model so you pay only for what you use, using managed services to reduce the cost of ownership, and applying commitment-based discounts where usage is predictable. Tagging resources consistently allows costs to be attributed to teams, products, and environments, which in turn drives accountability. Right-sizing compute, deleting idle resources, and moving infrequently accessed data to cheaper storage tiers are common, high-impact optimizations. Cost optimization is a continuous activity that pairs well with operational metrics and forecasting.\n\n## Sustainability Pillar (Detailed Guidance, Part 3)\nThe sustainability pillar focuses on minimizing the environmental impact of running cloud workloads. It encourages teams to understand their impact, establish sustainability goals, maximize utilization, anticipate and adopt new more efficient hardware and software offerings, use managed services, and reduce the downstream impact of cloud workloads. Maximizing utilization means running fewer, busier resources rather than many idle ones. Choosing regions and scheduling flexible workloads thoughtfully can also reduce impact. The pillar treats sustainability as a shared responsibility between the cloud provider, which improves the efficiency of the underlying infrastructure, and the customer, which improves the efficiency of the workloads that run on it.\n\n## Prompt Caching on Amazon Bedrock (Detailed Guidance, Part 3)\nPrompt caching is an Amazon Bedrock feature that lets applications cache a static prompt prefix so that repeated requests reusing that prefix can skip reprocessing it. This reduces time to first token and lowers input-token cost for workloads that send a large, stable context on every request, such as a system prompt, a long set of instructions, a knowledge document, or few-shot examples. To use prompt caching with the Converse API, the request places the static content first and then inserts a cache checkpoint block immediately after the content that should be cached. On a cache hit, the response usage block reports cache read input tokens; on the write that populates the cache, it reports cache write input tokens. Each model has a minimum number of tokens that the cached prefix must contain before caching is triggered, and the cache entry has a time to live after which it expires. Prompt caching is available for on-demand invocation and is most effective when the same large prefix is reused many times within the cache lifetime.\n\n## Latency Benchmarking Methodology (Detailed Guidance, Part 3)\nBenchmarking model latency requires careful, repeatable methodology. Time to first token, sometimes called time to first byte, measures how quickly the model begins returning output and is heavily influenced by input length and any cached prefix. Time to last token measures the full response duration. Output tokens per second is computed from the output token count divided by the total streaming time. To produce trustworthy numbers, run many invocations per scenario, report percentiles such as the median, the ninetieth percentile, and the standard deviation rather than a single average, and hold inference parameters constant across compared scenarios. When comparing cached and uncached behavior, keep the model, region, prompt, and inference configuration identical so that the cache is the only variable. Warm the cache with an initial request before measuring cache hit latency, because the first request pays the write cost while later requests enjoy the read benefit.\n\n## Inference Parameters and Their Effects (Detailed Guidance, Part 3)\nTemperature, top-p, and top-k shape how a model samples its next token. Temperature scales the probability distribution: lower values make output more deterministic and focused, while higher values make it more diverse and creative. Top-p, also called nucleus sampling, restricts sampling to the smallest set of tokens whose cumulative probability exceeds the configured threshold. Top-k restricts sampling to the k most likely tokens. For reproducible benchmarks, these parameters must be set explicitly and held constant, because letting them vary or default silently makes results impossible to compare. Not every model supports every parameter; top-k in particular is passed through an additional model request fields channel and should be omitted for models that do not accept it, to avoid validation errors."}
diff --git a/model-latency-benchmarking/dummy-prompts-for-benchmarking.jsonl b/model-latency-benchmarking/dummy-prompts-for-benchmarking.jsonl
index c11d72b08..12c24006e 100644
--- a/model-latency-benchmarking/dummy-prompts-for-benchmarking.jsonl
+++ b/model-latency-benchmarking/dummy-prompts-for-benchmarking.jsonl
@@ -1,5 +1,4 @@
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.meta.llama3-1-70b-instruct-v1:0", "region": "us-east-2", "inference_profile": "optimized"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0", "region": "us-east-2", "inference_profile": "optimized"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0", "region": "us-east-2", "inference_profile": "standard"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "amazon.nova-micro-v1:0", "region": "us-east-1", "inference_profile": "standard"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "anthropic.claude-3-5-sonnet-20241022-v2:0", "region": "us-west-2", "inference_profile": "standard"}
\ No newline at end of file
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-opus-4-5-20251101-v1:0", "region": "us-east-1", "inference_profile": "standard"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-sonnet-4-5-20250929-v1:0", "region": "us-east-1", "inference_profile": "optimized"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "region": "us-west-2", "inference_profile": "standard"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.amazon.nova-pro-v1:0", "region": "us-east-1", "inference_profile": "standard"}
diff --git a/model-latency-benchmarking/latency-benchmarking-tool.ipynb b/model-latency-benchmarking/latency-benchmarking-tool.ipynb
index 2e34ccbe8..7081761c0 100644
--- a/model-latency-benchmarking/latency-benchmarking-tool.ipynb
+++ b/model-latency-benchmarking/latency-benchmarking-tool.ipynb
@@ -50,6 +50,27 @@
     "Note: if you configure `\"inference_profile\": \"optimized\"`, you must use `us-east-2` region only because optimized inference is currently only available in `us-east-2` region. "
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Enabling prompt caching\n",
+    "\n",
+    "Prompt caching is optional and turned off by default, so existing datasets run unchanged. When enabled, the tool caches a large static prefix so repeated requests that reuse it skip reprocessing, which lowers Time to First Token and input-token cost.\n",
+    "\n",
+    "To enable prompt caching, add these optional fields to a scenario line in your JSONL dataset:\n",
+    "\n",
+    "- `prompt_caching`: set to `true` to cache the static prefix for that scenario (default `false`).\n",
+    "- `cached_context`: the long, static text to cache. Required when `prompt_caching` is `true`, and it must meet the model minimum cacheable token count (for example, 4,096 tokens for Claude Haiku 4.5) for caching to trigger.\n",
+    "- `cache_ttl`: how long the cache entry lives, either `5m` (default) or `1h`.\n",
+    "\n",
+    "You can also flip the global `PROMPT_CACHING` toggle in the Configuration cell to enable caching for every scenario by default. A per-scenario `prompt_caching` value always overrides the global toggle.\n",
+    "\n",
+    "Prompt caching works for on-demand invocation only and only on caching-capable models, so set `prompt_caching: true` only on models that support it. The file `caching-demo-prompts-for-benchmarking.jsonl` ships ready-to-run cached and uncached pairs for Claude Haiku 4.5 and Amazon Nova Pro.\n",
+    "\n",
+    "When caching is active, each invocation also reports `cache_read_input_tokens`, `cache_write_input_tokens`, and `Cache_Hit_Rate`, and the analysis reports Time to First Token separately for cached and uncached calls.\n"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -148,7 +169,17 @@
     "TEMPERATURE = 1\n",
     "TOP_P = 1\n",
     "TOP_K = 250\n",
-    "EXPERIMENT_NAME = '<name-and-version-of-your-experiment>' # your custom experiment name"
+    "\n",
+    "# Some Bedrock models (for example Amazon Nova and the latest Claude models)\n",
+    "# reject temperature and topP in the same request, so the tool sends only one\n",
+    "# sampling parameter. Set this to 'temperature' or 'topP'.\n",
+    "INFERENCE_SAMPLING = 'temperature'\n",
+    "EXPERIMENT_NAME = '<name-and-version-of-your-experiment>' # your custom experiment name\n",
+    "\n",
+    "# Global prompt caching default toggle. Shipped default is\n",
+    "# False so existing datasets behave exactly as before. A per-scenario\n",
+    "# `prompt_caching` field in the JSONL always overrides this global default.\n",
+    "PROMPT_CACHING = False"
    ]
   },
   {
@@ -180,6 +211,46 @@
     "from threading import Lock\n",
     "from typing import List, Dict\n",
     "\n",
+    "# ---------------------------------------------------------------------------\n",
+    "# Minimum required boto3 version\n",
+    "# Minimum boto3 that supports the cachePoint block in the Converse API and the\n",
+    "# latest listed Bedrock models (Claude Opus/Sonnet/Haiku 4.5, Claude 3.7\n",
+    "# Sonnet, Amazon Nova). The tool checks this before running any benchmark.\n",
+    "# This value is parseable by check_boto3_version below.\n",
+    "# ---------------------------------------------------------------------------\n",
+    "MIN_BOTO3_VERSION = \"1.40.0\"\n",
+    "\n",
+    "def _parse_version(version):\n",
+    "    \"\"\"Parse a dotted version string into a 3-tuple of ints (major, minor, patch).\n",
+    "\n",
+    "    Non-numeric suffixes are stripped so values like '1.40.0' or '1.40.0rc1'\n",
+    "    parse cleanly. Missing components default to 0 so the result always has\n",
+    "    three elements and tuple comparison is well defined.\"\"\"\n",
+    "    parts = []\n",
+    "    for token in str(version).strip().split('.'):\n",
+    "        digits = ''.join(ch for ch in token if ch.isdigit())\n",
+    "        parts.append(int(digits) if digits else 0)\n",
+    "    while len(parts) < 3:\n",
+    "        parts.append(0)\n",
+    "    return tuple(parts[:3])\n",
+    "\n",
+    "def check_boto3_version(installed, minimum):\n",
+    "    \"\"\"Return (is_satisfied, message) from a pure comparison of version tuples.\n",
+    "\n",
+    "    is_satisfied is True if and only if the installed version is greater than\n",
+    "    or equal to the minimum. When unsatisfied, the message states the required\n",
+    "    minimum version so the user knows the target to upgrade to.\"\"\"\n",
+    "    is_satisfied = _parse_version(installed) >= _parse_version(minimum)\n",
+    "    if is_satisfied:\n",
+    "        message = f\"boto3 {installed} satisfies the minimum required version {minimum}.\"\n",
+    "    else:\n",
+    "        message = (\n",
+    "            f\"boto3 {installed} is below the required minimum version {minimum}. \"\n",
+    "            f\"Please upgrade boto3 to {minimum} or later before running this tool \"\n",
+    "            f\"(for example: pip install --upgrade 'boto3>={minimum}').\"\n",
+    "        )\n",
+    "    return is_satisfied, message\n",
+    "\n",
     "logging_lock = Lock()\n",
     "os.makedirs(f\"{directory}\", exist_ok=True)\n",
     "os.makedirs(f\"{directory}-analysis\", exist_ok=True)\n",
@@ -211,23 +282,89 @@
     "        content = file.read()\n",
     "    return content\n",
     "\n",
-    "def get_body(model_id, file_path, prompt, max_tokens):\n",
-    "    body = [\n",
+    "def _is_anthropic_model(model_id):\n",
+    "    '''Return True when the target model is an Anthropic Claude model.'''\n",
+    "    normalized = (model_id or '').lower()\n",
+    "    return 'anthropic' in normalized or 'claude' in normalized\n",
+    "\n",
+    "def _build_cache_point_block(cache_ttl, model_id):\n",
+    "    '''Return a Converse cachePoint content block for the target model.\n",
+    "\n",
+    "    The Converse API expresses a cache checkpoint as a content block of the\n",
+    "    form {'cachePoint': {'type': 'default'}}. The default cache duration is\n",
+    "    five minutes. The optional 'ttl' field requests an extended duration and\n",
+    "    is only supported for Anthropic models, so it is attached only for the\n",
+    "    '1h' option on Anthropic models. Every other case uses a plain cachePoint\n",
+    "    with the default duration. cache_ttl is validated upstream by\n",
+    "    validate_caching_fields, so it is one of {'5m', '1h'} here.'''\n",
+    "    block = {'cachePoint': {'type': 'default'}}\n",
+    "    if cache_ttl == '1h' and _is_anthropic_model(model_id):\n",
+    "        block['cachePoint']['ttl'] = '1h'\n",
+    "    return block\n",
+    "\n",
+    "def _model_supports_top_k(model_id):\n",
+    "    '''Return True when the target model accepts top_k.\n",
+    "\n",
+    "    top_k is not part of the standard Converse inferenceConfig and is passed\n",
+    "    through additionalModelRequestFields. The Anthropic Claude family accepts\n",
+    "    top_k there, while other families (for example Amazon Nova) reject the\n",
+    "    field and fail validation. We gate on the model identifier so top_k is\n",
+    "    sent only when supported and omitted otherwise.'''\n",
+    "    normalized = (model_id or '').lower()\n",
+    "    return 'anthropic' in normalized or 'claude' in normalized\n",
+    "\n",
+    "def build_converse_request(prompt, cached_context, prompt_caching, cache_ttl, max_tokens, temperature, top_p, top_k, model_id):\n",
+    "    '''Build the Converse request parts for a single invocation.\n",
+    "\n",
+    "    Returns (messages, inferenceConfig, additionalModelRequestFields). This\n",
+    "    replaces the legacy get_body and fixes the temperature/top_p defect. The\n",
+    "    benchmark function passes modelId and performanceConfig.latency separately\n",
+    "    when it calls the model.\n",
+    "\n",
+    "    Behavior:\n",
+    "      - inferenceConfig sends only one sampling parameter (temperature or\n",
+    "        topP) because some models reject both together; neither is hardcoded.\n",
+    "      - When prompt_caching is True, the user message places the cached_context\n",
+    "        text block first, then exactly one cachePoint block carrying the TTL\n",
+    "        derived from cache_ttl, then the user prompt.\n",
+    "      - When prompt_caching is False, no cachePoint block is included and the\n",
+    "        message matches the original single-text structure\n",
+    "        with no cachePoint block.\n",
+    "      - top_k is passed through additionalModelRequestFields only when it is\n",
+    "        set and supported by the target model, otherwise it is omitted and\n",
+    "        this return value is None.'''\n",
+    "    if prompt_caching:\n",
+    "        content = [\n",
+    "            {'text': cached_context},\n",
+    "            _build_cache_point_block(cache_ttl, model_id),\n",
+    "            {'text': prompt},\n",
+    "        ]\n",
+    "    else:\n",
+    "        content = [\n",
+    "            {'text': prompt},\n",
+    "        ]\n",
+    "\n",
+    "    messages = [\n",
     "        {\n",
     "            'role': 'user',\n",
-    "            'content': [\n",
-    "                {\n",
-    "                'text': prompt\n",
-    "                },\n",
-    "            ]\n",
+    "            'content': content,\n",
     "        },\n",
     "    ]\n",
-    "    inferenceConfig={\n",
-    "        'maxTokens': max_tokens,\n",
-    "        'temperature': 0,\n",
-    "        'topP': 1\n",
-    "    }\n",
-    "    return body, inferenceConfig\n",
+    "\n",
+    "    # Some models reject temperature and topP together, so send only the\n",
+    "    # configured sampling parameter (see INFERENCE_SAMPLING in the config cell).\n",
+    "    sampling = globals().get('INFERENCE_SAMPLING', 'temperature')\n",
+    "    inferenceConfig = {'maxTokens': max_tokens}\n",
+    "    if sampling == 'topP':\n",
+    "        inferenceConfig['topP'] = top_p\n",
+    "    else:\n",
+    "        inferenceConfig['temperature'] = temperature\n",
+    "\n",
+    "    additionalModelRequestFields = None\n",
+    "    if top_k is not None and _model_supports_top_k(model_id):\n",
+    "        additionalModelRequestFields = {'top_k': top_k}\n",
+    "\n",
+    "    return messages, inferenceConfig, additionalModelRequestFields\n",
     "\n",
     "def read_jsonl_files(directory_path):\n",
     "    all_data = []\n",
@@ -243,11 +380,107 @@
     "                        print(f\"Error decoding JSON in file {filename}: {e}\")\n",
     "    return all_data\n",
     "\n",
+    "def parse_scenario(raw, file_path, global_prompt_caching):\n",
+    "    \"\"\"Map a raw JSONL object to a Scenario dict with caching fields resolved.\n",
+    "\n",
+    "    Backward compatibility is enforced here. A raw object\n",
+    "    that contains only the original fields produces a Scenario that behaves\n",
+    "    exactly like prompt_caching=False when the global default is False\n",
+    "    so existing datasets keep working unchanged.\n",
+    "\n",
+    "    Optional caching fields and their defaults:\n",
+    "      - prompt_caching: bool, default = global_prompt_caching\n",
+    "      - cache_ttl: str, default \\\"5m\\\"\n",
+    "      - cached_context: str or None, default None\n",
+    "\n",
+    "    All existing fields are preserved: text_prompt, expected_output_tokens,\n",
+    "    task_type, model_id, region, inference_profile.\"\"\"\n",
+    "    scenario = {\n",
+    "        'file_path': file_path,\n",
+    "        'text_prompt': raw.get('text_prompt'),\n",
+    "        'expected_output_tokens': raw.get('expected_output_tokens', 100),\n",
+    "        'task_type': raw.get('task_type'),\n",
+    "        'model_id': raw.get('model_id'),\n",
+    "        'region': raw.get('region'),\n",
+    "        'inference_profile': raw.get('inference_profile', 'optimized'),\n",
+    "    }\n",
+    "    # Resolve the effective prompt caching value: the per-scenario field always\n",
+    "    # wins when present, otherwise fall back to the global default.\n",
+    "    scenario['prompt_caching'] = raw.get('prompt_caching', global_prompt_caching)\n",
+    "    scenario['cache_ttl'] = raw.get('cache_ttl', '5m')\n",
+    "    scenario['cached_context'] = raw.get('cached_context', None)\n",
+    "    return scenario\n",
+    "\n",
+    "def validate_caching_fields(scenario):\n",
+    "    \"\"\"Return (is_valid, error_message) for a scenario's caching fields.\n",
+    "\n",
+    "    Validation runs per scenario before invocation and\n",
+    "    MUST NOT raise. The run loop turns an invalid result\n",
+    "    into an error status and descriptive message; this function only reports\n",
+    "    validity. Rules:\n",
+    "      - When prompt_caching is True, cached_context must be a non-empty\n",
+    "        string; None, empty string, and whitespace-only are all treated as\n",
+    "        missing and are invalid.\n",
+    "      - cache_ttl must be one of {\\\"5m\\\", \\\"1h\\\"}.\n",
+    "      - Otherwise the scenario is valid.\"\"\"\n",
+    "    prompt_caching = scenario.get('prompt_caching', False)\n",
+    "    cache_ttl = scenario.get('cache_ttl', '5m')\n",
+    "    cached_context = scenario.get('cached_context', None)\n",
+    "\n",
+    "    if prompt_caching and (cached_context is None or not str(cached_context).strip()):\n",
+    "        return False, (\n",
+    "            'Prompt caching is enabled but cached_context is empty or missing. '\n",
+    "            'Provide a non-empty cached_context or disable prompt_caching for '\n",
+    "            'this scenario.'\n",
+    "        )\n",
+    "\n",
+    "    if cache_ttl not in ('5m', '1h'):\n",
+    "        return False, (\n",
+    "            f\"Unsupported cache_ttl value '{cache_ttl}'. cache_ttl must be \"\n",
+    "            f\"either '5m' or '1h'.\"\n",
+    "        )\n",
+    "\n",
+    "    return True, ''\n",
+    "\n",
+    "def extract_usage_metrics(usage):\n",
+    "    \"\"\"Extract token counts from a Converse usage block.\n",
+    "\n",
+    "    Returns a dict with input_tokens, output_tokens, cache_read_input_tokens,\n",
+    "    and cache_write_input_tokens. The usage block is the 'usage' object found\n",
+    "    in the converse_stream metadata event. This factors the usage parsing out\n",
+    "    of benchmark so it can be tested directly.\n",
+    "\n",
+    "    Behavior:\n",
+    "      - input_tokens and output_tokens are read from 'inputTokens' and\n",
+    "        'outputTokens'.\n",
+    "      - cache_read_input_tokens and cache_write_input_tokens are read from\n",
+    "        'cacheReadInputTokens' and 'cacheWriteInputTokens' and default to 0\n",
+    "        when the corresponding field is absent.\n",
+    "      - Missing fields never raise; usage may be None or empty.\"\"\"\n",
+    "    usage = usage or {}\n",
+    "    return {\n",
+    "        'input_tokens': usage.get('inputTokens'),\n",
+    "        'output_tokens': usage.get('outputTokens'),\n",
+    "        'cache_read_input_tokens': usage.get('cacheReadInputTokens', 0) or 0,\n",
+    "        'cache_write_input_tokens': usage.get('cacheWriteInputTokens', 0) or 0,\n",
+    "    }\n",
+    "\n",
+    "def derive_cache_hit_rate(cache_read_input_tokens, total_input_tokens):\n",
+    "    \"\"\"Return the per-invocation Cache_Hit_Rate.\n",
+    "\n",
+    "    Cache_Hit_Rate is cache_read_input_tokens divided by total_input_tokens\n",
+    "    when total_input_tokens is greater than zero, and 0.0 otherwise. Returning\n",
+    "    0.0 on a zero (or missing) total avoids a divide-by-zero and keeps the\n",
+    "    metric well defined for every invocation.\"\"\"\n",
+    "    if total_input_tokens and total_input_tokens > 0:\n",
+    "        return cache_read_input_tokens / total_input_tokens\n",
+    "    return 0.0\n",
+    "\n",
     "def post_iteration(scenario_config):\n",
     "    logging.info(f'Sleeping for {scenario_config[\"sleep_between_invocations\"]} seconds.')\n",
     "    time.sleep(scenario_config[\"sleep_between_invocations\"])\n",
     "\n",
-    "def benchmark(bedrock, file_path, prompt, latency_inference_profile, max_tokens, model_id=\"\", stream=True, sleep_on_throttling=5):\n",
+    "def benchmark(bedrock, file_path, prompt, latency_inference_profile, max_tokens, model_id=\"\", stream=True, sleep_on_throttling=5, prompt_caching=False, cached_context=None, cache_ttl=\"5m\", temperature=0, top_p=1, top_k=None):\n",
     "    accept = 'application/json'\n",
     "    content_type = 'application/json'\n",
     "    api_call_status = 'Success'\n",
@@ -256,20 +489,38 @@
     "    dt = datetime.fromtimestamp(time.time(), tz=pytz.utc)\n",
     "    job_timestamp_iso = dt.strftime('%Y-%m-%dT%H:%M:%SZ')\n",
     "\n",
-    "    body, inference_config = get_body(model_id, file_path, prompt, max_tokens)\n",
+    "    body, inference_config, additional_model_request_fields = build_converse_request(\n",
+    "        prompt=prompt,\n",
+    "        cached_context=cached_context,\n",
+    "        prompt_caching=prompt_caching,\n",
+    "        cache_ttl=cache_ttl,\n",
+    "        max_tokens=max_tokens,\n",
+    "        temperature=temperature,\n",
+    "        top_p=top_p,\n",
+    "        top_k=top_k,\n",
+    "        model_id=model_id,\n",
+    "    )\n",
     "    output_token_size, input_token_size = None, None\n",
+    "    # Cache token fields default to zero so the return schema stays stable even\n",
+    "    # on a failed invocation or when the response omits cache usage fields.\n",
+    "    # so the CSV schema stays stable.\n",
+    "    cache_read_input_tokens, cache_write_input_tokens = 0, 0\n",
     "\n",
     "    while True:\n",
     "        try:\n",
     "            start = time.time()\n",
-    "            response = bedrock.converse_stream(\n",
-    "                messages=body,\n",
-    "                modelId=model_id,\n",
-    "                inferenceConfig=inference_config,\n",
-    "                performanceConfig={\n",
-    "                        'latency': latency_inference_profile\n",
-    "                    }\n",
-    "            )\n",
+    "            converse_kwargs = {\n",
+    "                'messages': body,\n",
+    "                'modelId': model_id,\n",
+    "                'inferenceConfig': inference_config,\n",
+    "                'performanceConfig': {'latency': latency_inference_profile},\n",
+    "            }\n",
+    "            # Pass additionalModelRequestFields (for example top_k) only when the\n",
+    "            # request builder produced it, so models that reject the field are\n",
+    "            # not sent an empty/invalid parameter.\n",
+    "            if additional_model_request_fields is not None:\n",
+    "                converse_kwargs['additionalModelRequestFields'] = additional_model_request_fields\n",
+    "            response = bedrock.converse_stream(**converse_kwargs)\n",
     "            first_byte = None\n",
     "            event_stream = response.get('stream')\n",
     "            for event in event_stream:   \n",
@@ -283,8 +534,11 @@
     "                elif 'metadata' in event:\n",
     "                    metadata = event['metadata']\n",
     "                    if 'usage' in metadata:\n",
-    "                        output_token_size = metadata['usage'].get('outputTokens', None)\n",
-    "                        input_token_size = metadata['usage'].get('inputTokens', None)\n",
+    "                        usage_metrics = extract_usage_metrics(metadata['usage'])\n",
+    "                        output_token_size = usage_metrics['output_tokens']\n",
+    "                        input_token_size = usage_metrics['input_tokens']\n",
+    "                        cache_read_input_tokens = usage_metrics['cache_read_input_tokens']\n",
+    "                        cache_write_input_tokens = usage_metrics['cache_write_input_tokens']\n",
     "            last_byte = time.time()\n",
     "            duration_to_first_byte = round(first_byte - start, 2)\n",
     "            duration_to_last_byte = round(last_byte - start, 2)\n",
@@ -296,7 +550,7 @@
     "            break\n",
     "        else:\n",
     "            break\n",
-    "    return duration_to_first_byte, duration_to_last_byte, job_timestamp_iso, api_call_status, full_error_message, output_token_size, input_token_size\n",
+    "    return duration_to_first_byte, duration_to_last_byte, job_timestamp_iso, api_call_status, full_error_message, output_token_size, input_token_size, cache_read_input_tokens, cache_write_input_tokens\n",
     "\n",
     "def execute_benchmark(client, scenarios, scenario_config, num_parallel_calls=4, early_break=False):\n",
     "    pp = pprint.PrettyPrinter(indent=2)\n",
@@ -306,27 +560,78 @@
     "        local_client = get_bedrock_client(scenario['region'])\n",
     "        local_invocations = []\n",
     "        file_path = scenario['file_path']\n",
-    "        prompt = scenario['prompt']\n",
-    "        \n",
+    "        # Resolve fields tolerant of both parse_scenario output (text_prompt,\n",
+    "        # inference_profile, expected_output_tokens) and pre-derived keys so the\n",
+    "        # run loop can build scenarios directly via parse_scenario.\n",
+    "        prompt = scenario.get('prompt', scenario.get('text_prompt'))\n",
+    "        latency_inference_profile = scenario.get('latency_inference_profile', scenario.get('inference_profile', 'optimized'))\n",
+    "        configured_output_tokens = scenario.get('configured_output_tokens_for_request', scenario.get('expected_output_tokens', 100))\n",
+    "        stream = scenario.get('stream', True)\n",
+    "        prompt_caching = scenario.get('prompt_caching', PROMPT_CACHING)\n",
+    "        cached_context = scenario.get('cached_context', None)\n",
+    "        cache_ttl = scenario.get('cache_ttl', '5m')\n",
+    "\n",
+    "        # Validate caching fields before any Bedrock call.\n",
+    "        # An invalid configuration is recorded as a CachingConfigError invocation\n",
+    "        # and the scenario is skipped without calling Bedrock, so the run continues\n",
+    "        # and the CSV schema stays stable.\n",
+    "        is_valid, validation_error = validate_caching_fields(scenario)\n",
+    "        if not is_valid:\n",
+    "            invocation = {\n",
+    "                'time_to_first_byte': None,\n",
+    "                'time_to_last_byte': None,\n",
+    "                'job_timestamp_iso': get_timestamp(),\n",
+    "                'configured_output_tokens_for_request': configured_output_tokens,\n",
+    "                'model_input_tokens': None,\n",
+    "                'model_output_tokens': None,\n",
+    "                'model': scenario['model_id'],\n",
+    "                'region': scenario['region'],\n",
+    "                'invocation_id': 0,\n",
+    "                'api_call_status': 'CachingConfigError',\n",
+    "                'full_error_message': validation_error,\n",
+    "                'TEMPERATURE': TEMPERATURE,\n",
+    "                'TOP_P': TOP_P,\n",
+    "                'TOP_K': TOP_K,\n",
+    "                'EXPERIMENT_NAME': EXPERIMENT_NAME,\n",
+    "                'task_type': scenario['task_type'],\n",
+    "                'inference_profile': latency_inference_profile,\n",
+    "                'prompt_caching': prompt_caching,\n",
+    "                'cache_ttl': cache_ttl,\n",
+    "                'cache_read_input_tokens': 0,\n",
+    "                'cache_write_input_tokens': 0,\n",
+    "                'cache_hit_rate': derive_cache_hit_rate(0, None),\n",
+    "            }\n",
+    "            local_invocations.append(invocation)\n",
+    "            with logging_lock:\n",
+    "                logging.error(f\"Skipping scenario {scenario['model_id']} due to caching config error: {validation_error}\")\n",
+    "            return local_invocations\n",
+    "\n",
     "        for invocation_id in range(scenario_config[\"invocations_per_scenario\"]):\n",
     "            try:\n",
     "                time_to_first_byte, time_to_last_byte, job_timestamp_iso, api_call_status, \\\n",
-    "                full_error_message, model_output_tokens, model_input_tokens = benchmark(\n",
+    "                full_error_message, model_output_tokens, model_input_tokens, \\\n",
+    "                cache_read_input_tokens, cache_write_input_tokens = benchmark(\n",
     "                    local_client,\n",
     "                    file_path,\n",
     "                    prompt,\n",
-    "                    latency_inference_profile=scenario['latency_inference_profile'],\n",
-    "                    max_tokens=scenario['configured_output_tokens_for_request'],\n",
+    "                    latency_inference_profile=latency_inference_profile,\n",
+    "                    max_tokens=configured_output_tokens,\n",
     "                    model_id=scenario['model_id'],\n",
-    "                    stream=scenario['stream'],\n",
-    "                    sleep_on_throttling=scenario_config['sleep_between_invocations']\n",
+    "                    stream=stream,\n",
+    "                    sleep_on_throttling=scenario_config['sleep_between_invocations'],\n",
+    "                    prompt_caching=prompt_caching,\n",
+    "                    cached_context=cached_context,\n",
+    "                    cache_ttl=cache_ttl,\n",
+    "                    temperature=TEMPERATURE,\n",
+    "                    top_p=TOP_P,\n",
+    "                    top_k=TOP_K\n",
     "                )\n",
     "\n",
     "                invocation = {\n",
     "                    'time_to_first_byte': time_to_first_byte,\n",
     "                    'time_to_last_byte': time_to_last_byte,\n",
     "                    'job_timestamp_iso': job_timestamp_iso,\n",
-    "                    'configured_output_tokens_for_request': scenario['configured_output_tokens_for_request'],\n",
+    "                    'configured_output_tokens_for_request': configured_output_tokens,\n",
     "                    'model_input_tokens': model_input_tokens,\n",
     "                    'model_output_tokens': model_output_tokens,\n",
     "                    'model': scenario['model_id'],\n",
@@ -339,7 +644,12 @@
     "                    'TOP_K': TOP_K,\n",
     "                    'EXPERIMENT_NAME': EXPERIMENT_NAME,\n",
     "                    'task_type': scenario['task_type'],\n",
-    "                    'inference_profile': scenario['latency_inference_profile'],\n",
+    "                    'inference_profile': latency_inference_profile,\n",
+    "                    'prompt_caching': prompt_caching,\n",
+    "                    'cache_ttl': cache_ttl,\n",
+    "                    'cache_read_input_tokens': cache_read_input_tokens,\n",
+    "                    'cache_write_input_tokens': cache_write_input_tokens,\n",
+    "                    'cache_hit_rate': derive_cache_hit_rate(cache_read_input_tokens, model_input_tokens),\n",
     "                }\n",
     "                local_invocations.append(invocation)\n",
     "                \n",
@@ -350,8 +660,36 @@
     "                post_iteration(scenario_config=scenario_config)\n",
     "                \n",
     "            except Exception as e:\n",
+    "                # Fail-soft: any failed invocation records its status and full\n",
+    "                # error message, then the run continues. Cache\n",
+    "                # token columns are zeroed so the CSV schema stays stable.\n",
     "                with logging_lock:\n",
     "                    logging.error(f\"Error while processing scenario: {scenario['model_id']}. Error: {e}\")\n",
+    "                invocation = {\n",
+    "                    'time_to_first_byte': None,\n",
+    "                    'time_to_last_byte': None,\n",
+    "                    'job_timestamp_iso': get_timestamp(),\n",
+    "                    'configured_output_tokens_for_request': configured_output_tokens,\n",
+    "                    'model_input_tokens': None,\n",
+    "                    'model_output_tokens': None,\n",
+    "                    'model': scenario['model_id'],\n",
+    "                    'region': scenario['region'],\n",
+    "                    'invocation_id': invocation_id,\n",
+    "                    'api_call_status': 'InvocationError',\n",
+    "                    'full_error_message': str(e),\n",
+    "                    'TEMPERATURE': TEMPERATURE,\n",
+    "                    'TOP_P': TOP_P,\n",
+    "                    'TOP_K': TOP_K,\n",
+    "                    'EXPERIMENT_NAME': EXPERIMENT_NAME,\n",
+    "                    'task_type': scenario['task_type'],\n",
+    "                    'inference_profile': latency_inference_profile,\n",
+    "                    'prompt_caching': prompt_caching,\n",
+    "                    'cache_ttl': cache_ttl,\n",
+    "                    'cache_read_input_tokens': 0,\n",
+    "                    'cache_write_input_tokens': 0,\n",
+    "                    'cache_hit_rate': derive_cache_hit_rate(0, None),\n",
+    "                }\n",
+    "                local_invocations.append(invocation)\n",
     "                \n",
     "        return local_invocations\n",
     "\n",
@@ -383,29 +721,30 @@
     "        return all_invocations\n",
     "\n",
     "if __name__ == \"__main__\":\n",
+    "    # Version gate: stop before any invocation when boto3 is too old to support\n",
+    "    # cachePoint in Converse and the latest listed models.\n",
+    "    is_boto3_satisfied, boto3_version_message = check_boto3_version(boto3.__version__, MIN_BOTO3_VERSION)\n",
+    "    print(boto3_version_message)\n",
+    "    if not is_boto3_satisfied:\n",
+    "        logging.error(boto3_version_message)\n",
+    "        raise SystemExit(boto3_version_message)\n",
+    "\n",
     "    use_cases_scenarios = []\n",
     "\n",
     "    # Read the JSONL file and process each line\n",
     "    with open(file_path, 'r', encoding='utf-8') as f:\n",
     "        for line in f:\n",
-    "            file = json.loads(line.strip())\n",
-    "            prompt = file.get('text_prompt')\n",
-    "            task_type = file.get('task_type')\n",
-    "            model_id = file.get('model_id')\n",
-    "            region = file.get('region')\n",
-    "            latency_inference_profile = file.get('inference_profile', 'optimized')\n",
-    "\n",
-    "            out_tokens = file.get('expected_output_tokens', 100)\n",
-    "            use_cases_scenarios.append({\n",
-    "                \"file_path\": file_path,\n",
-    "                \"configured_output_tokens_for_request\": out_tokens,\n",
-    "                \"prompt\": prompt,\n",
-    "                \"stream\": True,\n",
-    "                \"model_id\": model_id,\n",
-    "                \"region\": region,\n",
-    "                \"task_type\": task_type,\n",
-    "                \"latency_inference_profile\": latency_inference_profile\n",
-    "            })\n",
+    "            raw = json.loads(line.strip())\n",
+    "            # Build each scenario via parse_scenario so it carries the resolved\n",
+    "            # caching fields (prompt_caching, cache_ttl, cached_context) with the\n",
+    "            # global PROMPT_CACHING default applied. Datasets without the new\n",
+    "            # fields behave exactly as before.\n",
+    "            use_cases_scenarios.append(parse_scenario(raw, file_path, PROMPT_CACHING))\n",
+    "\n",
+    "    # The main-thread client below is a placeholder; each scenario opens its\n",
+    "    # own regional client inside process_scenario. Derive a region for it from\n",
+    "    # the first parsed scenario so the variable is always defined.\n",
+    "    region = use_cases_scenarios[0]['region'] if use_cases_scenarios else 'us-east-1'\n",
     "\n",
     "    # Main loop\n",
     "    run_count = 1\n",
@@ -505,6 +844,19 @@
     "\n",
     "def calculate_metrics(df, group_columns):\n",
     "    \"\"\"Calculate latency metrics grouped by model, region, and inference profile.\"\"\"\n",
+    "    # Be robust to older CSVs that predate the cache columns: if a cache\n",
+    "    # column is absent in the combined DataFrame, treat it as 0 so analysis\n",
+    "    # of pre-existing result files still works.\n",
+    "    for cache_col in ['cache_read_input_tokens', 'cache_write_input_tokens']:\n",
+    "        if cache_col not in df.columns:\n",
+    "            df[cache_col] = 0\n",
+    "    df['cache_read_input_tokens'] = pd.to_numeric(df['cache_read_input_tokens'], errors='coerce').fillna(0)\n",
+    "    df['cache_write_input_tokens'] = pd.to_numeric(df['cache_write_input_tokens'], errors='coerce').fillna(0)\n",
+    "\n",
+    "    # Derive is_cached per invocation: a cache hit means cache read tokens > 0\n",
+    "    # A cache hit means cache read tokens are above zero.\n",
+    "    df['is_cached'] = df['cache_read_input_tokens'] > 0\n",
+    "\n",
     "    metrics = df.groupby(group_columns).agg({\n",
     "        'time_to_first_byte': ['count', 'mean', 'median', \n",
     "                              lambda x: x.quantile(0.9), \n",
@@ -527,6 +879,32 @@
     "    otps_metrics.columns = ['OTPS_mean', 'OTPS_p50', 'OTPS_p90', 'OTPS_std']\n",
     "    \n",
     "    metrics = pd.concat([metrics, otps_metrics], axis=1)\n",
+    "\n",
+    "    # TTFT reported separately for cached vs uncached invocations\n",
+    "    # Each group is reindexed to the full metrics index so\n",
+    "    # a model with no cached (or no uncached) samples yields NaN instead of\n",
+    "    # dropping from the table.\n",
+    "    ttft_cached_mean = df[df['is_cached']].groupby(group_columns)['time_to_first_byte'].mean().round(3)\n",
+    "    ttft_uncached_mean = df[~df['is_cached']].groupby(group_columns)['time_to_first_byte'].mean().round(3)\n",
+    "    metrics['TTFT_cached_mean'] = ttft_cached_mean.reindex(metrics.index)\n",
+    "    metrics['TTFT_uncached_mean'] = ttft_uncached_mean.reindex(metrics.index)\n",
+    "\n",
+    "    # Aggregated cache token columns.\n",
+    "    cache_metrics = df.groupby(group_columns).agg({\n",
+    "        'cache_read_input_tokens': ['mean'],\n",
+    "        'cache_write_input_tokens': ['mean'],\n",
+    "    }).round(3)\n",
+    "    cache_metrics.columns = ['avg_cache_read_input_tokens', 'avg_cache_write_input_tokens']\n",
+    "    metrics = pd.concat([metrics, cache_metrics], axis=1)\n",
+    "\n",
+    "    # Aggregated cache_hit_rate as total cache read tokens over total input\n",
+    "    # tokens per group; 0.0 when a group has no input tokens.\n",
+    "    grouped = df.groupby(group_columns)\n",
+    "    total_cache_read = grouped['cache_read_input_tokens'].sum()\n",
+    "    total_input = grouped['model_input_tokens'].sum()\n",
+    "    cache_hit_rate = (total_cache_read / total_input).where(total_input > 0, 0.0).round(3)\n",
+    "    metrics['cache_hit_rate'] = cache_hit_rate.reindex(metrics.index)\n",
+    "\n",
     "    return metrics\n",
     "\n",
     "def create_performance_summary_tables(df, metrics, pdf):\n",
diff --git a/model-latency-benchmarking/readme.md b/model-latency-benchmarking/readme.md
index c7fe36bc4..6225abb63 100644
--- a/model-latency-benchmarking/readme.md
+++ b/model-latency-benchmarking/readme.md
@@ -8,51 +8,84 @@ This tool helps you benchmark model latency metrics for Large Language Models (L
 2. Measures how many tokens the model generates per second (Output Tokens Per Second)
 3. Tests different model versions and settings
 4. Handles multiple API calls at once
-5. Provides analysis and statistics that you can use to take decisions
+5. Supports optional, opt-in prompt caching per scenario and captures cache metrics
+6. Provides analysis and statistics that you can use to take decisions
 
-## How to use it
+## Prerequisites
+
+- AWS account with Amazon Bedrock access.
+- Model access enabled for every model you reference. The latest models (Claude Opus 4.5, Claude Sonnet 4.5, Claude Haiku 4.5, Claude 3.7 Sonnet, and Amazon Nova Pro) require model access to be enabled in the account and region where you run the benchmark. Enable access in the Amazon Bedrock console under Model access before running the tool.
+- A recent `boto3` version. The notebook declares a minimum required `boto3` version that supports the latest Bedrock models and prompt caching through the Converse API. If your installed version is below that minimum, the notebook reports the requirement before invoking any model. Upgrade with `pip install -U boto3`.
+- A file with your prompts in `JSONL` format (see the required format below).
+- Access to the AWS region you want to use.
 
-1. Make sure you have:
-   - AWS account with Amazon Bedrock access
-   - A file with your prompts (in `JSONL format` check required format below)
-   - Access to the AWS region you want to use
+## How to use it
 
-2. Set up your test:
+1. Set up your test:
    - Put your prompts in a JSONL file
    - Change settings in the first code cell (like file paths and test details)
+   - Some models reject `temperature` and `topP` together; set `INFERENCE_SAMPLING` in the configuration cell to `'temperature'` (default) or `'topP'`.
 
-3. Run the code:
+2. Run the code:
    - All cells will run automatically
    - Results will be saved in your chosen folder
    - You'll get a log file with details about what happened
 
-4. Check your results:
+3. Check your results:
    - Look at the CSV files for detailed metrics
    - Review the final analysis for overall performance
 
 ## Required Dataset Format
 
-Your input JSONL file should contain one JSON object per line with the following fields:
+Your input JSONL file should contain one JSON object per line. The following fields are supported. Existing datasets keep working unchanged: the caching fields are optional and additive, and a scenario without them behaves exactly as before.
+
+| Field | Type | Required | Default | Notes |
+|-------|------|----------|---------|-------|
+| `text_prompt` | string | yes | - | The user prompt. |
+| `expected_output_tokens` | int | no | 100 | Maximum output tokens for the request. |
+| `task_type` | string | no | - | Currently supports `Text-Generation`. |
+| `model_id` | string | yes | - | Model ID or cross-region inference profile ID. |
+| `region` | string | yes | - | AWS region where the model is invoked. |
+| `inference_profile` | string | no | `optimized` | Latency setting passed via Converse `performanceConfig`, either `optimized` or `standard`. |
+| `prompt_caching` | bool | no | `false` | Opt-in prompt caching for the scenario. When omitted, caching is off. |
+| `cache_ttl` | string | no | `5m` | Cache checkpoint time-to-live. Allowed values are `5m` and `1h`. The `1h` extended TTL is only supported on Anthropic models; on other models the tool falls back to the default `5m`. |
+| `cached_context` | string | no | none | The long, static prompt prefix to cache. Required when `prompt_caching` is `true`. |
 
 ```json
 {
     "text_prompt": "Your question or instruction here",
-    "expected_output_tokens": 50,  // number of tokens expected in output
-    "task_type": "Text-Generation",  // currently supports Text-Generation
-    "model_id": "us.meta.llama3-1-70b-instruct-v1:0",  // model identifier
-    "region": "us-west-2", // region where you want to benchmark model latency metrics
-    "inference_profile": "optimized"  // optimization setting 
+    "expected_output_tokens": 50,
+    "task_type": "Text-Generation",
+    "model_id": "us.anthropic.claude-haiku-4-5-20251001-v1:0",
+    "region": "us-east-1",
+    "inference_profile": "optimized"
 }
 ```
 
 #### Example entries from the test dataset:
 
 ```json
-{"text_prompt": "Summarize the key features of cloud computing in one sentence.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.meta.llama3-1-70b-instruct-v1:0", "region": "us-east-2", "inference_profile": "optimized"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0", "region": "us-east-2", "inference_profile": "optimized"}
-{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0", "region": "us-east-2", "inference_profile": "standard"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-opus-4-5-20251101-v1:0", "region": "us-east-1", "inference_profile": "standard"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-sonnet-4-5-20250929-v1:0", "region": "us-east-1", "inference_profile": "optimized"}
+{"text_prompt": "Explain the concept of machine learning in simple terms.", "expected_output_tokens": 50, "task_type": "Text-Generation", "model_id": "us.amazon.nova-pro-v1:0", "region": "us-east-1", "inference_profile": "standard"}
+```
+
+#### Example entry with prompt caching enabled:
+
+```json
+{"text_prompt": "Using the reference document above, summarize the key points.", "expected_output_tokens": 200, "task_type": "Text-Generation", "model_id": "us.anthropic.claude-haiku-4-5-20251001-v1:0", "region": "us-east-1", "inference_profile": "standard", "prompt_caching": true, "cache_ttl": "5m", "cached_context": "<your long static context here>"}
 ```
 
+## Prompt caching
+
+Prompt caching reuses a static prompt prefix (`cached_context`) across requests to lower Time to First Token and input-token cost. Set `prompt_caching` to `true` and provide the prefix in `cached_context`; the optional fields are described in the dataset table above. Caching is off by default, so existing datasets are unaffected.
+
+- Available for on-demand invocation only.
+- Caching only triggers when the cached prefix meets the model's minimum token threshold (for example, 4,096 tokens for Claude Haiku 4.5). Confirm current thresholds and supported models in the [Amazon Bedrock prompt caching documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html).
+- Output adds `cache_read_input_tokens`, `cache_write_input_tokens`, and `Cache_Hit_Rate`, and the analysis reports Time to First Token for cached and uncached calls separately. Existing columns are unchanged.
+
+To see it in action, point the dataset path at `caching-demo-prompts-for-benchmarking.jsonl`, which pairs cached and uncached scenarios for Claude Haiku 4.5 and Amazon Nova Pro.
+
 ## Important notes
 
 - Test results depend on your specific prompts and setup
@@ -60,4 +93,4 @@ Your input JSONL file should contain one JSON object per line with the following
 - More test runs give more accurate results
 
 ## Need help?
-Check the comments in the code for more detailed information about each part of the framework or open an issue.
\ No newline at end of file
+Check the comments in the code for more detailed information about each part of the framework or open an issue.