Add cell outputs

Author: kacperlukawski

tutorials/49_TurboQuant_Quantization_with_HuggingFace.ipynb

@@ -8,7 +8,7 @@
     "\n",
     "- **Level**: Advanced\n",
     "- **Time to complete**: 20 min\n",
-    "- **Nodes Used**: [`HuggingFaceLocalChatGenerator`](https://docs.haystack.deepset.ai/docs/huggingfacelocalchatgenerator)\n",
+    "- **Components Used**: [`HuggingFaceLocalChatGenerator`](https://docs.haystack.deepset.ai/docs/huggingfacelocalchatgenerator)\n",
     "- **Goal**: Apply TurboQuant KV cache compression to a local LLM and measure its memory and throughput impact with Haystack."
    ]
   },
@@ -41,14 +41,29 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T13:26:59.381856Z",
+     "start_time": "2026-04-03T13:26:24.012627Z"
+    }
+   },
    "source": [
     "%%bash\n",
     "\n",
     "pip install -q haystack-ai turboquant-vllm"
-   ]
+   ],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\u001B[1m[\u001B[0m\u001B[34;49mnotice\u001B[0m\u001B[1;39;49m]\u001B[0m\u001B[39;49m A new release of pip is available: \u001B[0m\u001B[31;49m25.0.1\u001B[0m\u001B[39;49m -> \u001B[0m\u001B[32;49m26.0.1\u001B[0m\n",
+      "\u001B[1m[\u001B[0m\u001B[34;49mnotice\u001B[0m\u001B[1;39;49m]\u001B[0m\u001B[39;49m To update, run: \u001B[0m\u001B[32;49mpip install --upgrade pip\u001B[0m\n"
+     ]
+    }
+   ],
+   "execution_count": 1
   },
   {
    "cell_type": "markdown",
@@ -61,9 +76,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T13:26:59.418920Z",
+     "start_time": "2026-04-03T13:26:59.390162Z"
+    }
+   },
    "source": [
     "import time\n",
     "\n",
@@ -76,7 +94,9 @@
     "    if first_token_time is None:\n",
     "        first_token_time = now\n",
     "    last_token_time = now"
-   ]
+   ],
+   "outputs": [],
+   "execution_count": 2
   },
   {
    "cell_type": "markdown",
@@ -95,9 +115,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T13:27:40.845609Z",
+     "start_time": "2026-04-03T13:26:59.424193Z"
+    }
+   },
    "source": [
     "from transformers import DynamicCache\n",
     "from turboquant_vllm import CompressedDynamicCache\n",
@@ -107,7 +130,18 @@
     "# and not `compressed` directly.\n",
     "cache = DynamicCache()\n",
     "compressed = CompressedDynamicCache(cache, head_dim=128, bits=4)"
-   ]
+   ],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/kacper.lukawski/Projects/haystack-tutorials/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
+      "  from .autonotebook import tqdm as notebook_tqdm\n"
+     ]
+    }
+   ],
+   "execution_count": 3
   },
   {
    "cell_type": "markdown",
@@ -120,12 +154,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T13:28:00.561590Z",
+     "start_time": "2026-04-03T13:27:40.873264Z"
+    }
+   },
    "source": [
     "from haystack.components.generators.chat import HuggingFaceLocalChatGenerator\n",
-    "from haystack.utils import Secret\n",
     "\n",
     "generator = HuggingFaceLocalChatGenerator(\n",
     "    model=\"Qwen/Qwen3-4B-Thinking-2507\",\n",
@@ -136,7 +172,9 @@
     "    },\n",
     "    streaming_callback=timing_callback,\n",
     ")"
-   ]
+   ],
+   "outputs": [],
+   "execution_count": 4
   },
   {
    "cell_type": "markdown",
@@ -149,9 +187,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T14:06:57.416718Z",
+     "start_time": "2026-04-03T13:28:00.574408Z"
+    }
+   },
    "source": [
     "from haystack.dataclasses import ChatMessage\n",
     "\n",
@@ -160,17 +201,57 @@
     "    ChatMessage.from_user(\"What is the capital of France?\"),\n",
     "])\n",
     "total_time = time.perf_counter() - start"
-   ]
+   ],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Loading checkpoint shards: 100%|██████████| 3/3 [00:00<00:00, 55.96it/s]\n",
+      "Device set to use mps\n"
+     ]
+    }
+   ],
+   "execution_count": 5
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T14:06:59.733732Z",
+     "start_time": "2026-04-03T14:06:58.328820Z"
+    }
+   },
    "source": [
     "reply = output[\"replies\"][0]\n",
     "print(reply.text)"
-   ]
+   ],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Okay, the user is asking, \"What is the capital of France?\" This seems like a straightforward geography question. Let me recall... I know that Paris is the capital of France. But wait, I should make sure I'm not mixing it up with other countries. For example, London is the capital of the UK, and Berlin is for Germany. Yeah, France's capital is definitely Paris.\n",
+      "\n",
+      "Hmm, why would someone ask this? Maybe they're a student studying for a test, or someone learning English as a second language. They might be confirming basic facts. Or perhaps they're testing me to see if I know the answer correctly. Either way, I should give a clear and confident answer.\n",
+      "\n",
+      "I remember that historically, Paris has been the capital since the Middle Ages. It's also known for the Eiffel Tower, the Louvre Museum, and the French Revolution. So, there's no doubt here. But I should double-check to avoid any mistakes. Let me think... Yes, all reliable sources say Paris is the capital. No recent changes either—France hasn't moved its capital. \n",
+      "\n",
+      "The user might also be confused if they heard about \"République française\" or something else, but no, the capital is still Paris. I should mention it's in the Île-de-France region to be precise, but maybe that's extra. The question is simple, so the answer should be concise.\n",
+      "\n",
+      "Wait, is there any trick here? Like, does France have multiple capitals? No, Paris is the only capital. Sometimes people confuse it with other cities like Lyon or Marseille, but those are major cities, not capitals. So, no confusion here.\n",
+      "\n",
+      "I think the best response is to state clearly that Paris is the capital of France. Maybe add a brief note about its significance to be helpful. Like, it's the political, cultural, and economic center. But the user just asked for the capital, so keep it short. Don't overcomplicate it.\n",
+      "\n",
+      "Also, the user might be non-native English speaker, so I should use simple language. No need for complex terms. Just \"Paris\" is enough, but adding \"the capital city of France\" makes it clear.\n",
+      "\n",
+      "Let me phrase it: \"The capital of France is Paris.\" Done. That's accurate and straightforward. No need for more details unless the user asks follow-ups. \n",
+      "\n",
+      "Wait, just to be thorough—did France ever have another capital? Like, during the French Revolution, they moved temporarily? No, Paris remained the capital. There\n"
+     ]
+    }
+   ],
+   "execution_count": 6
   },
   {
    "cell_type": "markdown",
@@ -187,9 +268,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T14:07:00.078667Z",
+     "start_time": "2026-04-03T14:06:59.919485Z"
+    }
+   },
    "source": [
     "tokens = reply.meta[\"usage\"][\"completion_tokens\"]\n",
     "if first_token_time is not None and last_token_time is not None:\n",
@@ -198,7 +282,20 @@
     "    print(f\"Tokens: {tokens}\")\n",
     "    print(f\"Speed: {tokens / generation_time:.1f} tok/s\")\n",
     "print(f\"Total time: {total_time:.3f}s\")"
-   ]
+   ],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "TTFT: 81.985s\n",
+      "Tokens: 512\n",
+      "Speed: 0.4 tok/s\n",
+      "Total time: 1425.370s\n"
+     ]
+    }
+   ],
+   "execution_count": 7
   },
   {
    "cell_type": "markdown",
@@ -211,12 +308,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2026-04-03T14:07:00.526515Z",
+     "start_time": "2026-04-03T14:07:00.121042Z"
+    }
+   },
    "source": [
     "compressed.vram_bytes()"
-   ]
+   ],
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "20680704"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "execution_count": 8
   },
   {
    "cell_type": "markdown",