Skip to content

Commit 0cdb016

Browse files
committed
fix: move large context to user message in compression example
- Only USER messages are compressed, not system messages - Update example to put context in user message - Add clarifying comment about compression behavior
1 parent a608583 commit 0cdb016

1 file changed

Lines changed: 34 additions & 29 deletions

File tree

example/compression.py

Lines changed: 34 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@
55
2. Set a custom compression rate
66
3. Access compression metrics from the response
77
8-
Note: Compression works on INPUT tokens, so this example includes a large
9-
context document to demonstrate meaningful compression savings.
8+
IMPORTANT: Only USER messages are compressed. System messages are not compressed.
9+
This example includes a large context in the user message to demonstrate meaningful
10+
compression savings.
1011
"""
1112

1213
import os
@@ -24,20 +25,20 @@
2425
LARGE_CONTEXT = """
2526
The History and Impact of Artificial Intelligence
2627
27-
Artificial intelligence (AI) has evolved from a theoretical concept to a
28-
transformative technology that influences nearly every aspect of modern life.
29-
The field began in earnest in the 1950s when pioneers like Alan Turing and
28+
Artificial intelligence (AI) has evolved from a theoretical concept to a
29+
transformative technology that influences nearly every aspect of modern life.
30+
The field began in earnest in the 1950s when pioneers like Alan Turing and
3031
John McCarthy laid the groundwork for machine intelligence.
3132
32-
Early developments focused on symbolic reasoning and expert systems. These
33-
rule-based approaches dominated the field through the 1970s and 1980s, with
34-
systems like MYCIN demonstrating practical applications in medical diagnosis.
35-
However, these early systems were limited by their inability to learn from data
33+
Early developments focused on symbolic reasoning and expert systems. These
34+
rule-based approaches dominated the field through the 1970s and 1980s, with
35+
systems like MYCIN demonstrating practical applications in medical diagnosis.
36+
However, these early systems were limited by their inability to learn from data
3637
and adapt to new situations.
3738
38-
The resurgence of neural networks in the 1980s and 1990s, particularly with
39-
backpropagation algorithms, opened new possibilities. Yet it wasn't until the
40-
2010s, with the advent of deep learning and the availability of massive datasets
39+
The resurgence of neural networks in the 1980s and 1990s, particularly with
40+
backpropagation algorithms, opened new possibilities. Yet it wasn't until the
41+
2010s, with the advent of deep learning and the availability of massive datasets
4142
and computational power, that AI truly began to revolutionize industries.
4243
4344
Modern AI applications span numerous domains:
@@ -48,21 +49,21 @@
4849
- Finance leverages AI for fraud detection, algorithmic trading, and risk assessment
4950
- Transportation is being transformed by autonomous vehicles and traffic optimization
5051
51-
The development of large language models like GPT, BERT, and others has
52-
particularly accelerated progress in natural language understanding and generation.
53-
These models, trained on vast amounts of text data, can perform a wide range of
52+
The development of large language models like GPT, BERT, and others has
53+
particularly accelerated progress in natural language understanding and generation.
54+
These models, trained on vast amounts of text data, can perform a wide range of
5455
language tasks with remarkable proficiency.
5556
56-
Despite remarkable progress, significant challenges remain. Issues of bias,
57-
interpretability, safety, and ethical considerations continue to be areas of
58-
active research and debate. The AI community is working to ensure that these
59-
powerful technologies are developed and deployed responsibly, with consideration
57+
Despite remarkable progress, significant challenges remain. Issues of bias,
58+
interpretability, safety, and ethical considerations continue to be areas of
59+
active research and debate. The AI community is working to ensure that these
60+
powerful technologies are developed and deployed responsibly, with consideration
6061
for their societal impact.
6162
62-
Looking forward, AI is expected to continue advancing rapidly, with potential
63-
breakthroughs in areas like artificial general intelligence, quantum machine
64-
learning, and brain-computer interfaces. The integration of AI into daily life
65-
will likely deepen, raising important questions about human-AI collaboration,
63+
Looking forward, AI is expected to continue advancing rapidly, with potential
64+
breakthroughs in areas like artificial general intelligence, quantum machine
65+
learning, and brain-computer interfaces. The integration of AI into daily life
66+
will likely deepen, raising important questions about human-AI collaboration,
6667
workforce transformation, and the future of human cognition itself.
6768
"""
6869

@@ -72,20 +73,24 @@
7273
print()
7374

7475
# Example: Request with compression enabled and large input
75-
print("Example: Large context with compression enabled")
76+
print("Example: Large user message with compression enabled")
7677
print("-" * 70)
7778
print(f"Input context length: {len(LARGE_CONTEXT)} characters")
7879
print()
7980

81+
# NOTE: Only USER messages are compressed
82+
# Put the large context in the user message to demonstrate compression
83+
user_message = f"""Here is some context about AI:
84+
85+
{LARGE_CONTEXT}
86+
87+
Based on this context, summarize the key milestones in AI development in 3 bullet points."""
88+
8089
response = edgee.send(
8190
model="gpt-4o",
8291
input={
8392
"messages": [
84-
{"role": "system", "content": LARGE_CONTEXT},
85-
{
86-
"role": "user",
87-
"content": "Based on the context above, summarize the key milestones in AI development in 3 bullet points.",
88-
},
93+
{"role": "user", "content": user_message},
8994
],
9095
"enable_compression": True,
9196
"compression_rate": 0.5,

0 commit comments

Comments
 (0)