Skip to content

Commit f2e3548

Browse files
committed
docs: update compression example with larger input context
- Add substantial AI history document as context (~3000+ chars) - Demonstrate meaningful compression on large input - Show percentage of tokens saved - Explain that compression works on input tokens
1 parent ba76500 commit f2e3548

1 file changed

Lines changed: 74 additions & 5 deletions

File tree

example/compression.py

Lines changed: 74 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
11
"""Example: Token compression with Edgee Gateway SDK
22
33
This example demonstrates how to:
4-
1. Enable compression for a request
4+
1. Enable compression for a request with a large input context
55
2. Set a custom compression rate
66
3. Access compression metrics from the response
7+
8+
Note: Compression works on INPUT tokens, so this example includes a large
9+
context document to demonstrate meaningful compression savings.
710
"""
811

912
import os
@@ -17,18 +20,73 @@
1720
# Initialize the client
1821
edgee = Edgee(os.environ.get("EDGEE_API_KEY"))
1922

23+
# Large context document to demonstrate input compression
24+
LARGE_CONTEXT = """
25+
The History and Impact of Artificial Intelligence
26+
27+
Artificial intelligence (AI) has evolved from a theoretical concept to a
28+
transformative technology that influences nearly every aspect of modern life.
29+
The field began in earnest in the 1950s when pioneers like Alan Turing and
30+
John McCarthy laid the groundwork for machine intelligence.
31+
32+
Early developments focused on symbolic reasoning and expert systems. These
33+
rule-based approaches dominated the field through the 1970s and 1980s, with
34+
systems like MYCIN demonstrating practical applications in medical diagnosis.
35+
However, these early systems were limited by their inability to learn from data
36+
and adapt to new situations.
37+
38+
The resurgence of neural networks in the 1980s and 1990s, particularly with
39+
backpropagation algorithms, opened new possibilities. Yet it wasn't until the
40+
2010s, with the advent of deep learning and the availability of massive datasets
41+
and computational power, that AI truly began to revolutionize industries.
42+
43+
Modern AI applications span numerous domains:
44+
- Natural language processing enables machines to understand and generate human language
45+
- Computer vision allows machines to interpret visual information from the world
46+
- Robotics combines AI with mechanical systems for autonomous operation
47+
- Healthcare uses AI for diagnosis, drug discovery, and personalized treatment
48+
- Finance leverages AI for fraud detection, algorithmic trading, and risk assessment
49+
- Transportation is being transformed by autonomous vehicles and traffic optimization
50+
51+
The development of large language models like GPT, BERT, and others has
52+
particularly accelerated progress in natural language understanding and generation.
53+
These models, trained on vast amounts of text data, can perform a wide range of
54+
language tasks with remarkable proficiency.
55+
56+
Despite remarkable progress, significant challenges remain. Issues of bias,
57+
interpretability, safety, and ethical considerations continue to be areas of
58+
active research and debate. The AI community is working to ensure that these
59+
powerful technologies are developed and deployed responsibly, with consideration
60+
for their societal impact.
61+
62+
Looking forward, AI is expected to continue advancing rapidly, with potential
63+
breakthroughs in areas like artificial general intelligence, quantum machine
64+
learning, and brain-computer interfaces. The integration of AI into daily life
65+
will likely deepen, raising important questions about human-AI collaboration,
66+
workforce transformation, and the future of human cognition itself.
67+
"""
68+
2069
print("=" * 70)
2170
print("Edgee Token Compression Example")
2271
print("=" * 70)
2372
print()
2473

25-
# Example 1: Request with compression enabled
26-
print("Example 1: Request with compression enabled")
74+
# Example: Request with compression enabled and large input
75+
print("Example: Large context with compression enabled")
2776
print("-" * 70)
77+
print(f"Input context length: {len(LARGE_CONTEXT)} characters")
78+
print()
79+
2880
response = edgee.send(
2981
model="gpt-4o",
3082
input={
31-
"messages": [{"role": "user", "content": "Explain quantum computing in simple terms."}],
83+
"messages": [
84+
{"role": "system", "content": LARGE_CONTEXT},
85+
{
86+
"role": "user",
87+
"content": "Based on the context above, summarize the key milestones in AI development in 3 bullet points.",
88+
},
89+
],
3290
"enable_compression": True,
3391
"compression_rate": 0.5,
3492
},
@@ -51,7 +109,18 @@
51109
print(f" Input tokens: {response.compression.input_tokens}")
52110
print(f" Saved tokens: {response.compression.saved_tokens}")
53111
print(f" Compression rate: {response.compression.rate:.2%}")
54-
print(f" Token savings: {response.compression.saved_tokens} tokens saved!")
112+
savings_pct = (
113+
(response.compression.saved_tokens / response.compression.input_tokens * 100)
114+
if response.compression.input_tokens > 0
115+
else 0
116+
)
117+
print(f" Savings: {savings_pct:.1f}% of input tokens saved!")
118+
print()
119+
print(f" 💡 Without compression, this request would have used")
120+
print(f" {response.compression.input_tokens} input tokens.")
121+
print(
122+
f" With compression, only {response.compression.input_tokens - response.compression.saved_tokens} tokens were processed!"
123+
)
55124
else:
56125
print("No compression data available in response.")
57126
print("Note: Compression data is only returned when compression is enabled")

0 commit comments

Comments
 (0)