Skip to content

Commit a6e2137

Browse files
thsackosthsackos
committed
Add index, first post
1 parent ebf1025 commit a6e2137

5 files changed

Lines changed: 154 additions & 0 deletions

File tree

‎out/www/2025/09/25/abstractionholics-anonymous.html‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
<!DOCTYPE html>
2+
<html lang="en-US"><head>
3+
<meta charset="utf-8">
4+
<meta name="viewport" content="width=device-width, initial-scale=1">
5+
<title>Sackosoft</title>
6+
<meta name="description" content="Sackosoft Blog">
7+
<link rel="icon" href="/favicon.png" type="image/png">
8+
<link rel="canonical" href="https://sackosoft.github.io">
9+
<link rel="stylesheet" href="/css/main.css">
10+
</head>
11+
<body>
12+
<header>
13+
<nav>
14+
<a class="title" href="/">sackosoft</a>
15+
</nav>
16+
</header>
17+
<main>
18+
<h1 id="abstractionholics-anonymous">Abstractionholics Anonymous</h1>
19+
<p>Hello, everyone, my name is Theodore, and I’m an
20+
Abstractionholic.</p>
21+
<blockquote>
22+
<p>The first step to recovery is admitting that one has a problem.
23+
Admitting that one is powerless over their desire to abstract. Admitting
24+
that one’s usage of abstraction has made their life unmanageable.</p>
25+
</blockquote>
26+
<p>I’m here to admit that I have a problem. I am powerless over my
27+
compulsion toward abstraction.</p>
28+
<h2 id="the-dream-stream">The Dream Stream</h2>
29+
<p>Let’s take a step back. I think event streaming (à la Apache Kafka)
30+
is great. When a problem can be decomposed into a pipeline of simple
31+
operations, then one can handle massive throughput with relatively
32+
simple components connected by partitioned and ordered event streams.
33+
This design allows the components in the pipeline to add batching,
34+
caching and maintain idempotency with relatively little effort. However,
35+
I’ve yet to find a system which provides all of the following:</p>
36+
<ul>
37+
<li>User-defined partitioning.</li>
38+
<li>Within a partition, in-order delivery.</li>
39+
<li>Throughput up to millions of requests per second.</li>
40+
<li><strong>Correctness-preserving horizontal scaling</strong>.</li>
41+
</ul>
42+
<p>Scaling a topic by adding partitions is commonly supported; however,
43+
applications which rely on user-defined partitioning and are sensitive
44+
to event order may struggle to gracefully handle events moving between
45+
partitions.</p>
46+
<p>Teams have a few options to handle this, none of which seem
47+
ideal:</p>
48+
<ul>
49+
<li>Over provision partitions to avoid needing to perform scaling.
50+
Underutilize these resources indefinitely.
51+
<ul>
52+
<li>If you’re trying to empty out an uncomfortably full bank account,
53+
this is a great option for you.</li>
54+
</ul></li>
55+
<li>Remove user-defined partitioning, rely on external systems to manage
56+
operations.
57+
<ul>
58+
<li>In-memory caching, batching and idempotency become difficult or
59+
impossible. Concurrent consumers must coordinate transactions, and this
60+
contention is expensive to scale.</li>
61+
</ul></li>
62+
<li>Build a stream management layer to handle upgrades.
63+
<ul>
64+
<li>I’ve not seen this done in practice, but imagine it to be the most
65+
practical solve this problem. However, it comes at increased development
66+
effort and operational complexity.</li>
67+
</ul></li>
68+
</ul>
69+
<p>My “Dream Stream” is a system which manages event partitioning so
70+
that scaling is completely opaque to consumers. And I think I have some
71+
good ideas for how that might work. So for the past few months, I’ve
72+
been trying to put my money where my mouth is.</p>
73+
<h2 id="realization">Realization</h2>
74+
<p>Look, I’ve only been a software developer for seven years. I’ve had
75+
the fortunate opportunity to build some pretty neat things, but I have a
76+
<strong>lot</strong> left to learn. My recent venture into building the
77+
<a href="https://github.com/sackosoft/frieze">Dream Stream</a> has made
78+
clear to me an unhealthy relationship with Abstraction.</p>
79+
<p>The first meaningful commit was <a
80+
href="https://github.com/sackosoft/frieze/commit/f0a4ecb394f8438a23533cec06e4637b975d6ec3">add
81+
the initial hosting abstractions</a>! Wow! “It’s poor design for an
82+
application to know about the source of configuration,” I told myself!
83+
“An application shouldn’t have to know that it is using a UDP socket to
84+
talk to another node, it should be abstracted!” After all, you never
85+
know how much time you can save if you decide to switch to something
86+
else later! Wow.</p>
87+
<p>Embarassingly, these hosting abstractions consumed my focus for the
88+
first week or so of development on the project.</p>
89+
<p>Eventually, I moved on to add implementations of <a
90+
href="https://ferd.ca/interval-tree-clocks.html">Interval Tree
91+
Clocks</a> and the <a
92+
href="https://en.wikipedia.org/wiki/SWIM_Protocol">SWIM Protocol</a>
93+
with the impression that I would need these components to build the
94+
Dream Stream.</p>
95+
<figure>
96+
<img src="/2025/09/25/swim-demo.gif"
97+
alt="An animated video of the SWIM protocol simulator in the abandoned sackosoft/frieze project. A user interacts with buttons to add nodes to a circle representing a cluster. Each node that joins contacts a seed node to join the cluster. The nodes then gossip information between themselves as they run a failure detection protocol based on pinging each other. The user clicks a button to stop a node, within a few seconds some nodes learn of the node failure due to failed ping requests, they gossip this information to other nodes, and each node eventually drops the information for the stopped node." />
98+
<figcaption aria-hidden="true">An animated video of the SWIM protocol
99+
simulator in the abandoned sackosoft/frieze project. A user interacts
100+
with buttons to add nodes to a circle representing a cluster. Each node
101+
that joins contacts a seed node to join the cluster. The nodes then
102+
gossip information between themselves as they run a failure detection
103+
protocol based on pinging each other. The user clicks a button to stop a
104+
node, within a few seconds some nodes learn of the node failure due to
105+
failed ping requests, they gossip this information to other nodes, and
106+
each node eventually drops the information for the stopped
107+
node.</figcaption>
108+
</figure>
109+
<p>While these were great learning experiences, they make it so
110+
painfully obvious that I only have one ‘tool in the toolbox’, so to say.
111+
Every step along the way I was drawing boxes and arrows; pondering
112+
metaphyical code responsibilities coming up with ways to split things
113+
into neat little bundles. It works. It gets stuff done. I get paid big
114+
bucks to do it. But, man, I can’t help but feel that every box creates
115+
friction. Every every bundle imposes constraints. Every decision made to
116+
offer myself flexibility for a future decision may actually defer the
117+
benefit of making a decision at all.</p>
118+
<p>I can’t help but feel there must be a better way. I have a problem.
119+
I’m an Abstractionholic.</p>
120+
</main>
121+
</body>
122+
</html>

‎out/www/2025/09/25/swim-demo.gif‎

1.41 MB
Loading

‎out/www/css/main.css‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.title {
2+
}
3+
4+
.post-list {
5+
}

‎out/www/favicon.png‎

2.24 KB
Loading

‎out/www/index.html‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
<!DOCTYPE html>
2+
<html lang="en-US"><head>
3+
<meta charset="utf-8">
4+
<meta name="viewport" content="width=device-width, initial-scale=1">
5+
<title>Sackosoft</title>
6+
<meta name="description" content="Sackosoft Blog">
7+
<link rel="icon" href="/favicon.png" type="image/png">
8+
<link rel="canonical" href="https://sackosoft.github.io">
9+
<link rel="stylesheet" href="/css/main.css">
10+
</head>
11+
<body>
12+
<header>
13+
<nav>
14+
<a class="title" href="/">sackosoft</a>
15+
</nav>
16+
</header>
17+
<main>
18+
<ul class="post-list">
19+
<li>
20+
<time class="meta" datetime="2025-09-25">Sep 25, 2025</time><h2>
21+
<a href="/2025/09/25/abstractionholics-anonymous.html">Abstractionhoics Anonymous</a>
22+
</h2>
23+
</li>
24+
</ul>
25+
</main>
26+
</body>
27+
</html>

0 commit comments

Comments
 (0)