Skip to content

Commit d0ea85d

Browse files
ullaskunder3ullaskunder3
committed
added new blog on fix github action
1 parent 88583e4 commit d0ea85d

4 files changed

Lines changed: 332 additions & 3 deletions

‎data/posts.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -913,5 +913,19 @@
913913
"webhooks",
914914
"architecture"
915915
]
916+
},
917+
{
918+
"id": 63,
919+
"title": "How I Re-Architected My Blog Pipeline for Instant Publishing, Better SEO, and Zero Rate Limits",
920+
"subtitle": "A story about fixing race conditions, avoiding Next.js API rate limits, and building a perfectly synchronized GitHub Actions webhook pipeline.",
921+
"slug": "fixing-next-js-github-api-rate-limits-and-rss-race-conditions",
922+
"date": "2026-07-08",
923+
"tags": [
924+
"nextjs",
925+
"github-actions",
926+
"automation",
927+
"architecture",
928+
"seo"
929+
]
916930
}
917931
]

‎data/posts/building-a-zero-dependency-event-driven-github-profile-updater-with-actions-and-webhooks.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ tag:
1010
- webhooks
1111
- architecture
1212
prev: go-integer-types-explained-from-bits-and-bytes-to-int64
13-
next: graphics-template
13+
next: fixing-next-js-github-api-rate-limits-and-rss-race-conditions
1414
---
1515

1616
I build things. I write about building things. But manually copying links from my new blog posts and pasting them into my GitHub profile README? That is not engineering. That is data entry.
@@ -227,4 +227,4 @@ The final execution time? **Under 15 seconds.** The compute cost? Practically ze
227227

228228
By avoiding black-box templates and writing the glue code myself, I ended up with a system that is perfectly tailored, completely secure, and infinitely easier to debug.
229229

230-
No templates. Just code.
230+
No templates. Just code.

‎data/posts/fixing-next-js-github-api-rate-limits-and-rss-race-conditions.mdx‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
---
2+
title: How I Re-Architected My Blog Pipeline for Instant Publishing, Better SEO, and Zero Rate Limits
3+
subtitle: A story about fixing race conditions, avoiding Next.js API rate limits, and building a perfectly synchronized GitHub Actions webhook pipeline.
4+
id: 63
5+
date: '2026-07-08'
6+
tag:
7+
- nextjs
8+
- github-actions
9+
- automation
10+
- architecture
11+
- seo
12+
prev: building-a-zero-dependency-event-driven-github-profile-updater-with-actions-and-webhooks
13+
next: graphics-template
14+
---
15+
16+
# How I Re-Architected My Blog Pipeline for Instant Publishing, Better SEO, and Zero Rate Limits
17+
18+
A couple of weeks ago, I wrote about [building a zero-dependency, event-driven GitHub profile updater](/blog/building-a-zero-dependency-event-driven-github-profile-updater-with-actions-and-webhooks). The system worked. Push a blog, fire a webhook, update the profile README. Clean.
19+
20+
Then I pushed a new post and watched my profile show stale data.
21+
22+
The system had two hidden flaws that I had completely overlooked. This post is about finding them, understanding why they happened, and the exact code changes I made to fix them.
23+
24+
---
25+
26+
## The Setup (Context)
27+
28+
My blog architecture spans three repositories:
29+
30+
```text
31+
blog-api → Headless CMS. Stores .mdx files and posts.json.
32+
ullaskunder → Next.js portfolio. Fetches content from blog-api via GitHub API.
33+
ullaskunder3 → GitHub Profile README. Auto-updated by a webhook from blog-api.
34+
```
35+
36+
The original data flow looked like this:
37+
38+
```text
39+
[ Push to blog-api ]
40+
|
41+
| 1. GitHub Action fires webhook
42+
v
43+
[ ullaskunder3 Profile Action ]
44+
|
45+
| 2. Fetches rss.xml from ullaskunder.com
46+
v
47+
[ Parses XML, updates README.md ]
48+
```
49+
50+
Two problems hid inside this flow.
51+
52+
---
53+
54+
## Problem 1: The RSS Race Condition
55+
56+
### What went wrong
57+
58+
When I pushed a new blog post to `blog-api`, two things happened simultaneously:
59+
60+
1. The GitHub Action in `blog-api` fired the webhook to `ullaskunder3` *instantly*.
61+
2. Vercel detected the push and started rebuilding my Next.js site.
62+
63+
The webhook completed in under 5 seconds. The Vercel build took 1-2 minutes.
64+
65+
The profile action was fetching `rss.xml` from the live site. But the live site had not finished deploying yet. So the action read the *old* RSS feed, found no new post, and committed nothing.
66+
67+
### My misconception
68+
69+
I assumed that when I push to `blog-api`, the `rss.xml` on my live site updates at the same moment. I was treating two independent systems — GitHub and Vercel — as if they were synchronous.
70+
71+
They are not. GitHub Actions and Vercel deployments run on completely separate infrastructure with no coordination.
72+
73+
### My first hack
74+
75+
I added a `sleep` timer to the `blog-api` trigger workflow:
76+
77+
```yaml
78+
# ❌ The old approach (blog-api/.github/workflows/trigger-profile.yml)
79+
jobs:
80+
notify-profile:
81+
runs-on: ubuntu-latest
82+
steps:
83+
- name: Wait for Next.js Deployment
84+
run: sleep 180 # Sit idle for 3 minutes, hoping Vercel finishes
85+
86+
- name: Ping Profile Repository
87+
run: |
88+
curl -f -i -s -L \
89+
-X POST \
90+
-H "Accept: application/vnd.github+json" \
91+
-H "Authorization: Bearer ${{ secrets.PROFILE_UPDATE_TOKEN }}" \
92+
-H "X-GitHub-Api-Version: 2022-11-28" \
93+
https://api.github.com/repos/ullaskunder3/ullaskunder3/dispatches \
94+
-d '{"event_type": "blog_published"}'
95+
```
96+
97+
This is bad engineering for three reasons:
98+
99+
1. **Wasted compute.** Three minutes of an idle CI runner per push.
100+
2. **Fragile.** If Vercel takes 4 minutes one day, the profile still gets stale data.
101+
3. **Not event-driven.** A `sleep` is a poll. I built this system to avoid polling.
102+
103+
### The actual fix
104+
105+
The problem was not timing. The problem was the *data source*.
106+
107+
My profile action was fetching from `ullaskunder.com/rss.xml` (the frontend). But the data originates from `blog-api/data/posts.json` (the CMS). The CMS updates the moment I push. Vercel does not.
108+
109+
Solution: skip the frontend entirely. Fetch the raw JSON from the CMS.
110+
111+
**Step 1:** Remove the sleep timer from `blog-api`:
112+
113+
```yaml
114+
# ✅ The new approach (blog-api/.github/workflows/trigger-profile.yml)
115+
jobs:
116+
notify-profile:
117+
runs-on: ubuntu-latest
118+
steps:
119+
- name: Ping Profile Repository
120+
run: |
121+
curl -f -i -s -L \
122+
-X POST \
123+
-H "Accept: application/vnd.github+json" \
124+
-H "Authorization: Bearer ${{ secrets.PROFILE_UPDATE_TOKEN }}" \
125+
-H "X-GitHub-Api-Version: 2022-11-28" \
126+
https://api.github.com/repos/ullaskunder3/ullaskunder3/dispatches \
127+
-d '{"event_type": "blog_published"}'
128+
```
129+
130+
No waiting. The webhook fires the instant I push.
131+
132+
**Step 2:** Rewrite the profile action's Python script to parse JSON instead of XML.
133+
134+
Here is the old script that fetched from the live site:
135+
136+
```python
137+
# ❌ Old: Fetching from the Next.js frontend (subject to Vercel build delays)
138+
import urllib.request, xml.etree.ElementTree as ET
139+
from datetime import datetime
140+
141+
url = 'https://www.ullaskunder.com/rss.xml'
142+
req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
143+
xml_data = urllib.request.urlopen(req).read()
144+
root = ET.fromstring(xml_data)
145+
146+
all_posts = []
147+
for item in root.findall('./channel/item'):
148+
title = item.find('title').text
149+
link = item.find('link').text
150+
pubDate = item.find('pubDate').text
151+
dt = datetime.strptime(pubDate, '%a, %d %b %Y %H:%M:%S %Z')
152+
all_posts.append({'title': title, 'link': link, 'dt': dt})
153+
```
154+
155+
And here is the new script that fetches directly from the CMS:
156+
157+
```python
158+
# ✅ New: Fetching raw JSON from the CMS repo (instant, no build required)
159+
import urllib.request, json
160+
from datetime import datetime
161+
162+
url = 'https://raw.githubusercontent.com/ullaskunder3/blog-api/main/data/posts.json'
163+
req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
164+
json_data = urllib.request.urlopen(req).read()
165+
posts_list = json.loads(json_data)
166+
167+
all_posts = []
168+
for post in posts_list:
169+
title = post['title']
170+
slug = post['slug']
171+
link = f'https://www.ullaskunder.com/blog/{slug}'
172+
date_str = post['date']
173+
dt = datetime.strptime(date_str, '%Y-%m-%d')
174+
all_posts.append({'title': title, 'link': link, 'dt': dt})
175+
```
176+
177+
Two key differences:
178+
179+
1. **Data source changed.** `raw.githubusercontent.com` serves the file the moment it is merged to `main`. No build step. No delay.
180+
2. **URL construction changed.** The RSS feed provided the full `<link>` tag. The JSON only provides a `slug`. So the script now constructs the URL dynamically: `f'https://www.ullaskunder.com/blog/{slug}'`.
181+
182+
The updated data flow:
183+
184+
```text
185+
[ Push to blog-api ]
186+
|
187+
| 1. Webhook fires instantly (no sleep)
188+
v
189+
[ ullaskunder3 Profile Action ]
190+
|
191+
| 2. Fetches posts.json from raw.githubusercontent.com
192+
| (Available immediately, no Vercel build needed)
193+
v
194+
[ Parses JSON, constructs URLs from slugs ]
195+
|
196+
v
197+
[ Updates README.md in under 10 seconds ]
198+
```
199+
200+
The race condition is gone. The profile action no longer depends on Vercel at all.
201+
202+
---
203+
204+
## Problem 2: The Next.js Rate Limit Trap
205+
206+
### What went wrong
207+
208+
While auditing the pipeline, I looked at how my Next.js portfolio actually fetches blog content. I found this at the top of `app/blogs/[slug]/page.tsx`:
209+
210+
```typescript
211+
// ❌ Old: Forces server-side rendering on every request
212+
export const dynamic = "force-dynamic";
213+
```
214+
215+
### My misconception
216+
217+
I thought `force-dynamic` was the safest way to guarantee users always see the latest blog post. I did not think about what it actually does under the hood.
218+
219+
Here is what `force-dynamic` tells Next.js:
220+
221+
> Do not cache this page. On every single request, spin up the server, fetch the data from GitHub, parse the Markdown, render the HTML, and send it to the user.
222+
223+
Every page view was making a live API call to GitHub. My portfolio fetches `.mdx` files from the `blog-api` repository using the GitHub Contents API with a Personal Access Token (PAT). That means every visitor was consuming one of my 5,000 requests/hour.
224+
225+
This is the kind of flaw that works perfectly in development and during low traffic, but breaks the moment a post gets shared on social media.
226+
227+
### The actual numbers
228+
229+
| Metric | `force-dynamic` |
230+
| --- | --- |
231+
| API calls per visitor | 1 |
232+
| Rate limit | 5,000/hour |
233+
| Max concurrent readers | ~83/minute before crash |
234+
| TTFB (Time to First Byte) | 300-500ms (server render) |
235+
| SEO impact | Slow TTFB hurts Core Web Vitals |
236+
237+
83 readers per minute sounds comfortable until you realize a single Hacker News or Reddit post can send thousands of visitors in an hour.
238+
239+
### The fix: Incremental Static Regeneration
240+
241+
I replaced `force-dynamic` with a single line:
242+
243+
```typescript
244+
// ✅ New: Cache the page, revalidate every 60 seconds
245+
export const revalidate = 60;
246+
```
247+
248+
What ISR does:
249+
250+
1. **First request:** Next.js renders the page server-side, caches the full HTML at Vercel's Edge CDN.
251+
2. **Subsequent requests (within 60 seconds):** Vercel serves the cached HTML instantly. Zero API calls.
252+
3. **First request after 60 seconds:** Vercel serves the cached version to the user (still fast), but triggers a background revalidation. Next.js silently fetches the latest data from GitHub and updates the cache.
253+
4. **Next request:** Gets the freshly cached version.
254+
255+
The user never waits. GitHub never gets hammered.
256+
257+
### The numbers after ISR
258+
259+
| Metric | `force-dynamic` (Before) | `revalidate = 60` (After) |
260+
| --- | --- | --- |
261+
| API calls per visitor | 1 | 0 (served from cache) |
262+
| API calls per 60 seconds | Unbounded | 1 |
263+
| Max concurrent readers | ~83/min | Unlimited (Edge CDN) |
264+
| TTFB | 300-500ms | <50ms |
265+
| SEO impact | Poor | Excellent |
266+
267+
### When ISR is not enough
268+
269+
ISR has one trade-off: content staleness. If I fix a typo in a blog post, the live site could show the old version for up to 60 seconds. For my use case, this is completely acceptable.
270+
271+
If you need truly instant updates (e.g., a stock ticker or live score), ISR is the wrong tool. Use `force-dynamic` or WebSockets instead. But for blog content that changes a few times a month, a 60-second cache window is the sweet spot between freshness and performance.
272+
273+
---
274+
275+
## The Final Architecture
276+
277+
All three repositories are now perfectly synchronized:
278+
279+
```text
280+
[ Push .mdx + posts.json to blog-api ]
281+
|
282+
|──── Vercel detects push, rebuilds ullaskunder.com (background)
283+
|
284+
|──── GitHub Action fires webhook instantly
285+
| |
286+
| v
287+
| [ ullaskunder3 Profile Action ]
288+
| |
289+
| | Fetches posts.json from raw.githubusercontent.com
290+
| |
291+
| v
292+
| [ README.md updated in <10 seconds ]
293+
|
294+
v
295+
[ Vercel build completes (~1-2 min) ]
296+
|
297+
| ISR cache expires after 60s
298+
| Background revalidation fetches fresh data
299+
v
300+
[ ullaskunder.com/blog shows new post ]
301+
```
302+
303+
### What changed across each repository
304+
305+
**`blog-api`** — Removed the `sleep 180` from the trigger workflow. The webhook now fires immediately.
306+
307+
**`ullaskunder3`** — Rewrote the Python script from XML/RSS parsing to direct JSON parsing. Data source changed from `ullaskunder.com/rss.xml` to `raw.githubusercontent.com/.../posts.json`.
308+
309+
**`ullaskunder`** — Replaced `export const dynamic = "force-dynamic"` with `export const revalidate = 60` in the blog route. Pages are now cached at the Edge.
310+
311+
### The result
312+
313+
Profile updates: instant. Page loads: under 50ms. API rate limit risk: eliminated. Compute waste: zero.
314+
315+
Three small code changes. Two misconceptions corrected.

‎data/posts/graphics-template.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ tag:
99
- react
1010
- hooks
1111
- guide
12-
prev: building-a-zero-dependency-event-driven-github-profile-updater-with-actions-and-webhooks
12+
prev: fixing-next-js-github-api-rate-limits-and-rss-race-conditions
1313
next: how-to-write-makdown
1414
---
1515

0 commit comments

Comments
 (0)