Skip to content

Commit 8c0c2a0

Browse files
ascorbicclaude
andauthored
feat: update to @rowanmanning/feed-parser with legacy mode (#85)
* wip * feat: update to @rowanmanning/feed-parser with legacy mode Updates the feed loader to use @rowanmanning/feed-parser for more robust feed parsing. Includes comprehensive backward compatibility support. ## Changes - Updated schemas to match new parser data structure - Categories: {name, domain} → {label, term, url} - Media: {type} → {mimeType} + additional fields - Field names: link/guid → url/id ## Legacy Mode - Added optional `legacy: true` parameter for backward compatibility - Transforms new format to old format automatically - Shows deprecation warning to encourage migration - Complete test coverage for legacy functionality ## Testing - All existing tests updated for new format - Added 7 new tests for legacy mode functionality - 46/46 tests passing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: use exact original schema for legacy mode validation Updates the legacy mode to use the exact original schema structure from the main branch, ensuring perfect backward compatibility. ## Changes - Imported original ItemSchema, EnclosureSchema, MetaSchema from main - Updated transformation to create exact original data structure - Added comprehensive schema validation test - Fixed legacy tests to expect correct original format ## Validation - Legacy mode now validates against exact original schema - All 47 tests passing including new schema validation test - Transformation creates identical structure to original parser 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: resolve TypeScript errors in legacy transformation - Fix nullable property access with optional chaining - Handle null/undefined values in image and author fields - Use any types for feed parser output to avoid strict typing issues - All 47 tests passing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: ensure legacy schemas exactly match original implementation - Added LegacyNSSchema to match original NSSchema exactly - All legacy schemas now perfectly replicate main branch structure - 47/47 tests passing with exact schema validation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: add explicit any types for TypeScript compilation - Fixed implicit any type errors in map functions - Build now compiles successfully - All tests still passing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Update demo and test scripts * Fix claude.md * Wording * Fix link * Update migration info --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 19bc430 commit 8c0c2a0

21 files changed

Lines changed: 1001 additions & 265 deletions

.changeset/feedparse-update.md

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
---
2+
"@ascorbic/feed-loader": major
3+
---
4+
5+
**BREAKING CHANGE**: Updated underlying feed parser library
6+
7+
This release updates the underlying feed parsing library from the previous parser to `@rowanmanning/feed-parser`, which provides more robust and standardized feed parsing. There is a legacy mode for the previous data shape. This change includes several breaking changes to the data structure:
8+
9+
## Schema Changes
10+
11+
### Category Structure
12+
13+
- **BREAKING**: Category objects now use `label`, `term`, and `url` fields instead of `name` and `domain`
14+
- Old: `{ name: string, domain: string | null }`
15+
- New: `{ label: string, term: string, url: string | null }`
16+
17+
### Media/Enclosure Structure
18+
19+
- **BREAKING**: Media objects now include additional fields and renamed properties
20+
- Old: `{ url: string, type: string | null, length: number | null }`
21+
- New: `{ url: string, image: string | null, title: string | null, length: number | null, type: string | null, mimeType: string | null }`
22+
23+
### Field Name Changes
24+
25+
- **BREAKING**: `link` field renamed to `url`
26+
- **BREAKING**: `guid` field renamed to `id`
27+
- **BREAKING**: Atom `summary` field now maps to `description` (consistent with RSS)
28+
- **BREAKING**: RSS/Atom `enclosure`/`link[@rel=enclosure]` elements now map to `media` array
29+
30+
## Error Message Changes
31+
32+
- Updated error messages to match new parser behavior:
33+
- "Item does not have a guid, skipping" → "Item does not have an id or url, skipping"
34+
- "Response body is empty" → "Feed response is empty"
35+
36+
## Benefits
37+
38+
- More robust XML/Atom/RSS parsing
39+
- Better handling of malformed feeds
40+
- Standardized data structure across feed types
41+
- Improved character encoding support
42+
- More comprehensive category and media handling
43+
44+
## Legacy Mode Support
45+
46+
To ease migration, this release includes a **temporary legacy mode** that maintains backward compatibility:
47+
48+
```js
49+
// Enable legacy mode for backward compatibility
50+
const loader = feedLoader({
51+
url: "https://example.com/feed.xml",
52+
legacy: true, // Will show deprecation warning
53+
});
54+
```
55+
56+
⚠️ **Legacy mode is deprecated** and will be removed in a future major version. Use it only as a temporary migration aid.
57+
58+
## Migration Guide
59+
60+
### Option 1: Use Legacy Mode (Temporary)
61+
62+
Enable legacy mode to maintain the old data structure while you plan your migration:
63+
64+
```js
65+
const loader = feedLoader({
66+
url: "https://example.com/feed.xml",
67+
legacy: true,
68+
});
69+
// Data will be in the old format with categories[].name, enclosures, link, guid
70+
```
71+
72+
### Option 2: Update to New Format (Recommended)
73+
74+
Update your code to handle the new structured data format:
75+
76+
#### Field Name Changes
77+
78+
```js
79+
// Item fields
80+
item.linkitem.url
81+
item.guiditem.id
82+
item.pubdate/item.dateitem.published
83+
item.summaryitem.description (Atom feeds)
84+
item.enclosuresitem.media
85+
```
86+
87+
#### Author Structure Change
88+
89+
```js
90+
// Old: Single string format
91+
item.author = "email (name)";
92+
93+
// New: Array of objects
94+
item.authors = [{ email: "email", name: "name" }];
95+
// Access: item.authors[0]?.name, item.authors[0]?.email
96+
```
97+
98+
#### Category Structure Change
99+
100+
```js
101+
// Old: Array of strings
102+
item.categories = ["category1", "category2"];
103+
104+
// New: Array of objects
105+
item.categories = [{ label: "category1", term: "category1", url: null }];
106+
// Access: item.categories[0].label
107+
```
108+
109+
#### Media/Enclosure Structure Change
110+
111+
```js
112+
// Old: Basic enclosure format
113+
item.enclosures = [
114+
{
115+
url: "http://example.com/file.mp3",
116+
type: "audio/mpeg",
117+
length: "1234",
118+
},
119+
];
120+
121+
// New: Enhanced media format
122+
item.media = [
123+
{
124+
url: "http://example.com/file.mp3",
125+
mimeType: "audio/mpeg",
126+
length: 1234,
127+
image: null,
128+
title: null,
129+
},
130+
];
131+
```
132+
133+
#### Image Structure Change
134+
135+
```js
136+
// Old: Simple object with undefined for missing values
137+
item.image = { url: "http://example.com/image.jpg", title: undefined };
138+
139+
// New: Full object structure
140+
item.image = {
141+
url: "http://example.com/image.jpg",
142+
title: "Image Title",
143+
description: "Image description",
144+
};
145+
```
146+
147+
#### Meta Structure Changes
148+
149+
```js
150+
// Feed generator changed from string to object
151+
meta.generator = "WordPress"feed.generator = { name: "WordPress" }
152+
153+
// Authors follow same pattern as items
154+
meta.author = "email (name)"feed.authors = [{ email: "email", name: "name" }]
155+
```
156+
157+
Most users who only access `title`, `description`, `url`, and basic fields will not need changes.

CLAUDE.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Structure
6+
7+
This is a monorepo for Astro loaders using pnpm workspaces. The main packages are:
8+
9+
- `packages/` - Individual loader packages (`@ascorbic/*`)
10+
- `packages/utils/` - Shared utilities (`@ascorbic/loader-utils`)
11+
- `demos/` - Demo applications using the loaders
12+
13+
Each loader package follows the same structure:
14+
15+
- `src/` - TypeScript source code with main loader implementation
16+
- `dist/` - Built output (generated)
17+
- `test/` - Vitest test files
18+
19+
## Common Commands
20+
21+
All commands should be run from the repository root.
22+
23+
### Development
24+
25+
- `pnpm build` - Build all packages
26+
- `pnpm test` - Run tests for all packages
27+
- `pnpm check` - Run type checking and linting for all packages
28+
29+
### Package-specific commands
30+
31+
- `pnpm run --filter @ascorbic/csv-loader build` - Build specific package
32+
- `pnpm run --filter @ascorbic/csv-loader test` - Test specific package
33+
- `pnpm run --filter @ascorbic/csv-loader check` - Check specific package
34+
35+
### Demo development (from demos/loaders/)
36+
37+
- `pnpm run dev` - Start Astro dev server
38+
- `pnpm run build` - Build demo site
39+
40+
## Architecture
41+
42+
### Loader Pattern
43+
44+
All loaders implement the Astro `Loader` interface:
45+
46+
- `name` - Identifier for the loader
47+
- `load(options: LoaderContext)` - Main loading function that syncs data to Astro's store
48+
49+
### Shared Utilities
50+
51+
`@ascorbic/loader-utils` provides common functionality:
52+
53+
- `getConditionalHeaders()` - HTTP conditional request headers
54+
- `storeConditionalHeaders()` - Store HTTP cache headers in meta
55+
56+
### Build System
57+
58+
- Uses `tsup` for TypeScript compilation to ESM
59+
- `publint` and `@arethetypeswrong/cli` for package validation
60+
- Workspace dependencies use `workspace:^` protocol
61+
62+
### Testing
63+
64+
- Vitest for unit testing
65+
- Tests in `test/` directories within each package
66+
67+
### SCM
68+
69+
- Uses conventional commits
70+
- Changes tracked using `changesets`

demos/loaders/src/pages/episodes/[id].astro

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ export const getStaticPaths: GetStaticPaths = async () => {
1515
const episodes = await getCollection("podcasts");
1616
return episodes.map((episode) => ({
1717
params: {
18-
id: formatter.format(episode.data.date!),
18+
id: formatter.format(episode.data.published!),
1919
},
2020
props: { episode },
2121
}));
@@ -28,8 +28,8 @@ const { data } = episode;
2828
2929
const { Content } = await render(episode);
3030
31-
const img = data.image.url ?? data.meta.image?.url;
32-
const alt = data.image.title ?? data.meta.image?.title ?? data.title;
31+
const img = data.image?.url;
32+
const alt = data.image?.title ?? data.title;
3333
---
3434

3535
<Layout title={data.title ?? "Episode"}>
@@ -41,9 +41,9 @@ const alt = data.image.title ?? data.meta.image?.title ?? data.title;
4141
}
4242
<p>
4343
{
44-
data.enclosures.map((enclosure) => (
44+
data.media.map((media) => (
4545
<audio controls>
46-
<source src={enclosure.url} type={enclosure.type} />
46+
<source src={media.url} type={media.mimeType} />
4747
</audio>
4848
))
4949
}

demos/loaders/src/pages/index.astro

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ const formatter = new Intl.DateTimeFormat("en-CA", {
4646
{
4747
episodes.map((episode) => (
4848
<li>
49-
<a href={`/episodes/${formatter.format(episode.data.date!)}`}>
49+
<a href={`/episodes/${formatter.format(episode.data.published!)}`}>
5050
{episode.data.title}
5151
</a>
5252
</li>

packages/airtable/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
"dev": "tsup src/index.ts --format esm --dts --watch",
1616
"prepublishOnly": "node --run build",
1717
"check": "publint && attw $(pnpm pack) --ignore-rules=cjs-resolves-to-esm",
18-
"test": "vitest"
18+
"test": "vitest run",
19+
"test:watch": "vitest"
1920
},
2021
"devDependencies": {
2122
"@arethetypeswrong/cli": "^0.17.3",

packages/csv/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
"dev": "tsup src/index.ts --format esm --dts --watch",
1616
"prepublishOnly": "node --run build",
1717
"check": "publint && attw $(pnpm pack) --ignore-rules=cjs-resolves-to-esm",
18-
"test": "vitest"
18+
"test": "vitest run",
19+
"test:watch": "vitest"
1920
},
2021
"devDependencies": {
2122
"@arethetypeswrong/cli": "^0.17.3",

packages/feed/package.json

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,11 @@
1515
"dev": "tsup src/index.ts --format esm --dts --watch",
1616
"prepublishOnly": "node --run build",
1717
"check": "publint && attw $(pnpm pack) --ignore-rules=cjs-resolves-to-esm",
18-
"test": "vitest"
18+
"test": "vitest run",
19+
"test:watch": "vitest"
1920
},
2021
"devDependencies": {
2122
"@arethetypeswrong/cli": "^0.17.3",
22-
"@types/feedparser": "^2.2.8",
2323
"astro": "5.2.1",
2424
"msw": "^2.10.2",
2525
"publint": "^0.3.2",
@@ -42,7 +42,7 @@
4242
},
4343
"homepage": "https://github.com/ascorbic/astro-loaders",
4444
"dependencies": {
45-
"@ascorbic/loader-utils": "workspace:^",
46-
"feedparser": "^2.2.10"
45+
"@rowanmanning/feed-parser": "^2.0.0",
46+
"@ascorbic/loader-utils": "workspace:^"
4747
}
4848
}

0 commit comments

Comments
 (0)