Skip to content

Commit f4f7956

Browse files
committed
feat: Add Markdown Import feature (closes #484)
1 parent d0b29bd commit f4f7956

25 files changed

Lines changed: 1183 additions & 9 deletions

File tree

Readme.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ This also includes source code snippets. Highlighting is done via [highlight.js]
1414
## Features
1515

1616
- **Modern Markdown Editor** - Write blog posts with a feature-rich markdown editor
17+
- **Markdown Import** - Automatically import blog posts from external repositories (e.g., GitHub)
1718
- **Bookmarks** - Allow readers to save their favorite articles
1819
- **Drafts** - Save work in progress and continue later
1920
- **Scheduled Publishing** - Plan ahead and publish automatically
@@ -41,6 +42,7 @@ This also includes source code snippets. Highlighting is done via [highlight.js]
4142
- [Storage Provider](./docs/Storage/Readme.md)
4243
- [Media Upload](./docs/Media/Readme.md)
4344
- [Search Engine Optimization (SEO)](./docs/SEO/Readme.md)
45+
- [Markdown Import](./docs/Features/MarkdownImport/Readme.md)
4446
- [Advanced Features](./docs/Features/AdvancedFeatures.md)
4547

4648
## Installation
Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
# Markdown Import
2+
3+
The Markdown Import feature allows you to automatically import blog posts from external sources (such as GitHub repositories) by periodically scanning for markdown files. This enables you to author and version control your blog posts externally while having them automatically synchronized to your blog.
4+
5+
## Overview
6+
7+
The Markdown Import job runs every 15 minutes (when enabled) and:
8+
1. Fetches markdown files from the configured source URL
9+
2. Parses metadata from each file's header section
10+
3. Creates new blog posts or updates existing ones based on the `ExternalId`
11+
4. Clears the cache to reflect changes
12+
13+
## Configuration
14+
15+
Add the following section to your `appsettings.json` file:
16+
17+
```json
18+
{
19+
"MarkdownImport": {
20+
"Enabled": true,
21+
"SourceType": "FlatDirectory",
22+
"Url": "https://raw.githubusercontent.com/yourusername/blog-posts/main/posts/"
23+
}
24+
}
25+
```
26+
27+
### Configuration Properties
28+
29+
| Property | Type | Description | Default |
30+
|----------|------|-------------|---------|
31+
| `Enabled` | boolean | Enable or disable the markdown import feature | `false` |
32+
| `SourceType` | string | Type of source provider (currently only `FlatDirectory` is supported) | `"FlatDirectory"` |
33+
| `Url` | string | Base URL where markdown files are located | `""` |
34+
35+
## Markdown File Format
36+
37+
Each markdown file must follow this three-section format, with sections separated by `----------`:
38+
39+
```markdown
40+
----------
41+
id: unique-blog-post-id
42+
title: Your Blog Post Title
43+
tags: tag1, tag2, tag3
44+
image: https://example.com/preview-image.webp
45+
fallbackimage: https://example.com/fallback-image.jpg
46+
published: true
47+
updatedDate: 2026-01-25T20:30:00Z
48+
authorName: John Doe
49+
----------
50+
This is the **short description** of your blog post.
51+
It can contain *markdown* formatting and will be displayed in blog post previews.
52+
----------
53+
This is the main content of your blog post.
54+
55+
## You can use headings
56+
57+
- Bullet points
58+
- Code blocks
59+
- Images
60+
- All markdown features supported by the blog
61+
```
62+
63+
### Metadata Fields
64+
65+
#### Required Fields
66+
67+
- **id**: Unique identifier for the blog post (used to track and update posts). Must be unique across all markdown files. Example: `my-first-post`
68+
- **title**: The title of the blog post
69+
- **image**: URL to the preview image (used in blog post cards and social media)
70+
- **published**: Boolean value (`true` or `false`) indicating whether the post should be published
71+
72+
#### Optional Fields
73+
74+
- **tags**: Comma-separated list of tags
75+
- **fallbackimage**: URL to a fallback image (used if the primary image fails to load)
76+
- **updatedDate**: ISO 8601 formatted date (e.g., `2026-01-25T20:30:00Z`). If not provided, current time is used
77+
- **authorName**: Name of the author. Useful when `UseMultiAuthorMode` is enabled
78+
79+
### Content Sections
80+
81+
After the metadata header, the file must contain two content sections:
82+
83+
1. **Short Description** (first section after header): A brief summary shown in blog post listings
84+
2. **Main Content** (second section after header): The full blog post content
85+
86+
Both sections support full markdown syntax.
87+
88+
## How It Works
89+
90+
### Import Process
91+
92+
1. The job fetches all `.md` files from the configured URL
93+
2. Files are processed in alphabetical order
94+
3. For each file:
95+
- The markdown is parsed into metadata, short description, and content
96+
- The system checks if a blog post with the same `ExternalId` exists
97+
- If it exists, the post is updated with new content
98+
- If it doesn't exist, a new blog post is created
99+
4. After successful imports, the cache is cleared
100+
101+
### Manual Import Trigger
102+
103+
In addition to the automatic 15-minute schedule, you can manually trigger an import from the **Settings** page in the admin area:
104+
105+
1. Log in to your blog
106+
2. Navigate to **Settings** (when logged in)
107+
3. Click the **"Run Import"** button in the Markdown Import row
108+
4. The import job will start immediately
109+
110+
This is useful when:
111+
- You've just pushed new markdown files and want them imported right away
112+
- You're testing the import configuration
113+
- You need to re-import files after making corrections
114+
115+
### Update Behavior
116+
117+
When a markdown file is re-imported (same `id` as an existing post):
118+
- All content is updated from the markdown file
119+
- The `ExternalId` remains unchanged
120+
- **⚠️ Manual edits made through the blog UI will be overwritten**
121+
122+
**Critical Warning**: If you edit an imported blog post through the blog's UI (using the built-in editor), those changes will be **permanently lost** the next time the import job runs (either automatically every 15 minutes or when manually triggered).
123+
124+
**Best Practice**: Always treat your external markdown repository as the **single source of truth** for imported posts. Make all edits to imported posts in your external repository, not in the blog UI.
125+
126+
If you need to stop auto-importing a specific post while retaining your manual edits:
127+
1. Remove the markdown file from the external source, OR
128+
2. Change the `id` field in the markdown file (this will create a new post on next import)
129+
3. The original imported post (with your manual edits) will remain unchanged in the blog
130+
131+
### Error Handling
132+
133+
The import job is designed to be resilient:
134+
- If a file fails to parse, an error is logged and the job continues with other files
135+
- If the source URL is unreachable, the error is logged and the job completes without changes
136+
- Invalid field values are logged as warnings but won't crash the job
137+
138+
## Example Workflows
139+
140+
### GitHub Repository Setup
141+
142+
1. Create a repository for your blog posts (e.g., `blog-posts`)
143+
2. Create a `posts/` directory
144+
3. Add markdown files following the format above
145+
4. Configure your blog's `appsettings.json` to point to the raw GitHub URL:
146+
147+
```json
148+
{
149+
"MarkdownImport": {
150+
"Enabled": true,
151+
"SourceType": "FlatDirectory",
152+
"Url": "https://raw.githubusercontent.com/yourusername/blog-posts/main/posts/"
153+
}
154+
}
155+
```
156+
157+
### Example Markdown File
158+
159+
File: `2026-01-my-first-imported-post.md`
160+
161+
```markdown
162+
----------
163+
id: 2026-01-my-first-imported-post
164+
title: Getting Started with Markdown Import
165+
tags: tutorial, markdown, automation
166+
image: https://images.unsplash.com/photo-1499750310107-5fef28a66643
167+
fallbackimage: https://via.placeholder.com/800x400
168+
published: true
169+
updatedDate: 2026-01-25T10:00:00Z
170+
authorName: Jane Developer
171+
----------
172+
Learn how to use the markdown import feature to manage your blog posts in a Git repository.
173+
This short description appears in blog listings.
174+
----------
175+
# Introduction
176+
177+
This is the full blog post content. You can use any markdown syntax here.
178+
179+
## Why Use Markdown Import?
180+
181+
- Version control your blog posts with Git
182+
- Write in your favorite editor
183+
- Collaborate with others using pull requests
184+
- Automate your blogging workflow
185+
186+
## Code Example
187+
188+
```csharp
189+
public class BlogPost
190+
{
191+
public string Title { get; set; }
192+
public string Content { get; set; }
193+
}
194+
```
195+
196+
That's all there is to it!
197+
198+
## Troubleshooting
199+
200+
### Posts Not Importing
201+
202+
1. Check that `Enabled` is set to `true` in configuration
203+
2. Verify the `Url` is accessible and returns a directory listing with `.md` files
204+
3. Check application logs for error messages
205+
4. Ensure markdown files follow the correct format
206+
207+
### Parsing Errors
208+
209+
Common issues:
210+
- Missing required fields (`id`, `title`, `image`, `published`)
211+
- Malformed header section (missing `----------` delimiters)
212+
- Invalid date format in `updatedDate` field
213+
- Empty content sections
214+
215+
Check the application logs for specific error messages indicating which file and what field caused the issue.
216+
217+
### Updates Not Reflecting
218+
219+
- The job runs every 15 minutes, so changes may take time to appear
220+
- Check that the `id` field in your markdown matches the `ExternalId` of the existing post
221+
- Clear the blog cache manually if needed
222+
223+
## Limitations
224+
225+
- **Flat Directory Only**: Currently only supports flat directory structures (all files in one directory)
226+
- **Public URLs**: The URL must be publicly accessible (no authentication support yet)
227+
- **No Conflict Resolution**: External source is always the source of truth; manual edits are overwritten

docs/Setup/Configuration.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,12 @@ The appsettings.json file has a lot of options to customize the content of the b
6767
"ContainerName": "",
6868
"CdnEndpoint": ""
6969
},
70-
"UseMultiAuthorMode": false
70+
"UseMultiAuthorMode": false,
71+
"MarkdownImport": {
72+
"Enabled": false,
73+
"SourceType": "FlatDirectory",
74+
"Url": ""
75+
}
7176
}
7277
```
7378

@@ -113,3 +118,7 @@ The appsettings.json file has a lot of options to customize the content of the b
113118
| ContainerName | string | The container name for the image storage provider |
114119
| CdnEndpoint | string | Optional CDN endpoint to use for uploaded images. If set, the blog will return this URL instead of the storage account URL for uploaded assets. |
115120
| UseMultiAuthorMode | boolean | The default value is `false`. If set to `true` then author name will be associated with blog posts at the time of creation. This author name will be fetched from the identity provider's `name` or `nickname` or `preferred_username` claim property. |
121+
| [MarkdownImport](./../Features/MarkdownImport/Readme.md) | node | Configuration for the markdown import feature. If left empty or `Enabled` is `false`, the feature is disabled. |
122+
| Enabled | boolean | Enable or disable automatic markdown import from external sources |
123+
| SourceType | string | Type of the markdown source (currently only `FlatDirectory` is supported) |
124+
| Url | string | Base URL where markdown files are located |

src/LinkDotNet.Blog.Domain/BlogPost.cs

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ public sealed partial class BlogPost : Entity
4040

4141
public string? AuthorName { get; private set; }
4242

43+
public string? ExternalId { get; private set; }
44+
4345
private string GenerateSlug()
4446
{
4547
if (string.IsNullOrWhiteSpace(Title))
@@ -95,7 +97,8 @@ public static BlogPost Create(
9597
DateTime? scheduledPublishDate = null,
9698
IEnumerable<string>? tags = null,
9799
string? previewImageUrlFallback = null,
98-
string? authorName = null)
100+
string? authorName = null,
101+
string? externalId = null)
99102
{
100103
if (scheduledPublishDate is not null && isPublished)
101104
{
@@ -116,7 +119,8 @@ public static BlogPost Create(
116119
IsPublished = isPublished,
117120
Tags = tags?.Select(t => t.Trim()).ToImmutableArray() ?? [],
118121
ReadingTimeInMinutes = ReadingTimeCalculator.CalculateReadingTime(content),
119-
AuthorName = authorName
122+
AuthorName = authorName,
123+
ExternalId = externalId
120124
};
121125

122126
return blogPost;
@@ -148,5 +152,6 @@ public void Update(BlogPost from)
148152
Tags = from.Tags;
149153
ReadingTimeInMinutes = from.ReadingTimeInMinutes;
150154
AuthorName = from.AuthorName;
155+
ExternalId = from.ExternalId;
151156
}
152157
}
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
namespace LinkDotNet.Blog.Domain.MarkdownImport;
2+
3+
public sealed record MarkdownContent(
4+
MarkdownMetadata Metadata,
5+
string ShortDescription,
6+
string Content);
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
using System;
2+
using System.Collections.Generic;
3+
4+
namespace LinkDotNet.Blog.Domain.MarkdownImport;
5+
6+
public sealed record MarkdownMetadata(
7+
string Id,
8+
string Title,
9+
string Image,
10+
bool Published,
11+
IReadOnlyCollection<string> Tags,
12+
string? FallbackImage,
13+
DateTime? UpdatedDate,
14+
string? AuthorName);

0 commit comments

Comments
 (0)