Skip to content

Commit 6a3fb67

Browse files
Shivampal157xmnlab
andauthored
feat: migrate blog posts from ipynb/md to qmd format (#263)
Co-authored-by: Ivan Ogasawara <ivan.ogasawara@gmail.com>
1 parent 4bc3cd1 commit 6a3fb67

82 files changed

Lines changed: 9645 additions & 7591 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/main.yaml

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,10 @@ jobs:
3333
cancel-in-progress: true
3434
defaults:
3535
run:
36-
shell: bash -l {0}
36+
# bash -el required so conda activation persists (README: IMPORTANT)
37+
shell: bash -el {0}
3738
steps:
38-
- uses: actions/checkout@v3
39+
- uses: actions/checkout@v4
3940

4041
- uses: conda-incubator/setup-miniconda@v3
4142
with:
@@ -45,18 +46,26 @@ jobs:
4546
activate-environment: osl-web
4647
auto-update-conda: true
4748
conda-solver: libmamba
49+
conda-remove-defaults: true
4850

4951
- name: Install dependencies
5052
run: |
5153
poetry check
5254
poetry install
5355
python -m nltk.downloader punkt
5456
57+
# Render blog .qmd → .md so Build uses correct index.md (with YAML)
58+
- name: Pre-build blog (quarto + inject)
59+
run: makim pages.pre-build
60+
61+
# Skip mkdocs-build in pre-commit so we don't fail when repo's md != pre-build output
5562
- name: Linter
5663
if: ${{ github.event_name == 'pull_request' }}
64+
env:
65+
PRE_COMMIT_SKIP: mkdocs-build
5766
run: |
5867
pre-commit install
59-
pre-commit run --all-file --verbose
68+
pre-commit run --all-files --verbose
6069
6170
- name: Build the book
6271
run: |

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,3 +128,9 @@ dmypy.json
128128
# Pyre type checker
129129
.pyre/
130130
.vscode
131+
132+
/.quarto/
133+
**/*.quarto_ipynb
134+
135+
# llm
136+
.codex

.makim.yaml

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,39 @@ groups:
55
help: pre-build step
66
backend: bash
77
run: |
8+
set -e
89
mkdir -p build
9-
# Directory to search for .ipynb files
10+
# Directory to search for blog sources
1011
export SEARCH_DIR="pages/blog"
1112
12-
# Find all .ipynb files, excluding .ipynb_checkpoints,
13-
# and convert them to Markdown named 'index.md'
13+
# 1) Convert legacy .ipynb posts to Markdown (for now)
14+
# This keeps existing notebook-based posts working while we
15+
# migrate everything to Quarto.
1416
find "$SEARCH_DIR" -path "*/.ipynb_checkpoints/*" -prune -o -name \
1517
"*.ipynb" -exec sh -c \
1618
'jupyter nbconvert --to markdown --template=theme/custom-markdown.tpl --output-dir "$(dirname "$0")" --output "index" "$0"' {} \;
1719
18-
# remove console colors from md files
19-
find "$SEARCH_DIR" -name \
20-
"index.md" -exec sh -c \
20+
# 2) Convert .qmd blog posts to Markdown using Quarto
21+
# Run quarto render from inside each blog folder (Quarto rejects --output path).
22+
if ! command -v quarto >/dev/null 2>&1; then
23+
echo "[EE] Quarto CLI is required but not found on PATH. Install it (e.g. conda install quarto) and retry."
24+
exit 1
25+
fi
26+
mapfile -d "" qmd_files < <(find "$SEARCH_DIR" -name "*.qmd" -print0)
27+
for qmd_path in "${qmd_files[@]}"; do
28+
dir=$(dirname "$qmd_path")
29+
base=$(basename "$qmd_path" .qmd)
30+
(
31+
cd "$dir"
32+
quarto render "$base.qmd" --to gfm -M template=default
33+
)
34+
done
35+
36+
# 3) Ensure generated index.md have YAML from .qmd (Quarto may drop it)
37+
python scripts/inject-qmd-yaml-into-md.py
38+
39+
# 4) Remove console colors from generated md files
40+
find "$SEARCH_DIR" -name "index.md" -exec sh -c \
2141
'cat "$(dirname "$0")/index.md" | python scripts/clean-output.py > "$(dirname "$0")/temp_index.md" && mv "$(dirname "$0")/temp_index.md" "$(dirname "$0")/index.md"' {} \;
2242
2343
build:

CONTRIBUTING.md

Lines changed: 45 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -42,15 +42,13 @@ dependencies and activate it.
4242

4343
### Creating a New Blog Post
4444

45-
1. **Prepare the Blog Post:**
45+
1. **Prepare the Blog Post (Quarto-first workflow):**
4646

4747
- Navigate to `pages/blog` and create a new folder with a slugified version
4848
of your blog post's title. Use
4949
[https://slugify.online/](https://slugify.online/) to generate a slug.
50-
- Inside this folder, create your blog post file:
51-
- For Markdown: `index.md`
52-
- For Jupyter Notebooks: `index.ipynb` (use Jupyter Lab to create this
53-
directly)
50+
- Inside this folder, create your blog post as a **Quarto document**:
51+
- `index.qmd`
5452

5553
2. **Include a Header Image:**
5654
- Place a header image (either `header.png` or `header.jpg`) in your blog
@@ -70,8 +68,8 @@ dependencies and activate it.
7068

7169
### Metadata and Formatting
7270

73-
- **Markdown Posts:** Add a metadata block at the beginning of your `index.md`
74-
file:
71+
- **Quarto (`.qmd`) Posts:** Add a metadata block at the beginning of your
72+
`index.qmd` file:
7573

7674
```markdown
7775
---
@@ -87,20 +85,54 @@ dependencies and activate it.
8785
---
8886
```
8987

90-
- **Jupyter Notebook Posts:** The first cell of your `index.ipynb` should be in
91-
`raw` mode containing the same metadata as above.
88+
The body of the file uses standard Markdown plus any Quarto features you need
89+
(code chunks, figures, etc.). During the build, `makim pages.build` will
90+
render `index.qmd` to `index.md` using Quarto so that MkDocs can consume the
91+
generated Markdown.
9292

9393
3. **Building and Viewing:**
94-
- If using a Jupyter Notebook, run `makim pages.build` to convert the
95-
notebook into a Markdown file (`index.md`).
96-
- Add the generated `index.md` to your repository as it will be used to
97-
render the webpage.
94+
- Run `makim pages.build` to render `index.qmd` to `index.md` and build the
95+
site. The generated `index.md` is used to render the webpage.
96+
97+
### Regenerating blog Markdown (for CI)
98+
99+
CI expects the rendered `index.md` files under `pages/blog/*/` to be committed.
100+
If you change `.qmd` content or metadata, regenerate and push the markdown:
101+
102+
```bash
103+
# From repo root with conda env active (e.g. on WSL or Linux)
104+
$ makim pages.pre-build
105+
$ git add pages/blog/*/index.md
106+
$ git commit -m "chore: sync rendered blog index.md from qmd"
107+
$ git push
108+
```
109+
110+
On Windows, use WSL or a Linux environment so `makim pages.pre-build` (Quarto)
111+
runs correctly.
112+
113+
### Commit messages
114+
115+
Keep commit messages professional and descriptive. Do not add tool or editor
116+
tags (e.g. "Made-with: Cursor") to commit messages.
117+
118+
To fix existing commits that contain such a line (e.g. before pushing a PR):
119+
120+
```bash
121+
# Rebase the last N commits (replace 3 with how many need fixing)
122+
$ git rebase -i HEAD~3
123+
# In the editor, change 'pick' to 'reword' for each commit whose message you want to fix. Save and close.
124+
# For each chosen commit, Git will open the message: remove the "Made-with: ..." line, save and close.
125+
# Then force-push your branch (only for your own PR branch):
126+
$ git push --force-with-lease
127+
```
98128

99129
## Final Steps
100130

101131
Before submitting your blog post:
102132

103133
- Ensure all files are added to your repository.
134+
- For new or migrated posts, confirm that `index.qmd` exists and that
135+
`makim pages.build` successfully generates the corresponding `index.md`.
104136
- Submit a pull request to the main `opensciencelabs.github.io` repository for
105137
review.
106138

_quarto.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Project-level Quarto config. Ensures GFM output keeps YAML front matter
2+
# so blog index.md have title, date, authors, tags, etc. for MkDocs.
3+
format:
4+
gfm:
5+
variant: +yaml_metadata_block

bkp/blogs/call-for-interns-2024-01/index.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -209,8 +209,7 @@ requirements.
209209
way to define targets and dependencies. Instead of using the Makefile format,
210210
it uses yaml format.
211211
- **Categories:** DevOps, Automation
212-
- **Organization/Project Webpage URL:**
213-
[https://osl-incubator.github.io/makim/](https://osl-incubator.github.io/makim/)
212+
- **Organization/Project Webpage URL:** [https://makim.org/](https://makim.org/)
214213
- **Contact:** Ivan Ogasawara
215214
[ivan.ogasawara@gmail.com](mailto:ivan.ogasawara@gmail.com)
216215
- **Project Ideas URL:**

bkp/opportunities/internship/cycles/2024-01.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -309,7 +309,7 @@ requirements.
309309
- **Description**: Makim (or makim) is based on make and focus on improve the
310310
way to define targets and dependencies. Instead of using the Makefile format,
311311
it uses yaml format.
312-
- **Organization/Project Webpage URL**: <https://osl-incubator.github.io/makim/>
312+
- **Organization/Project Webpage URL**: <https://makim.org/>
313313
- **Contact**: Ivan Ogasawara (ivan.ogasawara@gmail.com)
314314
- **Project Ideas URL**: <https://github.com/osl-incubator/makim/issues/74>
315315
- **Application Record**:

bkp/opportunities/internship/cycles/2024-02.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ requirements.
315315
- **Description**: Makim (or makim) is based on make and focus on improve the
316316
way to define targets and dependencies. Instead of using the Makefile format,
317317
it uses yaml format.
318-
- **Organization/Project Webpage URL**: <https://osl-incubator.github.io/makim/>
318+
- **Organization/Project Webpage URL**: <https://makim.org/>
319319
- **Contact**: Ivan Ogasawara (ivan.ogasawara@gmail.com)
320320
- **Project Ideas URL**:
321321
<https://github.com/osl-incubator/makim/wiki/OSL-Internship-%E2%80%90-2024-%E2%80%90-2nd-Cycle>

conda/dev.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@ dependencies:
77
- pip
88
- poetry
99
- nodejs
10+
- quarto

0 commit comments

Comments
 (0)