Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .csync.env.example
Original file line number Diff line number Diff line change
@@ -1,7 +1,15 @@
# Confluence credentials

# --- Data Center ---
# Personal Access Token (Profile → Personal Access Tokens in Confluence DC)
CONFLUENCE_TOKEN=

# Or basic auth — less preferred, PAT is better
# Or basic auth — less preferred, PAT is better (requires --unsafe-auth)
# CONFLUENCE_USERNAME=user@example.com
# CONFLUENCE_PASSWORD=

# --- Cloud (yoursite.atlassian.net) ---
# API token from https://id.atlassian.com/manage-profile/security/api-tokens
# plus your Atlassian account email. No --unsafe-auth needed.
# CONFLUENCE_TOKEN=
# CONFLUENCE_EMAIL=you@example.com
5 changes: 5 additions & 0 deletions .py-conf-sync.config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@
confluence_url: https://confluence.example.com
jira_url: https://jira.example.com # optional — enables Jira macro links on pull

# For Atlassian Cloud, use your site URL instead (auto-detected from *.atlassian.net):
# confluence_url: https://yoursite.atlassian.net/wiki
# jira_url: https://yoursite.atlassian.net
# instance_type: cloud # optional override — only needed for Cloud behind a custom domain

pages:
- page_id: "123456"
file_path: docs/onboarding.md
Expand Down
55 changes: 47 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ This file provides context for Claude Code when working in this project.

## What this project does

`py_conf_sync.py` is a single-file CLI tool that syncs Confluence Data Center pages
with Markdown files in a git repository. It pulls pages down from Confluence (converting
`py_conf_sync.py` is a single-file CLI tool that syncs Confluence pages (Data Center
or Cloud) with Markdown files in a git repository. It pulls pages down from Confluence (converting
storage format to Markdown), allows local editing, then pushes changes back (converting
Markdown back to Confluence storage format).

Expand Down Expand Up @@ -48,10 +48,20 @@ Dependabot regenerates them automatically on its weekly dep-bump PRs.

Everything lives in `py_conf_sync.py`. The main sections are:

- **`ConfluenceClient`** — thin `requests` wrapper around the Confluence REST API.
Three methods: `get_page()`, `update_page()`, and `upload_attachment()`. Auth is
- **`ConfluenceClient`** — thin `requests` wrapper around the Confluence DC REST API
(v1). Three methods: `get_page()`, `update_page()`, and `upload_attachment()`. Auth is
either a PAT Bearer token or basic auth, loaded from `.env` via `python-dotenv`.

- **`CloudConfluenceClient`** — subclass for Atlassian Cloud. Overrides `get_page()`
and `update_page()` to use the v2 API (`/api/v2/pages/{id}`); inherits
`upload_attachment()` because v2 has no attachment upload endpoint (Cloud still uses
the v1 path). Auth is basic auth with `CONFLUENCE_EMAIL` + `CONFLUENCE_TOKEN`
(Atlassian API token) — no `--unsafe-auth` needed. Selected by `_is_cloud()`:
explicit `instance_type` config field wins, else hostname ends with `.atlassian.net`.
`_get_client()` appends `/wiki` to the base URL if missing and writes it back to
`config["confluence_url"]` (conversion code builds/matches attachment download URLs
from that value).

- **`storage_to_markdown()`** — converts Confluence storage format (XHTML + `ac:*`/`ri:*`
macros) to Markdown. Named macro patterns are applied in order before markdownify runs.
Many macros have full round-trip support; unknown macros are stripped by `_MACRO_RE`.
Expand Down Expand Up @@ -113,6 +123,12 @@ pages:
`page_id` is always treated as a string. `file_path` is relative to wherever the script
is run from (intended to be the repo root).

For Atlassian Cloud, set `confluence_url: https://yoursite.atlassian.net/wiki` (the
`/wiki` suffix is appended automatically if omitted) and `jira_url` to the bare site URL.
Cloud is auto-detected from `*.atlassian.net` hostnames; the optional
`instance_type: cloud | datacenter` field overrides detection (needed for Cloud behind
a custom domain).

**`img_dir`**: On pull, if an attachment image's filename exists as `{img_dir}/{filename}`
locally, the Markdown image reference uses the local relative path instead of the Confluence
download URL. On push, local image references are uploaded as Confluence attachments before
Expand All @@ -121,19 +137,27 @@ converting to storage format.
## Credentials (`.csync.env`)

```
# Data Center
CONFLUENCE_TOKEN=<personal access token>
# or (requires --unsafe-auth flag at runtime)
CONFLUENCE_USERNAME=user@example.com
CONFLUENCE_PASSWORD=<password>

# Cloud — API token from id.atlassian.com + account email; no --unsafe-auth needed
CONFLUENCE_TOKEN=<api token>
CONFLUENCE_EMAIL=user@example.com
```

Credentials are **never created by `init`** — users set them up manually by copying
`.csync.env.example` to `~/.csync.env` and filling in their PAT.
`.csync.env.example` to `~/.csync.env` and filling in their token.

The credential file is located by searching in order: `~/.csync.env` (preferred), then
the current working directory, then the script's own directory.

Basic auth is disabled by default and requires `--unsafe-auth` to activate.
Password basic auth (DC only) is disabled by default and requires `--unsafe-auth` to
activate. Cloud's email + API token basic auth is not gated — it is Atlassian's
recommended method for scripts. Cloud with a missing `CONFLUENCE_EMAIL` or
`CONFLUENCE_TOKEN` is a hard error.

## Known limitations / gotchas

Expand Down Expand Up @@ -180,10 +204,25 @@ Basic auth is disabled by default and requires `--unsafe-auth` to activate.

## Confluence API reference

Data Center (v1, base `{confluence_url}/rest/api`):

- Get page: `GET /rest/api/content/{id}?expand=body.storage,version,title`
- Update page: `PUT /rest/api/content/{id}` with JSON body (see `update_page()`)
- Upload attachment: `POST /rest/api/content/{id}/child/attachment`
- Space content: `GET /rest/api/space/{KEY}/content`

All endpoints are Confluence DC REST API v1 (`/rest/api/`). This tool does not use the
v2 API (`/api/v2/`) introduced in newer DC versions.
Cloud (base `{confluence_url}` = `https://site.atlassian.net/wiki`):

- Get page: `GET /api/v2/pages/{id}?body-format=storage` — response carries `title`,
`version.number`, and `body.storage.value` in the same shapes the v1 consumer code
reads, so `cmd_pull`/`cmd_push` are client-agnostic.
- Update page: `PUT /api/v2/pages/{id}` with `{id, status: "current", title, body, version}`
- Upload attachment: `POST /rest/api/content/{id}/child/attachment` — still v1; the v2
API has no attachment upload endpoint.
- Download attachment content: `GET /rest/api/content/{id}/child/attachment/{attId}/download`
(redirects to a signed media URL). The legacy `/download/attachments/{id}/{file}` path
returns 401 for API-token auth on Cloud — it only accepts browser-session cookies —
which is why `download_attachment_text()` is overridden in the Cloud client.

Atlassian is deprecating Cloud v1 content endpoints on a rolling timeline, which is why
Cloud page reads/writes use v2 while DC stays on v1.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ major:
$(call _release,$(NEW_VERSION))

define _release
sed -i 's/__version__ = "$(CURRENT_VERSION)"/__version__ = "$(1)"/' py_conf_sync.py
sed -i.bak 's/__version__ = "$(CURRENT_VERSION)"/__version__ = "$(1)"/' py_conf_sync.py && rm py_conf_sync.py.bak
git add py_conf_sync.py
git commit -m "chore: bump version to $(1)"
git tag v$(1)
Expand Down
72 changes: 56 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,15 @@ Or rebuild explicitly without running a command:
./csync --rebuild status
```

> **Image build fails downloading Chromium?** If `playwright install chromium` dies with
> `Client network socket disconnected before secure TLS connection was established` on
> every mirror, your network is advertising IPv6 that doesn't actually route (common on
> home networks; frequently reported from Docker Desktop on Apple Silicon Macs, but it is
> a network issue, not an architecture one — playwright's Node-based downloader tries IPv6
> first and never falls back to IPv4). Fix: in Docker Desktop go to
> **Settings → Resources → Network** and set **Default networking mode** to **IPv4 only**,
> then Apply & restart and rebuild.

## Commands

```
Expand Down Expand Up @@ -219,9 +228,9 @@ Confluence will render correctly, and pulling that page back produces the same M
### Mermaid diagrams

Fenced ` ```mermaid ` blocks are automatically rendered to PNG images and uploaded as
Confluence attachments on push. Confluence Data Center does not natively render Mermaid,
so this gives you diagrams in git-tracked Markdown that display correctly on the published
page — without any Confluence app or plugin.
Confluence attachments on push. Confluence (Data Center and Cloud alike) does not natively
render Mermaid, so this gives you diagrams in git-tracked Markdown that display correctly
on the published page — without any Confluence app or plugin.

**On push:** each ` ```mermaid ` block is rendered to a retina-quality PNG by headless
Chromium (bundled in the Docker image via Playwright and a bundled `mermaid.min.js` — no
Expand Down Expand Up @@ -310,17 +319,38 @@ img_dir: assets/images

## Credentials

Credentials are **never created automatically** by `csync init`. Set them up by hand:

```bash
cp .csync.env.example ~/.csync.env
# edit ~/.csync.env and fill in your credentials (see below)
```

### Data Center

Use a Personal Access Token (PAT) — it's scoped, revocable, and doesn't expose your password:

`Confluence → Profile → Personal Access Tokens → Create token`

Credentials are **never created automatically** by `csync init`. Set them up by hand:
```
CONFLUENCE_TOKEN=<your PAT>
```

```bash
cp .csync.env.example ~/.csync.env
# edit ~/.csync.env and fill in CONFLUENCE_TOKEN
### Cloud

Use an Atlassian API token together with your Atlassian account email:

`https://id.atlassian.com/manage-profile/security/api-tokens → Create API token`

```
CONFLUENCE_TOKEN=<your API token>
CONFLUENCE_EMAIL=you@example.com
```

Cloud authenticates with HTTP basic auth (email + API token) — this is Atlassian's
recommended method for scripts and does **not** require `--unsafe-auth`, which only
gates password-based basic auth on Data Center.

### Credential file location

The tool searches for `.csync.env` in this order and uses the first file found:
Expand All @@ -344,13 +374,23 @@ without this flag and the command will exit with an error. PAT is strongly prefe

## Version support

This tool targets **Confluence Data Center** and uses the v1 REST API
(`/rest/api/content/`). It has been tested against **DC 9.2.x**.
The tool supports both **Confluence Data Center** and **Confluence Cloud**.
The instance type is auto-detected from the `confluence_url` hostname
(`*.atlassian.net` → Cloud); set `instance_type: cloud` or
`instance_type: datacenter` in the config to override (e.g. for a Cloud site
behind a custom domain).

The v1 API has been stable since Confluence 6.x and remains fully supported
in the 9.x line, so any reasonably modern DC instance should work without
changes.

**Confluence Cloud is not currently supported.** Cloud uses a different base
URL structure and OAuth-based authentication model. Cloud support is planned
once the relevant infrastructure migration is complete.
| | Data Center | Cloud |
|---|---|---|
| `confluence_url` | `https://confluence.example.com` | `https://yoursite.atlassian.net/wiki` (`/wiki` is appended automatically if omitted) |
| Auth | Bearer PAT (`CONFLUENCE_TOKEN`) | Basic auth: `CONFLUENCE_EMAIL` + `CONFLUENCE_TOKEN` (API token) |
| Page read/write | v1 REST API (`/rest/api/content/`) | v2 REST API (`/api/v2/pages/`) |
| Attachment upload | v1 REST API | v1 REST API (v2 has no upload endpoint) |

Data Center has been tested against **DC 9.2.x**. The v1 API has been stable
since Confluence 6.x and remains fully supported in the 9.x line, so any
reasonably modern DC instance should work without changes.

On Cloud, page reads and writes use the v2 API (the v1 content endpoints are
being deprecated by Atlassian); attachment upload still uses the v1 endpoint
because v2 does not provide one.
14 changes: 12 additions & 2 deletions csync
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,17 @@ else
IMAGE="$LOCAL_IMAGE"
fi

# Mount ~/.csync.env into the container at the same path so _find_env_file() can locate it
# Mount ~/.csync.env into the container at the same path so _find_env_file() can locate it.
# If it's a FIFO (e.g. a 1Password-served environment), Docker's VM file sharing cannot
# pass it through — read it once on the host and inject the values as environment
# variables instead (py_conf_sync accepts credentials from the environment). The content
# is passed via process substitution on the exec line below: the fd only stays open
# through exec if the substitution is part of that command, not a prior array assignment.
home_env_args=()
if [[ -f "$HOME/.csync.env" ]]; then
env_file_content=""
if [[ -p "$HOME/.csync.env" ]]; then
env_file_content="$(cat "$HOME/.csync.env")"
elif [[ -f "$HOME/.csync.env" ]]; then
home_env_args+=(-v "$HOME/.csync.env:$HOME/.csync.env" -e "HOME=$HOME")
fi

Expand All @@ -36,11 +44,13 @@ for arg in "$@"; do
fi
done

# An empty env-file is a harmless no-op, so the substitution is unconditional.
exec docker run --rm \
--network=host \
--user "$(id -u):$(id -g)" \
-v "$(pwd):/workspace" \
-w /workspace \
--env-file <(printf '%s\n' "$env_file_content") \
"${home_env_args[@]+"${home_env_args[@]}"}" \
"${extra_mounts[@]+"${extra_mounts[@]}"}" \
"$IMAGE" \
Expand Down
Loading