Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
dist
node_modules
ctrf
coverage
dist
.plans
CLAUDE.md
*.plan
83 changes: 83 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,10 @@ Explore more <a href="https://www.ctrf.io/integrations">integrations</a> <br/>
- **Send AI Test Summary to Slack**: Automatically send AI test summary to a Slack channel.
- **Send Failed Test Details to Slack**: Automatically send failed test details to a Slack channel.
- **Build your own Slack message**: Create and customize your own Slack test reports with our flexible templating system.
- **Threading Support**: Post messages as threaded replies to an existing thread, or auto-thread multi-failure reports.
- **Tagging**: Tag users, channels and groups in the message.
- **Conditional Notifications**: Use the `--onFailOnly` option to send notifications only if tests fail.
- **Dry Run**: Preview the Slack message payload without sending it.

![Example view](assets/results.png)

Expand Down Expand Up @@ -329,6 +331,68 @@ And finally, you can tag a group by using the `\!subteam^` symbol with the group
npx slack-ctrf results /path/to/ctrf-file.json -s "<\!subteam^0123456789> please review the results"
```

## Threading Support

You can post messages as threaded replies to group related messages together.

### Post to a Thread

Use the `--thread-ts` (or `--tt`) option to reply to an existing thread:

```sh
npx slack-ctrf results /path/to/ctrf-report.json --thread-ts "1234567890.123456"
```

### Get Message Timestamp

Use the `--return-ts` (or `--rt`) flag to output the message timestamp. This is useful for capturing the timestamp of a parent message to thread subsequent commands under it:

```sh
# Send initial summary and capture its timestamp
THREAD_TS=$(npx slack-ctrf results /path/to/ctrf-report.json --return-ts)

# Thread a follow-up report
npx slack-ctrf failed /path/to/ctrf-report.json --thread-ts "$THREAD_TS"
```

**Note:** `--return-ts` only works with OAuth token authentication (not webhooks).

### Reply Broadcast

To send a threaded reply and also broadcast it to the main channel, use the `--reply-broadcast` (or `--rb`) flag:

```sh
npx slack-ctrf results /path/to/ctrf-report.json --thread-ts "1234567890.123456" --reply-broadcast
```

### Auto-Threading

When using the `failed` or `ai` commands with multiple failures, you can enable auto-threading to automatically post a summary to the channel and thread individual failure details under it. Use the `--auto-thread` (or `--at`) flag to opt in:

```sh
npx slack-ctrf failed /path/to/ctrf-report.json --auto-thread
```

## Dry Run

To preview the Slack message payload without sending it, use the `--dry-run` (or `--dr`) flag:

```sh
npx slack-ctrf results /path/to/ctrf-report.json --dry-run
```

The payload will be printed to stdout. No message is sent to Slack.

## Limit Failure Reports

When using the `failed` or `ai` commands, you can cap the number of individual failure messages sent with `--max-reports` (or `--mr`):

```sh
npx slack-ctrf failed /path/to/ctrf-report.json --max-reports 5
```

Defaults to 10. If more tests fail than the limit, a notice is appended listing the overflow count.

## Options

- `--onFailOnly, -f`: Send notification only if there are failed tests.
Expand All @@ -338,6 +402,25 @@ npx slack-ctrf results /path/to/ctrf-file.json -s "<\!subteam^0123456789> please
- `--webhook-url, -w`: Incoming webhook URL
- `--oauth-token, -o`: OAuth token
- `--channel-id, -ch`: Channel ID
- `--thread-ts, --tt`: Thread timestamp to reply to an existing thread
- `--return-ts, --rt`: Output the message timestamp (OAuth only)
- `--reply-broadcast, --rb`: Also send threaded reply to the channel
- `--auto-thread, --at`: Thread multi-message reports under a summary (default: false)
- `--dry-run, --dr`: Print the Slack message payload instead of sending it
- `--max-reports, --mr`: Maximum number of individual failure messages to send (default: 10)
- `--max-retries`: Number of retry attempts on rate limit or timeout errors (default: 3)

## Environment Variables

CLI flags take precedence over environment variables.

- `SLACK_WEBHOOK_URL`: Incoming webhook URL
- `SLACK_OAUTH_TOKEN`: OAuth token
- `SLACK_CHANNEL_ID`: Channel ID
- `SLACK_THREAD_TS`: Thread timestamp to reply to an existing thread
- `SLACK_AUTO_THREAD`: Set to `true` to enable automatic threading
- `SLACK_DRY_RUN`: Set to `true` to enable dry run mode
- `SLACK_MAX_RETRIES`: Number of retry attempts (default: 3)

## Merge reports

Expand Down
62 changes: 18 additions & 44 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 26 additions & 27 deletions src/blocks.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,21 @@ describe('Blocks', () => {
const blocks = createTestResultBlocks(mockSummary, mockBuildInfo)

expect(blocks).toHaveLength(1)
expect(blocks[0].type).toBe(BLOCK_TYPES.SECTION)
expect(blocks[0].text.type).toBe(TEXT_TYPES.MRKDWN)
expect(blocks[0].text.text).toContain(`${EMOJIS.TEST_TUBE} 13`)
expect(blocks[0].text.text).toContain(`${EMOJIS.CHECK_MARK} 10`)
expect(blocks[0].text.text).not.toContain(EMOJIS.FALLEN_LEAF)

expect(blocks[0].accessory).toHaveProperty('alt_text', 'Pie Chart')
expect(blocks[0].accessory).toHaveProperty('type', 'image')
expect(blocks[0].accessory.image_url).toContain(
expect(blocks[0]!.type).toBe(BLOCK_TYPES.SECTION)
expect(blocks[0]!.text!.type).toBe(TEXT_TYPES.MRKDWN)
expect(blocks[0]!.text!.text).toContain(`${EMOJIS.TEST_TUBE} 13`)
expect(blocks[0]!.text!.text).toContain(`${EMOJIS.CHECK_MARK} 10`)
expect(blocks[0]!.text!.text).not.toContain(EMOJIS.FALLEN_LEAF)

expect(blocks[0]!.accessory).toHaveProperty('alt_text', 'Pie Chart')
expect(blocks[0]!.accessory).toHaveProperty('type', 'image')
expect(blocks[0]!.accessory!.image_url).toContain(
'https://quickchart.io/chart?'
)

expect(blocks[0].text.text).toContain(MESSAGES.RESULT_PASSED)
expect(blocks[0].text.text).toContain(mockBuildInfo)
expect(blocks[0].text.text).toContain('00:00:25') // Duration formatted
expect(blocks[0]!.text!.text).toContain(MESSAGES.RESULT_PASSED)
expect(blocks[0]!.text!.text).toContain(mockBuildInfo)
expect(blocks[0]!.text!.text).toContain('00:00:25') // Duration formatted
})

it('should create blocks for failed tests', () => {
Expand All @@ -57,15 +57,14 @@ describe('Blocks', () => {
other: 0,
tests: 11,
start: 1706644023000,
stop: 1706644024000, // 1 second later
stop: 1706644048000,
}

const blocks = createTestResultBlocks(mockSummary, mockBuildInfo)

expect(blocks).toHaveLength(1)
expect(blocks[0].text.text).toContain(`${EMOJIS.X_MARK} 2`)
expect(blocks[0].text.text).toContain('*Result:* 2 failed tests')
expect(blocks[0].text.text).toContain('00:00:01') // Duration formatted
expect(blocks[0]!.text!.text).toContain(`${EMOJIS.X_MARK} 2`)
expect(blocks[0]!.text!.text).toContain('*Result:* 2 failed tests')
})

it('should handle duration less than one second', () => {
Expand All @@ -82,7 +81,7 @@ describe('Blocks', () => {

const blocks = createTestResultBlocks(mockSummary, mockBuildInfo)

expect(blocks[0].text.text).toContain(MESSAGES.DURATION_LESS_THAN_ONE)
expect(blocks[0]!.text!.text).toContain(MESSAGES.DURATION_LESS_THAN_ONE)
})

it('should include flaky tests count when provided', () => {
Expand All @@ -104,7 +103,7 @@ describe('Blocks', () => {
flakyCount
)

expect(blocks[0].text.text).toContain(
expect(blocks[0]!.text!.text).toContain(
`${EMOJIS.FALLEN_LEAF} ${flakyCount}`
)
})
Expand Down Expand Up @@ -140,14 +139,14 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain(mockBuildInfo)

const allBlockText = blocks
.filter(block => block.text?.text)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(allBlockText).toContain('Test 1')
Expand Down Expand Up @@ -188,8 +187,8 @@ describe('Blocks', () => {
const blocks = createMessageBlocks(options)

expect(blocks.length).toBeGreaterThan(0)
expect(blocks[0].type).toBe(BLOCK_TYPES.HEADER)
expect(blocks[0].text.text).toContain('Test Results')
expect(blocks[0]!.type).toBe(BLOCK_TYPES.HEADER)
expect(blocks[0]!.text!.text).toContain('Test Results')
})

it('should include prefix and suffix when provided', () => {
Expand All @@ -205,7 +204,7 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain('Prefix message')
Expand All @@ -223,7 +222,7 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain('Missing environment properties')
Expand Down Expand Up @@ -273,7 +272,7 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain(testName)
Expand All @@ -294,7 +293,7 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain(testName)
Expand Down Expand Up @@ -344,7 +343,7 @@ describe('Blocks', () => {

const blockTexts = blocks
.filter(block => block.type === BLOCK_TYPES.SECTION)
.map(block => block.text.text)
.map(block => block.text!.text)
.join('\n')

expect(blockTexts).toContain(testName)
Expand Down
Loading