Skip to content

Commit 89f36e8

Browse files
authored
Merge pull request #13 from SpecterOps/go
Replace PowerShell with Go Port
2 parents b1cefe0 + 2f99eb1 commit 89f36e8

76 files changed

Lines changed: 35485 additions & 1237 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# GitHub Actions CI: Integration Tests with Samba AD DC + SQL Server
2+
3+
## Context
4+
5+
MSSQLHound needs CI that validates all expected edges are created against a real AD environment. The integration test framework already handles AD object creation via LDAP, SQL setup/teardown, collector execution, and edge validation — we just need to provide the infrastructure.
6+
7+
**Architecture**: SQL Server installed on the runner host (via apt) + Samba AD DC in a Docker container. SQL Server is configured for AD auth via a keytab from `samba-tool`. The integration test framework (`TestIntegrationAll`) handles everything else.
8+
9+
## Network and domain config
10+
11+
| Setting | Value |
12+
|---------|-------|
13+
| Subnet | `10.2.0.0/20` |
14+
| DC IP | `10.2.10.100` |
15+
| SQL Server | On the runner host (localhost) |
16+
| Domain FQDN | `MAYYHEM.COM` |
17+
| NetBIOS | `MAYYHEM` |
18+
| Admin account | `MAYYHEM\domainadmin` (sysadmin on SQL) |
19+
| Password (all) | `password` |
20+
| SQL auth mode | Mixed mode (Windows + SQL) |
21+
22+
## Files to create
23+
24+
- `.github/workflows/ci.yml` (new)
25+
26+
## Plan
27+
28+
### 1. Create `.github/workflows/ci.yml` with two jobs
29+
30+
**Job 1: `unit-tests`** — Fast validation of all edge creation logic
31+
- `ubuntu-latest`, checkout, setup Go, `go test -v -count=1 ./...`
32+
33+
**Job 2: `integration-tests`** — Full pipeline against real AD + SQL Server on `ubuntu-22.04`
34+
35+
#### Step 1: Checkout + Setup Go
36+
37+
#### Step 2: Start Samba AD DC
38+
```bash
39+
docker network create --subnet=10.2.0.0/20 adnet
40+
41+
docker run -d --privileged \
42+
--name dc --hostname DC \
43+
--network adnet --ip 10.2.10.100 \
44+
-e REALM='MAYYHEM.COM' \
45+
-e DOMAIN='MAYYHEM' \
46+
-e ADMIN_PASS='password' \
47+
-e DNS_FORWARDER='8.8.8.8' \
48+
-p 389:389/tcp -p 389:389/udp \
49+
-p 636:636/tcp \
50+
-p 88:88/tcp -p 88:88/udp \
51+
-p 464:464/tcp -p 464:464/udp \
52+
diegogslomp/samba-ad-dc
53+
```
54+
55+
#### Step 3: Wait for Samba DC readiness
56+
Poll `samba-tool domain info` up to 60 attempts / 2s.
57+
58+
#### Step 4: Create domainadmin user + SQL Server service account + keytab
59+
```bash
60+
# Create domainadmin user
61+
docker exec dc samba-tool user create domainadmin 'password' --use-username-as-cn
62+
docker exec dc samba-tool group addmembers "Domain Admins" domainadmin
63+
64+
# Create SQL Server service account
65+
docker exec dc samba-tool user create sqlsvc 'password' --use-username-as-cn
66+
HOSTNAME=$(hostname)
67+
docker exec dc samba-tool spn add MSSQLSvc/${HOSTNAME}.mayyhem.com sqlsvc
68+
docker exec dc samba-tool spn add MSSQLSvc/${HOSTNAME}.mayyhem.com:1433 sqlsvc
69+
70+
# Export keytab
71+
docker exec dc samba-tool domain exportkeytab /tmp/mssql.keytab --principal=sqlsvc
72+
docker exec dc samba-tool domain exportkeytab /tmp/mssql.keytab \
73+
--principal=MSSQLSvc/${HOSTNAME}.mayyhem.com
74+
docker exec dc samba-tool domain exportkeytab /tmp/mssql.keytab \
75+
--principal=MSSQLSvc/${HOSTNAME}.mayyhem.com:1433
76+
docker cp dc:/tmp/mssql.keytab /tmp/mssql.keytab
77+
```
78+
79+
#### Step 5: Configure host DNS + Kerberos
80+
```bash
81+
# DNS
82+
echo "10.2.10.100 dc.mayyhem.com dc mayyhem.com" | sudo tee -a /etc/hosts
83+
sudo sed -i '1i nameserver 10.2.10.100' /etc/resolv.conf
84+
85+
# Kerberos
86+
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y krb5-user
87+
cat <<'EOF' | sudo tee /etc/krb5.conf
88+
[libdefaults]
89+
default_realm = MAYYHEM.COM
90+
dns_lookup_realm = false
91+
dns_lookup_kdc = false
92+
93+
[realms]
94+
MAYYHEM.COM = {
95+
kdc = 10.2.10.100
96+
admin_server = 10.2.10.100
97+
default_domain = mayyhem.com
98+
}
99+
100+
[domain_realm]
101+
.mayyhem.com = MAYYHEM.COM
102+
mayyhem.com = MAYYHEM.COM
103+
EOF
104+
```
105+
106+
#### Step 6: Install SQL Server 2022
107+
```bash
108+
curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | \
109+
sudo gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg
110+
curl -fsSL https://packages.microsoft.com/config/ubuntu/22.04/mssql-server-2022.list | \
111+
sudo tee /etc/apt/sources.list.d/mssql-server-2022.list
112+
sudo apt-get update
113+
sudo apt-get install -y mssql-server
114+
sudo MSSQL_SA_PASSWORD='password' MSSQL_PID='Developer' \
115+
/opt/mssql/bin/mssql-conf setup accept-eula
116+
```
117+
118+
#### Step 7: Enable mixed mode auth
119+
```bash
120+
sudo /opt/mssql/bin/mssql-conf set sqlagent.enabled true
121+
sudo /opt/mssql/bin/mssql-conf set network.kerberoskeytabfile /var/opt/mssql/secrets/mssql.keytab
122+
sudo /opt/mssql/bin/mssql-conf set network.privilegedadaccount sqlsvc
123+
124+
# Enable mixed mode (SQL + Windows auth)
125+
# SQL Server Linux uses the MSSQL_SA_PASSWORD being set during setup to enable mixed mode.
126+
# To explicitly toggle it post-setup if needed:
127+
sudo /opt/mssql/bin/mssql-conf set sqlagent.enabled true
128+
```
129+
130+
Note: SQL Server on Linux enables mixed mode auth when SA password is set during setup. The `MSSQL_SA_PASSWORD` in step 6 handles this.
131+
132+
#### Step 8: Configure SQL Server keytab + restart
133+
```bash
134+
sudo mkdir -p /var/opt/mssql/secrets
135+
sudo cp /tmp/mssql.keytab /var/opt/mssql/secrets/mssql.keytab
136+
sudo chown mssql:mssql /var/opt/mssql/secrets/mssql.keytab
137+
sudo chmod 400 /var/opt/mssql/secrets/mssql.keytab
138+
sudo systemctl restart mssql-server
139+
sleep 5
140+
```
141+
142+
#### Step 9: Create domainadmin as SQL sysadmin
143+
```bash
144+
# Install sqlcmd
145+
curl -fsSL https://packages.microsoft.com/config/ubuntu/22.04/prod.list | \
146+
sudo tee /etc/apt/sources.list.d/mssql-release.list
147+
sudo apt-get update
148+
sudo ACCEPT_EULA=Y apt-get install -y mssql-tools18
149+
150+
export PATH="$PATH:/opt/mssql-tools18/bin"
151+
sqlcmd -S localhost -U sa -P 'password' -C -Q "
152+
CREATE LOGIN [MAYYHEM\domainadmin] FROM WINDOWS;
153+
ALTER SERVER ROLE [sysadmin] ADD MEMBER [MAYYHEM\domainadmin];
154+
"
155+
```
156+
157+
#### Step 10: Verify AD auth
158+
```bash
159+
echo 'password' | kinit Administrator@MAYYHEM.COM
160+
klist
161+
```
162+
163+
#### Step 11: Run integration tests
164+
```bash
165+
MSSQL_SERVER=localhost \
166+
MSSQL_USER=sa \
167+
MSSQL_PASSWORD='password' \
168+
MSSQL_DOMAIN=mayyhem.com \
169+
MSSQL_DC=10.2.10.100 \
170+
LDAP_USER='Administrator@mayyhem.com' \
171+
LDAP_PASSWORD='password' \
172+
MSSQL_SKIP_DOMAIN=false \
173+
MSSQL_ACTION=all \
174+
MSSQL_SKIP_HTML=true \
175+
go test -v -count=1 -tags integration -timeout 30m \
176+
-run TestIntegrationAll ./internal/collector/...
177+
```
178+
179+
Since `MSSQL_DOMAIN=mayyhem.com``substituteDomain()` extracts `MAYYHEM` as NetBIOS, which matches the hardcoded `MAYYHEM\` references in the SQL scripts. No substitution gap.
180+
181+
## Critical files
182+
183+
| File | Role |
184+
|------|------|
185+
| [integration_setup_test.go](internal/collector/integration_setup_test.go) | Config loading, LDAP object creation, SQL setup orchestration |
186+
| [integration_sql_test.go](internal/collector/integration_sql_test.go) | Embedded SQL scripts with `FROM WINDOWS` + `$Domain` references |
187+
| [edge_integration_test.go](internal/collector/edge_integration_test.go) | `TestIntegrationAll` entry point, edge validation |
188+
| [edge_test_data_test.go](internal/collector/edge_test_data_test.go) | 200+ edge test case definitions |
189+
190+
## Risks and mitigations
191+
192+
| Risk | Mitigation |
193+
|------|------------|
194+
| Port 53 conflict on runner | Don't publish port 53; use `/etc/hosts` + `/etc/resolv.conf` |
195+
| SQL Server 2022 not on Ubuntu 24.04 | Pin `ubuntu-22.04` |
196+
| Samba DC slow to start | 120s timeout with polling |
197+
| `FROM WINDOWS` fails if DNS broken | Verify `kinit` works before running tests |
198+
| `diegogslomp/samba-ad-dc` unavailable | Fall back to `craftdock/samba-ad-dc` |
199+
| Mixed mode not enabled | SA password set during setup enables it automatically |
200+
201+
## Verification
202+
203+
1. Unit tests locally: `go test -v ./...`
204+
2. Push to branch and verify both jobs pass
205+
3. Integration test output includes edge coverage report from `TestIntegrationAll`

.claude/plans/logging.md

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Logging Overhaul Plan
2+
3+
## Context
4+
MSSQLHound currently uses raw `fmt.Printf`/`fmt.Println` for all logging (~149 calls across 10 files). Messages have no timestamps, no log levels, and inconsistent formatting. The goal is to add UTC timestamps and log levels to every message using Go's stdlib `log/slog` (available since Go 1.21, project uses Go 1.24).
5+
6+
## Approach: Use `log/slog` from stdlib
7+
8+
No new packages or dependencies. Create a `*slog.Logger` in `main()`, propagate via struct fields.
9+
10+
### Output format
11+
```
12+
INFO 2026-03-30T14:22:01Z Processing 5 SQL Server(s)...
13+
INFO 2026-03-30T14:22:01Z [corp.local] Enumerating MSSQL SPNs from Active Directory...
14+
VERBOSE 2026-03-30T14:22:01Z [corp.local] Found SPNs count=12 host=sql01.corp.local
15+
WARNING 2026-03-30T14:22:02Z [sql01.corp.local] SPN enumeration failed error="connection refused"
16+
DEBUG 2026-03-30T14:22:02Z [sql01.corp.local] EPA TLS handshake complete cipher=0x1301
17+
```
18+
19+
- Custom `slog.Handler` implementation in a new file `internal/logging/handler.go`
20+
- Format: `LEVEL TIMESTAMP [target] message attrs...`
21+
- Level name is left-aligned, right-padded to 7 chars (longest: `WARNING` and `VERBOSE`)
22+
- Output goes to **stderr** (standard for logs; keeps stdout clean for data output like tables)
23+
- Timestamps in UTC RFC3339 format
24+
- **Target context**: when processing a specific server or domain, a `[target]` tag appears after the timestamp. Implemented via `logger.With("target", serverName)` to create sub-loggers. The custom handler renders the `target` attr specially in brackets, separate from other attrs.
25+
- Messages without a target (startup, global config) omit the bracket section
26+
27+
### Colors (ANSI escape codes)
28+
- Auto-detect TTY on stderr (`os.Stderr.Fd()` + `isatty` check or `golang.org/x/term.IsTerminal`). Disable colors when piped.
29+
- **Level colors**:
30+
- `ERROR` — red (ANSI 31)
31+
- `WARNING` — yellow (ANSI 33)
32+
- `INFO` — white/default (no color)
33+
- `VERBOSE` — dim/gray (ANSI 90)
34+
- `DEBUG` — magenta (ANSI 35)
35+
- **Timestamp** — light blue (ANSI 94)
36+
- **Target** `[brackets]` — deterministic color per unique target string. Hash the target name to pick from a palette of primary/secondary ANSI colors (red 31, green 32, yellow 33, blue 34, magenta 35, cyan 36). Same target always gets the same color. Use a simple hash (e.g., `fnv32(target) % len(palette)`).
37+
- **Message text** — default/no color
38+
- **Attrs** (`key=value`) — dim (ANSI 2) for the key, default for value
39+
40+
### Custom slog levels
41+
| Name | slog.Level value | Meaning |
42+
|---|---|---|
43+
| `ERROR` | `slog.LevelError` (8) | Error conditions |
44+
| `WARNING` | `slog.LevelWarn` (4) | Warnings |
45+
| `INFO` | `slog.LevelInfo` (0) | Normal status/progress |
46+
| `VERBOSE` | `slog.Level(-2)` | Detailed progress (was `logVerbose`) |
47+
| `DEBUG` | `slog.LevelDebug` (-4) | EPA/TLS/NTLM diagnostics (was `logDebug`/`logf`) |
48+
49+
### Level mapping from current code
50+
| Current pattern | New level |
51+
|---|---|
52+
| `fmt.Printf(...)` (status/progress) | `INFO` |
53+
| `fmt.Printf("Warning: ...")` | `WARNING` |
54+
| `fmt.Printf("ERROR: ...")` | `ERROR` |
55+
| `logVerbose(...)` | `VERBOSE` |
56+
| `logDebug(...)` / `logf(...)` (EPA diagnostics) | `DEBUG` |
57+
58+
### Flag behavior
59+
- No flags: minimum level = INFO
60+
- `--verbose`: minimum level = VERBOSE (shows VERBOSE + INFO + WARNING + ERROR)
61+
- `--debug`: minimum level = DEBUG (shows everything)
62+
- `--debug` additionally sets `debug=true` on subsystems (controls EPA test behavior beyond just logging)
63+
64+
## Implementation Phases
65+
66+
### Phase 0: Custom handler (`internal/logging/`)
67+
68+
**New file: [internal/logging/handler.go](go/internal/logging/handler.go)**
69+
- Implement `slog.Handler` that formats: `LEVEL TIMESTAMP [target] message attrs...`
70+
- Define custom level constants: `LevelVerbose = slog.Level(-2)`
71+
- Level name mapping: `-2``VERBOSE`, `slog.LevelWarn``WARNING`
72+
- Left-align level name, right-pad to 7 chars
73+
- Special handling for `target` attr: rendered as `[value]` before the message, not as `key=value`
74+
- Other attrs appended as `key=value` after the message
75+
- Thread-safe writer (mutex around writes)
76+
- `WithAttrs` / `WithGroup` support for creating sub-loggers (e.g., `logger.With("target", server)`)
77+
- ANSI color support: detect TTY via `golang.org/x/term.IsTerminal(int(os.Stderr.Fd()))`
78+
- Color each element per the palette defined above (level, timestamp, target, attrs)
79+
- Target color: `fnv32a(targetString) % 6` maps to one of [blue 34, cyan 36, bright green 92, bright blue 94, bright cyan 96, bright white 97]. These avoid red (ERROR), yellow (WARNING), magenta (DEBUG), and gray (VERBOSE).
80+
- Accept a `NoColor bool` option to force colors off (for tests or `--no-color` flag)
81+
82+
### Phase 1: Logger setup in main (`cmd/mssqlhound/`)
83+
84+
**[main.go](go/cmd/mssqlhound/main.go)**
85+
- Create `slog.LevelVar` and `*slog.Logger` with custom `logging.NewHandler(os.Stderr, ...)` in `main()`
86+
- Add `PersistentPreRunE` to set level from `--verbose`/`--debug` flags
87+
- Pass logger to `run()` and subcommands
88+
- Convert 11 `fmt.Printf`/`fmt.Println` calls to `logger.Info`/`logger.Warn`
89+
- Keep `fmt.Fprintf(os.Stderr, ...)` for cobra error at line 105 (logger may not exist)
90+
91+
**[cmd_test_epa_matrix.go](go/cmd/mssqlhound/cmd_test_epa_matrix.go)**
92+
- Accept logger parameter from main
93+
- Convert 8 `fmt.Printf` calls
94+
95+
### Phase 2: Collector (`internal/collector/`)
96+
97+
**[collector.go](go/internal/collector/collector.go)**
98+
- Add `Logger *slog.Logger` field to `Config` struct
99+
- Convert 75 `fmt.Printf`/`fmt.Println` calls to appropriate slog levels
100+
- Convert 49 `c.logVerbose(...)` calls to `c.config.Logger.Log(ctx, logging.LevelVerbose, ...)`
101+
- Remove `logVerbose` method (line 6128)
102+
- When processing a server, create a sub-logger: `serverLog := c.config.Logger.With("target", server.ConnectionString)` and use it for all per-server messages
103+
- For domain-level operations: `domainLog := c.config.Logger.With("target", domain)`
104+
- Pass logger to `mssql.Client` and `wmi` calls
105+
106+
### Phase 3: MSSQL client (`internal/mssql/`)
107+
108+
**[client.go](go/internal/mssql/client.go)** — 15 fmt calls + 7 logf calls
109+
- Add `logger *slog.Logger` field, `SetLogger` method
110+
- Default to `slog.Default()` in `NewClient`
111+
- Replace `logVerbose`/`logDebug` methods with `c.logger.Debug()`
112+
- Change `epaTLSDialer.logf` and `epaTDSDialer.logf` fields from `func(string, ...interface{})` to `*slog.Logger`
113+
- Update dialer `d.logf(...)` calls to `d.logger.Debug(...)`
114+
115+
**[epa_tester.go](go/internal/mssql/epa_tester.go)** — 69 logf calls + 2 fmt calls
116+
- Add `Logger *slog.Logger` to `EPATestConfig`
117+
- Remove the `logf` closure (line 91-95)
118+
- Convert all 69 `logf(...)` calls to `config.Logger.Debug(...)` with `"component", "epa"` attr
119+
- Convert 2 direct `fmt.Printf` calls
120+
121+
**[epa_auth_provider.go](go/internal/mssql/epa_auth_provider.go)** — 2 fmt calls
122+
- Add `logger *slog.Logger` field
123+
- Convert 2 `fmt.Printf("[EPA-auth] ...")` calls
124+
125+
**[powershell_fallback.go](go/internal/mssql/powershell_fallback.go)** — 1 fmt call
126+
- Add `logger *slog.Logger` field, `SetLogger` method
127+
- Remove `logVerbose` method, convert call to `p.logger.Debug()`
128+
129+
### Phase 4: Supporting packages
130+
131+
**[epamatrix/epamatrix.go](go/internal/epamatrix/epamatrix.go)** — 24 fmt calls
132+
- Add `Logger *slog.Logger` to `MatrixConfig`
133+
- Convert all 24 calls
134+
135+
**[wmi/wmi_windows.go](go/internal/wmi/wmi_windows.go)** — 7 fmt calls
136+
- Change `GetLocalGroupMembers` and `GetLocalGroupMembersWithFallback` signatures: replace `verbose bool` with `logger *slog.Logger`
137+
- Update [wmi/wmi_stub.go](go/internal/wmi/wmi_stub.go) signatures to match
138+
- Update caller in [collector.go:1854](go/internal/collector/collector.go#L1854)
139+
140+
### NOT changed
141+
- **[epamatrix/table.go](go/internal/epamatrix/table.go)**`PrintResultsTable`/`Summarize` write formatted table data to `io.Writer`. This is data output, not logging.
142+
- All `fmt.Errorf(...)` calls (error construction, not logging)
143+
- All `fmt.Sprintf(...)` calls (string building)
144+
145+
## Verification
146+
1. `go build ./...` compiles cleanly
147+
2. `go vet ./...` passes
148+
3. Run with no flags — only INFO+ messages appear, each with UTC timestamp and level
149+
4. Run with `--verbose` — DEBUG messages appear
150+
5. Run with `--debug` — EPA diagnostic messages appear with `component=epa` attribute
151+
6. Table output (EPA matrix) still renders correctly to stdout without log formatting

0 commit comments

Comments
 (0)