Skip to content

Commit 8a877ec

Browse files
moshloopmoshloop
committed
chore: update godocs and new helper methods
1 parent cf67870 commit 8a877ec

25 files changed

Lines changed: 1385 additions & 79 deletions

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,6 @@
1212
*.out
1313

1414
# Intellij
15-
.idea/
15+
.idea/
16+
.vscode/
17+
cliff.toml

‎CLAUDE.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
flanksource/commons is a Go utility library providing common functionality for the Flanksource ecosystem. It's a collection of reusable modules organized by concern: core utilities, infrastructure helpers, development tools, data processing, and system integration.
8+
9+
## Key Commands
10+
11+
```bash
12+
# Testing
13+
make test # Run all tests with verbose output
14+
go test ./... # Run all tests (standard Go)
15+
go test ./logger/... # Run tests for a specific package
16+
17+
# Linting and Code Quality
18+
make lint # Run golangci-lint with project-specific rules
19+
20+
# Dependency Management
21+
make tidy # Clean up go.mod dependencies
22+
go mod tidy # Standard Go dependency cleanup
23+
24+
# Build
25+
go build # Build the dependency installer binary
26+
go run main.go <dep> # Download and install a dependency (e.g., go run main.go jq)
27+
```
28+
29+
## Architecture and Code Organization
30+
31+
### Module Structure
32+
The codebase follows a flat package structure where each directory represents a focused utility module:
33+
34+
- **HTTP Client (`http/`)**: Enhanced HTTP client with authentication middleware pattern. Supports Digest, NTLM, OAuth2, and custom auth. Uses functional options for configuration.
35+
- **Logger (`logger/`)**: Dual logging system supporting both logrus and slog backends. Global logger instance with interface-based design for flexibility.
36+
- **Collections (`collections/`)**: Generic utilities for maps, slices, sets, and priority queues. Heavy use of Go generics.
37+
- **Dependency Management (`deps/`)**: Template-based system for downloading and installing external binaries. Supports cross-platform paths and verification.
38+
- **Properties (`properties/`)**: Global configuration management with environment variable support and structured properties.
39+
40+
### Key Patterns
41+
42+
1. **Global State Pattern**: Logger and Properties packages use global instances initialized via `Init()` functions. Always check if initialization is needed before using these packages.
43+
44+
2. **Middleware Pattern**: HTTP client uses layered middleware for auth, tracing, and logging:
45+
```go
46+
client := http.NewClient().
47+
WithLogger(logger).
48+
WithAuth(username, password).
49+
WithTrace(true)
50+
```
51+
52+
3. **Interface-First Design**: Most packages expose interfaces (e.g., `Logger` interface) allowing for mock implementations in tests.
53+
54+
4. **Builder Pattern**: Used extensively in HTTP client and dependency configurations for fluent API design.
55+
56+
### Testing Approach
57+
58+
- Uses testify for assertions in most tests
59+
- Some modules (like `diff/`) use Ginkgo/Gomega for BDD-style tests
60+
- Test files are colocated with source files following `*_test.go` convention
61+
- No global test setup/teardown - each test is self-contained
62+
63+
### Important Development Notes
64+
65+
1. **Go Version**: Requires Go 1.23.0+ (uses newer generic features)
66+
2. **Linting**: The project has custom golangci-lint rules that disable certain checks for HTTP-related naming
67+
3. **Imports**: When adding new functionality, prefer using existing utilities from within the project (e.g., use `collections` package for slice operations)
68+
4. **Error Handling**: The codebase uses explicit error returns rather than panics - maintain this pattern
69+
5. **Logging**: Use the `logger` package for all logging needs, avoid fmt.Print statements
70+
71+
### Common Tasks
72+
73+
To add a new utility module:
74+
1. Create a new directory at the root level
75+
2. Follow the existing pattern of having a main file with the package name
76+
3. Include comprehensive tests in `*_test.go` files
77+
4. Update imports in other packages if the utility is widely useful
78+
79+
To modify the HTTP client:
80+
1. Changes go in `http/client.go` or `http/middleware.go`
81+
2. Authentication providers are in `http/auth_*.go` files
82+
3. Always maintain backward compatibility with the builder pattern
83+
84+
To work with the dependency installer:
85+
1. Dependencies are defined in `deps/binary.go`
86+
2. Binary definitions include version, download URLs, and install paths
87+
3. Use templates for cross-platform path handling

‎collections/collections.go‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,28 @@
1+
// Package collections provides generic utilities for working with
2+
// collections like slices, maps, sets, and priority queues.
3+
//
4+
// The package leverages Go generics to provide type-safe operations
5+
// without runtime overhead. It includes utilities for common operations
6+
// like filtering, mapping, grouping, and set operations.
7+
//
8+
// Slice Operations:
9+
//
10+
// numbers := []int{1, 2, 3, 4, 5}
11+
// evens := collections.Filter(numbers, func(n int) bool { return n%2 == 0 })
12+
// squares := collections.Map(numbers, func(n int) int { return n * n })
13+
//
14+
// Map Operations:
15+
//
16+
// data := map[string]int{"a": 1, "b": 2}
17+
// keys := collections.Keys(data)
18+
// values := collections.Values(data)
19+
//
20+
// Set Operations:
21+
//
22+
// set1 := collections.NewSet(1, 2, 3)
23+
// set2 := collections.NewSet(2, 3, 4)
24+
// union := set1.Union(set2) // {1, 2, 3, 4}
25+
// intersect := set1.Intersect(set2) // {2, 3}
126
package collections
227

328
import (
@@ -19,7 +44,15 @@ func takeSliceArg(arg interface{}) (out []interface{}, ok bool) {
1944
return out, true
2045
}
2146

22-
// ToString takes an object and tries to convert it to a string
47+
// ToString converts various types to their string representation.
48+
// Handles slices by joining elements with commas, fmt.Stringer types,
49+
// strings, booleans, and falls back to fmt.Sprintf for other types.
50+
//
51+
// Example:
52+
//
53+
// collections.ToString([]int{1, 2, 3}) // "1, 2, 3"
54+
// collections.ToString(true) // "true"
55+
// collections.ToString(nil) // ""
2356
func ToString(i interface{}) string {
2457
if slice, ok := takeSliceArg(i); ok {
2558
s := ""

‎collections/maps.go‎

Lines changed: 74 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,13 @@ import (
88
"strings"
99
)
1010

11-
// ToGenericMap converts a map[string]string to a map[string]interface{}
11+
// ToGenericMap converts a map[string]string to a map[string]interface{}.
12+
// Useful when you need to pass string maps to functions expecting generic maps.
13+
//
14+
// Example:
15+
//
16+
// strMap := map[string]string{"name": "John", "age": "30"}
17+
// generic := collections.ToGenericMap(strMap)
1218
func ToGenericMap(m map[string]string) map[string]interface{} {
1319
var out = map[string]interface{}{}
1420
for k, v := range m {
@@ -17,7 +23,14 @@ func ToGenericMap(m map[string]string) map[string]interface{} {
1723
return out
1824
}
1925

20-
// ToStringMap converts a map[string]interface{} to a map[string]string by using each objects String() fn
26+
// ToStringMap converts a map[string]interface{} to a map[string]string.
27+
// Each value is converted using fmt.Sprintf("%v", value).
28+
//
29+
// Example:
30+
//
31+
// data := map[string]interface{}{"count": 42, "active": true}
32+
// strings := collections.ToStringMap(data)
33+
// // Result: {"count": "42", "active": "true"}
2134
func ToStringMap(m map[string]interface{}) map[string]string {
2235
var out = make(map[string]string)
2336
for k, v := range m {
@@ -26,7 +39,17 @@ func ToStringMap(m map[string]interface{}) map[string]string {
2639
return out
2740
}
2841

29-
// ToBase64Map converts a map[string]interface{} to a map[string]string by converting []byte to base64
42+
// ToBase64Map converts a map[string]interface{} to a map[string]string.
43+
// []byte values are encoded as base64, other types use fmt.Sprintf.
44+
//
45+
// Example:
46+
//
47+
// data := map[string]interface{}{
48+
// "text": "hello",
49+
// "binary": []byte{0x48, 0x65, 0x6c, 0x6c, 0x6f},
50+
// }
51+
// encoded := collections.ToBase64Map(data)
52+
// // Result: {"text": "hello", "binary": "SGVsbG8="}
3053
func ToBase64Map(m map[string]interface{}) map[string]string {
3154
var out = make(map[string]string)
3255
for k, v := range m {
@@ -40,7 +63,8 @@ func ToBase64Map(m map[string]interface{}) map[string]string {
4063
return out
4164
}
4265

43-
// ToByteMap converts a map[string]interface{} to a map[string]string by converting []byte to base64
66+
// ToByteMap converts a map[string]interface{} to a map[string][]byte.
67+
// String values are converted to []byte, []byte values are kept as-is.
4468
func ToByteMap(m map[string]interface{}) map[string][]byte {
4569
var out = make(map[string][]byte)
4670
for k, v := range m {
@@ -54,8 +78,15 @@ func ToByteMap(m map[string]interface{}) map[string][]byte {
5478
return out
5579
}
5680

57-
// MergeMap will merge map b into a.
58-
// On key collision, map b takes precedence.
81+
// MergeMap merges two maps, with values from b taking precedence.
82+
// Returns the merged map (modifies map a in place).
83+
//
84+
// Example:
85+
//
86+
// defaults := map[string]int{"timeout": 30, "retries": 3}
87+
// overrides := map[string]int{"timeout": 60}
88+
// config := collections.MergeMap(defaults, overrides)
89+
// // Result: {"timeout": 60, "retries": 3}
5990
func MergeMap[T1 comparable, T2 any](a, b map[T1]T2) map[T1]T2 {
6091
if a == nil {
6192
a = make(map[T1]T2)
@@ -72,10 +103,14 @@ func MergeMap[T1 comparable, T2 any](a, b map[T1]T2) map[T1]T2 {
72103
return a
73104
}
74105

75-
// KeyValueSliceToMap takes in a list of strings in a=b format
76-
// and returns a map from it.
106+
// KeyValueSliceToMap converts a slice of "key=value" strings to a map.
107+
// Strings without '=' are treated as keys with empty values.
108+
//
109+
// Example:
77110
//
78-
// Any string that's not in a=b format will be ignored.
111+
// args := []string{"env=prod", "debug=true", "verbose"}
112+
// config := collections.KeyValueSliceToMap(args)
113+
// // Result: {"env": "prod", "debug": "true", "verbose": ""}
79114
func KeyValueSliceToMap(in []string) map[string]string {
80115
sanitized := make(map[string]string, len(in))
81116
for _, item := range in {
@@ -89,15 +124,27 @@ func KeyValueSliceToMap(in []string) map[string]string {
89124
return sanitized
90125
}
91126

92-
// SelectorToMap returns a map from a selector string.
93-
// i.e. "a=b,c=d" -> {"a": "b", "c": "d"}
127+
// SelectorToMap parses a comma-separated selector string into a map.
128+
// Commonly used for Kubernetes-style label selectors.
129+
//
130+
// Example:
131+
//
132+
// selector := "app=nginx,env=prod,tier=frontend"
133+
// labels := collections.SelectorToMap(selector)
134+
// // Result: {"app": "nginx", "env": "prod", "tier": "frontend"}
94135
func SelectorToMap(selector string) (matchLabels map[string]string) {
95136
labels := strings.Split(selector, ",")
96137
return KeyValueSliceToMap(labels)
97138
}
98139

99-
// SortedMap takes a map and returns a sorted key value string
100-
// i.e. {"a": "b", "c": "d"} -> "a=b,c=d"
140+
// SortedMap converts a map to a sorted key=value string representation.
141+
// Keys are sorted alphabetically for consistent output.
142+
//
143+
// Example:
144+
//
145+
// labels := map[string]string{"tier": "web", "app": "nginx"}
146+
// result := collections.SortedMap(labels)
147+
// // Result: "app=nginx,tier=web"
101148
func SortedMap(m map[string]string) string {
102149
keys := MapKeys(m)
103150
sort.Slice(keys, func(i, j int) bool { return keys[i] < keys[j] })
@@ -110,7 +157,13 @@ func SortedMap(m map[string]string) string {
110157
return strings.Join(items, ",")
111158
}
112159

113-
// MapToIni takes a map and converts it into an INI formatted string
160+
// MapToIni converts a map to INI format with one key=value pair per line.
161+
//
162+
// Example:
163+
//
164+
// config := map[string]string{"host": "localhost", "port": "8080"}
165+
// ini := collections.MapToIni(config)
166+
// // Result: "host=localhost\nport=8080\n"
114167
func MapToIni(Map map[string]string) string {
115168
str := ""
116169
for k, v := range Map {
@@ -137,6 +190,13 @@ func IniToMap(path string) map[string]string {
137190
return result
138191
}
139192

193+
// MapKeys returns a slice containing all keys from the map.
194+
// The order of keys is non-deterministic.
195+
//
196+
// Example:
197+
//
198+
// users := map[int]string{1: "Alice", 2: "Bob", 3: "Charlie"}
199+
// ids := collections.MapKeys(users) // [1, 2, 3] (order may vary)
140200
func MapKeys[K comparable, T any](m map[K]T) []K {
141201
keys := make([]K, 0, len(m))
142202
for k := range m {

‎collections/priorityqueue.go‎

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,6 @@
22
// Use of this source code is governed by a BSD-style
33
// license that can be found in the LICENSE file.
44

5-
// Package priorityqueue implements a priority queue backed by binary queue.
6-
//
7-
// A thread-safe priority queue based on a priority queue.
8-
// The elements of the priority queue are ordered by a comparator provided at queue construction time.
9-
//
10-
// The heap of this queue is the least/smallest element with respect to the specified ordering.
11-
// If multiple elements are tied for least value, the heap is one of those elements arbitrarily.
12-
//
13-
// Structure is thread safe.
14-
//
15-
// References: https://en.wikipedia.org/wiki/Priority_queue
165
package collections
176

187
import (
@@ -182,6 +171,20 @@ type queueItem[T comparable] struct {
182171
var _ queues.Queue[int] = (*Queue[int])(nil)
183172

184173
// Queue holds elements in an array-list
174+
// Queue is a thread-safe priority queue implementation.
175+
// Elements are ordered by a comparator function and can optionally
176+
// be deduplicated using an equality function.
177+
//
178+
// Example:
179+
//
180+
// queue, _ := collections.NewQueue(collections.QueueOpts[int]{
181+
// Comparator: func(a, b int) int { return a - b }, // Min heap
182+
// Dedupe: true,
183+
// Equals: func(a, b int) bool { return a == b },
184+
// })
185+
// queue.Enqueue(5)
186+
// queue.Enqueue(1)
187+
// item, _ := queue.Dequeue() // Returns 1
185188
type Queue[T comparable] struct {
186189
heap *binaryheap.Heap[queueItem[T]]
187190
Comparator utils.Comparator[T]
@@ -191,13 +194,25 @@ type Queue[T comparable] struct {
191194
Dedupe bool
192195
}
193196

197+
// QueueOpts configures a priority queue.
194198
type QueueOpts[T comparable] struct {
195199
Comparator utils.Comparator[T]
196200
Dedupe bool
197201
Equals func(a, b T) bool
198202
Metrics MetricsOpts[T]
199203
}
200204

205+
// NewQueue creates a new priority queue with the specified options.
206+
// Returns an error if Dedupe is true but no Equals function is provided.
207+
//
208+
// Example:
209+
//
210+
// // Create a min-heap priority queue for tasks
211+
// queue, err := collections.NewQueue(collections.QueueOpts[Task]{
212+
// Comparator: func(a, b Task) int {
213+
// return a.Priority - b.Priority // Lower priority first
214+
// },
215+
// })
201216
func NewQueue[T comparable](opts QueueOpts[T]) (*Queue[T], error) {
202217
if opts.Dedupe && opts.Equals == nil {
203218
return nil, errors.New("dedupe requires Equals function")

0 commit comments

Comments
 (0)