Skip to content

add token vault feature#4

Open
psarna wants to merge 3 commits intomasterfrom
tok1
Open

add token vault feature#4
psarna wants to merge 3 commits intomasterfrom
tok1

Conversation

@psarna
Copy link
Copy Markdown
Collaborator

@psarna psarna commented Apr 21, 2026

Adds token vault capabilities which translate auth tokens
on the fly, so that the proxees do not actually ever see
the real tokens.

Flow:

$ ./sandworm token add bearer --host api.github.com --secret "$GITHUB_API_TOKEN"
created bearer mapping "api.github.com" for api.github.com * using proxy token "default"
$ ./sandworm token list
default
  - bearer api.github.com *
$ ./sandworm --domains api.github.com --translate-tokens
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Starting proxy" addr=0.0.0.0:2137 mode=allow allowed_domains=[api.github.com] allowed_cidrs=[] blocked_domains=[] blocked_cidrs=[]
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Starting admin panel" addr=127.0.0.1:2138
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Proxy started successfully" port=2137 admin_bind=127.0.0.1 mode=allow domains=[api.github.com] cidrs=[] blocked_domains=[] blocked_cidrs=[] translate_tokens=true translate_tokens_db=./sandworm.db

time=2026-04-27T09:52:49.262+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:58068
time=2026-04-27T09:52:59.189+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:43258
time=2026-04-27T09:53:02.509+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:54504
$ . <(./sandworm token config)
$ echo $HTTPS_PROXY
http://x:sand_-khhQ0jx0wOPxaoyGajlCdchfBWB-ZXu@127.0.0.1:2137

$ curl -sSfL https://api.github.com/user | jq -r '.name,.company'
Piotr Sarna
poolside

Revoking path:

$ ./sandworm token revoke
revoked proxy token "default"
$ ./sandworm token list
default
  - no mappings

$ curl -sSfL https://api.github.com/user | jq -r '.name,.company'
curl: (22) The requested URL returned error: 401

@psarna psarna force-pushed the tok1 branch 14 times, most recently from 3b159d0 to 72fa00c Compare April 27, 2026 08:00
Adds token vault capabilities which translate auth tokens
on the fly, so that the proxees do not actually ever see
the real tokens.

Flow:
```sh
$ ./sandworm token add bearer --host api.github.com --secret "$GITHUB_API_TOKEN"
created bearer mapping "api.github.com" for api.github.com * using proxy token "default"
$ ./sandworm token list
default
  - bearer api.github.com *
```

```sh
$ ./sandworm --domains api.github.com --translate-tokens
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Starting proxy" addr=0.0.0.0:2137 mode=allow allowed_domains=[api.github.com] allowed_cidrs=[] blocked_domains=[] blocked_cidrs=[]
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Starting admin panel" addr=127.0.0.1:2138
time=2026-04-27T09:45:37.688+02:00 level=INFO msg="Proxy started successfully" port=2137 admin_bind=127.0.0.1 mode=allow domains=[api.github.com] cidrs=[] blocked_domains=[] blocked_cidrs=[] translate_tokens=true translate_tokens_db=./sandworm.db

time=2026-04-27T09:52:49.262+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:58068
time=2026-04-27T09:52:59.189+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:43258
time=2026-04-27T09:53:02.509+02:00 level=INFO msg="Proxy request" method=CONNECT host=api.github.com:443 url=//api.github.com:443 remote=127.0.0.1:54504
```

```sh
$ . <(./sandworm token config)
$ echo $HTTPS_PROXY
http://x:sand_-khhQ0jx0wOPxaoyGajlCdchfBWB-ZXu@127.0.0.1:2137

$ curl -sSfL https://api.github.com/user | jq -r '.name,.company'
Piotr Sarna
poolside

Revoking path:
```
$ ./sandworm token revoke
revoked proxy token "default"
$ ./sandworm token list
default
  - no mappings

$ curl -sSfL https://api.github.com/user | jq -r '.name,.company'
curl: (22) The requested URL returned error: 401

```
@yangchi
Copy link
Copy Markdown
Contributor

yangchi commented Apr 27, 2026

O_O this is beyond my know how . I will take a look but it will be slow

(Update: looked. Still don't think I'm the right person to look. I have nothing useful to offer)

Copy link
Copy Markdown

@bronsted bronsted left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very cool.

A few places, it might be a good idea to add a few more tests touching some of the edge cases.

Lets try it!

Comment thread pkg/proxy/proxy.go Outdated
}

func NewProxy(port int, adminBind string, mode string, allowedDomains []string, allowedCIDRs []string, blockedDomains []string, blockedCIDRs []string, ipProxyRange string, ipPorts []int, dnsPort int) *Proxy {
func NewProxy(port int, adminBind string, mode string, allowedDomains []string, allowedCIDRs []string, blockedDomains []string, blockedCIDRs []string, ipProxyRange string, ipPorts []int, dnsPort int, tokenDB string) (*Proxy, error) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list of arguments is becoming a bit long. Consider using an argument struct or an option pattern

Comment thread cmd/sandworm/token_commands.go Outdated
}

proxyURL := fmt.Sprintf("http://x:%s@%s:%d", token.Token, args.BindHost, args.Port)
fmt.Fprintln(os.Stdout, "# sandworm token config")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we might as well use fmt.Println here.

Comment thread cmd/sandworm/token_commands.go Outdated

tokens := store.ListTokens()
if len(tokens) == 0 {
fmt.Fprintln(os.Stdout, "no proxy tokens")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should be an error or go to stderr (I am not sure)

Use: "list",
Short: "List proxy tokens",
RunE: func(cmd *cobra.Command, cmdArgs []string) error {
store, err := proxy.OpenTranslationStore(args.DB)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the body of this function should go into a separate function - that also makes it easier to test

Comment thread pkg/proxy/mitm.go Outdated
continue
}
for _, value := range values {
if _, err := fmt.Fprintf(writer, "%s: %s\r\n", name, value); err != nil {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cant remember if HTTP has special format/rules for writing headers - maybe we can use the function in the http package https://cs.opensource.google/go/go/+/refs/tags/go1.26.2:src/net/http/header.go;l=85 - otherwise consider adding a comment why we cannot use it

Comment thread pkg/proxy/token_translation_test.go Outdated
defer upstream.Close()

upstreamURL, err := url.Parse(upstream.URL)
if err != nil {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is equivalent to:
require.NoError(t, err, "parse upstream url)

Comment thread pkg/proxy/token_translation_test.go Outdated
t.Fatalf("create rule: %v", err)
}

ctx, cancel := context.WithCancel(context.Background())
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use t.Context() - that will also be automatically cancelled

Comment thread pkg/proxy/token_translation_test.go Outdated
}
for input, want := range cases {
if got := NormalizePathPattern(input); got != want {
t.Fatalf("NormalizePathPattern(%q) = %q, want %q", input, got, want)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail the whole test on the first failure case. Consider using

assert.Equal(t, got, want, "NormalizePathPattern(%q) = %q, input)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants