Skip to content

Added function caller info for RunAiQuery#367

Open
satti-hari-krishna-reddy wants to merge 8 commits intoShuffle:mainfrom
satti-hari-krishna-reddy:observ
Open

Added function caller info for RunAiQuery#367
satti-hari-krishna-reddy wants to merge 8 commits intoShuffle:mainfrom
satti-hari-krishna-reddy:observ

Conversation

@satti-hari-krishna-reddy
Copy link
Copy Markdown
Collaborator

@satti-hari-krishna-reddy satti-hari-krishna-reddy commented Apr 8, 2026

What this PR does

I've added Context-Based Caller Tracking to bring visibility and observability into which features/functions are actually using RunAiQuery.

  • Explicitly requiring a caller in the context forces future features to include it. This means they automatically get tracked and prevents "silent" new AI calls.
  • Updates other functions to propagate context

Comment thread ai.go
cachedChatMd5 := md5.Sum([]byte(systemMessage))
// Does this mean that two same orgs that has same system message results in the same Md5 id ?

cachedChatMd5 := md5.Sum([]byte(systemMessage+org))
@frikky
Copy link
Copy Markdown
Member

frikky commented Apr 9, 2026

When you add this many context changes, please describe them and why. I don't get it :)

Fundmental changes need describing

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds context-based caller tracking to RunAiQuery for better observability of which features/functions are invoking LLM calls, and propagates context.Context through related helper functions.

Changes:

  • Introduces EnsureContextWithCaller and enforces presence of a "caller" value in context for RunAiQuery.
  • Threads context.Context through multiple AI-related helper functions and call sites.
  • Adds AI query logging (caller + org + estimated token counts) and adjusts cache key derivation to incorporate org.
Comments suppressed due to low confidence (1)

ai.go:9333

  • RunAiQuery’s cache key is still too coarse and can cause unrelated prompts to share cached chat history (and potentially leak data across requests) because it’s based only on systemMessage+org. In particular, calls that pass an incomingRequest often pass empty systemMessage/userMessage (e.g., RunAiQuery(ctx, "", "", chatCompletion)), which will write under a key derived only from org_id, and later calls with empty systemMessage will read/append that history. Consider hashing the actual request (e.g., model + full messages) and/or including userMessage (and possibly rootCaller) in the cache key; alternatively, disable cache when systemMessage is empty or when incomingRequest is provided.
	// Rerun with the same chat IF POSSIBLE
	// This makes it so that the chance of getting the right result is lower
	// Does this mean that two same orgs that has same system message results in the same Md5 id ?

	cachedChatMd5 := md5.Sum([]byte(systemMessage+org))
	cachedChat := fmt.Sprintf("chat-%x", cachedChatMd5)

	if len(incomingRequest) > 0 {
		chatCompletion = incomingRequest[0]
	} else {
		if len(systemMessage) > 0 {
			chatCompletion.Messages = append(chatCompletion.Messages, openai.ChatCompletionMessage{
				Role:    openai.ChatMessageRoleSystem,
				Content: systemMessage,
			})
		}

		data, err := GetCache(ctx, cachedChat)
		if err == nil {
			oldChat := openai.ChatCompletionRequest{}
			cacheData := []byte(data.([]uint8))
			err = json.Unmarshal(cacheData, &oldChat)
			if err != nil {
				log.Printf("[ERROR] Failed to unmarshal cached chat: %s", err)
			}


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ai.go
Comment on lines +9241 to +9242
estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5))
totalEst := estSysTokens + estUserTokens
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, the new token estimate lines use space indentation and will be adjusted by gofmt. Running gofmt/goimports will align these with the surrounding code style.

Suggested change
estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5))
totalEst := estSysTokens + estUserTokens
estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5))
totalEst := estSysTokens + estUserTokens

Copilot uses AI. Check for mistakes.
Comment thread ai.go
Comment on lines +85 to +89

if len(ctx.Value("caller").(string)) == 0 {
ctx = context.WithValue(ctx, "caller", callerName)
}

Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EnsureContextWithCaller can panic when ctx has no "caller" value (including the common case where ctx is nil and you set it to context.Background()). The expression ctx.Value("caller").(string) will panic if the key is missing or not a string. Use a safe type assertion (v, _ := ctx.Value(...).(string)) before checking length, and only call context.WithValue when v is empty.

Suggested change
if len(ctx.Value("caller").(string)) == 0 {
ctx = context.WithValue(ctx, "caller", callerName)
}
caller, _ := ctx.Value("caller").(string)
if len(caller) == 0 {
ctx = context.WithValue(ctx, "caller", callerName)
}

Copilot uses AI. Check for mistakes.
Comment thread ai.go Outdated
Comment thread ai.go
Comment on lines +9195 to +9197
rootCaller, ok := ctx.Value("caller").(string)
org, ok := ctx.Value("org_id").(string)

Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New context keys are plain strings ("caller", "org_id"). Go’s context package recommends using an unexported typed key to avoid collisions between packages. Consider defining a private key type and constants (e.g., type ctxKey string; const ctxCallerKey ctxKey = "caller"), and using those consistently in EnsureContextWithCaller / RunAiQuery / handlers.

Copilot uses AI. Check for mistakes.
Comment thread ai.go
Comment on lines 23 to 29
"strings"
"sync"
"time"

"runtime"
"math"
openai "github.com/sashabaranov/go-openai"
uuid "github.com/satori/go.uuid"
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import ordering/grouping is inconsistent after adding math/runtime (stdlib imports should be sorted, and typically separated from third-party imports). Running gofmt/goimports will reorder these cleanly (e.g., math before runtime, and third-party imports in a separate group).

Copilot uses AI. Check for mistakes.
Comment thread ai.go
Comment on lines +9204 to +9206
if ok {
callerName = runtime.FuncForPC(pc).Name()
}
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few new space-indented lines in RunAiQuery (e.g., around the runtime.Caller block) that will be reformatted by gofmt (tabs/alignments). Please run gofmt on this function to keep formatting consistent and avoid noisy diffs.

Suggested change
if ok {
callerName = runtime.FuncForPC(pc).Name()
}
if ok {
callerName = runtime.FuncForPC(pc).Name()
}

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants