Added function caller info for RunAiQuery#367
Added function caller info for RunAiQuery#367satti-hari-krishna-reddy wants to merge 8 commits intoShuffle:mainfrom
Conversation
| cachedChatMd5 := md5.Sum([]byte(systemMessage)) | ||
| // Does this mean that two same orgs that has same system message results in the same Md5 id ? | ||
|
|
||
| cachedChatMd5 := md5.Sum([]byte(systemMessage+org)) |
|
When you add this many context changes, please describe them and why. I don't get it :) Fundmental changes need describing |
There was a problem hiding this comment.
Pull request overview
Adds context-based caller tracking to RunAiQuery for better observability of which features/functions are invoking LLM calls, and propagates context.Context through related helper functions.
Changes:
- Introduces
EnsureContextWithCallerand enforces presence of a"caller"value in context forRunAiQuery. - Threads
context.Contextthrough multiple AI-related helper functions and call sites. - Adds AI query logging (caller + org + estimated token counts) and adjusts cache key derivation to incorporate org.
Comments suppressed due to low confidence (1)
ai.go:9333
- RunAiQuery’s cache key is still too coarse and can cause unrelated prompts to share cached chat history (and potentially leak data across requests) because it’s based only on systemMessage+org. In particular, calls that pass an incomingRequest often pass empty systemMessage/userMessage (e.g., RunAiQuery(ctx, "", "", chatCompletion)), which will write under a key derived only from org_id, and later calls with empty systemMessage will read/append that history. Consider hashing the actual request (e.g., model + full messages) and/or including userMessage (and possibly rootCaller) in the cache key; alternatively, disable cache when systemMessage is empty or when incomingRequest is provided.
// Rerun with the same chat IF POSSIBLE
// This makes it so that the chance of getting the right result is lower
// Does this mean that two same orgs that has same system message results in the same Md5 id ?
cachedChatMd5 := md5.Sum([]byte(systemMessage+org))
cachedChat := fmt.Sprintf("chat-%x", cachedChatMd5)
if len(incomingRequest) > 0 {
chatCompletion = incomingRequest[0]
} else {
if len(systemMessage) > 0 {
chatCompletion.Messages = append(chatCompletion.Messages, openai.ChatCompletionMessage{
Role: openai.ChatMessageRoleSystem,
Content: systemMessage,
})
}
data, err := GetCache(ctx, cachedChat)
if err == nil {
oldChat := openai.ChatCompletionRequest{}
cacheData := []byte(data.([]uint8))
err = json.Unmarshal(cacheData, &oldChat)
if err != nil {
log.Printf("[ERROR] Failed to unmarshal cached chat: %s", err)
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5)) | ||
| totalEst := estSysTokens + estUserTokens |
There was a problem hiding this comment.
Similarly, the new token estimate lines use space indentation and will be adjusted by gofmt. Running gofmt/goimports will align these with the surrounding code style.
| estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5)) | |
| totalEst := estSysTokens + estUserTokens | |
| estUserTokens := int(math.Ceil(float64(len(userMessage)) / 3.5)) | |
| totalEst := estSysTokens + estUserTokens |
|
|
||
| if len(ctx.Value("caller").(string)) == 0 { | ||
| ctx = context.WithValue(ctx, "caller", callerName) | ||
| } | ||
|
|
There was a problem hiding this comment.
EnsureContextWithCaller can panic when ctx has no "caller" value (including the common case where ctx is nil and you set it to context.Background()). The expression ctx.Value("caller").(string) will panic if the key is missing or not a string. Use a safe type assertion (v, _ := ctx.Value(...).(string)) before checking length, and only call context.WithValue when v is empty.
| if len(ctx.Value("caller").(string)) == 0 { | |
| ctx = context.WithValue(ctx, "caller", callerName) | |
| } | |
| caller, _ := ctx.Value("caller").(string) | |
| if len(caller) == 0 { | |
| ctx = context.WithValue(ctx, "caller", callerName) | |
| } |
| rootCaller, ok := ctx.Value("caller").(string) | ||
| org, ok := ctx.Value("org_id").(string) | ||
|
|
There was a problem hiding this comment.
New context keys are plain strings ("caller", "org_id"). Go’s context package recommends using an unexported typed key to avoid collisions between packages. Consider defining a private key type and constants (e.g., type ctxKey string; const ctxCallerKey ctxKey = "caller"), and using those consistently in EnsureContextWithCaller / RunAiQuery / handlers.
| "strings" | ||
| "sync" | ||
| "time" | ||
|
|
||
| "runtime" | ||
| "math" | ||
| openai "github.com/sashabaranov/go-openai" | ||
| uuid "github.com/satori/go.uuid" |
There was a problem hiding this comment.
Import ordering/grouping is inconsistent after adding math/runtime (stdlib imports should be sorted, and typically separated from third-party imports). Running gofmt/goimports will reorder these cleanly (e.g., math before runtime, and third-party imports in a separate group).
| if ok { | ||
| callerName = runtime.FuncForPC(pc).Name() | ||
| } |
There was a problem hiding this comment.
There are a few new space-indented lines in RunAiQuery (e.g., around the runtime.Caller block) that will be reformatted by gofmt (tabs/alignments). Please run gofmt on this function to keep formatting consistent and avoid noisy diffs.
| if ok { | |
| callerName = runtime.FuncForPC(pc).Name() | |
| } | |
| if ok { | |
| callerName = runtime.FuncForPC(pc).Name() | |
| } |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
What this PR does
I've added Context-Based Caller Tracking to bring visibility and observability into which features/functions are actually using RunAiQuery.