Skip to content

Add note qc feature#155

Merged
jspada200 merged 10 commits into
mainfrom
AddNoteQCFeature
May 15, 2026
Merged

Add note qc feature#155
jspada200 merged 10 commits into
mainfrom
AddNoteQCFeature

Conversation

@jspada200
Copy link
Copy Markdown
Collaborator

@jspada200 jspada200 commented May 14, 2026

Summary

This is a heavy but cool one! This change adds a note QC workflow. It allows users to add checks to notes with natural language then these checks run on publish. For example, you can verify a note vs the transcript.

image

Checks can be added via the settings UI

2026-05-14 14 34 31

These checks run in the publish UI. They can update the note itself or the other fields on the note.

The checks have access to tools to browse the prodtracking software. An example of where this can be used is to find attachments to plug into the note links, tos, or CCs,

2026-05-14 14 35 23

Testing

  • I have tested these changes locally
  • [ X ] I have run all relevant automated tests
  • [ X ] I have verified this does not break existing workflows
  • For changes that can be tested in UI, I have included screenshots or gif animations of the changes.

How I Tested

  • Created a transcript missing info and the QC catches it
  • Created a note mentioning a person who was not tagged
  • Checked content of a note to ensure it contained the requested information.

jspada200 added 7 commits May 13, 2026 15:52
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
Signed-off-by: James Spadafora <spadjv@gmail.com>
@jspada200 jspada200 requested review from camerontarget14 and richardssam and removed request for richardssam May 14, 2026 21:52
@jspada200 jspada200 self-assigned this May 14, 2026
@jspada200 jspada200 changed the title Draft: Add note qc feature Add note qc feature May 14, 2026
@camerontarget14
Copy link
Copy Markdown
Collaborator

LVGTM!!! Very neat idea James!

Copy link
Copy Markdown
Collaborator

@camerontarget14 camerontarget14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things maybe worth looking at before merge (from my AI and brushed up by me):

  • generate_with_tools and generate_structured_with_tools duplicate the tool-call loop almost verbatim — consider extracting a shared helper.

  • json.loads(tc.function.arguments or "{}") will throw on malformed JSON; consider wrapping and returning an error result to the tool channel so the model can recover.

  • If max_iterations is hit mid tool-calling in generate_structured_with_tools, you proceed to instructor extraction without a clean final assistant turn. Worth at least a log warning, ideally a forced tool_choice="none" final pass.

As an aside:

  • max_iterations bounds turns but not size — if tool_executor returns large prodtracking blobs, messages grows every iteration and you can chew through context fast. Probably worth truncating tool results before appending, or tracking approximate token usage.

jspada200 added 2 commits May 14, 2026 15:54
Signed-off-by: James Spadafora <spadjv@gmail.com>
Copy link
Copy Markdown
Collaborator

@camerontarget14 camerontarget14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@jspada200 jspada200 merged commit b4bc1cb into main May 15, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants