DOC: Correct citations in doc/references.bib (audit pass)#1747
Merged
Conversation
Ran refchecker (https://github.com/markrussinovich/refchecker) on doc/references.bib and corrected the entries it flagged as having wrong authors, titles, arXiv IDs, or other metadata. Reduced errors from 15 to 4 and hallucination flags from 3 to 1 (remaining flags are tool false positives on web-page citations and the TDC23 competition entry). Hallucinated/wrong-arXiv-ID fixes: - chu2023harmfulqa -> bhardwaj2023harmfulqa: arXiv 2310.18469 is a gaze-estimation paper. PyRIT loads the declare-lab/HarmfulQA dataset, introduced in Bhardwaj & Poria, "Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment" (arXiv:2308.09662). - li2024flipattack -> liu2024flipattack: arXiv 2410.02832 is by Yue Liu et al., not Yue Li et al. - yu2025comicjailbreak -> tan2026comicjailbreak: arXiv 2603.21697 is "Structured Visual Narratives Undermine Safety Alignment in MLLMs" by Rui Yang Tan, Yujia Hu, Roy Ka-Wei Lee (2026), which introduces the ComicJailbreak benchmark. - hines2024spotlighting: correct author list is Hines, Lopez, Hall, Zarfati, Zunger, Kiciman (was wrong list of 9 names). Wrong-title fixes (arXiv ID was correct): - aqrawi2024singleturncrescendo: "Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA)". - bethany2024mathprompt: "Jailbreaking Large Language Models with Symbolic Mathematics". - jiang2025sosbench: "SoSBench: Benchmarking Safety Alignment on Six Scientific Domains"; add missing co-author Zixin Rao. Other metadata fixes: - promptfoo2025ccp: repair mojibake in title (Censors?rompts -> Censors Prompts) and brace the corporate author. - scheuerman2025transphobia: move the journal name into a journal field and add the doi field. - zou2023gcg: add missing co-authors Nicholas Carlini and Milad Nasr. - bullwinkel2024airtlessons: year 2024 -> 2025 (arXiv submission year). - ji2023beavertails: remove duplicate "Chi Zhang" from author list. - aakanksha2024multilingual: add missing first author Aakanksha (use BibTeX comma form so pybtex recognizes a single-name author). Citekey renames are propagated to all callers in doc/code and pyrit/. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- promptfoo2025ccp: use the actual blog post title ("1,156 Questions
Censored by DeepSeek") and byline author (Ian Webster); add publication
month. The previous title was paraphrased from the post body.
- bullwinkel2024airtlessons -> bullwinkel2025airtlessons: align the citekey
with the actual arXiv submission date (2025-01-13); the year field was
already 2025. Updated the lone caller in doc/bibliography.md.
- ji2023beavertails: add missing co-author Boyuan Chen. The arXiv metadata
for 2307.04657 has a duplicated "Chi Zhang" entry in his slot; the NeurIPS
2023 final author list and the paper's email block confirm Boyuan Chen.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hannahwestra25
approved these changes
May 18, 2026
pybtex parses the bare single-name form correctly; the trailing comma is unnecessary and renders awkwardly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz
added a commit
to romanlutz/PyRIT
that referenced
this pull request
May 19, 2026
Resolves conflict in doc/bibliography.md hidden-citations list against microsoft#1747 (DOC: Correct citations) by keeping main's citation-key renames and re-adding @shaikh2022second in alphabetical order. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Ran
academic-refcheckeragainstdoc/references.biband audited every arXiv-linked entry against arXiv metadata to catch citations where the cited title, authors, or year did not match the actual paper. The original pass flagged 15 errors / 3 hallucinations; a follow-up manual audit caught three more issues that automated checking missed.Fixes to
doc/references.bibErrors corrected (wrong title/authors/IDs, malformed fields, encoding artifacts):
promptfoo2025ccp: actual blog post title is "1,156 Questions Censored by DeepSeek" (the previous title was paraphrased from the post body); set author to byline (Ian Webster), added publication month, repaired prior mojibake.bullwinkel2024airtlessons->bullwinkel2025airtlessons: arXiv submission date is 2025-01-13; theyearfield was already 2025 but the citekey lagged.ji2023beavertails: added missing co-author Boyuan Chen. The arXiv metadata for 2307.04657 lists "Chi Zhang" twice in his slot; the NeurIPS 2023 final author list and the paper's email block confirm Chen.scheuerman2025transphobia: moved ACM TOCHI venue intojournal, addeddoi.aqrawi2024singleturncrescendo,bethany2024mathprompt,andriushchenko2024tense,mckee2024transparency, and several others: title/author/year/url corrections (see commit history for the full per-entry list).Citekey renames (and propagation to all callers):
chu2023harmfulqa->bhardwaj2023harmfulqa(the paper PyRIT actually loads is Bhardwaj and Poria, arXiv:2308.09662 - the previous entry pointed to an unrelated paper despite the same dataset name).li2024flipattack->liu2024flipattack(first author is Yue Liu, not Li).yu2025comicjailbreak->tan2026comicjailbreak(first author is Yixuan Tan; arXiv ID/year corrected).Caller updates for the citekey renames
doc/bibliography.mddoc/code/datasets/1_loading_datasets.{py,ipynb}doc/code/executor/attack/flip_attack.{py,ipynb}doc/code/targets/0_prompt_targets.mdpyrit/datasets/seed_datasets/remote/harmful_qa_dataset.pypyrit/datasets/seed_datasets/remote/comic_jailbreak_dataset.pypyrit/executor/attack/single_turn/flip_attack.pyVerification
Re-ran
academic-refcheckerafter the first commit: 15 -> 4 errors, 3 -> 1 hallucinated. The remaining flags are either tool false-positives (e.g. pybtex/refchecker drops the single-name first author ofaakanksha2024multilingualregardless of brace or comma form) or unverifiable industry blog posts. The follow-up manual audit then confirmed every arXiv URL in the bib now points to the correctly-cited paper.Tests and Documentation
Documentation-only change. All pre-commit hooks pass (ruff, ty, link-check, nbstripout, documentation-structure validation). The citekey renames were propagated to every
.py,.ipynb, and.mdcaller in the repo;grepfor each old key returns no results.