Fix on-demand subfile updates and handle related oversized Mongo writes

[enzok]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the reliability and user experience of on-demand analysis features. It addresses potential data storage issues by implementing specific error handling for oversized MongoDB documents and refactors the data update mechanism for greater consistency. Furthermore, it refines the logic for handling subfile categories and improves the presentation of generated string data in the user interface.

Highlights

• MongoDB Write Error Handling: Implemented robust error handling for MongoDB writes, specifically catching DocumentTooLarge errors to prevent failures when on-demand generated data exceeds MongoDB's document size limit. An appropriate HTTP 413 error is returned to the user in such cases.
• Refactored On-Demand Data Updates: Introduced a new recursive helper function, _set_service_by_sha256, to standardize and simplify the process of updating service-specific data within nested analysis report structures, improving code maintainability and correctness.
• Dynamic Subfile Category Handling: Corrected the logic for determining the category of subfiles when generating on-demand analysis, ensuring that non-static categories are preserved and target.file is used appropriately for selfextracted storage.
• Improved UI for On-Demand Generation and String Display: Updated the UI to dynamically pass the correct tab name to on-demand generation URLs for 'Strings' and 'Floss' services. Additionally, the display of strings (both general and .NET) in the UI has been enhanced for better readability and formatting.

Changelog

• web/analysis/views.py
  • Added import for DocumentTooLarge from pymongo.errors and defined MONGO_DOCUMENT_TOO_LARGE_ERRORS for centralized error handling.
  • Modified the on_demand function to conditionally set the category to target.file only when the original category is static and selfextracted storage is involved, preserving other categories.
  • Introduced a recursive helper function _set_service_by_sha256 to abstract the logic for finding and updating service data within complex nested data structures.
  • Replaced manual iteration loops with calls to _set_service_by_sha256 for updating service data in CAPE, procdump, procmemory, dropped, and extracted_files categories.
  • Wrapped the mongo_update_one call in a try-except block to catch MONGO_DOCUMENT_TOO_LARGE_ERRORS and return an HTTP 413 status, or a generic HTTP 500 for other exceptions during data storage.
• web/templates/analysis/generic/_subfile_info.html
  • Updated the 'Gen Strings' and 'Gen Floss' buttons to use the dynamic tab_name in their on_demand URLs instead of a hardcoded 'static' category.
  • Modified the display of sub_file.strings to iterate over each string and render it within a <pre> tag, ensuring proper formatting and line breaks.
  • Modified the display of sub_file.dotnet_strings to iterate over each string and render it within a <pre> tag, ensuring proper formatting and line breaks.
  • Added a trailing newline character to the end of the file.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request primarily addresses on-demand subfile updates, improves error handling for oversized MongoDB writes, and refactors update logic into a recursive function. It also correctly removes the |safe filter in the subfile information template, enhancing security against Cross-Site Scripting (XSS). However, a data integrity vulnerability exists in the on_demand view where an insufficiently validated category parameter could corrupt analysis documents in MongoDB. Additionally, a likely typo using file.sha256 instead of sub_file.sha256 for the .NET strings collapse ID may cause UI issues. Further improvements are needed in exception handling robustness and adopting standard logging practices.