Skip to content

static-analysis-pipeline.md

Latest commit

 

History

History
157 lines (125 loc) · 7.08 KB

static-analysis-pipeline.md

File metadata and controls

157 lines (125 loc) · 7.08 KB
Raw

Reusable Pattern: Static Artifact Analysis Pipeline

A language-agnostic pseudocode template for "this kind of problem" in malware analysis: safely turning a raw artifact into observable indicators and a report, without executing it.

It was distilled from the course's expert reference script (code/basic_analysis.py) and has already been re-applied, unchanged in shape, to a different artifact type (code/pdf_analysis.py). Use it as the starting skeleton for any new static-analysis task.

The invariant

Never execute the artifact. Read its bytes only. Static analysis is safe precisely because nothing runs. Every function below operates on bytes already in memory.

The five stages

Acquire  →  Extract  →  Fingerprint  →  Detect  →  Report
Stage Responsibility One job
Acquire Load the artifact's raw bytes, safely read-binary + error handling
Extract Pull observable clues from the bytes printable strings (extendable)
Fingerprint Produce identity for known-bad comparison MD5 + SHA-256
Detect Apply heuristics to the extracted features keyword / rule matching
Report Present findings in a readable, shareable form the only stage that prints

Pseudocode

╔══════════════════════════════════════════════════════════════════════╗
║  STATIC ARTIFACT ANALYSIS PIPELINE                                     ║
║  Invariant: NEVER execute the artifact. Read its bytes only.           ║
╚══════════════════════════════════════════════════════════════════════╝

# ---- CONFIG (hoist the knobs to the top so they're easy to tune) ----
CONSTANT MIN_STRING_LENGTH     = 4
CONSTANT SUSPICIOUS_KEYWORDS   = [ ... task-specific terms ... ]
CONSTANT PRINTABLE_ASCII_RANGE = bytes 0x20..0x7E

# ---- STEP 1: ACQUIRE (load without executing) ----
FUNCTION acquire(file_path):
    """Read the artifact's raw bytes. Return the bytes, or NOTHING on failure."""
    TRY:
        OPEN file_path in READ-BINARY mode AS f
        data <- f.read_all_bytes()
        RETURN data
    CATCH input/output OR access error:
        LOG "Error reading file: " + file_path
        RETURN NOTHING            # caller MUST check before proceeding

# ---- STEP 2: EXTRACT FEATURES (pull observable clues from raw bytes) ----
FUNCTION extract_strings(data, min_length = MIN_STRING_LENGTH):
    """Return every run of printable ASCII at least min_length long."""
    pattern <- "PRINTABLE_ASCII char, repeated >= min_length times"
    RETURN find_all(pattern, data)        # list of byte-strings

# ---- STEP 3: FINGERPRINT (identity for known-bad comparison) ----
FUNCTION compute_hashes(data):
    """Return identity hashes used to match against malware databases."""
    md5    <- MD5(data)    as hex
    sha256 <- SHA256(data) as hex
    RETURN (md5, sha256)

# ---- STEP 4: DETECT INDICATORS (apply heuristics to features) ----
FUNCTION detect_suspicious(strings, keyword_list = SUSPICIOUS_KEYWORDS):
    """Return the subset of strings that contain any suspicious keyword."""
    hits <- empty list
    FOR each s IN strings:
        text <- decode s to string SAFELY (ignore undecodable bytes)
        FOR each keyword IN keyword_list:
            IF lowercase(keyword) is found inside lowercase(text):
                APPEND text TO hits
                BREAK             # one match per feature is enough
    RETURN hits

# ---- STEP 5: REPORT (present findings; the ONLY stage that prints) ----
FUNCTION generate_report(file_path, md5, sha256, strings, indicators):
    """Emit a readable summary of all findings."""
    PRINT "Analysis Report for: " + file_path
    PRINT "MD5 Hash: "    + md5
    PRINT "SHA-256 Hash: " + sha256
    PRINT "Extracted Strings:"
    FOR each s IN strings: PRINT decode s SAFELY
    PRINT "Suspicious Indicators Found:"
    IF indicators non-empty:
        FOR each i IN indicators: PRINT i
    ELSE:
        PRINT "None found."

# ---- ORCHESTRATION (wire stages in order, with a guard) ----
MAIN (only when run directly):
    file_path <- "<the artifact>"

    data <- acquire(file_path)
    IF data is NOTHING:
        PRINT "Exiting due to read error."
        STOP                                # guard: never continue blind

    strings    <- extract_strings(data)
    md5,sha256 <- compute_hashes(data)
    indicators <- detect_suspicious(strings, SUSPICIOUS_KEYWORDS)
    generate_report(file_path, md5, sha256, strings, indicators)

Extension points — flexing it for new problems

The skeleton stays fixed; you plug new capability into the matching slot. When a new requirement appears, ask: "which of the five stages does this belong to?"

Slot Part 1 (current) How to extend
Acquire one local file many files / a folder; pull from quarantine
Extract printable strings parse file header (PE/ELF/Mach-O); compute entropy; regex for IPs/URLs/emails; list imports
Fingerprint MD5 + SHA-256 add SHA-1; fuzzy hashing (ssdeep) for variant matching
Detect keyword substring match regex IoC rules; YARA matching; VirusTotal hash lookup
Report print to console write file / JSON / CSV; return a structured object for other tools
Orchestrate single main, one file loop over many files; aggregate; set exit codes

Transferable conventions (carry into every implementation)

  1. One stage → one function (single responsibility).
  2. Acquire-without-executing — the static-analysis safety invariant.
  3. Defensive I/O — try/except, return nothing on failure, guard in main.
  4. Separate compute from present — analysis returns data; only the report prints.
  5. Hoist configuration — tunables live at the top, not buried in logic.
  6. Decode bytes safely — ignore undecodable bytes so one bad byte can't crash a run.
  7. Orchestrate behind a "run directly" guard — file works as both script and import.
  8. Fixed, meaningful order — Acquire → Extract → Fingerprint → Detect → Report.

Proof it generalizes

Stage basic_analysis.py (suspicious.bin) pdf_analysis.py (suspicious.pdf)
Acquire read_file read_file (identical)
Extract extract_strings extract_strings (identical)
Fingerprint compute_hashes compute_hashes (identical)
Detect detect_suspicious, keywords for generic binaries detect_suspicious, keywords for PDF attacks
Report generate_report generate_report (identical)
Orchestrate main, path = suspicious.bin main, path = suspicious.pdf

Only the path and the keyword list changed. That is the pattern working as intended.