Skip to content

FLAG_SYSTEM_IMPROVEMENT_PLAN.md

Latest commit

 

History

History
272 lines (191 loc) · 8.84 KB

FLAG_SYSTEM_IMPROVEMENT_PLAN.md

File metadata and controls

272 lines (191 loc) · 8.84 KB

Clone Check Flag System Improvement Plan

Executive Summary

The current flagging system treats normal vehicle maintenance as suspicious behavior. This creates noise and reduces trust. We need to refocus on genuine fraud indicators while celebrating good maintenance practices.

Core Philosophy Change

OLD: Flag anything unusual NEW: Flag only genuine fraud/cloning indicators and serious safety issues that aren't being addressed

Category 1: REMOVE THESE FLAGS ENTIRELY

These are normal vehicle behavior and should NOT be presented as concerns:

1. ❌ same_day_fail_pass (Currently 3 points)

Why Remove: This is GOOD BEHAVIOR - garage found issues, fixed them same day, re-tested. This is EXACTLY what should happen.

Real-world: For a 15-year-old car:

  • Brake pads wearing out → EXPECTED
  • Tyres needing replacement → EXPECTED
  • Suspension bushings worn → EXPECTED
  • Fixed immediately → EXCELLENT MAINTENANCE

Decision: Remove completely or make purely informational with positive framing: "Quick turnaround repairs completed"

2. ❌ advisory_escalation (Currently 8 points)

Why Remove: Advisory items GET WORSE over time - that's how wear works!

Real-world:

  • Year 1: "Brake pads wearing thin" (advisory)
  • Year 2: "Brake pads below minimum" (fail)
  • This is NORMAL WEAR, not fraud

Decision: Remove completely. This is how mechanical parts work.

3. ❌ low_mileage_for_age (Currently 2 points)

Why Remove: Many vehicles are genuinely low mileage (elderly owners, second cars, stored vehicles)

Decision: Remove as a "concern". Could be informational only: "Below average mileage for age"

4. ❌ frequent_test_center_changes (Currently 3 points)

Why Remove: People move, shops close, prices vary, quality varies

Decision: Remove entirely unless combined with same-day different locations

5. ❌ abnormal_test_frequency (Currently 2 points)

Why Remove: Failures require retests. This is normal.

Decision: Remove entirely

6. ❌ last_minute_mot (Currently 0 points already)

Why Remove: Everyone procrastinates. This is human nature.

Decision: Already 0 points, just remove from output

7. ❌ seasonal_test_bias / weekend_testing_pattern

Why Remove: People test when convenient. Working people prefer weekends.

Decision: Remove entirely. This is meaningless.

Category 2: KEEP THESE - REAL FRAUD INDICATORS

These ARE legitimate concerns and should remain (or be strengthened):

✅ multiple_test_locations_same_day (50 points) - KEEP

Why: Physically impossible. Clear cloning indicator.

✅ mileage_rollback (45 points) - KEEP

Why: Clear fraud. Odometer has been tampered with.

✅ manufacturer_mismatch (40 points) - KEEP

Why: VIN says Ford, MOT says BMW = identity fraud

✅ color_change (25 points) - KEEP WITH CONTEXT

Why: Color changes without documentation can indicate cloning Improve: Only flag if no supporting evidence (respray is normal for old cars)

✅ registration_change_detected (Critical) - KEEP & STRENGTHEN

Why: This is a PRIMARY cloning indicator

✅ zero_mileage_reading (30 points) - KEEP WITH CONTEXT

Why: Often indicates odometer replacement/tampering Improve: If surrounded by normal progression and only 1 instance, might be legitimate cluster replacement

✅ impossible_mileage_increase (35 points) - KEEP

Why: 200k miles in 6 months is impossible for private vehicles

Category 3: MODIFY THESE - NEEDS SMARTER LOGIC

These CAN indicate problems but need context:

🔄 dangerous_defect_ignored (Currently 25 points)

Current Problem: Flags if not fixed within 1 year Reality: If fixed at next MOT (within 12-15 months), this is NORMAL

New Logic:

# Only flag if:
# 1. Dangerous defect appears in FAILED test
# 2. NOT fixed at next test (or skipped MOT entirely)
# 3. OR appears in 2+ consecutive tests without being addressed

# DO NOT FLAG if:
# - Found in advisory
# - Fixed at next annual MOT
# - Only one occurrence

🔄 recurring_same_failure (Currently 18 points)

Current Problem: Flags if same issue appears twice Reality: Some issues are chronic or complex (rust, emissions)

New Logic:

# Only flag if:
# 1. Same MAJOR failure 3+ times
# 2. AND never properly resolved
# 3. AND spanning 2+ years

# DO NOT FLAG if:
# - Issue was fixed but recurred after 2+ years (could be different component)
# - Minor advisory items

🔄 mot_history_gap (Currently 10 points)

Current Problem: Flags any gap over 1 year Reality: SORN (Statutory Off Road Notification), stored vehicles, COVID, etc.

New Logic:

# Only flag if:
# 1. Gap coincides with suspicious mileage patterns
# 2. Gap is during typical "cloning window" (after acquisition, before sale)
# 3. No SORN record and gap unexplained

# DO NOT FLAG if:
# - Car is old and might be stored
# - Single gap with reasonable explanation
# - SORN period indicated in DVLA data

🔄 same_mileage_consecutive (Currently 12 points)

Current Problem: Flags identical mileage Reality: Stored vehicles, barely driven during COVID, etc.

New Logic:

# Only flag if:
# 1. Multiple consecutive tests with identical mileage AND
# 2. Surrounded by high mileage periods (indicates disconnected odometer)

# DO NOT FLAG if:
# - Single occurrence
# - Vehicle age suggests might be stored/rarely used
# - Low absolute mileage (classic car territory)

Category 4: NEW FLAGS TO ADD

Missing important indicators:

➕ repeated_major_structural_rust

Why: Major chassis rust that keeps appearing suggests serious structural issues Logic: Same structural component fails for rust 2+ times

➕ high_value_theft_marker

Why: Certain makes/models are commonly cloned (Range Rover, BMW M series, etc.) Logic: Cross-reference with theft statistics for make/model

➕ export_marker_with_uk_tests

Why: Car marked for export but still being tested in UK is suspicious Logic: DVLA shows export marker but MOT tests continue

Severity Calibration

Current system gives points to everything. New system:

CRITICAL (Block/Warn) - 40+ points:

  • multiple_test_locations_same_day
  • mileage_rollback (significant, 10k+ miles)
  • manufacturer_mismatch
  • registration_change_detected

HIGH (Strong Warning) - 20-39 points:

  • impossible_mileage_increase
  • zero_mileage_reading (multiple occurrences)
  • color_change (undocumented)

MEDIUM (Investigate) - 10-19 points:

  • dangerous_defect_ignored (using new logic)
  • recurring_same_failure (using new logic)
  • suspicious_mileage_pattern (multiple minor anomalies)

LOW (Minor Note) - 1-9 points:

  • NOTHING. If it's this minor, don't show it.

INFORMATIONAL (0 points):

  • Same-day fail then pass → Show as POSITIVE: "Maintenance: Issues identified and resolved same day"
  • Mileage below average → "Lower than average mileage for vehicle age"

Risk Score Recalibration

Current scale is too sensitive. Suggestions:

0-10: Excellent - No issues detected 11-25: Good - Minor points to note 26-45: Caution - Some concerns warrant attention 46-70: High Risk - Significant irregularities detected 71-100: Critical - Major red flags, avoid purchase

Real-World Test Case: The User's Example

Vehicle: 15-year-old car Current Flags:

  • Same-day fail then pass (2 occurrences) → 6 points total
  • Brake pads, tyres, suspension items replaced

Current System Says: "Issues detected" Reality: This car is WELL MAINTAINED

New System Should Say:

  • Risk Score: 5-10 (Excellent)
  • Summary: "All checks look standard, nothing of concern"
  • Optional Info: "Vehicle shows good maintenance history with issues addressed promptly"

Implementation Priority

Phase 1 (Immediate - High Impact):

  1. ❌ Remove same_day_fail_pass flag entirely
  2. ❌ Remove advisory_escalation flag entirely
  3. 🔄 Fix dangerous_defect_ignored logic (only flag if not fixed for 2+ years)
  4. 📊 Recalibrate risk score thresholds

Phase 2 (Short term): 5. 🔄 Improve recurring_same_failure logic 6. 🔄 Improve mot_history_gap logic 7. ❌ Remove noise flags (seasonal bias, weekend patterns, etc.)

Phase 3 (Medium term): 8. ➕ Add new meaningful flags 9. 🎯 Add vehicle age awareness throughout system 10. 📝 Improve flag descriptions to be more accurate

Summary

Current System: Flags normal maintenance as concerning New System: Celebrates good maintenance, flags real fraud

Current Risk Assessment: 15-year-old well-maintained car = "Issues detected" New Risk Assessment: 15-year-old well-maintained car = "Excellent condition"

Focus: Cloning detection, odometer fraud, identity mismatches Ignore: Normal wear, maintenance, human behavior patterns