Skip to content
OSSF Scorecard

Ignore pip-compile annotation diffs #10

Sign in to view logs

Ignore pip-compile annotation diffs

Ignore pip-compile annotation diffs #10

Triggered via push January 14, 2026 02:33
@TechNaviiTechNavii
pushed 80ecdd6
main
Status Success
Total duration 14s
Artifacts –

ossf-scorecard.yml

on: push
OSSF Scorecard Analysis
9s
OSSF Scorecard Analysis
Track Score History
Track Score History
Fit to window
Zoom out
Zoom in

Annotations

999 warnings and 1 notice
OSSF Scorecard Analysis
No files were found with the provided path: scorecard-results.sarif. No artifacts will be uploaded.
test_qa_thinking_level_trimmed (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_qa_thinking_level_blank (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_qa_provider_invalid_characters (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_qa_provider_allows_slash (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_count_errors_all_failed_statuses (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_case_insensitive (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_missing_attempts_key (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_non_dict_summary (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_failed_statuses_constant (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_basic (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_malformed_json (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_status_normalization (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_from_empty_string (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_missing_status (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_empty_attempts (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_from_none (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_timezone_normalization (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_valid_iso_timestamp (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_none_timestamp_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_utc_timezone (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_z_suffix (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_invalid_timestamp_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_timezone (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_empty_string_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_extract_openai_style_tokens (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_mixed_style_tokens_openai_preferred (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_zero_values (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_from_none (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_from_empty_dict (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_partial_anthropic (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_anthropic_style_tokens (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_partial_openai (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_with_extra_fields (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_get_session_context_manager (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_get_session_auto_commit (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_get_session_rollback_on_exception (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_qa_stream_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_error_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_error_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_multiple_attempts (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_cleanup_after_disconnect (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_multiple_attempts (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_cleanup_after_disconnect (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_missing_run_id_closes_with_4404 (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_missing_run_id_closes_with_4404 (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_runs_list_run_id_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_health_status_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_leaderboard_runs_count_is_int (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_qa_runs_list_run_id_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_run_detail_nonexistent_returns_404 (tests.test_api_contracts.TestRunDetailEndpointContract) failed: tests.test_api_contracts.TestRunDetailEndpointContract#L0
junit.xml [took 0s]
test_run_detail_error_response_shape (tests.test_api_contracts.TestRunDetailEndpointContract) failed: tests.test_api_contracts.TestRunDetailEndpointContract#L0
junit.xml [took 0s]
test_404_responses_have_detail_field (tests.test_api_contracts.TestErrorResponseConsistency) failed: tests.test_api_contracts.TestErrorResponseConsistency#L0
junit.xml [took 0s]
test_422_responses_have_detail_field (tests.test_api_contracts.TestErrorResponseConsistency) failed: tests.test_api_contracts.TestErrorResponseConsistency#L0
junit.xml [took 0s]
test_qa_create_run_invalid_provider_format_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_empty_models_array_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_zero_samples_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_whitespace_only_model_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_empty_body_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_missing_models_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_qa_create_run_negative_max_tokens_returns_422 (tests.test_api_contracts.TestQACreateRunValidationContract) failed: tests.test_api_contracts.TestQACreateRunValidationContract#L0
junit.xml [took 0s]
test_incomplete_endpoint_missing_run_returns_404 (tests.test_api_contracts.TestRetryResumeEndpointsContract) failed: tests.test_api_contracts.TestRetryResumeEndpointsContract#L0
junit.xml [took 0s]
test_api_errors_endpoint_missing_run_returns_404 (tests.test_api_contracts.TestRetryResumeEndpointsContract) failed: tests.test_api_contracts.TestRetryResumeEndpointsContract#L0
junit.xml [took 0s]
test_qa_api_errors_endpoint_missing_run_returns_404 (tests.test_api_contracts.TestRetryResumeEndpointsContract) failed: tests.test_api_contracts.TestRetryResumeEndpointsContract#L0
junit.xml [took 0s]
test_qa_runs_list_run_entry_shape (tests.test_api_contracts.TestQARunsListEndpointContract) failed: tests.test_api_contracts.TestQARunsListEndpointContract#L0
junit.xml [took 0s]
test_qa_runs_list_with_limit_parameter (tests.test_api_contracts.TestQARunsListEndpointContract) failed: tests.test_api_contracts.TestQARunsListEndpointContract#L0
junit.xml [took 0s]
test_qa_runs_list_returns_200_with_runs_array (tests.test_api_contracts.TestQARunsListEndpointContract) failed: tests.test_api_contracts.TestQARunsListEndpointContract#L0
junit.xml [took 0s]
test_health_returns_200_with_status_ok (tests.test_api_contracts.TestHealthEndpointContract) failed: tests.test_api_contracts.TestHealthEndpointContract#L0
junit.xml [took 0s]
test_leaderboard_returns_200_with_models (tests.test_api_contracts.TestLeaderboardEndpointContract) failed: tests.test_api_contracts.TestLeaderboardEndpointContract#L0
junit.xml [took 0s]
test_leaderboard_model_entry_shape (tests.test_api_contracts.TestLeaderboardEndpointContract) failed: tests.test_api_contracts.TestLeaderboardEndpointContract#L0
junit.xml [took 0s]
test_qa_leaderboard_returns_200_with_models (tests.test_api_contracts.TestQALeaderboardEndpointContract) failed: tests.test_api_contracts.TestQALeaderboardEndpointContract#L0
junit.xml [took 0s]
test_qa_runs_list_limit_zero (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_empty_model_id_capabilities (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_runs_list_limit_zero (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_runs_list_limit_very_large (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_runs_list_limit_negative (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_run_id_with_special_characters (tests.test_api_contracts.TestEdgeCases) failed: tests.test_api_contracts.TestEdgeCases#L0
junit.xml [took 0s]
test_runs_list_with_limit_parameter (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_runs_list_returns_200_with_runs_array (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_runs_list_run_entry_shape (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_openapi_paths_include_core_endpoints (tests.test_api_contracts.TestOpenAPISchemaContract) failed: tests.test_api_contracts.TestOpenAPISchemaContract#L0
junit.xml [took 0s]
test_openapi_schema_accessible (tests.test_api_contracts.TestOpenAPISchemaContract) failed: tests.test_api_contracts.TestOpenAPISchemaContract#L0
junit.xml [took 0s]
test_openapi_schema_has_required_fields (tests.test_api_contracts.TestOpenAPISchemaContract) failed: tests.test_api_contracts.TestOpenAPISchemaContract#L0
junit.xml [took 0s]
test_openapi_schema_version (tests.test_api_contracts.TestOpenAPISchemaContract) failed: tests.test_api_contracts.TestOpenAPISchemaContract#L0
junit.xml [took 0s]
test_capabilities_error_response_shape (tests.test_api_contracts.TestModelsCapabilitiesEndpointContract) failed: tests.test_api_contracts.TestModelsCapabilitiesEndpointContract#L0
junit.xml [took 0s]
test_capabilities_requires_model_id (tests.test_api_contracts.TestModelsCapabilitiesEndpointContract) failed: tests.test_api_contracts.TestModelsCapabilitiesEndpointContract#L0
junit.xml [took 0s]
test_capabilities_unknown_model_returns_404 (tests.test_api_contracts.TestModelsCapabilitiesEndpointContract) failed: tests.test_api_contracts.TestModelsCapabilitiesEndpointContract#L0
junit.xml [took 0s]
test_capabilities_empty_model_id_returns_400 (tests.test_api_contracts.TestModelsCapabilitiesEndpointContract) failed: tests.test_api_contracts.TestModelsCapabilitiesEndpointContract#L0
junit.xml [took 0s]
test_capabilities_whitespace_model_id_returns_400 (tests.test_api_contracts.TestModelsCapabilitiesEndpointContract) failed: tests.test_api_contracts.TestModelsCapabilitiesEndpointContract#L0
junit.xml [took 0s]
test_qa_run_detail_nonexistent_returns_404 (tests.test_api_contracts.TestQARunDetailEndpointContract) failed: tests.test_api_contracts.TestQARunDetailEndpointContract#L0
junit.xml [took 0s]
test_qa_run_detail_error_response_shape (tests.test_api_contracts.TestQARunDetailEndpointContract) failed: tests.test_api_contracts.TestQARunDetailEndpointContract#L0
junit.xml [took 0s]
test_validation_error_response_shape (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_empty_body_returns_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_samples_out_of_range_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_negative_max_tokens_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_missing_models_returns_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_temperature_out_of_range_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_empty_models_array_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_invalid_model_name_format_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_invalid_provider_format_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_iter_pdf_streams_skips_invalid_streams (tests.test_dataset) failed: tests.test_dataset#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_non_json (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_fetch_model_metadata_parses_pricing (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_request_errors (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_switch_lmstudio_model_rejects_invalid_model_id (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_call_openrouter_requires_api_key (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_parses_pricing (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_tasks_unloads_lmstudio_models_after_completion (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_lmstudio_endpoint_fails_when_base_url_empty (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_non_json (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_list_lmstudio_models_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_uses_configured_openrouter_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_settings_do_not_require_openrouter_api_key (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_api_token_blank_becomes_none (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_harness_openrouter_base_url_default (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_no_fallback_to_default_lmstudio_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_skips_when_only_lmstudio_models (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_lmstudio_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_settings_defaults (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_switch_lmstudio_model_invokes_lms_cli (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_expert_questions_openrouter_call_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_no_fallback_to_default_openrouter_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_expert_questions_lmstudio_allows_not_loaded_state (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_expert_questions_lmstudio_call_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_api_token_trimmed (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_lmstudio_base_url_default (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_request_errors (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_expert_questions_judge_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_question_benchmark_unloads_lmstudio_models_after_completion (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_harness_openrouter_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_harness_lmstudio_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_call_openrouter_uses_configured_openrouter_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_resume_from_without_dir_fails (tests.test_cli_entrypoints.TestResumeRetryArguments) failed: tests.test_cli_entrypoints.TestResumeRetryArguments#L0
junit.xml [took 0s]
test_resume_incomplete_flag_parses (tests.test_cli_entrypoints.TestResumeRetryArguments) failed: tests.test_cli_entrypoints.TestResumeRetryArguments#L0
junit.xml [took 0s]
test_retry_api_errors_flag_parses (tests.test_cli_entrypoints.TestResumeRetryArguments) failed: tests.test_cli_entrypoints.TestResumeRetryArguments#L0
junit.xml [took 0s]
test_missing_required_args_exits_nonzero (tests.test_cli_entrypoints.TestExitCodes) failed: tests.test_cli_entrypoints.TestExitCodes#L0
junit.xml [took 0s]
test_invalid_task_exits_one (tests.test_cli_entrypoints.TestExitCodes) failed: tests.test_cli_entrypoints.TestExitCodes#L0
junit.xml [took 0s]
test_help_exits_zero (tests.test_cli_entrypoints.TestExitCodes) failed: tests.test_cli_entrypoints.TestExitCodes#L0
junit.xml [took 0s]
test_unicode_in_model_name (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_invalid_temperature_type_fails (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_negative_samples_value (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_empty_task_string_fails (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_large_samples_value (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_task_with_special_chars_fails (tests.test_cli_entrypoints.TestEdgeCases) failed: tests.test_cli_entrypoints.TestEdgeCases#L0
junit.xml [took 0s]
test_load_incomplete_with_prompt_no_response (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_base_level_converted_to_none (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_dry_run_exits_zero (tests.test_cli_entrypoints.TestDryRunMode) failed: tests.test_cli_entrypoints.TestDryRunMode#L0
junit.xml [took 0s]
test_summary_uses_thinking_level_applied_or_requested (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_dry_run_nonexistent_task_fails (tests.test_cli_entrypoints.TestDryRunMode) failed: tests.test_cli_entrypoints.TestDryRunMode#L0
junit.xml [took 0s]
test_ignores_completed_attempts_with_response_json (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_load_incomplete_with_thinking_level (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_dry_run_with_include_tests (tests.test_cli_entrypoints.TestDryRunMode) failed: tests.test_cli_entrypoints.TestDryRunMode#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_ignores_non_directory_files (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_attempt_dir_path_included (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_slashes_in_model (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_insufficient_parts (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_sample_number (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_slashes_in_thinking_level (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_handles_multiple_incomplete_attempts (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_dry_run_outputs_prompt (tests.test_cli_entrypoints.TestDryRunMode) failed: tests.test_cli_entrypoints.TestDryRunMode#L0
junit.xml [took 0s]
test_ignores_directories_without_prompt (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_ignores_attempts_in_summary (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_ignores_completed_attempts_with_response_txt (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_invalid_sample_number_defaults_to_zero (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_dry_run_output_stable (tests.test_cli_entrypoints.TestOutputStability) failed: tests.test_cli_entrypoints.TestOutputStability#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_case_insensitive_status_matching (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_load_from_summary_json (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_attempts_json_takes_precedence (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_load_from_attempts_json (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_load_from_summary_json (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_help_format_stable (tests.test_cli_entrypoints.TestOutputStability) failed: tests.test_cli_entrypoints.TestOutputStability#L0
junit.xml [took 0s]
test_scan_error_logs_for_rate_limit (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_does_not_classify_user_code_failures_as_api_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_scan_error_logs_for_empty_response (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_ignores_directories_without_error_log (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_does_not_classify_syntax_errors_as_api_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_scan_error_logs_for_provider_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_multiple_underscores_in_model (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_scan_error_logs_for_429_status (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_edge_case_base_level (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_ignores_non_directory_files (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_load_from_attempts_json (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_fetch_model_metadata_parses_pricing (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_non_json (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_request_errors (tests.test_harness_metadata) failed: tests.test_harness_metadata#L0
junit.xml [took 0s]
test_require_api_token_blocks_when_invalid (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_mutating_routes_are_protected_by_dependency (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_blocks_when_missing (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_allows_when_valid (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_allows_when_unset (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_database_list_runs_returns_list (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_database_init_creates_runs_directory (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_progress_manager_stream_integration (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_progress_manager_multiple_subscribers (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_filesystem_run_persistence (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_health_endpoint (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_cors_middleware_configured (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_root_redirects_to_ui (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_ui_mount_exists (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_get_session_context_manager (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_get_session_auto_commit (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_get_session_rollback_on_exception (tests.test_database_utils.TestGetSessionIntegration) failed: tests.test_database_utils.TestGetSessionIntegration#L0
junit.xml [took 0s]
test_extract_tokens_from_none (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_partial_anthropic (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_from_empty_dict (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_partial_openai (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_with_extra_fields (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_anthropic_style_tokens (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_mixed_style_tokens_openai_preferred (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_tokens_zero_values (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_extract_openai_style_tokens (tests.test_database_utils.TestExtractUsageTokens) failed: tests.test_database_utils.TestExtractUsageTokens#L0
junit.xml [took 0s]
test_parse_empty_string_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_none_timestamp_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_valid_iso_timestamp (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_invalid_timestamp_fallback (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_z_suffix (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_utc_timezone (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_timezone_normalization (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_parse_timestamp_with_timezone (tests.test_database_utils.TestParseTimestamp) failed: tests.test_database_utils.TestParseTimestamp#L0
junit.xml [took 0s]
test_count_errors_malformed_json (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_from_empty_string (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_from_none (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_all_failed_statuses (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_basic (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_case_insensitive (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_invalid_argument_fails (tests.test_cli_entrypoints.TestArgumentParsing) failed: tests.test_cli_entrypoints.TestArgumentParsing#L0
junit.xml [took 0s]
test_count_errors_missing_attempts_key (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_conflicting_no_flag_parsing (tests.test_cli_entrypoints.TestArgumentParsing) failed: tests.test_cli_entrypoints.TestArgumentParsing#L0
junit.xml [took 0s]
test_failed_statuses_constant (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_missing_status (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_no_args_shows_error_or_help (tests.test_cli_entrypoints.TestArgumentParsing) failed: tests.test_cli_entrypoints.TestArgumentParsing#L0
junit.xml [took 0s]
test_help_exits_zero (tests.test_cli_entrypoints.TestHelpOutput) failed: tests.test_cli_entrypoints.TestHelpOutput#L0
junit.xml [took 0s]
test_help_contains_diff_options (tests.test_cli_entrypoints.TestHelpOutput) failed: tests.test_cli_entrypoints.TestHelpOutput#L0
junit.xml [took 0s]
test_help_contains_required_arguments (tests.test_cli_entrypoints.TestHelpOutput) failed: tests.test_cli_entrypoints.TestHelpOutput#L0
junit.xml [took 0s]
test_count_errors_empty_attempts (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_count_errors_status_normalization (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_help_contains_thinking_options (tests.test_cli_entrypoints.TestHelpOutput) failed: tests.test_cli_entrypoints.TestHelpOutput#L0
junit.xml [took 0s]
test_count_errors_non_dict_summary (tests.test_database_utils.TestCountErrorsFromSummary) failed: tests.test_database_utils.TestCountErrorsFromSummary#L0
junit.xml [took 0s]
test_help_contains_description (tests.test_cli_entrypoints.TestHelpOutput) failed: tests.test_cli_entrypoints.TestHelpOutput#L0
junit.xml [took 0s]
test_qa_thinking_level_trimmed (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_qa_thinking_level_blank (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_direct_script_invocation_works (tests.test_cli_entrypoints.TestModuleInvocation) failed: tests.test_cli_entrypoints.TestModuleInvocation#L0
junit.xml [took 0s]
test_qa_provider_allows_slash (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_module_invocation_works (tests.test_cli_entrypoints.TestModuleInvocation) failed: tests.test_cli_entrypoints.TestModuleInvocation#L0
junit.xml [took 0s]
test_qa_provider_invalid_characters (tests.test_qa_router) failed: tests.test_qa_router#L0
junit.xml [took 0s]
test_root_redirects_to_ui (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_never_crashes_on_arbitrary_text (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_handles_prefix_only_lines (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_cors_middleware_configured (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_never_crashes_on_patch_like_input (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_ui_mount_exists (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_never_crashes_on_binary_like_input (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_health_endpoint (tests.test_api_endpoints) failed: tests.test_api_endpoints#L0
junit.xml [took 0s]
test_leading_spaces_in_hunk_header (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_multiple_files (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_absolute_path_component_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_very_long_lines (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_double_dots_in_component_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_trailing_newline_preservation (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_context_only_patch (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_path_traversal_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_only_deletions (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_symlink_escape_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_zero_length_hunk (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_unicode_content (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_valid_path_join (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_invalid_run_id_special_chars (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_valid_run_id_simple (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_backslash (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_double_dots (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_valid_run_id_timestamp_format (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_empty (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_forward_slash (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_path_traversal (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_relative_escape_rejected (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_path_outside_root_rejected (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_valid_path_within_root (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[%2e%2e%2f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\/] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..././] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%2f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%252f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\..\\..\\windows\\system32\\config\\sam] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_encoded_attacks_blocked[run%5c..%5c..%5cwindows] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[../../../etc/passwd] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%00] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%5c] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[....////] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[....//] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[.%00./] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_encoded_attacks_blocked[run%2f..%2f..%2fetc] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[../] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..;] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_error_is_value_error (tests.test_path_security.TestPathTraversalError) failed: tests.test_path_security.TestPathTraversalError#L0
junit.xml [took 0s]
test_error_does_not_leak_path (tests.test_path_security.TestPathTraversalError) failed: tests.test_path_security.TestPathTraversalError#L0
junit.xml [took 0s]
test_task_id_pattern_matches_run_id_pattern (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_empty_string (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_no_trailing_newline (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_new_file_mode (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_only_additions (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_crlf_line_endings (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_mixed_whitespace_indentation (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_only_whitespace (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_missing_file_headers (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_no_newline_at_end_marker (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_deleted_file_mode (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_partial_diff_no_hunks (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_index_line (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_malformed_hunk_counts (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_rename_headers (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_hunk_header_with_function_context (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_git_binary_patch_marker (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_never_crashes_on_binary_like_input (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_handles_various_hunk_headers (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_patch_like_input (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_handles_malformed_hunk_headers (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_arbitrary_text (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_handles_multiple_hunks (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_qa_stream_cleanup_after_disconnect (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_missing_run_id_closes_with_4404 (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_missing_run_id_closes_with_4404 (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_error_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_cleanup_after_disconnect (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_run_stream_multiple_attempts (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_multiple_attempts (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_qa_stream_error_event_shape (tests.test_websocket_streams) failed: tests.test_websocket_streams#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_non_json (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_id_pattern_rejects_invalid_chars (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_fetch_model_metadata_skips_when_only_lmstudio_models (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_id_pattern_allows_valid_chars (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_expert_questions_openrouter_call_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_invalid_task_id_empty (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_switch_lmstudio_model_invokes_lms_cli (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_invalid_task_id_path_traversal (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_server_settings_defaults (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_invalid_task_id_absolute_path (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_harness_lmstudio_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_no_fallback_to_default_openrouter_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_valid_task_id (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_list_lmstudio_models_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_validation_independent_of_machine_paths (tests.test_path_traversal_regression.TestDeterministicBehavior) failed: tests.test_path_traversal_regression.TestDeterministicBehavior#L0
junit.xml [took 0s]
test_switch_lmstudio_model_rejects_invalid_model_id (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_same_invalid_input_returns_same_error (tests.test_path_traversal_regression.TestDeterministicBehavior) failed: tests.test_path_traversal_regression.TestDeterministicBehavior#L0
junit.xml [took 0s]
test_call_openrouter_uses_configured_openrouter_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_qa_api_errors_endpoint_rejects_traversal[run/../escape] (tests.test_path_traversal_regression.TestQARouteTraversalRejection) failed: tests.test_path_traversal_regression.TestQARouteTraversalRejection#L0
junit.xml [took 0s]
test_fetch_model_metadata_parses_pricing (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_qa_api_errors_endpoint_rejects_traversal[..%2f] (tests.test_path_traversal_regression.TestQARouteTraversalRejection) failed: tests.test_path_traversal_regression.TestQARouteTraversalRejection#L0
junit.xml [took 0s]
test_fetch_model_metadata_uses_configured_openrouter_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_qa_api_errors_endpoint_rejects_traversal[../] (tests.test_path_traversal_regression.TestQARouteTraversalRejection) failed: tests.test_path_traversal_regression.TestQARouteTraversalRejection#L0
junit.xml [took 0s]
test_expert_questions_lmstudio_call_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_qa_api_errors_endpoint_rejects_traversal[../../../etc/passwd] (tests.test_path_traversal_regression.TestQARouteTraversalRejection) failed: tests.test_path_traversal_regression.TestQARouteTraversalRejection#L0
junit.xml [took 0s]
test_server_api_token_trimmed (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[run_id/../../escape] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_expert_questions_lmstudio_allows_not_loaded_state (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[%2e%2e%2f] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_harness_openrouter_base_url_default (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_runs_api_errors_endpoint_rejects_traversal[..%2f..%2f] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_server_lmstudio_base_url_default (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[/absolute/path] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_server_settings_do_not_require_openrouter_api_key (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[..%5c..%5cwindows] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[run/../../../etc/passwd] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[....//] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[valid_prefix/../escape] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_errors_endpoint_rejects_traversal[run/../escape] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_errors_endpoint_rejects_traversal[../../../secret] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[..%2f..%2fetc] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[../etc/passwd] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_rejects_traversal_run_id[..\\windows\\system32] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_errors_endpoint_rejects_traversal[../] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_runs_api_errors_endpoint_rejects_traversal[..\\] (tests.test_path_traversal_regression.TestPathTraversalRejection) failed: tests.test_path_traversal_regression.TestPathTraversalRejection#L0
junit.xml [took 0s]
test_qa_invalid_run_id_error_is_generic (tests.test_path_traversal_regression.TestErrorMessageSafety) failed: tests.test_path_traversal_regression.TestErrorMessageSafety#L0
junit.xml [took 0s]
test_invalid_run_id_error_is_generic (tests.test_path_traversal_regression.TestErrorMessageSafety) failed: tests.test_path_traversal_regression.TestErrorMessageSafety#L0
junit.xml [took 0s]
test_code_and_qa_routes_handle_traversal_consistently[../] (tests.test_path_traversal_regression.TestCrossRouteConsistency) failed: tests.test_path_traversal_regression.TestCrossRouteConsistency#L0
junit.xml [took 0s]
test_code_and_qa_routes_handle_traversal_consistently[%2e%2e%2f] (tests.test_path_traversal_regression.TestCrossRouteConsistency) failed: tests.test_path_traversal_regression.TestCrossRouteConsistency#L0
junit.xml [took 0s]
test_code_and_qa_routes_handle_traversal_consistently[..\\] (tests.test_path_traversal_regression.TestCrossRouteConsistency) failed: tests.test_path_traversal_regression.TestCrossRouteConsistency#L0
junit.xml [took 0s]
test_run_creation_uses_safe_paths (tests.test_path_traversal_regression.TestArtifactBoundaryEnforcement) failed: tests.test_path_traversal_regression.TestArtifactBoundaryEnforcement#L0
junit.xml [took 0s]
test_generated_run_ids_are_deterministic_format (tests.test_path_traversal_regression.TestArtifactBoundaryEnforcement) failed: tests.test_path_traversal_regression.TestArtifactBoundaryEnforcement#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[run-with-dashes] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[run_with_underscores] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[20240101_120000_a1b2c3] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[test-run-456] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[abc123] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_valid_run_id_format_not_rejected_as_traversal[run_123] (tests.test_path_traversal_regression.TestValidRunIdFormats) failed: tests.test_path_traversal_regression.TestValidRunIdFormats#L0
junit.xml [took 0s]
test_models_capabilities_requires_model_id (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_leaderboard_structure (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_openapi_schema_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_task_discovery_returns_list (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_leaderboard_endpoint_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_task_discovery_no_duplicates (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_server_health_check (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_task_discovery_ids_are_strings (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_models_capabilities_invalid_model_id (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_runs_listing_endpoint_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_no_fallback_to_default_lmstudio_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_tasks_unloads_lmstudio_models_after_completion (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_expert_questions_judge_uses_configured_base_url (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_harness_openrouter_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_fetch_model_metadata_handles_request_errors (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_api_token_blank_becomes_none (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_run_question_benchmark_unloads_lmstudio_models_after_completion (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_call_openrouter_requires_api_key (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_lmstudio_endpoint_fails_when_base_url_empty (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_server_lmstudio_base_url_configurable_via_env (tests.test_external_endpoint_config) failed: tests.test_external_endpoint_config#L0
junit.xml [took 0s]
test_dict_args_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_message_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_format_args_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_filter_always_returns_true (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_structlog_api_error_redacted (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_structlog_request_headers_redacted (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_harness_logger_redacts_secrets (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_list_values_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_sensitive_field_name_fully_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_non_string_values_unchanged (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_error_field_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_nested_dict_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_event_field_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_generic_api_key_pattern_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_anthropic_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_safe_content_not_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_bearer_token_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_openai_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_case_insensitive_patterns (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_authorization_header_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_openrouter_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_multiple_secrets_all_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_password_in_url_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_short_tokens_not_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_url_token_params_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_same_input_same_output (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_error_messages_with_secrets_redacted (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_exception_traceback_redacted (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_no_false_positives_on_common_words (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_filter_installed (tests.test_log_redaction.TestInstallStdlibRedaction) failed: tests.test_log_redaction.TestInstallStdlibRedaction#L0
junit.xml [took 0s]
test_no_duplicate_filters (tests.test_log_redaction.TestInstallStdlibRedaction) failed: tests.test_log_redaction.TestInstallStdlibRedaction#L0
junit.xml [took 0s]
test_harness_metadata_structure (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_valid_request_passes (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_runs_listing_with_limit (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_model_with_colon_allowed (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_main_index_assets_served_with_200 (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_provider_invalid_characters (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_main_index_assets_exist_on_disk (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_main_index_html_served (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_js_assets_have_correct_content_type (tests.test_ui_static_assets.TestAssetContentTypes) failed: tests.test_ui_static_assets.TestAssetContentTypes#L0
junit.xml [took 0s]
test_css_assets_have_correct_content_type (tests.test_ui_static_assets.TestAssetContentTypes) failed: tests.test_ui_static_assets.TestAssetContentTypes#L0
junit.xml [took 0s]
test_thinking_level_trimmed (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_style_css_referenced_by_both (tests.test_ui_static_assets.TestAssetConsistency) failed: tests.test_ui_static_assets.TestAssetConsistency#L0
junit.xml [took 0s]
test_invalid_model_name (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_shared_assets_match (tests.test_ui_static_assets.TestAssetConsistency) failed: tests.test_ui_static_assets.TestAssetConsistency#L0
junit.xml [took 0s]
test_provider_allows_slash (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_extract_asset_references_finds_scripts (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_allowlist_enforced (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_extract_asset_references_deduplicates (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_headers_on_redirect_responses (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_extract_asset_references_finds_stylesheets (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_headers_consistent_across_code_and_qa_routers (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_extract_asset_references_strips_query_strings (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_qa_index_assets_exist_on_disk (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_qa_index_html_served (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_qa_index_assets_served_with_200 (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_schema_creation_succeeds (tests.test_migrations_smoke.TestMainDatabaseSchema) failed: tests.test_migrations_smoke.TestMainDatabaseSchema#L0
junit.xml [took 0s]
test_foreign_key_constraint_exists (tests.test_migrations_smoke.TestMainDatabaseSchema) failed: tests.test_migrations_smoke.TestMainDatabaseSchema#L0
junit.xml [took 0s]
test_schema_is_idempotent (tests.test_migrations_smoke.TestMainDatabaseSchema) failed: tests.test_migrations_smoke.TestMainDatabaseSchema#L0
junit.xml [took 0s]
test_attempts_table_has_expected_columns (tests.test_migrations_smoke.TestMainDatabaseSchema) failed: tests.test_migrations_smoke.TestMainDatabaseSchema#L0
junit.xml [took 0s]
test_runs_table_has_expected_columns (tests.test_migrations_smoke.TestMainDatabaseSchema) failed: tests.test_migrations_smoke.TestMainDatabaseSchema#L0
junit.xml [took 0s]
test_alembic_env_exists (tests.test_migrations_smoke.TestMigrationDirectoryExists) failed: tests.test_migrations_smoke.TestMigrationDirectoryExists#L0
junit.xml [took 0s]
test_migrations_readme_exists (tests.test_migrations_smoke.TestMigrationDirectoryExists) failed: tests.test_migrations_smoke.TestMigrationDirectoryExists#L0
junit.xml [took 0s]
test_migrations_directory_exists (tests.test_migrations_smoke.TestMigrationDirectoryExists) failed: tests.test_migrations_smoke.TestMigrationDirectoryExists#L0
junit.xml [took 0s]
test_schema_creation_succeeds (tests.test_migrations_smoke.TestQADatabaseSchema) failed: tests.test_migrations_smoke.TestQADatabaseSchema#L0
junit.xml [took 0s]
test_qa_runs_table_has_expected_columns (tests.test_migrations_smoke.TestQADatabaseSchema) failed: tests.test_migrations_smoke.TestQADatabaseSchema#L0
junit.xml [took 0s]
test_qa_attempts_table_has_expected_columns (tests.test_migrations_smoke.TestQADatabaseSchema) failed: tests.test_migrations_smoke.TestQADatabaseSchema#L0
junit.xml [took 0s]
test_qa_foreign_key_constraint_exists (tests.test_migrations_smoke.TestQADatabaseSchema) failed: tests.test_migrations_smoke.TestQADatabaseSchema#L0
junit.xml [took 0s]
test_can_insert_and_query_run_record (tests.test_migrations_smoke.TestDatabaseCRUDOperations) failed: tests.test_migrations_smoke.TestDatabaseCRUDOperations#L0
junit.xml [took 0s]
test_can_insert_and_query_qa_run_record (tests.test_migrations_smoke.TestDatabaseCRUDOperations) failed: tests.test_migrations_smoke.TestDatabaseCRUDOperations#L0
junit.xml [took 0s]
test_can_insert_run_with_attempts (tests.test_migrations_smoke.TestDatabaseCRUDOperations) failed: tests.test_migrations_smoke.TestDatabaseCRUDOperations#L0
junit.xml [took 0s]
test_cascade_delete_removes_attempts (tests.test_migrations_smoke.TestDatabaseCRUDOperations) failed: tests.test_migrations_smoke.TestDatabaseCRUDOperations#L0
junit.xml [took 0s]
test_run_record_fields_match_columns (tests.test_migrations_smoke.TestModelSchemaConsistency) failed: tests.test_migrations_smoke.TestModelSchemaConsistency#L0
junit.xml [took 0s]
test_qa_attempt_record_fields_match_columns (tests.test_migrations_smoke.TestModelSchemaConsistency) failed: tests.test_migrations_smoke.TestModelSchemaConsistency#L0
junit.xml [took 0s]
test_qa_run_record_fields_match_columns (tests.test_migrations_smoke.TestModelSchemaConsistency) failed: tests.test_migrations_smoke.TestModelSchemaConsistency#L0
junit.xml [took 0s]
test_attempt_record_fields_match_columns (tests.test_migrations_smoke.TestModelSchemaConsistency) failed: tests.test_migrations_smoke.TestModelSchemaConsistency#L0
junit.xml [took 0s]
test_attempt_dir_path_included (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_ignores_attempts_in_summary (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_invalid_sample_number_defaults_to_zero (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_summary_uses_thinking_level_applied_or_requested (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_load_incomplete_with_prompt_no_response (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_handles_multiple_incomplete_attempts (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_slashes_in_model (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_insufficient_parts (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_ignores_directories_without_prompt (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_base_level_converted_to_none (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_load_incomplete_with_thinking_level (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_slashes_in_thinking_level (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_sample_number (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_headers_on_error_responses (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_csp_restricts_form_action (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_websocket_connections (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_restricts_base_uri (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_prevents_clickjacking (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_data_uris_for_images (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_inline_scripts_for_gui (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_self_origin (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_headers_on_qa_runs (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_leaderboard (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_qa_leaderboard (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_runs_list (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_permissions_policy_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_referrer_policy_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_csp_not_on_json_api (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_x_xss_protection_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_x_content_type_options_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_x_frame_options_not_on_json_api (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_all_security_headers_on_ui (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_csp_on_ui_route (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_x_frame_options_on_ui_route (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_security_headers_on_qa_ui (tests.test_security_headers.TestSecurityHeadersOnQAUI) failed: tests.test_security_headers.TestSecurityHeadersOnQAUI#L0
junit.xml [took 0s]
test_exclude_paths (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_default_csp_is_defined (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_custom_headers_override (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_default_headers_match_constants (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_custom_csp_override (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_iter_pdf_streams_skips_invalid_streams (tests.test_dataset) failed: tests.test_dataset#L0
junit.xml [took 0s]
test_task_discovery_ids_are_strings (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_task_discovery_no_duplicates (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_server_health_check (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_openapi_schema_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_leaderboard_structure (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_task_discovery_returns_list (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_runs_listing_endpoint_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_runs_listing_with_limit (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_models_capabilities_requires_model_id (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_leaderboard_endpoint_accessible (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_harness_metadata_structure (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_ignores_completed_attempts_with_response_txt (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_models_capabilities_invalid_model_id (tests.test_critical_paths) failed: tests.test_critical_paths#L0
junit.xml [took 0s]
test_ignores_completed_attempts_with_response_json (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_shared_assets_match (tests.test_ui_static_assets.TestAssetConsistency) failed: tests.test_ui_static_assets.TestAssetConsistency#L0
junit.xml [took 0s]
test_ignores_non_directory_files (tests.test_harness_retry_resume.TestLoadIncompleteAttempts) failed: tests.test_harness_retry_resume.TestLoadIncompleteAttempts#L0
junit.xml [took 0s]
test_style_css_referenced_by_both (tests.test_ui_static_assets.TestAssetConsistency) failed: tests.test_ui_static_assets.TestAssetConsistency#L0
junit.xml [took 0s]
test_case_insensitive_status_matching (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_css_assets_have_correct_content_type (tests.test_ui_static_assets.TestAssetContentTypes) failed: tests.test_ui_static_assets.TestAssetContentTypes#L0
junit.xml [took 0s]
test_load_from_attempts_json (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_js_assets_have_correct_content_type (tests.test_ui_static_assets.TestAssetContentTypes) failed: tests.test_ui_static_assets.TestAssetContentTypes#L0
junit.xml [took 0s]
test_load_from_summary_json (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_qa_index_html_served (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_attempts_json_takes_precedence (tests.test_harness_retry_resume.TestLoadFailedAttempts) failed: tests.test_harness_retry_resume.TestLoadFailedAttempts#L0
junit.xml [took 0s]
test_ignores_non_directory_files (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_ignores_directories_without_error_log (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_qa_index_assets_exist_on_disk (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_parse_directory_name_edge_case_base_level (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_qa_index_assets_served_with_200 (tests.test_ui_static_assets.TestQAIndexAssets) failed: tests.test_ui_static_assets.TestQAIndexAssets#L0
junit.xml [took 0s]
test_main_index_html_served (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_scan_error_logs_for_429_status (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_main_index_assets_served_with_200 (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_load_from_attempts_json (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_main_index_assets_exist_on_disk (tests.test_ui_static_assets.TestMainIndexAssets) failed: tests.test_ui_static_assets.TestMainIndexAssets#L0
junit.xml [took 0s]
test_scan_error_logs_for_rate_limit (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_scan_error_logs_for_empty_response (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_scan_error_logs_for_provider_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_parse_directory_name_with_multiple_underscores_in_model (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_load_from_summary_json (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_does_not_classify_user_code_failures_as_api_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_empty_run_directory (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_does_not_classify_syntax_errors_as_api_error (tests.test_harness_retry_resume.TestLoadApiErrorAttempts) failed: tests.test_harness_retry_resume.TestLoadApiErrorAttempts#L0
junit.xml [took 0s]
test_api_token_trimmed (tests.test_server_config) failed: tests.test_server_config#L0
junit.xml [took 0s]
test_settings_defaults (tests.test_server_config) failed: tests.test_server_config#L0
junit.xml [took 0s]
test_api_token_blank_becomes_none (tests.test_server_config) failed: tests.test_server_config#L0
junit.xml [took 0s]
test_filesystem_run_persistence (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_database_init_creates_runs_directory (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_progress_manager_multiple_subscribers (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_progress_manager_stream_integration (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_database_list_runs_returns_list (tests.test_integration) failed: tests.test_integration#L0
junit.xml [took 0s]
test_rate_limit_cleanup_old_entries (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_retry_after_calculation (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_head_requests_bypass_size_limit (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_missing_content_length_header (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_options_requests_bypass_size_limit (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_invalid_content_length_header (tests.test_request_limits.TestEdgeCases) failed: tests.test_request_limits.TestEdgeCases#L0
junit.xml [took 0s]
test_factory_function (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_successful_responses_include_rate_limit_headers (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_x_forwarded_for_header (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_all_requests_limited_when_mutating_only_false (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_get_requests_not_limited_when_mutating_only (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_429_includes_retry_after_header (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_disabled_by_default (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_delete_requests_are_limited (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_429_includes_rate_limit_headers (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_env_var_configuration (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_over_limit_returns_429 (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_default_values (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_enabled_via_env_var (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_x_real_ip_header (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_exclude_paths (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_extract_asset_references_deduplicates (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_extract_asset_references_strips_query_strings (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_extract_asset_references_finds_scripts (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_extract_asset_references_finds_stylesheets (tests.test_ui_static_assets.TestAssetExtraction) failed: tests.test_ui_static_assets.TestAssetExtraction#L0
junit.xml [took 0s]
test_delete_leaderboard_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_runs_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_retry_api_errors_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_resume_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_retry_single_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_qa_post_retry_single_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_qa_post_runs_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_bearer_case_insensitivity (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_basic_auth_rejected (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_token_with_extra_whitespace (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_empty_bearer_token (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_no_space_after_bearer (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_delete_leaderboard_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_post_runs_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_runs_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_post_retry_single_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_retry_api_errors_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_resume_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_retry_single_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_retry_single_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_delete_leaderboard_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_post_runs_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_post_runs_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_post_retry_api_errors_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_post_resume_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_post_retry_single_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_post_retry_single_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_under_limit_allowed (tests.test_request_limits.TestRateLimitMiddleware) failed: tests.test_request_limits.TestRateLimitMiddleware#L0
junit.xml [took 0s]
test_qa_post_runs_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_size_limit_on_runs_endpoint (tests.test_request_limits.TestIntegrationWithRealApp) failed: tests.test_request_limits.TestIntegrationWithRealApp#L0
junit.xml [took 0s]
test_post_retry_api_errors_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_rate_limit_disabled_by_default (tests.test_request_limits.TestIntegrationWithRealApp) failed: tests.test_request_limits.TestIntegrationWithRealApp#L0
junit.xml [took 0s]
test_post_runs_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_delete_leaderboard_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_post_resume_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_router_lmstudio_has_timeout (tests.test_request_limits.TestTimeoutEnforcement) failed: tests.test_request_limits.TestTimeoutEnforcement#L0
junit.xml [took 0s]
test_post_retry_single_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_harness_requests_have_timeouts (tests.test_request_limits.TestTimeoutEnforcement) failed: tests.test_request_limits.TestTimeoutEnforcement#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_both_limits_enforced (tests.test_request_limits.TestCombinedMiddleware) failed: tests.test_request_limits.TestCombinedMiddleware#L0
junit.xml [took 0s]
test_model_capabilities_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_size_limit_before_rate_limit (tests.test_request_limits.TestCombinedMiddleware) failed: tests.test_request_limits.TestCombinedMiddleware#L0
junit.xml [took 0s]
test_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_content_length_header_triggers_early_rejection (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_qa_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_factory_function (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_get_requests_not_checked (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_exclude_paths (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_small_payload_allowed (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_env_var_invalid_value (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_large_payload_rejected_with_413 (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_env_var_configuration (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_default_max_size (tests.test_request_limits.TestRequestSizeLimitMiddleware) failed: tests.test_request_limits.TestRequestSizeLimitMiddleware#L0
junit.xml [took 0s]
test_security_headers_on_qa_ui (tests.test_security_headers.TestSecurityHeadersOnQAUI) failed: tests.test_security_headers.TestSecurityHeadersOnQAUI#L0
junit.xml [took 0s]
test_x_content_type_options_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_x_xss_protection_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_csp_not_on_json_api (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_permissions_policy_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_referrer_policy_present (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_x_frame_options_not_on_json_api (tests.test_security_headers.TestSecurityHeadersOnHealth) failed: tests.test_security_headers.TestSecurityHeadersOnHealth#L0
junit.xml [took 0s]
test_exclude_paths (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_default_headers_match_constants (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_custom_headers_override (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_custom_csp_override (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_default_csp_is_defined (tests.test_security_headers.TestSecurityHeadersMiddlewareUnit) failed: tests.test_security_headers.TestSecurityHeadersMiddlewareUnit#L0
junit.xml [took 0s]
test_all_security_headers_on_ui (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_x_frame_options_on_ui_route (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_csp_on_ui_route (tests.test_security_headers.TestSecurityHeadersOnUI) failed: tests.test_security_headers.TestSecurityHeadersOnUI#L0
junit.xml [took 0s]
test_headers_on_leaderboard (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_runs_list (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_qa_leaderboard (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_headers_on_qa_runs (tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints) failed: tests.test_security_headers.TestSecurityHeadersOnAPIEndpoints#L0
junit.xml [took 0s]
test_csp_allows_websocket_connections (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_inline_scripts_for_gui (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_data_uris_for_images (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_restricts_form_action (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_prevents_clickjacking (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_allows_self_origin (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_csp_restricts_base_uri (tests.test_security_headers.TestCSPDirectives) failed: tests.test_security_headers.TestCSPDirectives#L0
junit.xml [took 0s]
test_headers_on_redirect_responses (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_headers_consistent_across_code_and_qa_routers (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_headers_on_error_responses (tests.test_security_headers.TestSecurityHeadersConsistency) failed: tests.test_security_headers.TestSecurityHeadersConsistency#L0
junit.xml [took 0s]
test_extract_single_line_answer_strips_answer_prefix (harness.tests.test_expert_questions_answer_extraction) failed: harness.tests.test_expert_questions_answer_extraction#L0
junit.xml [took 0s]
test_post_retry_api_errors_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_post_resume_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_delete_leaderboard_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_health_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_run_incomplete_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_health_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_run_incomplete_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_scripts_define_repo_root_from_script_dir (tests.test_validator_script_portability.TestScriptPathResolution) failed: tests.test_validator_script_portability.TestScriptPathResolution#L0
junit.xml [took 0s]
test_scripts_cd_to_repo_root (tests.test_validator_script_portability.TestScriptPathResolution) failed: tests.test_validator_script_portability.TestScriptPathResolution#L0
junit.xml [took 0s]
test_typecheck_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[lint.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[test.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[typecheck.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_check_python_version_script_is_sourced (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_lint_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_test_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[typecheck.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[test.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[format.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[lint.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[check-deps.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_lint_from_nested_directory (tests.test_validator_script_portability.TestCrossDirectoryExecution) failed: tests.test_validator_script_portability.TestCrossDirectoryExecution#L0
junit.xml [took 0s]
test_post_runs_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_lint_from_tmp (tests.test_validator_script_portability.TestCrossDirectoryExecution) failed: tests.test_validator_script_portability.TestCrossDirectoryExecution#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_invalid_task_id_path_traversal (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_qa_post_runs_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_invalid_task_id_absolute_path (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_post_retry_single_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_invalid_task_id_empty (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_qa_post_retry_single_no_token_required (tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsNoTokenRequired#L0
junit.xml [took 0s]
test_valid_task_id (tests.test_path_security.TestValidateTaskId) failed: tests.test_path_security.TestValidateTaskId#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_error_is_value_error (tests.test_path_security.TestPathTraversalError) failed: tests.test_path_security.TestPathTraversalError#L0
junit.xml [took 0s]
test_post_retry_single_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_resume_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_post_runs_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_error_does_not_leak_path (tests.test_path_security.TestPathTraversalError) failed: tests.test_path_security.TestPathTraversalError#L0
junit.xml [took 0s]
test_qa_post_runs_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_run_id_pattern_allows_valid_chars (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_delete_leaderboard_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_task_id_pattern_matches_run_id_pattern (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_run_id_pattern_rejects_invalid_chars (tests.test_path_security.TestPatternRegex) failed: tests.test_path_security.TestPatternRegex#L0
junit.xml [took 0s]
test_qa_post_retry_single_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_path_traversal_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_post_retry_api_errors_correct_token (tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsCorrectToken#L0
junit.xml [took 0s]
test_valid_path_join (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_post_resume_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_absolute_path_component_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_qa_post_retry_single_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_symlink_escape_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_double_dots_in_component_rejected (tests.test_path_security.TestSafePathJoin) failed: tests.test_path_security.TestSafePathJoin#L0
junit.xml [took 0s]
test_post_runs_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_valid_path_within_root (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_post_retry_single_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_relative_escape_rejected (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_post_retry_api_errors_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_path_outside_root_rejected (tests.test_path_security.TestValidatePathWithinRoot) failed: tests.test_path_security.TestValidatePathWithinRoot#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_invalid_run_id_double_dots (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_qa_post_runs_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_delete_leaderboard_missing_token (tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsMissingToken#L0
junit.xml [took 0s]
test_qa_delete_leaderboard_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_post_retry_single_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_retry_single_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_retry_api_errors_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_post_retry_api_errors_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_resume_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_post_runs_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_qa_post_runs_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_delete_leaderboard_invalid_token (tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken) failed: tests.test_auth_permission_matrix.TestMutatingEndpointsInvalidToken#L0
junit.xml [took 0s]
test_empty_bearer_token (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_bearer_case_insensitivity (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_no_space_after_bearer (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_basic_auth_rejected (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_token_with_extra_whitespace (tests.test_auth_permission_matrix.TestAuthTokenEdgeCases) failed: tests.test_auth_permission_matrix.TestAuthTokenEdgeCases#L0
junit.xml [took 0s]
test_qa_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_health_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_run_incomplete_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithoutToken#L0
junit.xml [took 0s]
test_qa_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_health_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_invalid_run_id_path_traversal (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_backslash (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_empty (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_forward_slash (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_invalid_run_id_special_chars (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_valid_run_id_timestamp_format (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_valid_run_id_simple (tests.test_path_security.TestValidateRunId) failed: tests.test_path_security.TestValidateRunId#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..;] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%5c] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\..\\..\\windows\\system32\\config\\sam] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[.%00./] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%252f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_encoded_attacks_blocked[run%5c..%5c..%5cwindows] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..\\/] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[%2e%2e%2f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%2f] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[....////] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..././] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_encoded_attacks_blocked[run%2f..%2f..%2fetc] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[../] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[....//] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[../../../etc/passwd] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_common_traversal_attacks_blocked[..%00] (tests.test_path_security.TestRealWorldAttacks) failed: tests.test_path_security.TestRealWorldAttacks#L0
junit.xml [took 0s]
test_bounded_rapid_subscribe_unsubscribe (tests.test_progress_stress.TestBoundedStress) failed: tests.test_progress_stress.TestBoundedStress#L0
junit.xml [took 0s]
test_bounded_many_events_single_subscriber (tests.test_progress_stress.TestBoundedStress) failed: tests.test_progress_stress.TestBoundedStress#L0
junit.xml [took 0s]
test_cleanup_after_complete_and_unsubscribe (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_many_runs_no_memory_growth (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_queue_garbage_collected_after_unsubscribe (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_no_cleanup_with_active_subscribers (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_subscribe_to_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_unsubscribe_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_unsubscribe_wrong_queue (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_double_unsubscribe (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_rapid_complete_fail_race (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_get_run_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_publish_to_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_many_subscribers_receive_all_events (tests.test_progress_stress.TestManySubscribers) failed: tests.test_progress_stress.TestManySubscribers#L0
junit.xml [took 0s]
test_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_subscriber_churn_during_publishing (tests.test_progress_stress.TestManySubscribers) failed: tests.test_progress_stress.TestManySubscribers#L0
junit.xml [took 0s]
test_qa_list_runs_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_thread_safety (tests.test_progress_stress.TestQAProgressManagerStress) failed: tests.test_progress_stress.TestQAProgressManagerStress#L0
junit.xml [took 0s]
test_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_qa_many_subscribers (tests.test_progress_stress.TestQAProgressManagerStress) failed: tests.test_progress_stress.TestQAProgressManagerStress#L0
junit.xml [took 0s]
test_late_subscriber_gets_history (tests.test_progress_stress.TestEventOrdering) failed: tests.test_progress_stress.TestEventOrdering#L0
junit.xml [took 0s]
test_qa_run_api_errors_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_strict_ordering_under_load (tests.test_progress_stress.TestEventOrdering) failed: tests.test_progress_stress.TestEventOrdering#L0
junit.xml [took 0s]
test_qa_leaderboard_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_concurrent_thread_publishing (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_run_incomplete_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_thread_pool_publish_stress (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_model_capabilities_accessible (tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken) failed: tests.test_auth_permission_matrix.TestReadOnlyEndpointsAccessibleWithToken#L0
junit.xml [took 0s]
test_thread_publish_with_subscriber_churn (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_secure_run_handles_nonexistent_command (harness.tests.test_secure_execution) failed: harness.tests.test_secure_execution#L0
junit.xml [took 0s]
test_extract_single_line_answer_strips_answer_prefix (harness.tests.test_expert_questions_answer_extraction) failed: harness.tests.test_expert_questions_answer_extraction#L0
junit.xml [took 0s]
test_secure_run_limits (harness.tests.test_secure_execution) failed: harness.tests.test_secure_execution#L0
junit.xml [took 0s]
test_secure_run_limits (harness.tests.test_secure_execution) failed: harness.tests.test_secure_execution#L0
junit.xml [took 0s]
test_secure_run_handles_nonexistent_command (harness.tests.test_secure_execution) failed: harness.tests.test_secure_execution#L0
junit.xml [took 0s]
test_provider_invalid_characters (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_runs_list_returns_200_with_runs_array (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_valid_request_passes (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_runs_list_with_limit_parameter (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_thinking_level_trimmed (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_allowlist_enforced (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_model_with_colon_allowed (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_invalid_model_name (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_provider_allows_slash (tests.test_validators) failed: tests.test_validators#L0
junit.xml [took 0s]
test_require_api_token_blocks_when_invalid (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_blocks_when_missing (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_mutating_routes_are_protected_by_dependency (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_allows_when_unset (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_require_api_token_allows_when_valid (tests.test_auth) failed: tests.test_auth#L0
junit.xml [took 0s]
test_handles_prefix_only_lines (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_patch_like_input (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_arbitrary_text (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_binary_like_input (tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz) failed: tests.test_harness_patch_parser_fuzz.TestIsProbablyValidPatchFuzz#L0
junit.xml [took 0s]
test_very_long_lines (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_crlf_line_endings (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_rename_headers (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_only_additions (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_only_whitespace (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_unicode_content (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_context_only_patch (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_git_binary_patch_marker (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_partial_diff_no_hunks (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_no_newline_at_end_marker (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_zero_length_hunk (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_missing_file_headers (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_new_file_mode (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_leading_spaces_in_hunk_header (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_empty_string (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_mixed_whitespace_indentation (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_deleted_file_mode (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_only_deletions (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_index_line (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_hunk_header_with_function_context (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_multiple_files (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_trailing_newline_preservation (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_no_trailing_newline (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_malformed_hunk_counts (tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases) failed: tests.test_harness_patch_parser_fuzz.TestPatchParserEdgeCases#L0
junit.xml [took 0s]
test_runs_list_run_entry_shape (tests.test_api_contracts.TestRunsListEndpointContract) failed: tests.test_api_contracts.TestRunsListEndpointContract#L0
junit.xml [took 0s]
test_qa_run_detail_nonexistent_returns_404 (tests.test_api_contracts.TestQARunDetailEndpointContract) failed: tests.test_api_contracts.TestQARunDetailEndpointContract#L0
junit.xml [took 0s]
test_qa_run_detail_error_response_shape (tests.test_api_contracts.TestQARunDetailEndpointContract) failed: tests.test_api_contracts.TestQARunDetailEndpointContract#L0
junit.xml [took 0s]
test_422_responses_have_detail_field (tests.test_api_contracts.TestErrorResponseConsistency) failed: tests.test_api_contracts.TestErrorResponseConsistency#L0
junit.xml [took 0s]
test_404_responses_have_detail_field (tests.test_api_contracts.TestErrorResponseConsistency) failed: tests.test_api_contracts.TestErrorResponseConsistency#L0
junit.xml [took 0s]
test_create_run_empty_models_array_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_temperature_out_of_range_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_invalid_model_name_format_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_negative_max_tokens_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_samples_out_of_range_returns_400_or_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_invalid_provider_format_returns_400 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_validation_error_response_shape (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_missing_models_returns_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_create_run_empty_body_returns_422 (tests.test_api_contracts.TestCreateRunValidationContract) failed: tests.test_api_contracts.TestCreateRunValidationContract#L0
junit.xml [took 0s]
test_run_detail_error_response_shape (tests.test_api_contracts.TestRunDetailEndpointContract) failed: tests.test_api_contracts.TestRunDetailEndpointContract#L0
junit.xml [took 0s]
test_run_detail_nonexistent_returns_404 (tests.test_api_contracts.TestRunDetailEndpointContract) failed: tests.test_api_contracts.TestRunDetailEndpointContract#L0
junit.xml [took 0s]
test_runs_list_run_id_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_qa_runs_list_run_id_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_leaderboard_runs_count_is_int (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_health_status_is_string (tests.test_api_contracts.TestResponseTypeValidation) failed: tests.test_api_contracts.TestResponseTypeValidation#L0
junit.xml [took 0s]
test_leaderboard_model_entry_shape (tests.test_api_contracts.TestLeaderboardEndpointContract) failed: tests.test_api_contracts.TestLeaderboardEndpointContract#L0
junit.xml [took 0s]
test_leaderboard_returns_200_with_models (tests.test_api_contracts.TestLeaderboardEndpointContract) failed: tests.test_api_contracts.TestLeaderboardEndpointContract#L0
junit.xml [took 0s]
test_handles_multiple_hunks (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_patch_like_input (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_binary_like_input (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_handles_various_hunk_headers (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_handles_malformed_hunk_headers (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_never_crashes_on_arbitrary_text (tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz) failed: tests.test_harness_patch_parser_fuzz.TestNormalizePatchFormatFuzz#L0
junit.xml [took 0s]
test_bounded_rapid_subscribe_unsubscribe (tests.test_progress_stress.TestBoundedStress) failed: tests.test_progress_stress.TestBoundedStress#L0
junit.xml [took 0s]
test_bounded_many_events_single_subscriber (tests.test_progress_stress.TestBoundedStress) failed: tests.test_progress_stress.TestBoundedStress#L0
junit.xml [took 0s]
test_qa_many_subscribers (tests.test_progress_stress.TestQAProgressManagerStress) failed: tests.test_progress_stress.TestQAProgressManagerStress#L0
junit.xml [took 0s]
test_qa_thread_safety (tests.test_progress_stress.TestQAProgressManagerStress) failed: tests.test_progress_stress.TestQAProgressManagerStress#L0
junit.xml [took 0s]
test_thread_publish_with_subscriber_churn (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_thread_pool_publish_stress (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_concurrent_thread_publishing (tests.test_progress_stress.TestCrossThreadPublishing) failed: tests.test_progress_stress.TestCrossThreadPublishing#L0
junit.xml [took 0s]
test_double_unsubscribe (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_unsubscribe_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_subscribe_to_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_publish_to_nonexistent_run (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_rapid_complete_fail_race (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_unsubscribe_wrong_queue (tests.test_progress_stress.TestEdgeCases) failed: tests.test_progress_stress.TestEdgeCases#L0
junit.xml [took 0s]
test_no_cleanup_with_active_subscribers (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_many_runs_no_memory_growth (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_cleanup_after_complete_and_unsubscribe (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_queue_garbage_collected_after_unsubscribe (tests.test_progress_stress.TestMemoryCleanup) failed: tests.test_progress_stress.TestMemoryCleanup#L0
junit.xml [took 0s]
test_late_subscriber_gets_history (tests.test_progress_stress.TestEventOrdering) failed: tests.test_progress_stress.TestEventOrdering#L0
junit.xml [took 0s]
test_strict_ordering_under_load (tests.test_progress_stress.TestEventOrdering) failed: tests.test_progress_stress.TestEventOrdering#L0
junit.xml [took 0s]
test_many_subscribers_receive_all_events (tests.test_progress_stress.TestManySubscribers) failed: tests.test_progress_stress.TestManySubscribers#L0
junit.xml [took 0s]
test_subscriber_churn_during_publishing (tests.test_progress_stress.TestManySubscribers) failed: tests.test_progress_stress.TestManySubscribers#L0
junit.xml [took 0s]
test_progress_manager_is_thread_safe (tests.test_progress) failed: tests.test_progress#L0
junit.xml [took 0s]
test_qa_progress_manager_is_thread_safe (tests.test_progress) failed: tests.test_progress#L0
junit.xml [took 0s]
test_scripts_define_repo_root_from_script_dir (tests.test_validator_script_portability.TestScriptPathResolution) failed: tests.test_validator_script_portability.TestScriptPathResolution#L0
junit.xml [took 0s]
test_scripts_cd_to_repo_root (tests.test_validator_script_portability.TestScriptPathResolution) failed: tests.test_validator_script_portability.TestScriptPathResolution#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[typecheck.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[test.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_test_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_typecheck_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[lint.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[format.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[check-deps.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_validator_script_resolves_repo_root_dynamically[lint.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[test.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_check_python_version_script_is_sourced (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_script_uses_script_dir_for_path_resolution[typecheck.sh] (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_lint_script_changes_to_repo_root (tests.test_validator_script_portability.TestValidatorScriptPortability) failed: tests.test_validator_script_portability.TestValidatorScriptPortability#L0
junit.xml [took 0s]
test_lint_from_nested_directory (tests.test_validator_script_portability.TestCrossDirectoryExecution) failed: tests.test_validator_script_portability.TestCrossDirectoryExecution#L0
junit.xml [took 0s]
test_lint_from_tmp (tests.test_validator_script_portability.TestCrossDirectoryExecution) failed: tests.test_validator_script_portability.TestCrossDirectoryExecution#L0
junit.xml [took 0s]
test_openai_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_anthropic_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_url_token_params_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_safe_content_not_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_openrouter_key_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_authorization_header_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_multiple_secrets_all_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_generic_api_key_pattern_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_short_tokens_not_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_bearer_token_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_case_insensitive_patterns (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_password_in_url_redacted (tests.test_log_redaction.TestRedactString) failed: tests.test_log_redaction.TestRedactString#L0
junit.xml [took 0s]
test_exception_traceback_redacted (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_no_false_positives_on_common_words (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_same_input_same_output (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_error_messages_with_secrets_redacted (tests.test_log_redaction.TestRedactionDeterminism) failed: tests.test_log_redaction.TestRedactionDeterminism#L0
junit.xml [took 0s]
test_format_args_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_message_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_filter_always_returns_true (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_dict_args_redacted (tests.test_log_redaction.TestSecretRedactionFilter) failed: tests.test_log_redaction.TestSecretRedactionFilter#L0
junit.xml [took 0s]
test_harness_logger_redacts_secrets (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_structlog_request_headers_redacted (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_structlog_api_error_redacted (tests.test_log_redaction.TestRepresentativeLoggingPaths) failed: tests.test_log_redaction.TestRepresentativeLoggingPaths#L0
junit.xml [took 0s]
test_list_values_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_sensitive_field_name_fully_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_error_field_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_non_string_values_unchanged (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_nested_dict_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_event_field_redacted (tests.test_log_redaction.TestRedactSecretsProcessor) failed: tests.test_log_redaction.TestRedactSecretsProcessor#L0
junit.xml [took 0s]
test_no_duplicate_filters (tests.test_log_redaction.TestInstallStdlibRedaction) failed: tests.test_log_redaction.TestInstallStdlibRedaction#L0
junit.xml [took 0s]
test_filter_installed (tests.test_log_redaction.TestInstallStdlibRedaction) failed: tests.test_log_redaction.TestInstallStdlibRedaction#L0
junit.xml [took 0s]
576 tests found
There are 576 tests, see "Raw output" for the full list of tests.