Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions testing-v2/IMPROVEMENTS-LOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -1118,3 +1118,32 @@ After completing the iteration successfully, user provided GitHub samples showin
- Throughput & scaling (5 rules)
- Global distribution (6 rules)
- Monitoring & diagnostics (5 rules)

#### 2026-03-31: iteration-001-python - Gaming Leaderboard (Python) [skills loaded]

- **Scenario**: gaming-leaderboard
- **Iteration**: iteration-001-python
- **Skills loaded**: Yes
- **Result**: FAILED -- 21/94 tests passed (22.3%)
- **Score**: 1/10

**Results by Category**:
- api_contract: 10 passed, 35 failed, 0 skipped
- build_startup: 2 passed, 0 failed, 0 skipped
- cosmos_infrastructure: 3 passed, 9 failed, 1 skipped
- data_integrity: 4 passed, 1 failed, 0 skipped
- robustness: 4 passed, 27 failed, 0 skipped

**Issues Encountered**:
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGetPlayer::test_get_existing_player** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGetPlayer::test_get_player_has_required_fields** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGetPlayer::test_get_player_stats_updated_after_scores** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestSubmitScore::test_submit_score_returns_201** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestSubmitScore::test_submit_score_response_has_required_fields** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestSubmitScore::test_submit_score_returns_correct_data** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGlobalLeaderboard::test_global_leaderboard_returns_200** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGlobalLeaderboard::test_global_leaderboard_returns_array** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGlobalLeaderboard::test_global_leaderboard_entries_have_required_fields** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err
1. **testing-v2.scenarios.gaming-leaderboard.tests.test_api_contract.TestGlobalLeaderboard::test_global_leaderboard_sorted_descending** -- failed on setup with "AssertionError: Failed to submit score for player-001: 500 Internal Server Err

**Test Results**: 21 passed, 73 failed out of 94

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@echo off
uvicorn main:app --host 0.0.0.0 --port 8000 > "D:\a\cosmosdb-agent-kit\cosmosdb-agent-kit\testing-v2\scenarios\gaming-leaderboard\iterations\iteration-001-python\app-output.log" 2> "D:\a\cosmosdb-agent-kit\cosmosdb-agent-kit\testing-v2\scenarios\gaming-leaderboard\iterations\iteration-001-python\app-error.log"
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"succeeded": true,
"stderr_tail": "",
"build_command": "pip install -r requirements.txt",
"stdout_tail": "Downloading multidict-6.7.1-cp312-cp312-win_amd64.whl (46 kB)\nDownloading yarl-1.23.0-cp312-cp312-win_amd64.whl (87 kB)\nDownloading aiohappyeyeballs-2.6.1-py3-none-any.whl (15 kB)\nDownloading aiosignal-1.4.0-py3-none-any.whl (7.5 kB)\nDownloading annotated_doc-0.0.4-py3-none-any.whl (5.3 kB)\nDownloading attrs-26.1.0-py3-none-any.whl (67 kB)\nDownloading azure_core-1.39.0-py3-none-any.whl (218 kB)\nDownloading frozenlist-1.8.0-cp312-cp312-win_amd64.whl (44 kB)\nDownloading h11-0.16.0-py3-none-any.whl (37 kB)\nDownloading idna-3.11-py3-none-any.whl (71 kB)\nDownloading propcache-0.4.1-cp312-cp312-win_amd64.whl (41 kB)\nDownloading pydantic-2.12.5-py3-none-any.whl (463 kB)\nDownloading pydantic_core-2.41.5-cp312-cp312-win_amd64.whl (2.0 MB)\n ---------------------------------------- 2.0/2.0 MB 56.7 MB/s 0:00:00\nDownloading annotated_types-0.7.0-py3-none-any.whl (13 kB)\nDownloading requests-2.33.1-py3-none-any.whl (64 kB)\nDownloading charset_normalizer-3.4.6-cp312-cp312-win_amd64.whl (154 kB)\nDownloading urllib3-2.6.3-py3-none-any.whl (131 kB)\nDownloading certifi-2026.2.25-py3-none-any.whl (153 kB)\nDownloading starlette-1.0.0-py3-none-any.whl (72 kB)\nDownloading anyio-4.13.0-py3-none-any.whl (114 kB)\nDownloading typing_extensions-4.15.0-py3-none-any.whl (44 kB)\nDownloading typing_inspection-0.4.2-py3-none-any.whl (14 kB)\nDownloading httptools-0.7.1-cp312-cp312-win_amd64.whl (86 kB)\nDownloading python_dotenv-1.2.2-py3-none-any.whl (22 kB)\nDownloading watchfiles-1.1.1-cp312-cp312-win_amd64.whl (288 kB)\nDownloading websockets-16.0-cp312-cp312-win_amd64.whl (178 kB)\nInstalling collected packages: websockets, urllib3, typing-extensions, python-dotenv, propcache, multidict, idna, httptools, h11, frozenlist, charset_normalizer, certifi, attrs, annotated-types, annotated-doc, aiohappyeyeballs, yarl, uvicorn, typing-inspection, requests, pydantic-core, anyio, aiosignal, watchfiles, starlette, pydantic, azure-core, aiohttp, fastapi, azure-cosmos\n\nSuccessfully installed aiohappyeyeballs-2.6.1 aiohttp-3.13.4 aiosignal-1.4.0 annotated-doc-0.0.4 annotated-types-0.7.0 anyio-4.13.0 attrs-26.1.0 azure-core-1.39.0 azure-cosmos-4.15.0 certifi-2026.2.25 charset_normalizer-3.4.6 fastapi-0.135.2 frozenlist-1.8.0 h11-0.16.0 httptools-0.7.1 idna-3.11 multidict-6.7.1 propcache-0.4.1 pydantic-2.12.5 pydantic-core-2.41.5 python-dotenv-1.2.2 requests-2.33.1 starlette-1.0.0 typing-extensions-4.15.0 typing-inspection-0.4.2 urllib3-2.6.3 uvicorn-0.42.0 watchfiles-1.1.1 websockets-16.0 yarl-1.23.0",
"exit_code": 0
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
language: python
database: gaming-leaderboard-db
port: 8000
health: /health
build: pip install -r requirements.txt
run: uvicorn main:app --host 0.0.0.0 --port 8000
skills_loaded: true
Loading