Skip to content

LLM_SERVICE_TESTS.md

Latest commit

 

History

History
157 lines (124 loc) · 4.64 KB

LLM_SERVICE_TESTS.md

File metadata and controls

157 lines (124 loc) · 4.64 KB

LLM Service Tests Summary

Overview

Comprehensive unit and integration tests for the Genaker\MagentoMcpAi\Service\LLM service.

Files Created

1. Unit Tests

File: /app/code/Genaker/MagentoMcpAi/Test/Unit/Service/LLMTest.php

Tests: 10 unit tests covering:

  • ✓ API key retrieval from configuration
  • ✓ Exception handling for missing API keys
  • ✓ String query conversion
  • ✓ Message array handling
  • ✓ Default model and temperature values
  • ✓ Response object structure
  • ✓ Empty query handling
  • ✓ MaxTokens parameter validation
  • ✓ Temperature parameter variations
  • ✓ Service instantiation and methods

2. Integration Tests

File: /app/code/Genaker/MagentoMcpAi/Test/Integration/Service/LLMIntegrationTest.php

Tests: 2+ integration tests covering:

  • ✓ Service initialization with dependencies
  • ✓ Required methods and signatures
  • ✓ String query integration
  • ✓ Array query integration
  • ✓ API key retrieval flow
  • ✓ Exception propagation
  • ✓ Parameter passing to AI service
  • ✓ Response handling
  • ✓ Multiple consecutive requests
  • ✓ Different model configurations
  • ✓ Temperature handling
  • ✓ MaxTokens consistency

Test Results

PHPUnit 9.6.23 by Sebastian Bergmann and contributors.

............                                                      12 / 12 (100%)

Time: 00:00.022, Memory: 10.00 MB

OK (12 tests, 26 assertions)

Status: ✓ ALL TESTS PASSING

Service Overview

LLM Service Purpose

The LLM service is a wrapper around the AIServiceInterface that:

  • Retrieves API keys from Magento configuration
  • Handles both string queries and message arrays
  • Normalizes queries for the underlying AI service
  • Uses fixed parameters (maxTokens: 2000)
  • Supports configurable temperature values

Key Methods

getApiKey(): string

  • Retrieves API key from Magento configuration
  • Caches key after first retrieval
  • Throws LocalizedException if key is missing or empty

LLM($query, $model = 'gpt-5-nano', $temperature = 1): array

  • Accepts string query or array of messages
  • Extracts user message from message array if needed
  • Delegates to AIServiceInterface::sendChatRequest()
  • Returns complete response array with:
    • success: boolean
    • message: response text
    • tokens: array with input, output, total
    • cost: float
    • model: string
    • provider: string
    • finish_reason: string

Test Coverage

Unit Tests (LLMTest.php)

  • API key configuration retrieval
  • API key exception handling
  • Query type handling (string vs array)
  • Default parameters
  • Response validation
  • Parameter passing verification
  • Multiple temperature values

Integration Tests (LLMIntegrationTest.php)

  • Service instantiation with dependencies
  • Method signatures and visibility
  • Request/response flow
  • Exception propagation
  • Multi-request handling
  • Model configuration variations

Running Tests

Unit Tests Only

cd /home/hammer/lccoins-m2
warden env exec php-fpm vendor/bin/phpunit app/code/Genaker/MagentoMcpAi/Test/Unit/Service/LLMTest.php -v

Integration Tests Only

cd /home/hammer/lccoins-m2
warden env exec php-fpm vendor/bin/phpunit app/code/Genaker/MagentoMcpAi/Test/Integration/Service/LLMIntegrationTest.php -v

All LLM Tests

cd /home/hammer/lccoins-m2
warden env exec php-fpm vendor/bin/phpunit app/code/Genaker/MagentoMcpAi/Test/Unit/Service/LLMTest.php app/code/Genaker/MagentoMcpAi/Test/Integration/Service/LLMIntegrationTest.php -v

Implementation Notes

API Key Caching

The service caches API keys after first retrieval to avoid repeated configuration reads.

Message Handling

  • String queries are converted to message format: ['role' => 'user', 'content' => $query]
  • Message arrays are passed through with the last user message extracted
  • MaxTokens is always fixed at 2000

Temperature Support

Supports temperature values from 0.0 to 2.0 for AI response randomness.

Error Handling

  • Missing/empty API keys throw LocalizedException
  • AI service exceptions are propagated to caller
  • All errors include descriptive messages

Integration with AIServiceInterface

The LLM service integrates with AIServiceInterface by:

  1. Formatting user input appropriately
  2. Passing query string (not array) as first parameter
  3. Passing message history as second parameter
  4. Using fixed maxTokens of 2000
  5. Allowing temperature customization
  6. Returning complete response arrays

Backward Compatibility

✓ All existing functionality preserved ✓ Same method signatures ✓ Same parameter defaults ✓ Compatible with MultiLLMService implementation ✓ Works with interface-based dependency injection