Failing BDD TestFramework

[Mafti]

Currently testing the azure-pipeline, but am at the following error:

2020-02-18T13:04:55.9656326Z ##[section]Starting: Evaluate Test Results
2020-02-18T13:04:55.9803415Z ==============================================================================
2020-02-18T13:04:55.9803512Z Task         : PowerShell
2020-02-18T13:04:55.9803551Z Description  : Run a PowerShell script on Linux, macOS, or Windows
2020-02-18T13:04:55.9803615Z Version      : 2.163.1
2020-02-18T13:04:55.9803654Z Author       : Microsoft Corporation
2020-02-18T13:04:55.9803693Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
2020-02-18T13:04:55.9803757Z ==============================================================================
2020-02-18T13:04:57.6401119Z Generating script.
2020-02-18T13:04:57.7024110Z ========================== Starting Command Output ===========================
2020-02-18T13:04:57.7467964Z ##[command]"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\Agent\_work\_temp\30e5b1e6-96ae-4f99-8c0a-8e710a0bfd48.ps1'"
2020-02-18T13:04:58.8164611Z ##[error]python : 
2020-02-18T13:04:58.8231535Z ##[error]At C:\Agent\_work\_temp\30e5b1e6-96ae-4f99-8c0a-8e710a0bfd48.ps1:2 char:1
+ python -m outsystems.pipeline.evaluate_test_results --artifacts "Arti ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:String) [], RemoteException
2020-02-18T13:04:58.8342196Z ##[error]    + FullyQualifiedErrorId : NativeCommandError
2020-02-18T13:04:58.8757043Z ##[error] 
2020-02-18T13:04:58.8794019Z ##[error]PowerShell exited with code '1'.
2020-02-18T13:04:58.9319190Z ##[section]Finishing: Evaluate Test Results

[duarte-castano]

Hi @Mafti

The error shown does not give much information about the issue itself.

The evaluate_test_results function performs the following actions:

1. Loads the test endpoints (from the file generated by the generate_unit_testing_assembly function)
2. Runs all test suites found in the applications marked as test apps
3. Stores the value in a XML file having the JUNIT test results format

To use the evaluate_test_results function you'll have to:

• Have tests applications using the BBD Framework
• Run previously the generate_unit_testing_assembly function

If you are not using test applications using BDD Framework (available on Forge), you should not use this function on your pipeline.

Best regards,
Duarte

[Mafti]

Hi,

Exact my point, I have no idea how to deep dive into the actual error.
I do have a bdd-framework test application which is marked as such in the trigger-pipeline.

[duarte-castano]

Hi @Mafti

Can you check if this behavior still happens after applying our suggestion mentioned on the BDDFramework task also fetches the InternalError Page issue.

Thank you,
Duarte

[Mafti]

Hi,

I changed the outsystems code as suggested, but sadly, the error keeps on persisting :(

[Mafti]

@duarte-castano I think I might have found the issue.
When I run that script evaluate_test_results in powershell it will fail.
When I run the script evaluate_test_results in powershell as admin it will succeed.
I noticed the file "junit-result.xml" is being created, but it throws an exception when it tries to update that specific file.

I am not sure if this is an oversight on our side, or a missing instruction/elevated rights configuration on your side (that we need to take care of)

[Mafti]

@duarte-castano Hi, I might have gone the wrong path, so the latest state it's still failing.
However, when I execute the actual line which is calling the powershell-script
i.e. "...Powershell.exe ... guid.ps1" in a command-window on lifetime, It works as expected.
but running it via azure it throws the error :(