Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,11 @@ jobs:
- name: Build package
run: python setup.py sdist bdist_wheel

# - name: Publish package
# uses: pypa/gh-action-pypi-publish@release/v1
# with:
# user: ${{ secrets.PYPI_USERNAME }}
# password: ${{ secrets.PYPI_PASSWORD }}
- name: Publish package
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}

- name: Setup git credentials
uses: oleksiyrudenko/gha-git-credentials@v2-latest
Expand All @@ -69,11 +69,11 @@ jobs:
email: 'support@reportportal.io'
token: ${{ secrets.GITHUB_TOKEN }}

# - name: Tagging new version
# id: newVersionTag
# run: |
# git tag -a ${{ env.RELEASE_VERSION }} -m "Release ${{ env.RELEASE_VERSION }}"
# git push --tags
- name: Tagging new version
id: newVersionTag
run: |
git tag -a ${{ env.RELEASE_VERSION }} -m "Release ${{ env.RELEASE_VERSION }}"
git push --tags

- name: Checkout develop branch
uses: actions/checkout@v4
Expand Down Expand Up @@ -108,13 +108,13 @@ jobs:
version: ${{ env.RELEASE_VERSION }}
path: ./${{ env.CHANGE_LOG_FILE }}

# - name: Create Release
# id: createRelease
# uses: ncipollo/release-action@v1
# with:
# tag: ${{ env.RELEASE_VERSION }}
# name: Release ${{ env.RELEASE_VERSION }}
# body: ${{ steps.readChangelogEntry.outputs.changes }}
- name: Create Release
id: createRelease
uses: ncipollo/release-action@v1
with:
tag: ${{ env.RELEASE_VERSION }}
name: Release ${{ env.RELEASE_VERSION }}
body: ${{ steps.readChangelogEntry.outputs.changes }}

- name: Update version file
id: versionFileUpdate
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## [Unreleased]
### Added
- Too long BDD Step name truncation on reporting, by @HardNorth
- `use_index` parameter for `@pytest.mark.parametrize` decorator, which replaces parameter values with their indexes in the list, by @ramir-dn

## [5.5.0]
### Added
- Issue [#357](https://github.com/reportportal/agent-python-pytest/issues/357) `pytest-bdd` support, by @HardNorth
### Fixed
- Issue [#389](https://github.com/reportportal/agent-python-pytest/issues/389) `rp_tests_attributes` configuration parameter handling, by @HardNorth
Expand Down
4 changes: 4 additions & 0 deletions examples/bdd/features/long_step_scenario.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Feature: Test long step scenario

Scenario: The scenario
Given A very long step. Seriously, this step is ridiculously long. Our users require such a long step and here it is. This is not even a half of the step, it will last really long. I believe you will get tired before you reach the end of the step. Why? Because our users send here a megabyte-length JSONs or things like that, despite our words that this is not good for user experience, since it's hard to read such a long lines on UI and for backend performance, since we use this field in full-text search. OK I'm out of thought what else I can write here, but the step must go on. Probably you will see some random typing soon to get the step longer than it's right now. Because only now we crossed a half of the length we need. Alright, here we go: 39248cht42 3x,r093mhxr0,3hr c089r3423 xk309,,r3 k302uk032 3249xul398u3 at 34r9k39489 aumvi xkr293jed0 i 93u9f32u smhf 09ktc903 a tu09r328 ef5u0fu3v 0k8utf2 u59du9v u423kuc 9f5kv 39kd3uf39 -3940u5kfu5 b3-90485l-k3f. Are you tired? I'm too, but we still need a few words at the end.
30 changes: 30 additions & 0 deletions examples/bdd/step_defs/long_step_scenario_steps.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copyright 2025 EPAM Systems
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import logging

from pytest_bdd import given, scenarios

# Import the scenario from the feature file
scenarios("../features/long_step_scenario.feature")


LOGGER = logging.getLogger(__name__)


@given(
"A very long step. Seriously, this step is ridiculously long. Our users require such a long step and here it is. This is not even a half of the step, it will last really long. I believe you will get tired before you reach the end of the step. Why? Because our users send here a megabyte-length JSONs or things like that, despite our words that this is not good for user experience, since it's hard to read such a long lines on UI and for backend performance, since we use this field in full-text search. OK I'm out of thought what else I can write here, but the step must go on. Probably you will see some random typing soon to get the step longer than it's right now. Because only now we crossed a half of the length we need. Alright, here we go: 39248cht42 3x,r093mhxr0,3hr c089r3423 xk309,,r3 k302uk032 3249xul398u3 at 34r9k39489 aumvi xkr293jed0 i 93u9f32u smhf 09ktc903 a tu09r328 ef5u0fu3v 0k8utf2 u59du9v u423kuc 9f5kv 39kd3uf39 -3940u5kfu5 b3-90485l-k3f. Are you tired? I'm too, but we still need a few words at the end." # noqa: E501
)
def step_with_long_text():
LOGGER.info("Step with extremely long text executed successfully")
24 changes: 24 additions & 0 deletions examples/test_case_id/test_case_id_decorator_use_index_false.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""A simple example test with Test Case ID decorator using parameter values."""

# Copyright (c) 2023 https://reportportal.io .
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

import pytest

TEST_CASE_ID = "ISSUE-USE-INDEX-FALSE"


@pytest.mark.parametrize(("param1", "param2"), [("value1", "value2")])
@pytest.mark.tc_id(TEST_CASE_ID, parameterized=True, use_index=False)
def test_case_id_decorator(param1, param2):
assert True
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""A simple example test with Test Case ID decorator using parameter indices with selected params."""

# Copyright (c) 2023 https://reportportal.io .
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

import pytest

TEST_CASE_ID = "ISSUE-USE-INDEX-PARTIAL"


@pytest.mark.parametrize(("param1", "param2"), [("value1", "value2")])
@pytest.mark.tc_id(TEST_CASE_ID, parameterized=True, params=["param1"], use_index=True)
def test_case_id_decorator(param1, param2):
assert True
24 changes: 24 additions & 0 deletions examples/test_case_id/test_case_id_decorator_use_index_true.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""A simple example test with Test Case ID decorator using parameter indices."""

# Copyright (c) 2023 https://reportportal.io .
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

import pytest

TEST_CASE_ID = "ISSUE-USE-INDEX-TRUE"


@pytest.mark.parametrize(("param1", "param2"), [("value1", "value2")])
@pytest.mark.tc_id(TEST_CASE_ID, parameterized=True, use_index=True)
def test_case_id_decorator(param1, param2):
assert True
26 changes: 24 additions & 2 deletions pytest_reportportal/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -599,18 +599,26 @@ def _get_code_ref(self, item: Item) -> str:

def _get_test_case_id(self, mark, leaf: Dict[str, Any]) -> str:
parameters: Optional[Dict[str, Any]] = leaf.get("parameters", None)
parameters_indices: Optional[Dict[str, Any]] = leaf.get("parameters_indices") or {}
parameterized = True
selected_params: Optional[List[str]] = None
use_index = False
if mark is not None:
parameterized = mark.kwargs.get("parameterized", False)
selected_params: Optional[Union[str, List[str]]] = mark.kwargs.get("params", None)
use_index = mark.kwargs.get("use_index", False)
if selected_params is not None and not isinstance(selected_params, list):
selected_params = [selected_params]

param_str = None
if parameterized and parameters is not None and len(parameters) > 0:
if selected_params is not None and len(selected_params) > 0:
param_list = [str(parameters.get(param, None)) for param in selected_params]
if use_index:
param_list = [str((param, parameters_indices.get(param, None))) for param in selected_params]
else:
param_list = [str(parameters.get(param, None)) for param in selected_params]
elif use_index:
param_list = [str(param) for param in parameters_indices.items()]
else:
param_list = [str(param) for param in parameters.values()]
param_str = "[{}]".format(",".join(sorted(param_list)))
Expand Down Expand Up @@ -729,6 +737,19 @@ def _get_parameters(self, item) -> Optional[Dict[str, Any]]:
return None
return {str(k): v.replace("\0", "\\0") if isinstance(v, str) else v for k, v in params.items()}

def _get_parameters_indices(self, item) -> Optional[Dict[str, Any]]:
"""
Get params indices of item.

:param item: Pytest.Item
:return: dict of params indices
"""
indices = item.callspec.indices if hasattr(item, "callspec") else None
if not indices:
return None

return indices

def _process_test_case_id(self, leaf: Dict[str, Any]) -> str:
"""
Process Test Case ID if set.
Expand Down Expand Up @@ -793,6 +814,7 @@ def _process_metadata_item_start(self, leaf: Dict[str, Any]) -> None:
leaf["name"] = self._process_item_name(leaf)
leaf["description"] = self._get_item_description(item)
leaf["parameters"] = self._get_parameters(item)
leaf["parameters_indices"] = self._get_parameters_indices(item)
leaf["code_ref"] = self._get_code_ref(item)
leaf["test_case_id"] = self._process_test_case_id(leaf)
leaf["issue"] = self._process_issue(item)
Expand Down Expand Up @@ -1279,7 +1301,7 @@ def start_bdd_step(self, feature: Feature, scenario: Scenario, step: Step) -> No
if feature.background:
background_leaf = scenario_leaf["children"][feature.background]
self._finish_bdd_step(background_leaf, "PASSED")
item_id = reporter.start_nested_step(f"{step.keyword} {step.name}", timestamp())
item_id = reporter.start_nested_step(self._truncate_item_name(f"{step.keyword} {step.name}"), timestamp())
step_leaf["item_id"] = item_id
step_leaf["exec"] = ExecStatus.IN_PROGRESS

Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
dill>=0.3.6
pytest>=4.6.10
reportportal-client~=5.6.2
reportportal-client~=5.6.4
aenum>=3.1.0
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

from setuptools import setup

__version__ = "5.5.0"
__version__ = "5.5.1"


def read_file(fname):
Expand Down
30 changes: 30 additions & 0 deletions tests/integration/test_bdd.py
Original file line number Diff line number Diff line change
Expand Up @@ -1130,3 +1130,33 @@ def test_rp_tests_attributes_bdd_tags(mock_client_init):
assert {"key": "test_key", "value": "test_value"} in attributes
assert {"value": "ok"} in attributes
assert {"key": "key", "value": "value"} in attributes


@mock.patch(REPORT_PORTAL_SERVICE)
def test_long_step(mock_client_init):
mock_client = setup_mock_for_logging(mock_client_init)
result = utils.run_pytest_tests(tests=["examples/bdd/step_defs/long_step_scenario_steps.py"])
assert int(result) == 0, "Exit code should be 0 (no errors)"

# Verify scenario is correctly identified
scenario_call = mock_client.start_test_item.call_args_list[0]
assert scenario_call[1]["name"] == "Feature: Test long step scenario - Scenario: The scenario"
assert scenario_call[1]["item_type"] == "STEP"
assert scenario_call[1]["code_ref"] == "features/long_step_scenario.feature/[SCENARIO:The scenario]"

# Verify step is truncated but still identifiable
step_call = mock_client.start_test_item.call_args_list[1]
step_name = step_call[0][0]
assert step_name.startswith("Given A very long step.")
assert len(step_name) <= 1024, "Step name should be truncated to at most 1024 characters"
assert step_name.endswith("..."), "Truncated step should have ellipsis"

# Verify step details
assert step_call[0][2] == "step"
assert step_call[1]["parent_item_id"] == scenario_call[1]["name"] + "_1"
assert step_call[1]["has_stats"] is False

# Verify all steps pass
finish_calls = mock_client.finish_test_item.call_args_list
for call in finish_calls:
assert call[1]["status"] == "PASSED"
77 changes: 77 additions & 0 deletions tests/integration/test_case_id_use_index_report.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Copyright (c) 2023 https://reportportal.io .
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

"""This module includes integration tests for Test Case ID report with use_index parameter."""

from unittest import mock

import pytest

from examples.test_case_id import (
test_case_id_decorator_use_index_false,
test_case_id_decorator_use_index_partial_params,
test_case_id_decorator_use_index_true,
)
from tests import REPORT_PORTAL_SERVICE
from tests.helpers import utils


@mock.patch(REPORT_PORTAL_SERVICE)
@pytest.mark.parametrize(
["test", "expected_id"],
[
# Test use_index=True: should use parameter indices instead of values
# Format should be: ('param1', 0),('param2', 0) instead of value1,value2
(
"examples/test_case_id/test_case_id_decorator_use_index_true.py",
test_case_id_decorator_use_index_true.TEST_CASE_ID + "[('param1', 0),('param2', 0)]",
),
# Test use_index=False: should use parameter values (default behavior)
(
"examples/test_case_id/test_case_id_decorator_use_index_false.py",
test_case_id_decorator_use_index_false.TEST_CASE_ID + "[value1,value2]",
),
# Test use_index=True with selected params: should use index for selected param only
# Should be ('param1', 0) instead of value1
(
"examples/test_case_id/test_case_id_decorator_use_index_partial_params.py",
test_case_id_decorator_use_index_partial_params.TEST_CASE_ID + "[('param1', 0)]",
),
],
)
def test_use_index_parameters(mock_client_init, test, expected_id):
"""Verify the use_index parameter in Test Case ID functionality.

This test verifies that the new use_index parameter works correctly for parameterized tests.
When use_index=True, the test case ID should include parameter indices instead of parameter values.
This is useful for scenarios where parameter values might change between test runs but the indices
remain constant, ensuring that retry groups are properly maintained.

The test covers:
1. use_index=True with all parameters - should use (param_name, index) format
2. use_index=False - should use parameter values (default behavior)
3. use_index=True with selected parameters - should use indices only for selected params

:param mock_client_init: Pytest fixture for mocking the ReportPortal client
:param test: a test file path to run
:param expected_id: the expected Test Case ID format
"""
result = utils.run_pytest_tests(tests=[test])
assert int(result) == 0, "Exit code should be 0 (no errors)"

mock_client = mock_client_init.return_value
assert mock_client.start_test_item.call_count > 0, '"start_test_item" called incorrect number of times'

call_args = mock_client.start_test_item.call_args_list
step_call_args = call_args[-1][1]
assert step_call_args["test_case_id"] == expected_id
5 changes: 2 additions & 3 deletions tests/integration/test_pass_failed_skipped.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,8 @@ def test_simple_tests(mock_client_init, test, expected_run_status, expected_item
if i == 0:
actual_status = finish_test_step["status"]
assert (
actual_status == expected_item_status,
f'Invalid item status, actual "{actual_status}", expected: "{expected_item_status}"',
)
actual_status == expected_item_status
), f'Invalid item status, actual "{actual_status}", expected: "{expected_item_status}"'

finish_launch_call_args = mock_client.finish_launch.call_args_list
assert len(finish_launch_call_args) == 1
Expand Down