Skip to content

fix for issue NewValidation: CSCwp95515 - Policydist crash-loop on upgrade if configpushShardCont headTx is non-zero while tailTx is zero #253#260

Merged
monrog2 merged 16 commits into
masterfrom
issue253
Sep 19, 2025
Merged

fix for issue NewValidation: CSCwp95515 - Policydist crash-loop on upgrade if configpushShardCont headTx is non-zero while tailTx is zero #253#260
monrog2 merged 16 commits into
masterfrom
issue253

Conversation

@jeestr4d
Copy link
Copy Markdown
Collaborator

@jeestr4d jeestr4d commented Jul 3, 2025

fix for issue #253

Ran the test in a lab (using headTx == 0 and tailTx == 0) :

[Check 84/84] configpushShardCont headTx... FAIL - OUTAGE WARNING!!
dn headTx tailTx


configpush/sh-1 0 0
configpush/sh-10 0 0
configpush/sh-11 0 0
configpush/sh-12 0 0
configpush/sh-13 0 0
configpush/sh-14 0 0
configpush/sh-15 0 0
configpush/sh-16 0 0
configpush/sh-17 0 0
configpush/sh-18 0 0
configpush/sh-19 0 0
configpush/sh-2 0 0
configpush/sh-20 0 0
configpush/sh-21 0 0
configpush/sh-22 0 0
configpush/sh-23 0 0
configpush/sh-24 0 0
configpush/sh-25 0 0
configpush/sh-26 0 0
configpush/sh-27 0 0
configpush/sh-28 0 0
configpush/sh-29 0 0
configpush/sh-3 0 0
configpush/sh-30 0 0
configpush/sh-31 0 0
configpush/sh-32 0 0
configpush/sh-4 0 0
configpush/sh-5 0 0
configpush/sh-6 0 0
configpush/sh-7 0 0
configpush/sh-8 0 0
configpush/sh-9 0 0

Recommended Action: Contact Cisco TAC for Support before upgrade
Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#policydist-configpushshardcont-defect

pytest results:

python3 -m pytest tests/configpuh_shard_cont_check
============================================================================================== test session starts ===============================================================================================
platform darwin -- Python 3.9.6, pytest-8.2.0, pluggy-1.5.0
rootdir: /Users/jeestrad/Documents/Python/pre-upgrade/ACI-Pre-Upgrade-Validation-Script
configfile: pytest.ini
plugins: anyio-4.8.0
collected 6 items

tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs0-6.0(3a)-FAIL - OUTAGE WARNING!!] PASSED [ 16%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs1-5.2(6a)-PASS] PASSED [ 33%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs2-5.2(6b)-N/A] PASSED [ 50%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs3-6.1(6b)-N/A] PASSED [ 66%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs4-6.1(4b)-N/A] PASSED [ 83%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs5-6.1(4a)-N/A] PASSED [100%]

=============================================================================================== 6 passed in 0.03s ================================================================================================

jeestr4d and others added 9 commits July 2, 2025 18:08
…grade if configpushShardCont headTx is non-zero while tailTx is zero #253
* Introduced CSCwp22212 check for stale pconsRA objects

---------

Co-authored-by: Gabriel <gmonroy@cisco.com>
* Add ISIS DTEP Check

---------

Co-authored-by: Gabriel <gmonroy@cisco.com>
* feat: synth class + write back each function as json result

* fix: Clean up get_vpc_nodes() and its pytest

* feat: Split the main script execution code into each func and add pytest

* feat: Retire json_log and adopt json per rule for both PUV and regular use + linting

* feat: Support QA version and AciVersion instance as input in AciVersion (#259)

* feat: Support QA version and AciVersion instance as input in AciVersion

* fix: Use supported version format in older_than

* fix: Use ValueError in AciVersion

* feat: -c (cversion) / -d (debug_function) args into puv (#265)

* feat: Update synthMaintP with the latest schema with ruleId

* feat: Add decorator `@check_wrapper` for all check functions

A new decorator `@check_wrapper` is to move most of the I/O functionalities,
such as printing and writing the result into a file, outside of each
check function so that each check can focus on the validation logic
itself by minimizing the impact from a requirement change in the output
format and so on.
To support this, a new class `Result` is also introduced to make it
clear what a check function is expected to return. As long as `Result`
class is returned, the decorator `@check_wrapper` handles the printing
them to stdout and files.

* feat: rename synth class to AciResult + breakdown --puv into --api-only and --no-cleanup

* feat: Add --version and --total-checks input args

* feat: Do not touch log folders with some options like --version

* fix: ValueError when col and row len do not match + pytest

* fix: correct checks with mismatched col row length

* fix: cimc logic update for QA

---------

Co-authored-by: takishida <38262981+takishida@users.noreply.github.com>
Co-authored-by: tkishida <tkishida@cisco.com>
v3.0.0 to mark for PUV integration + accounting for the 7 new input arg options
…grade if configpushShardCont headTx is non-zero while tailTx is zero #253
Copy link
Copy Markdown
Collaborator Author

@jeestr4d jeestr4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • changed validation with new framework with decorator

  • added condition to also check for tailTx != 0 while headTx == 0

  • lab test (with modified conditions headTx or headTx == 0 )

[Check 86/86] configpushShardCont headTx... FAIL - OUTAGE WARNING!!
WARNING!!
dn headTx tailTx


configpush/sh-1 0 0
configpush/sh-10 0 0
configpush/sh-11 0 0

Recommended Action: Contact Cisco TAC for Support before upgrade
Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#policydist-configpushshardcont-defect

  • pytest results

tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs0-None-MANUAL CHECK REQUIRED]
---------------------------------------------------------------------------------------- live log setup -----------------------------------------------------------------------------------------
[14:45:26.216 INFO initialize:5272] Cleaning up previous run files in preupgrade_validator_logs/
[14:45:26.217 INFO initialize:5274] Creating directories preupgrade_validator_logs/ and preupgrade_validator_logs/json_results/
PASSED [ 11%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs1-6.0(3a)-FAIL - OUTAGE WARNING!!] PASSED [ 22%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs2-6.0(3a)-FAIL - OUTAGE WARNING!!] PASSED [ 33%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs3-5.2(6a)-PASS] PASSED [ 44%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs4-5.2(6b)-N/A] PASSED [ 55%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs5-6.1(6b)-N/A] PASSED [ 66%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs6-6.1(6b)-N/A] PASSED [ 77%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs7-6.1(4b)-N/A] PASSED [ 88%]
tests/configpuh_shard_cont_check/test_configpush_shard_cont_check.py::test_logic[icurl_outputs8-6.1(4a)-N/A] PASSED [100%]

======================================================================================= 9 passed in 0.03s =======================================================================================

@jeestr4d jeestr4d requested review from monrog2 and takishida July 23, 2025 21:05
Copy link
Copy Markdown
Collaborator

@monrog2 monrog2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, lets close on whether both tx scenarios are truly going to be flagged here, then get bug details to match before we push this out.

Comment thread aci-preupgrade-validation-script.py Outdated
Comment thread aci-preupgrade-validation-script.py Outdated
return Result(result=result, headers=headers, data=data, recommended_action=recommended_action, doc_url=doc_url)


@check_wrapper(check_title='configpushShardCont headTx')
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If both scenarios to be covered, title must change

Comment thread aci-preupgrade-validation-script.py Outdated
Comment thread docs/docs/validations.md Outdated
@monrog2
Copy link
Copy Markdown
Collaborator

monrog2 commented Sep 19, 2025

pytest output:

[Check  1/1] Policydist configpushShardCont Crash... Target version not supplied. Skipping.                         MANUAL CHECK REQUIRED
[Check  1/1] Policydist configpushShardCont Crash...                                                              FAIL - OUTAGE WARNING!!
  dn               headTx              tailTx
  --               ------              ------
  configpush/sh-2  576460752318322171  0
  configpush/sh-3  516460756318314171  0

  Recommended Action: Contact Cisco TAC for Support before upgrade
  Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#policydist-configpushshardcont-crash


[Check  1/1] Policydist configpushShardCont Crash...                                                                                 PASS
[Check  1/1] Policydist configpushShardCont Crash...                                                                                  N/A
[Check  1/1] Policydist configpushShardCont Crash...                                                                                  N/A

@monrog2 monrog2 merged commit 5a620c0 into master Sep 19, 2025
3 checks passed
@monrog2 monrog2 deleted the issue253 branch September 19, 2025 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NewValidation: CSCwp95515 - Policydist crash-loop on upgrade if configpushShardCont headTx is non-zero while tailTx is zero

4 participants