docs: use scipy optimizer in QAOA example#1801
Conversation
|
Hi @cmdr-chara , Thanks for making a PR. I'll take a look at this and get back to you. |
There was a problem hiding this comment.
Thanks! This is a promising start. Your scipy optimizer improves both the expectation value and the runtime performance of QAOA which is great. The success ratio appears to be worse than the random sampling approach though which is mysterious. Maybe there was a bug in this example prior to your contribution that has only manifested now.
Also I would much prefer if you commited the fully executed notebook to github so we can review the cell outputs (the matplotlib plots and the expectation values).
There was a problem hiding this comment.
One thing I did observe when running this locally was the the "success ratio" (printed by the validate_qaoa_experiment function) is lower. This success ratio is the proportion of the shots which land in the two basis states representing the optimal colorings. From the plot this is lower with your implementation than with the random sampling based approach.
This is a little puzzling as the scipy optimizer achieves a higher expecation value than random sampling (expected).
Lets look at the two plots below... The green bars represent "success" shots.
Your scipy based results: success ratio=0.3385, expectation value = 5.0935
Old random sampling based results: success ratio=0.4205, expectation value = 4.937499999999999
I suspect there is a bug somewhere as I would not expect the success ratio to be lower with a better optimizer. It could well have existed prior to your PR so it may be my fault.
Could you please investigate? Happy to discuss further if anything is unclear.
There was a problem hiding this comment.
I investigated this by checking both the plotted result bars and the sampled cut-value distribution.
The short version is that the two metrics are related but not identical. The scipy optimizer is maximizing expected max-cut energy over the sampled distribution. The success ratio reported by validate_qaoa_experiment only counts shots that land in the exact optimal colorings, which are the two green bars in the plot.
The plots show that the exact-optimal green bars can be lower, but that does not necessarily mean the energy is worse. In the executed notebook, much more probability mass moves into near-optimal cut=5 states, even though the exact-optimal cut=6 probability is lower. So the expected energy can improve while the exact-optimal success ratio decreases.
I added a distribution-by-cut-value diagnostic to make this visible numerically, because the bar chart alone shows basis-state counts but not the cut value represented by each non-optimal basis state.
This updates the QAOA MaxCut example to use
scipy.optimize.minimizefor the classical parameter search instead of sampling random angles.The refactor keeps full
QsysResultobjects out of the optimizer loop.eval_qaoa_energynow returns only the scalar energy, and the notebook runs one final QAOA circuit with the best parameters for validation and plotting.I also added SciPy to the examples dependency group because the notebook now imports it directly.
Testing:
python -m uv lock --checkpython -m uv run ruff format --checkpython -m uv run ruff checkSuccess ratio: 0.3845I also ran
python -m uv run pytest tests/integration/test_notebooks.pylocally on Windows/Python 3.13.7. It produced unrelated stochastic notebook snapshot diffs inadaptive-qpe,repeat-until-success,state_results, andswap_test. The changed QAOA notebook is excluded from that test by the project because it is long-running, so I executed it manually as above.AI disclosure: I used OpenAI Codex with GPT-5.5 Extra High to help refactor the QAOA optimizer loop and review the PR text. I manually inspected the diff, removed unnecessary generated wording, and ran the notebook/checks locally.
Closes #1545