feat(tidy3d-notebooks): GaussianBeam source gradient demonstration

[groberts-flex]

This notebook demonstrates optimizing GaussianBeam source gradients for a grating coupler alongside geometric parameters to target designs that are robust to source/packaging misalignments.

---

Note

Low Risk
Documentation and example-only changes with no runtime library or production code paths affected.

Overview
Adds a new inverse-design notebook that jointly optimizes grating coupler geometry and Gaussian beam source parameters (lateral position, tilt, waist) with Tidy3D Autograd, aimed at designs that tolerate source or packaging misalignment.

The example runs two related 2D grating-coupler problems: a partially etched top grating with per-tooth widths, and a variant with a lower silicon reflector (period and duty cycle) separated by an SiO₂ gap. Both maximize coupling into the backward waveguide mode while updating beam settings in the same optimization loop.

Docs: registers Autograd31GratingCouplerWithBeamOptimization in the Inverse Design example toctree in autograd.rst.

^{Reviewed by Cursor Bugbot for commit f9a596d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[marcorudolphflex]

Cool demo, thanks!
Some room for improvement:

1. Check CI-relevant lint/spellcheck
2. Add thumbnail image, add in metadata
3. Index the notebook in docs/features/autograd.rst
4. I see big vertical gaps in grating coupler design comparisons - let's save space here and make it better comparable
5. I would add some more references to other grating couplers and autograd tutorial notebooks.
6. rename to Autograd31... (parallel adjoint notebook is now Autograd30)
7. cell starting with def to_norm(value_phys, min_val, max_val): is very big and should be split semantically (like plotting/optimization/simulation setup) with some more descriptive md cells added in between. Would also add more comments or docstrings for some functions. We could also consider to move the "more interesting" or relevant functions like the "run..." ones to the cells where they are actually used as this is code to pay more attention to for the users. For the "boring" plot helpers, I would say it is okay to just define and hide all of them in one big cell with less explanation. Still I would let pay the users pay more attention to the "interesting" helpers with APIs which are actually relevant to handle with.
8. Can we explain the big dip in the Frozen-source optimization history? Is that more smooth with a smaller learning rate?
9. I think it's quite unusual in our notebooks to use ALL_CAPS variables? Consider making them lowercase. Also I would add some explanatory comments to most of them. Would move the plot-related variables (and the plot.rcParams calls) to the plotting helper cell.
10. use tidy3d adam instead of optax adam
11. remove local_gradient=True (or the local gradient option at all) - we should release this notebook after the tidy3d release of this feature anyways - or is it already released?
12. rename task_names starting with autograd35 - maybe remove this prefix completely to make this safe against some future migration
13. optional: log grad norm over optimization steps to get better feeling for convergence?
14. Use lower-case in title
15. I would make the introduction a little bit more detailed by introducing the concept how we will archieve with some more detail - I could imagine users would not directly get those short explanations like "improves robustness only through geometry changes"?

[github-actions]

Spell check passed successfully for 1 notebook(s).
Generated by GitHub Action run: https://github.com/flexcompute/tidy3d-notebooks/actions/runs/26976751970

[groberts-flex]

Thanks so much for the really helpful comments @marcorudolphflex

I put together another version of the notebook and left some responses to a couple of the comments below:

cell starting with def to_norm(value_phys, min_val, max_val): is very big and should be split semantically (like plotting/optimization/simulation setup) with some more descriptive md cells added in between. Would also add more comments or docstrings for some functions. We could also consider to move the "more interesting" or relevant functions like the "run..." ones to the cells where they are actually used as this is code to pay more attention to for the users. For the "boring" plot helpers, I would say it is okay to just define and hide all of them in one big cell with less explanation. Still I would let pay the users pay more attention to the "interesting" helpers with APIs which are actually relevant to handle with.

Broke this up a bit and added more markdown between the sections for explanations. Also expanded out notebook intro to have more references and introductory pieces.

Can we explain the big dip in the Frozen-source optimization history? Is that more smooth with a smaller learning rate?

Good question! this ended up being too aggressive of a learning rate. I turned it down and things look better now.

I ended up adding a few extra takeaways at the end of the notebook as the conclusion is a little nuanced. I think the story is still clear but let me know if it's not clear enough of a conclusion. I think a good takeaway for the time being at least is that you can optimize with the Gaussian beam parameters which was the motivation for the notebook.

[marcorudolphflex]

Looks great!
A few more minor comments:

1. Would remove this:

They are grouped together so the optimization and simulation code remains easier to follow.

2. The top thumbnail image looks quite large and may be positioned anyhwere in the middle/end of the first md cell.
3. avoid the plugin wording here?

using the adjoint plugin

I also wondered if the effect/benefit would be even stronger with more iterations? But would not see that blocking.

I like the new comparison plots!

[groberts-flex]

This took me a while to get updated, but ended up trying to simplify things a bit since after I ran for more iterations the effect was a little bit less clear. I ended up just doing a robustness optimization individually for source position and source tilt. And now I think things are clearer as you can see each of those robustness metrics improve over the baseline.

[marcorudolphflex]

This took me a while to get updated, but ended up trying to simplify things a bit since after I ran for more iterations the effect was a little bit less clear. I ended up just doing a robustness optimization individually for source position and source tilt. And now I think things are clearer as you can see each of those robustness metrics improve over the baseline.

Thanks for the effort on this!
some findings:

• task_name is still accepted by the robust async helper but is not used: the function coupling_efficiency_from_sim_async(..., task_name, ...) builds sims as a list and calls run_async(sims, verbose=False). The baseline/source-position/source-angle objectives pass descriptive names like grating_robustness_baseline, but those names are ignored for the batched robustness simulations.
• we do not longer want to use run_async for notebooks, instead just use the containerized run.
• eventhough the robustness metrics look fine, the overall performance seem to be fairly inferior to the baseline looking looking at the very last plots where our robust optimization outperforms the baseline only for some very big perturbations. Are such big perturbations realistic? I think it would be more convincing to have a better performance for smaller perturbations. The (non-log) mean over the perturbations looks worse than in the baseline. I see you changed the robust optimization does not longer start from the baseline and you changed from plain mean to log mean. Did this improve results and if so, how? In general, I'm not sure how much value our robustness metric has, better compare the (soft) min or so?

[groberts-flex]

Thanks for the review on this! I tried to get it to work better than baseline over smaller perturbations. There is a fundamental challenge here that is a little different than robust optimizations for geometries. If the source perturbation is small and ends up having a large overlap with the original source, then it limits how much the performance can actually drop. So I think the robust optimizations only show their benefit for fairly large perturbations which I think may not be as practical.

To simplify this whole thing I just took out the robust part and optimized two different grating couplers, one single layer one and one with a bottom reflector. And framed things more that the co-optimization of the source with the geometry can be a useful tool especially as grating geometry grows in complexity and it gets harder to intuitively place the source.

[marcorudolphflex]

Thanks for updating this!
I think it's good to show that in this "more plain way" to use source beam gradients as it focuses more on our new feature directly rather than distract from that with all the robustness.

Some comments from my side:

• would add the optimization parameters to the optimization cell as they act less globally here? I think this is more consistent with other notebooks.
• nit: thumbnail image is still good for this new version, but image filename is not any more as we do not apply robustness here
• do we really need this complicated casing in gradient_l2_norm? Looks like we only handle dicts here?
• similar finding for plot_history: isn't history always a dict?
• grad norm is always given for format_progress_line, so just no conditioning in the func and no default?
• from __future__ import annotations shouldn't be needed as we don't have type annotations here?
• this can be removed? Or is it intended to toggle? # reflector_gap = 1.6
• There could be more explanatory and result-summarizing text in the Final Comparison section. Would probably would make also sense to split the cells in there? Could also make sense to split the optimization result gathering and plotting for Stage 1 and Stage 2. Would check the overall notebook for some potential - except for the helper cells which may be less interesting for users and less necessary to comment/explain.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 6a05ac2. Configure here.}

[groberts-flex]

Thanks for the comments and review @marcorudolphflex! updated and re-ran the notebook, uploaded newest version here

[marcorudolphflex]

Thanks for the effort, great demo!