General payu docs for Run a Model section by ccarouge · Pull Request #1107 · ACCESS-NRI/ACCESS-Hive-Docs

ccarouge · 2025-12-16T23:20:52Z

ACCESS-Hive Docs

Thank you for submitting a pull request to the ACCESS-Hive Docs Project. 🎉

More assistance is available in the how to contribute section on ACCESS-Hive Docs

Description

Write a general payu doc page for the Run a Model section.

Fixes #978

❓ The issue #978 includes the issue #934 on hosting payu experiments on GitHub. However, the discussion on that issue does not seem to be finalised as to where to include this information. @atteggiani seems to prefer a tutorial page, which would not be the general payu page included in this PR. So for now, this PR does not address issue #934. ❓

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue, e.g. dead links)
New link / content

Checklist:

The new content is accessible and located in the appropriate section
My changes do not break navigation and do not generate new warnings
I have checked that the links are valid and point to the intended content
I have checked my code/text and corrected any misspellings

When your pull request is ready please request a review.

Unless there is a specific person you want your PR to be reviewd by, please select the Hive Docs Team: ACCESS-NRI/hivedocsteam. This ensures the load for reviewing pull requests is shared equitably.

github-actions · 2025-12-16T23:21:21Z

PR Preview
🚀 Preview of PR head commit `77eb63d` deployed to https://docs.access-hive.org.au/pr-previews/1107
2026-07-03 16:47 AEST
Preview generated through the Deploy to GitHub Pages workflow run 28643610391.

anton-seaice

I started reviewing , and now realised that this is Draft. Let me know when you want a review

jo-basevi

Sorry for the late initial review! I've just added a couple comments so far. Overall it seems to read well to me

ccarouge · 2026-01-20T06:15:24Z

@jo-basevi I have incorporated your feedback, can you please have a look and see if everything is resolved?

jo-basevi

Just had a re-read through and it all looks good to me. No extra comments sorry!

jo-basevi · 2026-03-06T05:32:15Z

+project: ol01
+```
+
+For model configurations and output to be saved to a `/scratch` storage allocation other than `project` (or your default if `project` is not set) then also set `shortpath` to the desired path. 


Yes short path will need be a full path. E.g. /scratch/PROJECT_CODE

jo-basevi · 2026-03-06T06:26:17Z

+    - restart002: 01/01/2006 (first restart date on or after 01/01/2005)  
+    - restart004: 01/01/2012 (first restart date on or after 01/01/2011)  
+    - restart005: 01/01/2015 (keeps immediate restarts before 01/01/2017)  


The time difference is always added to the last saved restart - so there will be at least 5 years between each saved restart. Each permanently saved restart is like a checkpoint.

Payu doesn't always know the model start time for an experiment and previous restarts may get deleted (maybe from scratch limits). For example if restart000 and restart002 were deleted, it could still follow on from restart004.

Does that make sense?

paigem · 2026-03-30T23:28:11Z

@ccarouge Just checking on the status of this PR. The Hive Docs team is happy to discuss any questions if helpful.

aidanheerdegen · 2026-04-06T23:07:29Z

I don't want to delay this any longer than necessary, but @Qian-HuiChen has recently added an interactive cloning mode that is likely to be the most widely used way of cloning a configuration or experiment from GitHub.

It's still being tested, but it's likely we'll release fairly soon as it's a nice feature for users. Consequently I think it should be included in these docs.

anton-seaice · 2026-04-06T23:09:42Z

I don't want to delay this any longer than necessary, but @Qian-HuiChen has recently added an interactive cloning mode that is likely to be the most widely used way of cloning a configuration or experiment from GitHub.

It's still being tested, but it's likely we'll release fairly soon as it's a nice feature for users. Consequently I think it should be included in these docs.

It's up to Clare - but it may be simplest to make a new issue for this (you know, scope creep and all :))

ccarouge · 2026-04-15T04:19:35Z

@paigem Sorry, the work plan + coordination in my team and team's projects took all my work. I'll get back to this soon.

@aidanheerdegen Let's see who gets there first for the interactive clone. If it's released before this doc is ready, then we should consider modifying the docs (I'd be more than happy for @Qian-HuiChen to contribute these modifications directly since I won't know how the interactive clone works). But as @anton-seaice said, this has been open for a long time so I'd rather try to close it instead of prolonging it if the interactive clone isn't released.

aidanheerdegen · 2026-04-15T04:24:31Z

I think the preference was to do the interactive clone instructions in a separate PR as this one is now rather daunting!

Qian-HuiChen · 2026-04-15T04:30:04Z

I am happy to leave payu clone interactive mode in a separate PR :)

ccarouge · 2026-04-22T06:25:12Z

@anton-seaice @aidanheerdegen @atteggiani @Qian-HuiChen , I've finally gone through all the comments (I think), so there is a new version to look at.

Main changes:

tried to add a blurb about the physics options for the model components in the "Terminology" section. Simplified the last section accordingly
changed payu clone to the interactive version. With colours in the terminal window, @atteggiani you may want to check my expansion on what spack and git had.
@aidanheerdegen I can't think of a neat way to indicate the work and archive sym links in the control directory are not always there in the diagram. So I left it untouched. Happy to hear some suggestions.
@aidanheerdegen @Qian-HuiChen In the interactive payu clone, one question is: "How would you like to name your local experiment directory?" but this "experiment directory" is what we've been calling the "control directory" I believe. Proof from payu clone output:
? How would you like to name your local experiment directory? diffuse_exps-1deg_jra55_ryf
To change directory to control directory run: cd diffuse_exps-1deg_jra55_ryf So even payu clone uses both terms for the same thing. I'm not asking for a change to payu clone, it's only to make you aware of the maybe confusing use of 2 terms for what seems to be the same thing.

Additionally, the model of having a separate workflow page and Run a Model pages doesn't work well for users. The Hive docs team is working on a different way to present the info. So this page is unlikely to be published as is (depends on timelines) but its content will definitely be used.

Qian-HuiChen · 2026-04-22T06:43:40Z

@ccarouge Thanks for adding payu clone interactive mode in the documents! It looks very nice 👍

Thanks for pointing out the inconsistent use of control directory and local experiment directory. I agree that it could be confusing. I have updated this question to

What would you like to name your local control directory?

in commit 271fac5. This will be included in the next payu release :)

aidanheerdegen

I have added some more comments. Hope that is helpful.

aidanheerdegen · 2026-04-24T04:13:53Z

Is there a source file that can be included as well so the diagram can be modified/updated as required?

Nope, need a Lucid Chart account to access the source.

Doh. Can you export an SVG that could be edited using something like Inkscape?

aidanheerdegen · 2026-04-24T04:19:46Z

+To get a local copy of a configuration, you need to:
+
+- identify the `<repository>` and `<branch>` name the configuration is stored under on GitHub. See the information on the [Run a Model][Run a Model] page of your chosen model for this step.
+- decide where on Gadi to store all your _payu_ experiments, `<configurations-directory>`, typically a folder under $HOME. This directory must exist before running _payu_.
+- decide on a name for your experiment, `<local-branch>`. It is recommended to choose a descriptive name, specific to your experiment.


Ok. But I think the name of the experiment and the control-directory should be swapped. People seem to get really confused on this point and name their branch the entire experiment name with the control directory name included.

It should be stressed that the control directory is the "top level" of the experiment "name-space". All experiments/branches will include the control directory name as a common element.

aidanheerdegen · 2026-04-24T04:28:35Z

+### Start the run from a specific restart file {: id='specific-restart'}
+
+To configure the experiment to start from specific restart files, add a [`restart:` entry](https://payu.readthedocs.io/en/stable/config.html#miscellaneous) to the `config.yaml` file, specifying the path to a folder containing existing restart files.
+Or, to do this automatically when setting up an experiment, add the `-r` flag to the `payu clone` command. 


Or, to do this automatically when setting up an experiment using payu clone interactive, give the restart path when prompted Do you want to specify a custom restart path?.

New `base_path` option is safer since payu will create a uniq folder for the experiment.

Clarify instructions for specifying a custom restart path during experiment setup.

anton-seaice · 2026-04-28T03:13:26Z

+As shown in the diagram, the general layout of a _payu_-supported model run consists of two main directories:
+
+- The _control_ directory contains the model configuration and is the directory from which the model run is started.  
+  {: #control-directory }. This directory contains information to manage the simulation, it also contains the scientific options for each model component that define the physics used in the model component or the diagnostics saved by the model component. These configuration options are specified in files located inside a subfolder of the _control_ directory, named according to the submodel's `name` specified in the `config.yaml` [`submodels` section](#submodels). To modify these options please refer to the configurations documentation of the respective model component, found on the [Run a Model][Run a Model] page for your chosen model.


Something doesn't render right

ccarouge · 2026-06-02T06:46:18Z

Thanks for everyone who contributed. Changes in the way the ACCESS-Hive docs are organised mean this page will not be published as is. Instead, we will use bits and pieces as include files into the Run a Model pages. I'm working on a new ESM1.5 page using the include files idea to test it out. And from that, I should be able to get an ESM1.6 page put together quickly (🤞 ).

Start general payu docs

7811e32