Create Initial Documentation

docs/source/index.rst

@@ -7,11 +7,17 @@ Welcome to the AiiDAlab ChemShell plugin documentation. Here we aim to provide
 a comprehensive guide on how to utilise the plugin to perform various ChemShell
 based workflows within the AiiDA/AiiDAlab workflow environment. This documentation
 refers to the AiiDAlab UI for the AiiDA ChemShell integration, for the core AiiDA
-ChemShell plugin please see `https://stfc.github.io/aiida-chemshell/`_\.
+ChemShell plugin please see `aiida-chemshell <https://stfc.github.io/aiida-chemshell/>`_\.
 
 
 .. toctree::
    :maxdepth: 2
    :caption: Contents:
 
+   user_guide/getting_started
+   user_guide/workflows
+   user_guide/history_page
+   user_guide/resource_management
+   user_guide/node_viewers
+
    api_docs/modules

docs/source/user_guide/getting_started.rst

@@ -0,0 +1,121 @@
+.. _getting_started:
+
+Getting Started
+===============
+
+ChemShell
+---------
+
+`ChemShell <https://chemshell.org/>`_ 
+is a feature rich multiscale chemical modelling environment that leverages the
+power of python scripting to design workflows encompassing a range of quantum mechanics (QM)
+and/or molecular mechanics (MM) software packages into a one-stop analysis tool. Focussing on
+multiscale simulation of complex systems using combined QM/MM methods it is fully scalable
+from your desktop to massively parallel supercomputers. ChemShell provides a suite of advanced
+modelling methods for geometry optimisation, energy surface mapping, molecular dynamics, monte
+carlo, free energy methods, excited states and more, all available for quantum, classical and
+hybrid QM/MM calculations.
+
+AiiDAlab
+--------
+
+`AiiDAlab <https://www.aiidalab.net/>`_ adapts the highly popular `AiiDA <https://www.aiida.net/>`_
+workflow management platform to provide an enhanced user interface (UI) based platform for
+carrying out complex computational scientific workflows. This plugin is designed to expose
+several common workflows utilising the ChemShell multiscale modelling software with a convenient
+and user friendly step-by-step approach.
+
+Running AiiDAlab
+~~~~~~~~~~~~~~~~
+
+It is generally recommended to run AiiDAlab through a container engine such as Docker or Apptainer,
+details on how to use Docker to run AiiDAlab can be found in the 
+`AiiDAlab documentation <https://aiidalab.readthedocs.io/en/latest/usage/access/local.html>`_\.
+Adaptations of the AiiDAlab docker images for use with other container engines such as Apptainer 
+are discussed in the 
+`Ada Lovelace Centre's AiiDAlab Development Guide <https://stfc.github.io/alc-ux/>`_\.
+In general the core docker image applicable to most use cases is ``aiidalab/full-stack:latest``
+however, many other options exist for more tailored startup environments.
+
+AiiDAlab provides a python module which encapsulates the creation and management of the required
+containerised environments for running AiiDAlab instances and controlling the AiiDA database 
+configuration. This can be installed through ``pip`` as,
+
+.. code:: bash 
+
+    pip install aiidalab-launch
+
+with further details given in the `documentation <https://aiidalab.readthedocs.io/en/latest/usage/access/launch.html>`_\.
+
+AiiDAlab ChemShell Containers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Multiple docker images are provided with this repository which build on the core foundation of the images
+provided by AiiDAlab bundling various components of the ChemShell workflows on-top of the core AiiDAlab
+application. At present two images are available; 
+`base <https://github.com/stfc/aiidalab-chemshell/pkgs/container/aiidalab-chemshell%2Fbase>`_ includes 
+the AiiDAlab ChemShell plugin pre-installed including all required dependencies,
+`full <https://github.com/stfc/aiidalab-chemshell/pkgs/container/aiidalab-chemshell%2Ffull>`_ 
+builds upon the base package including a working installation of ChemShell (version 25) configured
+with DL_POLY, NWChem and PySCF as available backends.
+
+To run one of the provided containers, first install and setup your desired container engine,
+then run the image as follows,
+
+.. code:: bash
+
+    docker run -it --rm -p 8888:8888 -v $HOME:/home/jovyan ghcr.io/stfc/aiidalab-chemshell/base:latest
+
+
+The additional run parameters will run the container interactively (``-it``), delete it when it is finished
+(``--rm``), expose the required port for the jupyter notebook instance (``-p 8888:8888``) and the final flag
+(``-v``) binds the home directory into the containers home directory so data can be made available and will
+persist beyond the container instance. For a more detailed description on how to configure 
+containers for AiiDAlab see `https://stfc.github.io/alc-ux <https://stfc.github.io/alc-ux>`_\.
+
+
+
+ChemShell Plugin
+----------------
+
+This AiiDAlab plugin is based around running workflows with the ChemShell multiscale chemical
+modelling software alongside providing UI components for managing common AiiDA components and
+tasks. The core component of the plugin is the **calculation workflow wizard** which enables
+the configuration and computation of several common scientific workflows with enhanced 
+visualisation for inputs/outputs and convenient workflow configuration options. A general
+outline of the application is given below with more details on the workflow configuration
+and AiiDA resource management steps given in :ref:`workflows` and :ref:`resource_management`
+respectively. 
+
+Home Page
+~~~~~~~~~
+
+.. figure:: ../../../images/screenshots/aiidalab_home.png
+    :width: 80%
+    :alt: AiiDAlab Home Page
+
+
+When the AiiDAlab application is first opened up in a browser it defaults to the AiiDAlab 
+`home page <https://aiidalab.readthedocs.io/en/latest/usage/home.html>`_ which consists 
+of some buttons to access general functionality including a terminal within the container
+instance and a store front for installing/managing plugins. Below these navigation components
+exists a list of banners for each plugin that is installed and registered in the current
+system. This is where users will access the specific plugins for ChemShell and other
+available workflows.
+
+The ChemShell plugin displays a simple start banner with several options for accessing
+different components within the plugin application. 
+
+.. figure:: ../../../images/screenshots/start_banner.png 
+    :width: 80%
+    :alt: AiiDAlab ChemShell plugin's start banner
+
+
+The navigation buttons access the following pages:
+
+- *New Calculation* -> Accesses the main workflow submission page. (:ref:`workflows`)
+- *History* -> Accesses previous calculations and their results. (:ref:`history_page`)
+- *Setup Resources* -> Accesses the AiiDA computer/code setup page. (:ref:`resource_management`)
+- *Documentation* -> Link to this documentation.
+
+Each of these pages is described in more detail throughout this documentation.

docs/source/user_guide/history_page.rst

@@ -0,0 +1,26 @@
+.. _history_page:
+
+Calculation History
+===================
+
+.. figure:: ../../../images/screenshots/history_page.png
+    :width: 80%
+    :alt: AiiDAlab ChemShell Process History Page
+
+
+This page allows users to search through previously submitted processes and displays
+key information such as AiiDA database references and the calculation results. The
+first half of the display contains a database search UI component which enables more
+tailored search queries, useful if the database is significantly large. By default
+it will simply search for all items in the database with the process key (i.e. all
+previously submitted workflow/calculation processes) which will include any processes submitted
+by the user not just those submitted through the AiiDAlab ChemShell interface. 
+
+Once a process has been selected it becomes visible in the tree view in the second half
+of the display. This visualiser mirrors that of the results view when the workflow was
+originally submitted via the main ChemShell UI. The main process displays its state and
+can be expanded to show a list of associated results objects for the job. Each of these
+objects can then be selected and will be shown in the visualiser at the bottom of the page. 
+This will either show a dedicated visualiser for the results object or if one is not 
+available it will show the AiiDA database reference for the object. Example for different
+supported visualisers are discussed in :ref:`node_viewers`\. 

docs/source/user_guide/node_viewers.rst

@@ -0,0 +1,73 @@
+.. _node_viewers:
+
+Data Node visualisation
+=======================
+
+The AiiDAlab ChemShell plugin supports a wide range of data visualisation options for both
+calculation inputs and outputs. A selection of currently supported visualisation widgets is
+given below.
+
+
+Structure Visualiser
+~~~~~~~~~~~~~~~~~~~~
+
+.. figure:: ../../../images/screenshots/results_structure_viewer.png
+    :width: 80%
+    :alt: View for a chemical structure
+
+
+One of the core abilities of the AiiDAlab ChemShell UI is to visualise chemical structures.
+This is the same visualiser as used in the structure input step but here it is used to
+visualise calculation results such as the optimised geometry of the given structure.
+
+
+Array Visualiser
+~~~~~~~~~~~~~~~~
+
+.. figure:: ../../../images/screenshots/results_array_viewer.png
+    :width: 80%
+    :alt: View for an array of data values
+
+Certain job types in ChemShell return an array of values, such as the forces on each
+atom after a single point calculation or geometry optimisation. These can be visualised
+in a table as shown here, 
+
+
+Folder/File Visualiser
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. figure:: ../../../images/screenshots/results_file_viewer.png
+    :width: 80%
+    :alt: View for a file within the retrieved objects list
+
+
+By default AiiDA produces a dictionary style folder object which contains all the files
+that were returned as part of the AiiDA (ChemShell) process. All the items within this 
+folder can be viewed as files and can be switched between using the drop down menu provided.
+As part of the retrieved objects dictionary that AiiDA returns for any ChemShell process
+the main ChemShell output log is included (*output.log*) and can be viewed directly
+in the UI.
+
+
+Single Value/AiiDA Node Viewer
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. figure:: ../../../images/screenshots/results_single_value_viewer.png
+    :width: 80%
+    :alt: View display for a single value AiiDA node
+
+
+Any value that is simply a single value (integer/floating point) or a type that doesn't have
+a dedicated visualiser is viewer as an AiiDA data noe. This includes the *type*, *uuid*, 
+*node pk* and the value if it is a simple data value. The example shown is for a single floating
+point number that corresponds to the final energy of the system that has been optimised. 
+
+.. note:: Energies outputted by ChemShell are typically in **atomic units** (Hartree). Common conversions 
+    are:
+
+    - 1 Hartree
+    - 27.211386 eV
+    - 2625.50 kJ/mol
+    - 627.509 kcal/mol
+    - 2.194746 wavenumbers
+    - 4.359745x10-18 Joules

docs/source/user_guide/resource_management.rst

@@ -0,0 +1,126 @@
+.. _resource_management:
+
+Resource Management
+===================
+
+AiiDA relies heavily on *computer* and *code* instances to be able to know where and how
+to run the underlying software stacks through any provided plugin. This page allows the
+user to configure the available *computer* and *code* instances within the AiiDAlab
+interface controlling which core software stacks the plugins have access to. Often the 
+provided container images come bundled with the local computer available as *localhost*
+and a selection of pre-installed software *code* instances, such as ChemShell if using the
+provided AiiDAlab ChemShell docker images. A list of all available codes can be seen at the
+bottom of the page including a search bar to search for a specific codes and the ability to
+hide certain codes from the AiiDAlab interface. The rest of this page is dedicated to 
+configuring new *computer* and *code* instances.
+
+Quick Setup
+-----------
+
+
+Manual Setup
+------------
+
+If your required software stack is not available within a quick setup database you will need
+to manually configure the different components. First you must check the box labelled *Tick checkbox
+to setup resource step by step* to enable to advanced configuration options. This will present
+three tabs which allow a user to configure SSH connections, computer and code instances respectively.
+
+
+SSH Connections
+~~~~~~~~~~~~~~~
+
+When setting up a completely fresh connection to a **remote HPC** the first step is to setup
+the ability to communicate with the remote server. AiiDA utilises SSH key authorisation by
+default to communicate with the remote machine in the background, if you already have 
+passwordless SSH authorisation enabled you can skip this step, otherwise you will need to
+configure the connection either through this input tab or externally outside AiiDAlab. 
+
+This tab utilises a pre-connection step based on a provided password which will then setup
+the SSH key authorisation with the remote machine. First ensure the *Verification mode* option
+if set to *Provide password to remote machine*. The user must then provide the hostname/address
+of the remote HPC machine alongside their username and password for the remote machine. 
+
+Once all the required fields have been filled click the *Setup ssh* button and AiiDA will run
+the required setup process and test the resulting connection in the background and will 
+notify the user on a successful setup.
+
+.. note:: 
+
+    Please ensure you remote resources support full passwordless SSH key based authorisation.
+    At present AiiDA is incompatible with system that require password authentication and 
+    whilst it can work with MFA based login methods these are not guaranteed. 
+
+
+Computer Instances
+~~~~~~~~~~~~~~~~~~
+
+Once a SSH connection has been configured either manually or through the provided UI utility, 
+a **computer** instance needs to be created to tell AiiDA how to run jobs on the remote 
+machine. A breakdown of the various inputs is given as follows:
+
+- **Computer name:** - the reference name given to the computer in the AiiDA database.
+- **Hostname:** - The address of the remote machine.
+- **Computer description**: A reference description applied to the computer in the AiiDA database.
+- **AiiDA working directory**: The directory on the remote machine where AiiDA jobs will run.
+- **Mpirun command:** - How to run mpi based jobs on the remote machine.
+- **#CPU(s) per node:** - Maximum number of cpus per node on the remote machine.
+- **Memory per node:** - Maximum memory per node on the remote machine.
+- **Transport type:** - How to connect to the remote machine, (SSH or local).
+- **Min. connection interval:** - The minimum time between connection requests sent to the remote machine.
+- **Scheduler:** - The jobs scheduler used on the remote system (use core.direct if no scheduler is implemented).
+- **Shebang:** - The top line of any scripts created specifying how they are interpreted.
+- **Use login shell** - Runs additional user configurations when connecting to the remote machine.
+- **Use double quotes to escape...** - Use double quotes instead of single for bash commands.
+- **Prepend text:** - Additional commands to run before the main executable is called.
+- **Append text:** - Additional commands to run after the main executable is called.
+
+An example for setting up the ChemShell code on a remote HPC that utilised the SLURM scheduler is given
+as follows, with any remaining fields left as their default values,
+
+.. code:: yaml
+
+    label: "remote1"
+    hostname: "remote1.ac.uk"
+    transport: "core.ssh"
+    scheduler: "core.slurm"
+    work_dir: "/home/user/.aiida_run"
+    mpirun_command: "srun "
+    mpiprocs_per_machine: "32"
+    prepend_text: "#SBATCH --partition=default"
+
+
+Clicking the *Setup computer* button will run AiiDA's computer setup and testing routines and will 
+provide a message if it is successful.  
+
+
+Code Instances
+~~~~~~~~~~~~~~
+
+Once AiiDA knows how to connect and talk to a computer it then needs to be able to call the
+relevant software executable which is where the *code* instance comes in. The Code tab 
+allows the setup to new code instances with the following inputs options, 
+
+- **AiiDA code label:** - The reference name for the code instance in the AiiDA database.
+- **Select computer:** - The computer instance on which the code is located.
+- **Code plugin:** - The AiiDA plugin used to handle the software.
+- **Code description:** - The reference description for the code instance in the AiiDA database.
+- **Absolute path to executable:** - The executable for the given software.
+- **Use double quotes to escape...** - Use double quotes instead of single for bash commands.
+- **Prepend text:** - Additional commands to run before the main executable is called.
+- **Append text:** - Additional commands to run after the main executable is called.
+
+An example for setting up the ChemShell code on a remote machine is given below,
+
+.. code:: yaml
+
+    label: "ChemShell (SCARF)",
+    description: "ChemShell 25.0.1 (parallel) compiled for SCARF",
+    filepath_executable: "chemsh.x",
+    default_calc_job_plugin: "chemshell",
+    prepend_text: "module load contrib/chemshell/25.0.1-intel",
+    append_text: "",
+
+
+Clicking the *Setup code* button will run AiiDA's code setup and testing routines and will 
+provide a message if it is successful.