diff --git a/docs/source/index.rst b/docs/source/index.rst index da47c38..52aa54f 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -7,11 +7,17 @@ Welcome to the AiiDAlab ChemShell plugin documentation. Here we aim to provide a comprehensive guide on how to utilise the plugin to perform various ChemShell based workflows within the AiiDA/AiiDAlab workflow environment. This documentation refers to the AiiDAlab UI for the AiiDA ChemShell integration, for the core AiiDA -ChemShell plugin please see `https://stfc.github.io/aiida-chemshell/`_\. +ChemShell plugin please see `aiida-chemshell `_\. .. toctree:: :maxdepth: 2 :caption: Contents: + user_guide/getting_started + user_guide/workflows + user_guide/history_page + user_guide/resource_management + user_guide/node_viewers + api_docs/modules diff --git a/docs/source/user_guide/getting_started.rst b/docs/source/user_guide/getting_started.rst new file mode 100644 index 0000000..7866dc3 --- /dev/null +++ b/docs/source/user_guide/getting_started.rst @@ -0,0 +1,121 @@ +.. _getting_started: + +Getting Started +=============== + +ChemShell +--------- + +`ChemShell `_ +is a feature rich multiscale chemical modelling environment that leverages the +power of python scripting to design workflows encompassing a range of quantum mechanics (QM) +and/or molecular mechanics (MM) software packages into a one-stop analysis tool. Focussing on +multiscale simulation of complex systems using combined QM/MM methods it is fully scalable +from your desktop to massively parallel supercomputers. ChemShell provides a suite of advanced +modelling methods for geometry optimisation, energy surface mapping, molecular dynamics, monte +carlo, free energy methods, excited states and more, all available for quantum, classical and +hybrid QM/MM calculations. + +AiiDAlab +-------- + +`AiiDAlab `_ adapts the highly popular `AiiDA `_ +workflow management platform to provide an enhanced user interface (UI) based platform for +carrying out complex computational scientific workflows. This plugin is designed to expose +several common workflows utilising the ChemShell multiscale modelling software with a convenient +and user friendly step-by-step approach. + +Running AiiDAlab +~~~~~~~~~~~~~~~~ + +It is generally recommended to run AiiDAlab through a container engine such as Docker or Apptainer, +details on how to use Docker to run AiiDAlab can be found in the +`AiiDAlab documentation `_\. +Adaptations of the AiiDAlab docker images for use with other container engines such as Apptainer +are discussed in the +`Ada Lovelace Centre's AiiDAlab Development Guide `_\. +In general the core docker image applicable to most use cases is ``aiidalab/full-stack:latest`` +however, many other options exist for more tailored startup environments. + +AiiDAlab provides a python module which encapsulates the creation and management of the required +containerised environments for running AiiDAlab instances and controlling the AiiDA database +configuration. This can be installed through ``pip`` as, + +.. code:: bash + + pip install aiidalab-launch + +with further details given in the `documentation `_\. + +AiiDAlab ChemShell Containers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Multiple docker images are provided with this repository which build on the core foundation of the images +provided by AiiDAlab bundling various components of the ChemShell workflows on-top of the core AiiDAlab +application. At present two images are available; +`base `_ includes +the AiiDAlab ChemShell plugin pre-installed including all required dependencies, +`full `_ +builds upon the base package including a working installation of ChemShell (version 25) configured +with DL_POLY, NWChem and PySCF as available backends. + +To run one of the provided containers, first install and setup your desired container engine, +then run the image as follows, + +.. code:: bash + + docker run -it --rm -p 8888:8888 -v $HOME:/home/jovyan ghcr.io/stfc/aiidalab-chemshell/base:latest + + +The additional run parameters will run the container interactively (``-it``), delete it when it is finished +(``--rm``), expose the required port for the jupyter notebook instance (``-p 8888:8888``) and the final flag +(``-v``) binds the home directory into the containers home directory so data can be made available and will +persist beyond the container instance. For a more detailed description on how to configure +containers for AiiDAlab see `https://stfc.github.io/alc-ux `_\. + + + +ChemShell Plugin +---------------- + +This AiiDAlab plugin is based around running workflows with the ChemShell multiscale chemical +modelling software alongside providing UI components for managing common AiiDA components and +tasks. The core component of the plugin is the **calculation workflow wizard** which enables +the configuration and computation of several common scientific workflows with enhanced +visualisation for inputs/outputs and convenient workflow configuration options. A general +outline of the application is given below with more details on the workflow configuration +and AiiDA resource management steps given in :ref:`workflows` and :ref:`resource_management` +respectively. + +Home Page +~~~~~~~~~ + +.. figure:: ../../../images/screenshots/aiidalab_home.png + :width: 80% + :alt: AiiDAlab Home Page + + +When the AiiDAlab application is first opened up in a browser it defaults to the AiiDAlab +`home page `_ which consists +of some buttons to access general functionality including a terminal within the container +instance and a store front for installing/managing plugins. Below these navigation components +exists a list of banners for each plugin that is installed and registered in the current +system. This is where users will access the specific plugins for ChemShell and other +available workflows. + +The ChemShell plugin displays a simple start banner with several options for accessing +different components within the plugin application. + +.. figure:: ../../../images/screenshots/start_banner.png + :width: 80% + :alt: AiiDAlab ChemShell plugin's start banner + + +The navigation buttons access the following pages: + +- *New Calculation* -> Accesses the main workflow submission page. (:ref:`workflows`) +- *History* -> Accesses previous calculations and their results. (:ref:`history_page`) +- *Setup Resources* -> Accesses the AiiDA computer/code setup page. (:ref:`resource_management`) +- *Documentation* -> Link to this documentation. + +Each of these pages is described in more detail throughout this documentation. \ No newline at end of file diff --git a/docs/source/user_guide/history_page.rst b/docs/source/user_guide/history_page.rst new file mode 100644 index 0000000..f8fcbfb --- /dev/null +++ b/docs/source/user_guide/history_page.rst @@ -0,0 +1,26 @@ +.. _history_page: + +Calculation History +=================== + +.. figure:: ../../../images/screenshots/history_page.png + :width: 80% + :alt: AiiDAlab ChemShell Process History Page + + +This page allows users to search through previously submitted processes and displays +key information such as AiiDA database references and the calculation results. The +first half of the display contains a database search UI component which enables more +tailored search queries, useful if the database is significantly large. By default +it will simply search for all items in the database with the process key (i.e. all +previously submitted workflow/calculation processes) which will include any processes submitted +by the user not just those submitted through the AiiDAlab ChemShell interface. + +Once a process has been selected it becomes visible in the tree view in the second half +of the display. This visualiser mirrors that of the results view when the workflow was +originally submitted via the main ChemShell UI. The main process displays its state and +can be expanded to show a list of associated results objects for the job. Each of these +objects can then be selected and will be shown in the visualiser at the bottom of the page. +This will either show a dedicated visualiser for the results object or if one is not +available it will show the AiiDA database reference for the object. Example for different +supported visualisers are discussed in :ref:`node_viewers`\. diff --git a/docs/source/user_guide/node_viewers.rst b/docs/source/user_guide/node_viewers.rst new file mode 100644 index 0000000..5e22d74 --- /dev/null +++ b/docs/source/user_guide/node_viewers.rst @@ -0,0 +1,73 @@ +.. _node_viewers: + +Data Node visualisation +======================= + +The AiiDAlab ChemShell plugin supports a wide range of data visualisation options for both +calculation inputs and outputs. A selection of currently supported visualisation widgets is +given below. + + +Structure Visualiser +~~~~~~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/results_structure_viewer.png + :width: 80% + :alt: View for a chemical structure + + +One of the core abilities of the AiiDAlab ChemShell UI is to visualise chemical structures. +This is the same visualiser as used in the structure input step but here it is used to +visualise calculation results such as the optimised geometry of the given structure. + + +Array Visualiser +~~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/results_array_viewer.png + :width: 80% + :alt: View for an array of data values + +Certain job types in ChemShell return an array of values, such as the forces on each +atom after a single point calculation or geometry optimisation. These can be visualised +in a table as shown here, + + +Folder/File Visualiser +~~~~~~~~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/results_file_viewer.png + :width: 80% + :alt: View for a file within the retrieved objects list + + +By default AiiDA produces a dictionary style folder object which contains all the files +that were returned as part of the AiiDA (ChemShell) process. All the items within this +folder can be viewed as files and can be switched between using the drop down menu provided. +As part of the retrieved objects dictionary that AiiDA returns for any ChemShell process +the main ChemShell output log is included (*output.log*) and can be viewed directly +in the UI. + + +Single Value/AiiDA Node Viewer +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/results_single_value_viewer.png + :width: 80% + :alt: View display for a single value AiiDA node + + +Any value that is simply a single value (integer/floating point) or a type that doesn't have +a dedicated visualiser is viewer as an AiiDA data noe. This includes the *type*, *uuid*, +*node pk* and the value if it is a simple data value. The example shown is for a single floating +point number that corresponds to the final energy of the system that has been optimised. + +.. note:: Energies outputted by ChemShell are typically in **atomic units** (Hartree). Common conversions + are: + + - 1 Hartree + - 27.211386 eV + - 2625.50 kJ/mol + - 627.509 kcal/mol + - 2.194746 wavenumbers + - 4.359745x10-18 Joules \ No newline at end of file diff --git a/docs/source/user_guide/resource_management.rst b/docs/source/user_guide/resource_management.rst new file mode 100644 index 0000000..4960510 --- /dev/null +++ b/docs/source/user_guide/resource_management.rst @@ -0,0 +1,126 @@ +.. _resource_management: + +Resource Management +=================== + +AiiDA relies heavily on *computer* and *code* instances to be able to know where and how +to run the underlying software stacks through any provided plugin. This page allows the +user to configure the available *computer* and *code* instances within the AiiDAlab +interface controlling which core software stacks the plugins have access to. Often the +provided container images come bundled with the local computer available as *localhost* +and a selection of pre-installed software *code* instances, such as ChemShell if using the +provided AiiDAlab ChemShell docker images. A list of all available codes can be seen at the +bottom of the page including a search bar to search for a specific codes and the ability to +hide certain codes from the AiiDAlab interface. The rest of this page is dedicated to +configuring new *computer* and *code* instances. + +Quick Setup +----------- + + +Manual Setup +------------ + +If your required software stack is not available within a quick setup database you will need +to manually configure the different components. First you must check the box labelled *Tick checkbox +to setup resource step by step* to enable to advanced configuration options. This will present +three tabs which allow a user to configure SSH connections, computer and code instances respectively. + + +SSH Connections +~~~~~~~~~~~~~~~ + +When setting up a completely fresh connection to a **remote HPC** the first step is to setup +the ability to communicate with the remote server. AiiDA utilises SSH key authorisation by +default to communicate with the remote machine in the background, if you already have +passwordless SSH authorisation enabled you can skip this step, otherwise you will need to +configure the connection either through this input tab or externally outside AiiDAlab. + +This tab utilises a pre-connection step based on a provided password which will then setup +the SSH key authorisation with the remote machine. First ensure the *Verification mode* option +if set to *Provide password to remote machine*. The user must then provide the hostname/address +of the remote HPC machine alongside their username and password for the remote machine. + +Once all the required fields have been filled click the *Setup ssh* button and AiiDA will run +the required setup process and test the resulting connection in the background and will +notify the user on a successful setup. + +.. note:: + + Please ensure you remote resources support full passwordless SSH key based authorisation. + At present AiiDA is incompatible with system that require password authentication and + whilst it can work with MFA based login methods these are not guaranteed. + + +Computer Instances +~~~~~~~~~~~~~~~~~~ + +Once a SSH connection has been configured either manually or through the provided UI utility, +a **computer** instance needs to be created to tell AiiDA how to run jobs on the remote +machine. A breakdown of the various inputs is given as follows: + +- **Computer name:** - the reference name given to the computer in the AiiDA database. +- **Hostname:** - The address of the remote machine. +- **Computer description**: A reference description applied to the computer in the AiiDA database. +- **AiiDA working directory**: The directory on the remote machine where AiiDA jobs will run. +- **Mpirun command:** - How to run mpi based jobs on the remote machine. +- **#CPU(s) per node:** - Maximum number of cpus per node on the remote machine. +- **Memory per node:** - Maximum memory per node on the remote machine. +- **Transport type:** - How to connect to the remote machine, (SSH or local). +- **Min. connection interval:** - The minimum time between connection requests sent to the remote machine. +- **Scheduler:** - The jobs scheduler used on the remote system (use core.direct if no scheduler is implemented). +- **Shebang:** - The top line of any scripts created specifying how they are interpreted. +- **Use login shell** - Runs additional user configurations when connecting to the remote machine. +- **Use double quotes to escape...** - Use double quotes instead of single for bash commands. +- **Prepend text:** - Additional commands to run before the main executable is called. +- **Append text:** - Additional commands to run after the main executable is called. + +An example for setting up the ChemShell code on a remote HPC that utilised the SLURM scheduler is given +as follows, with any remaining fields left as their default values, + +.. code:: yaml + + label: "remote1" + hostname: "remote1.ac.uk" + transport: "core.ssh" + scheduler: "core.slurm" + work_dir: "/home/user/.aiida_run" + mpirun_command: "srun " + mpiprocs_per_machine: "32" + prepend_text: "#SBATCH --partition=default" + + +Clicking the *Setup computer* button will run AiiDA's computer setup and testing routines and will +provide a message if it is successful. + + +Code Instances +~~~~~~~~~~~~~~ + +Once AiiDA knows how to connect and talk to a computer it then needs to be able to call the +relevant software executable which is where the *code* instance comes in. The Code tab +allows the setup to new code instances with the following inputs options, + +- **AiiDA code label:** - The reference name for the code instance in the AiiDA database. +- **Select computer:** - The computer instance on which the code is located. +- **Code plugin:** - The AiiDA plugin used to handle the software. +- **Code description:** - The reference description for the code instance in the AiiDA database. +- **Absolute path to executable:** - The executable for the given software. +- **Use double quotes to escape...** - Use double quotes instead of single for bash commands. +- **Prepend text:** - Additional commands to run before the main executable is called. +- **Append text:** - Additional commands to run after the main executable is called. + +An example for setting up the ChemShell code on a remote machine is given below, + +.. code:: yaml + + label: "ChemShell (SCARF)", + description: "ChemShell 25.0.1 (parallel) compiled for SCARF", + filepath_executable: "chemsh.x", + default_calc_job_plugin: "chemshell", + prepend_text: "module load contrib/chemshell/25.0.1-intel", + append_text: "", + + +Clicking the *Setup code* button will run AiiDA's code setup and testing routines and will +provide a message if it is successful. \ No newline at end of file diff --git a/docs/source/user_guide/workflows.rst b/docs/source/user_guide/workflows.rst new file mode 100644 index 0000000..6bddbf4 --- /dev/null +++ b/docs/source/user_guide/workflows.rst @@ -0,0 +1,155 @@ +.. _workflows: + +ChemShell Workflow Configuration +================================ + +.. figure:: ../../../images/screenshots/workflow_page.png + + +This is the main page for the AiiDAlab ChemShell plugin and allows users to configure +and submit ChemShell based computational calculations and view a summary of key results. +The page consists of a header with the same navigational components as the start banner +and a wizard style configuration menu enabling step-by-step configuration of complex +computational workflows which will utilise ChemShell's multiscale modelling capabilities. + +Configuration Wizard +-------------------- + +The primary component of this page is the configuration wizard which itself is broken +down into several steps each of which is required to submit the complete workflow. +Each step contains input fields which the user can interact with to configure the +workflow alongside a **submit** button which will *lock-in* the user's choices submitting +them to the underlying AiiDA engine which will communicate and manage the ChemShell +process. + +Structure Input +~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/structure_step.png + + +This is the first step and often the foundation of any workflow based on chemical +modelling. It provides several options for uploading a chemical structure to the +workflow as different tabs within the UI. The currently supported options are; +uploading a structure file (i.e. a .xyz chemical structure file), SMILES string +input (generate a generic 3D structure from a SMILES string) and a database +search option which enables the inclusion of structures already present in the +AiiDA database either from previous calculations, other plugins or that have +been included via the AiiDA command line interface (CLI). + +These input tabs are then followed by a visualisation box which allows the user +to dynamically visualise the structure they have uploaded (if it is in a supported +format). This box includes different visualisation options including options such +as whether to render the structure as *ball-and-stick*, *vdw* or *stick* representations +for example. It also exposes editing features, however, these are not fully +supported by the AiiDAlab interface. Therefore, whilst users can use the editing +features, to include the updated structure within the workflow configuration the +user will need to save the updated structure as a new structure file and upload this +new structure file via the *Upload Structure* tab. + +Whilst the physical creation/drawing of chemical structures is not directly supported +within the AiiDAlab ChemShell UI, there are many online applications which allow the +drawing of chemical structures and outputting them to *.xyz* files (or as a SMILES string) +which can then be imported into the workflow via the *Upload Structure* tab. Examples +include: + +- list +- of +- exmples + + +Workflow Setup +~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/workflow_step.png + + +This is the core step in configuring the different available workflows within the +AiiDAlab ChemShell plugin. It provides different tabs for each of the available +workflows (see :ref:`available_workflows`) each providing a tailored UI with various +input fields for the different components of the workflow setup. Typically these +will be pre-filled with sensible default values to quickly setup a reasonable +configuration for the given workflow but can then be altered by the user to +further tailor the workflow to suite their specific needs. + +Resource Setup +~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/code_step.png + +This step enables the user to define where and how the underlying ChemShell +software will run. The core inputs are the *AiiDA code instance* which tells +the AiiDA engine where the ChemShell executable exists and how to communicate +with it (more information on AiiDA code instances is given in +:ref:`resource_management`), and the number of CPU cores to provide for the +ChemShell calculation. + +In addition to these inputs it also provides inputs for a *label* and +*description* field which will be associated with the created AiiDA process +for improved future reference, see :ref:`history_page` for more information +on how these values are useful. + + +Job Monitor & Results +~~~~~~~~~~~~~~~~~~~~~ + +.. figure:: ../../../images/screenshots/results_step_waiting.png + +The final step for the wizard is a combined job monitor and results viewer. This +UI component is only available once all previous stages have been completed and a +workflow has been successfully submitted. Once submitted it will appear in the +UI with a key depicting what its status is; *created*, *finished, *excepted* etc. +Initially processes are given the *created* status which means they have successfully +been submitted to the AiiDA engine and are being carried out in the background. +Once finished they will typically be given one of two status keys, if the process +finished normally it will be labelled as *finished*. If if has failed for any reason +it will generally be given the *excepted* key. + +.. figure:: ../../../images/screenshots/results_step_finished.png + +Once a process has finished its associated results objects will be listed in the +displayed access tree under *outputs*. This may require a *refresh* to be properly +updated which can be carried out via the *refresh* button at the bottom of the UI. +Each of the listed results can then be clicked on and visualised in the display +either as references to their AiiDA database object or as a more detailed viewer +if one is supported for the data type. + + +.. note:: Energies outputted by ChemShell are typically in **atomic units** (Hartree). Common conversions + are: + + - 1 Hartree + - 27.211386 eV + - 2625.50 kJ/mol + - 627.509 kcal/mol + - 2.194746 wavenumbers + - 4.359745x10-18 Joules + + + +.. _available_workflows: + +Available Workflow Configurations +--------------------------------- + +Geometry Optimisation +~~~~~~~~~~~~~~~~~~~~~ + +This is the default base workflow provided with the AiiDAlab ChemShell plugin, it represents a geometry +optimisation calculation with optional additional steps such as the evaluation of vibrational frequencies +of the optimised structure. It can be configured for both QM, MM or QM/MM based methods with sensibly chosen +default parameters. The quality of the DFT portion of the calculation can be tuned with the *Basis Quality* +option, however this creates a trade-off with the time required for the calculations. + +The *MM* components of the workflow can be enabled by checking the *Use QM/MM* box which will enable the +remaining input fields. When using *MM* within a calculation the user must provide a pre-configured force +field in the required **DL_POLY** format, which needs to be provided in the *Force Field:* input section. +Additionally, the user needs to specify which atoms to apply the *QM* theory portion of a *QM/MM* calculation +method to, which can be provided as a comma separated list in the *QM Region:* input section. + + + +.. note:: + + More workflows are always being developed and added. If there are any that you would like to see within + this plugin get in touch via the GitHub discussions page. diff --git a/images/screenshots/aiidalab_home.png b/images/screenshots/aiidalab_home.png new file mode 100755 index 0000000..d721d49 Binary files /dev/null and b/images/screenshots/aiidalab_home.png differ diff --git a/images/screenshots/code_step.png b/images/screenshots/code_step.png new file mode 100755 index 0000000..fc26823 Binary files /dev/null and b/images/screenshots/code_step.png differ diff --git a/images/screenshots/history_page.png b/images/screenshots/history_page.png new file mode 100755 index 0000000..8f167fc Binary files /dev/null and b/images/screenshots/history_page.png differ diff --git a/images/screenshots/results_array_viewer.png b/images/screenshots/results_array_viewer.png new file mode 100755 index 0000000..06bc389 Binary files /dev/null and b/images/screenshots/results_array_viewer.png differ diff --git a/images/screenshots/results_file_viewer.png b/images/screenshots/results_file_viewer.png new file mode 100755 index 0000000..ff0de54 Binary files /dev/null and b/images/screenshots/results_file_viewer.png differ diff --git a/images/screenshots/results_single_value_viewer.png b/images/screenshots/results_single_value_viewer.png new file mode 100755 index 0000000..89210d1 Binary files /dev/null and b/images/screenshots/results_single_value_viewer.png differ diff --git a/images/screenshots/results_step_finished.png b/images/screenshots/results_step_finished.png new file mode 100755 index 0000000..b803456 Binary files /dev/null and b/images/screenshots/results_step_finished.png differ diff --git a/images/screenshots/results_step_waiting.png b/images/screenshots/results_step_waiting.png new file mode 100755 index 0000000..dd6c3d4 Binary files /dev/null and b/images/screenshots/results_step_waiting.png differ diff --git a/images/screenshots/results_structure_viewer.png b/images/screenshots/results_structure_viewer.png new file mode 100755 index 0000000..d7a4b5a Binary files /dev/null and b/images/screenshots/results_structure_viewer.png differ diff --git a/images/screenshots/start_banner.png b/images/screenshots/start_banner.png new file mode 100755 index 0000000..f41a593 Binary files /dev/null and b/images/screenshots/start_banner.png differ diff --git a/images/screenshots/structure_step.png b/images/screenshots/structure_step.png new file mode 100755 index 0000000..b74a04e Binary files /dev/null and b/images/screenshots/structure_step.png differ diff --git a/images/screenshots/workflow_page.png b/images/screenshots/workflow_page.png new file mode 100755 index 0000000..11d478a Binary files /dev/null and b/images/screenshots/workflow_page.png differ diff --git a/images/screenshots/workflow_step.png b/images/screenshots/workflow_step.png new file mode 100755 index 0000000..b6ad508 Binary files /dev/null and b/images/screenshots/workflow_step.png differ