S. Riette 14 oct 2022: documentation update

docs/CodingNorms.md

+# PHYEX developer documentation
+
+## ABOUT THIS DOCUMENT
+
+This document is intended for developers and integrators and describes the coding norms to use.
+
+This document is written using the markdown language. With pandoc, it can be converted to HTML (pandoc -s \<filename\>.md -o \<filename\>.html) or PDF (pandoc -s \<filename\>.md -o \<filename\>.pdf).
+
+## CODING NORMS
+
+### Namelists
+We must be able to reproduce (binary comparison of the output files) the model results before and after code modifications. It means that every modification must be controled by a namelist key (with the exception of bug corrections).
+
+### File names
+The fortran file names use a capital F letter (eg: foo.F90) except if working in a Meso-NH branch (mesonh\_\<commit\>) or in the folder (src/mesonh) specific to the Meso-NH model.
+
+Names for the module:
+
+  - modd\_ for module containing only variable declaration (eg: tuning parameters)
+  - modi\_ for module containing only interface declaration
+  - modn\_ for namelist declaration
+  - mode\_ for module containing executable source code (subroutine or function)
+
+### When using mode\_ or modi\_?
+When writing a new subroutine, should we put it in a module (in a mode\_ file) or should we write the subroutine in a file and write the interface bloc in another file (modi\_ file)?
+
+The answer depends on whether the routine is the 'main' routine of the parameterisation or not. If it is the 'main' routine, the interface bloc is declared apart, if not we can use a module.
+The idea behind is to break compilation dependency at the parameterisation level, and to isolate the interface declaration of the different routines that must be pluged in the hosting model.
+
+### Norm
+Several constraints are imposed:
+
+  - The code must be written with up to 132 characters per line.
+  - CODE IS IN CAPITAL LETTERS! comments in small letters
+  - All variables must be declared: IMPLICIT NONE
+  - except in rare cases, use automatic arrays, no allocatable
+  - dimensions of dummy argument arrays are explicit (no (:,:))
+  - use parenthesis when manipulating arrays (eg: A(:,:)=B(:,:)+C(:,:) instead of A=B+C)
+
+The variables are named according to the doctor norm:
+
+|Type / Status | INTEGER    | REAL       | LOGICAL     | CHARACTER      | TYPE             |
+|--------------|------------|------------|-------------|----------------|------------------|
+|Global        | N          | X          | L (not LP)  | C              | T (not TP,TS,TZ) |
+|Dummy argument| K          | P (not PP) | O           | H              | TP               |
+|Local         | I (not IS) | Z (not ZS) | G (not GS)  | Y (not YS, YP) | TZ               |
+|Loop control  | J (not JP) | -          | -           | -              | -                |
+
+Regarding array-syntax, code is written using array-syntax in the main branch and in mesonh specific branches based on the GPU branch, using array-syntax with mnh\_expand directives in the GPU branch, using DO loops in arome specific branches based on the GPU branch. If in doublt, check what is done in other routines in the brach you are working in.
+Be carrefull when using the mnh\_expand directives, code must respect some constraints:
+
+  - parenthesis after array variables are mandatory (no A=B+C, but A(:,:)=B(:,:)+C(:,:))
+  - no space between array variables and the opening parenthesis (no A (:)=B (:), but A(:)=B(:))
+  - same bounds as declared in the mnh\_expand directive should be used in the array-syntax (A(D%NIB;D%NIE)=...)
+
+A tool (verify\_mnh\_expand.py) can help at checking the validity of the written code.
+
+For the GPU branch (and branches on GPU, including model specific branches):
+
+  - except variables declared with the PARAMETER attribute, no variable from modules can be used in the physics. Varaibles must be put in a type received by interface.
+  - subroutines or functions must not be called from within a loop on horizontal or vertical dimensions (see below for exception)
+  - functions returning arrays must be rewritten as subroutine
+
+Call to external subroutine in loop on horizontal or vertical dimensions must be suppressed in the GPU version. If possible, the call must be put outside of the loop (acting on the full array as a whole) or the subroutine must be put in the CONTAINS part but, in this case, the included subroutine cannot use local array. There are 3 cases:
+
+  - the subroutine does't use local array: subroutine is put in an include file (with the .h extension) and included with the fortran INCLUDE statement.
+  - the subroutine use local arrays but it is called from only one place in the code: the source code of the subroutine is moved (no INCLUDE) in the CONTAINS part and the array declarations are moved in the main subroutine.
+  - the subroutine use local arrays and is called from several places: the previous technique is not recommended. The source code is put in an include file (with the .h extension) and an extra argument is provided to the subroutine and is used as a buffer so there is no more need to declare local arrays in the called subroutine.
+
+### Budgets
+
+In Meso-NH, the budget can be used in two ways:
+
+  - by giving to the budget machinery the tendency due to a given process
+  - by giving to the budget machinery the total tendency (S variable) before and after a given process. The budget machanism recomputes by difference the tendency only due to the given process.
+
+In AROME, we cannot provide the total tendency (S variable) before the process. This total tendency is stored internally by the machinery but cannot be set to a different value before doing a computation.
+
+The physics package must be usable from AROME and Meso-NH, several examples are given:
+
+Invalid for AROME:
+```
+budget_store_init(tempo_s)
+modification of tempo_s
+budget_store_end(tempo_s)
+```
+
+Valid:
+```
+budget_store_init(pronostic_s) #useless for AROME, but needed for Meso-NH
+modification of pronostic_s
+budget_store_end(pronostic_s)
+```
+
+Valid:
+```
+computation of delta tempo_s
+budget_store_add(delta tempo_s)
+```

docs/Developer.md

@@ -7,132 +7,87 @@ Developer who is interested in plugging the physics in a new model can refere to
 The topics covered are as follows:
 
   - [Package organisation](#package-organisation)
-  - [Code preparation](#code-preparation)
-  - [Coding norms](#coding-norms)
-  - [Pull requests](#pull-requests)
+  - [Contribution workflow for AROME-HARMONIE developers](#contribution-workflow-for-arome-harmonie-developers)
+  - [Contribution workflow for Méso-NH developers](#contribution-workflow-for-mesonh-developers)
+  - [Contribution workflow for other developers](#contribution-workflow-for-other-developers)
 
 This document is written using the markdown language. With pandoc, it can be converted to HTML (pandoc -s \<filename\>.md -o \<filename\>.html) or PDF (pandoc -s \<filename\>.md -o \<filename\>.pdf).
 
 ## PACKAGE ORGANISATION
 
-The package contains the folowing directories:
+The package contains two kinds of branches:
 
-  - docs: for documentation
-  - build: an autonomous build system is included in the package. Its usage is covered in the [Offline documentation](./Offline.md)
-  - src/common: the main source code which is the basis for all models
-  - src/\<model\>: the source code specific to one model that must replace source code found in the common directory
-
-In addition to this organisation, the package uses git branches. The main branches are as follows:
-
-  - main: source code without rewriting for GPU transformation (used for official Meso-NH source code)
-  - GPU: source code adapted for GPU transformations (used for official AROME source code, starting from the 48t3 cycle)
-  - arome\_\<commit\>: source code ready for inclusion in the AROME compilation environment (the generation of such a branch is described in [Code preparation](#code-preparation))
-  - testprogs\_data: modified source code used to generate samples for the test programs (more on this topic in the [Offline documentation](./Offline.md))
-
-## CODE PREPARATION
-
-The source code stored in the main and GPU branches must be usable by all the models. But these models can have contradictory constraints. To bypass this difficulty, the source code is preprocessed before being included in the compilation environment of each model.
+  - generic branches which contain codes for all the models and applications (eg: main and GPU branches)
+  - model specific branches which are automaticaly derived from generic branches (eg: arome\_\<commit\_hash\>, mesonh\_\<commit\_hash\>)
 
-This preprocessing step can be done on the fly (in this case the preprocessing tools must be available aside of the compilation tools), or the result of the preprocessing can be stored in the PHYEX package (in this case, the preprocessing is done once and can be used by several users).
-This second possibility is usefull to historize the source code really used during the model compilation and enables contributions to the PHYEX package without the need of the preprocessing tools.
+The directories found in the package are different depending on the kind (generic or model specific) branches.
 
-The preprocessed versions of the source code are put in branches named \<model\>\_\<commit\> where \<model\> is the name of the model for which the source code have been preprocessed and \<commit\> is the commit hash used as a basis.
+For model specific branches, only the source code adapted for a given model is present (on directory per parameterization and an aux directory). No compilation engine or scripts are present in these branches. They are intended to be included directly in the compilation system of the hosting model.
 
-The preprocessing tools are described in the [Tools documentation](./Tools.md).
+The generic branches contains the folowing directories:
 
-## CODING NORMS
-
-### File names
-The fortran file names use a capital F letter (eg: foo.F90) except if working a branch (mesonh\_\<commit\>) or in the folder (src/mesonh) specifci to the Meso-NH model.
-
-Names for the module:
-
-  - modd\_ for module containing only variable declaration (eg: tuning parameters)
-  - modi\_ for module containing only interface declaration
-  - modn\_ for namelist declaration
-  - mode\_ for module containing executable source code (subroutine or function)
-
-### When using mode\_ or modi\_?
-When writing a new subroutine, should we put it in a module (in a mode\_ file) or should we write the subroutine in a file and write the interface bloc in another file (modi\_ file)?
+  - docs: for documentation
+  - build: an autonomous build system is included in the package. Its usage is covered in the [Offline documentation](./Offline.md)
+  - src/common: the main source code which is the basis for all models
+  - src/\<model\>: the source code specific to one model that must replace or complement the source code found in the common directory
+  - tools: scripts to build model specific branches and run test cases (described in the [Integrator](./Integrator.md) documentation).
 
-The answer depends on whether the routine is the 'main' routine of the parameterisation or not. If it is the 'main' routine, the interface bloc is declared apart, if not we can use a module.
-The idea behind is to break compilation dependency at the parameterisation level, and to isolate the interface declaration of the different routines that must be pluged in the hosting model.
+Here is a short description of the different generic branches:
 
-### Norm
-Several constraints are imposed:
+  - main: source code without rewriting for GPU transformation
+  - GPU: source code adapted for GPU transformations
+  - testHUGE: modified source code to check the incomplete NPROMA feature
+  - testprogs\_data: modified source code used to generate samples for the test programs (more on this topic in the [Offline documentation](./Offline.md))
 
-  - The code must be written with up to 132 characters per line.
-  - CODE IS IN CAPITAL LETTERS! comments in small letters
-  - All variables must be declared: IMPLICIT NONE
-  - except in rare cases, use automatic arrays, no allocatable
-  - dimensions of dummy argument arrays are explicit (no (:,:))
-  - use parenthesis when manipulating arrays (eg: A(:,:)=B(:,:)+C(:,:) instead of A=B+C)
+## CONTRIBUTION WORKFLOW FOR AROME-HARMONIE DEVELOPERS
 
-The variables are named according to the doctor norm:
+The AROME build systems are evolving.
+Until cycle 49t1 (included), the physics source code is directly included in the source code tree.
+After cycle 49t1, the physics source code (as well as other model parts such as ectrans, fiat...) will be available through "boundles".
 
-|Type / Status | INTEGER    | REAL       | LOGICAL     | CHARACTER      | TYPE             |
-|--------------|------------|------------|-------------|----------------|------------------|
-|Global        | N          | X          | L (not LP)  | C              | T (not TP,TS,TZ) |
-|Dummy argument| K          | P (not PP) | O           | H              | TP               |
-|Local         | I (not IS) | Z (not ZS) | G (not GS)  | Y (not YS, YP) | TZ               |
-|Loop control  | J (not JP) | -          | -           | -              | -                |
+This evolution will impact the way to contribute to the PHYEX repository.
 
-Regarding array-syntax, code is written using array-syntax in the main branch and in mesonh specific branches based on the GPU branch, using array-syntax with mnh\_expand directives in the GPU branch, using DO loops in arome specific branches based on the GPU branch. If in doublt, check what is done in other routines in the brach you are working in.
-Be carrefull when using the mnh\_expand directives, code must respect some constraints:
+But, whatever is the cycle, the AROME-HARMONIE developers only see codes comming from arome specific branches (branches named arome\_\<commit\_hash\>). This code is ready for inclusion (array-syntax already transformed into DO loops for instance).
 
-  - parenthesis after array variables are mandatory (no A=B+C, but A(:,:)=B(:,:)+C(:,:))
-  - no space between array variables and the opening parenthesis (no A (:)=B (:), but A(:)=B(:))
-  - same bounds as declared in the mnh\_expand directive should be used in the array-syntax (A(D%NIB;D%NIE)=...)
+Said differently, developers do not need to manipulate code transformation tools.
 
-A tool (verify\_mnh\_expand.py) can help at checking the validity of the written code.
+## Until cycle 49t1
 
-For the GPU branch (and branches on GPU, including model specific branches):
+Workflow summary: because the physics source code is still included in the IAL source code, pull requests concerning the physics continue to follow the same path as before (ie pull requests are submitted to the IAL repository). Afterwards, the IAL integrator will submit a pull request to the PHYEX repository with only the relevant files.
 
-  - except variables declared with the PARAMETER attribute, no variable from modules can be used in the physics. Varaibles must be put in a type received by interface.
-  - subroutines or functions must not be called from within a loop on horizontal or vertical dimensions (see below for exception)
-  - functions returning arrays must be rewritten as subroutine
+![](./AROMEworkflow1.svg)
 
-Call to external subroutine in loop on horizontal or vertical dimensions must be suppressed in the GPU version. If possible, the call must be put outside of the loop (acting on the full array as a whole) or the subroutine must be put in the CONTAINS part but, in this case, the included subroutine cannot use local array. There are 3 cases:
+Workflow details (getting the source code in blue, pull request in red, integration in green):
 
-  - the subroutine does't use local array: subroutine is put in an include file (with the .h extension) and included with the fortran INCLUDE statement.
-  - the subroutine use local arrays but it is called from only one place in the code: the source code of the subroutine is moved (no INCLUDE) in the CONTAINS part and the array declarations are moved in the main subroutine.
-  - the subroutine use local arrays and is called from several places: the previous technique is not recommended. The source code is put in an include file (with the .h extension) and an extra argument is provided to the subroutine and is used as a buffer so there is no more need to declare local arrays in the called subroutine.
+  - 1: PHYEX administrator sends (pull request) the content of a specific arome branch to the IAL Integrator. The IAL integrator tags a new release of IAL.
+  - 2: AROME-HARMONIE developer forks the IAL repository
+  - 3: AROME-HARMONIE developer compiles, executes, modifies the source code in its environment
+  - 4: AROME-HARMONIE developer sends a pull request to the IAL repository
+  - 5: The IAL integrator extracts the physics source files and sends a pull request to the PHYEX repository
+  - 6: The PHYEX administrator checks and integrates the modifications in the GPU branch and, eventually, produce a new arome specific branch for future integration in IAL
 
-### Budgets
+## After cycle 49t1
 
-In Meso-NH, the budget can be used in two ways:
+Workflow summary: after the cycle 49t1 (starting from 49t2?), HARMONIE/AROME will become a boudle. Il will be built with source codes coming from various places. One of these places will be the PHYEX repository. Pull requests must be send to each modified boundles.
 
-  - by giving to the budget machinery the tendency due to a given process
-  - by giving to the budget machinery the total tendency (S variable) before and after a given process. The budget machanism recomputes by difference the tendency only due to the given process.
+![](./AROMEworkflow2.svg)
 
-In AROME, we cannot provide the total tendency (S variable) before the process. This total tendency is stored internally by the machinery but cannot be set to a different value before doing a computation.
+Workflow details (getting the source code in blue, pull request in red, integration in green):
 
-The physics package must be usable from AROME and Meso-NH, several examples are given:
+  - 1 and 2: AROME-HARMONIE developer forks the different repositories needed to build the model
+  - 3: AROME-HARMONIE developer compiles, executes, modifies the source code in its environment
+  - 4 and 5: AROME-HARMONIE developer sends pull requests to the different repositories where files have been modified
+  - 6: The PHYEX administrator checks the pull requests in the other applications, the IAL integrator integrates on the arome specific branch; then the PHYEX administrator integrates the modifications in the GPU branch and, eventually, produce a new arome specific branch for future integration in IAL
 
-Invalid for AROME:
-```
-budget_store_init(tempo_s)
-modification of tempo_s
-budget_store_end(tempo_s)
-```
 
-Valid:
-```
-budget_store_init(pronostic_s) #useless for AROME, but needed for Meso-NH
-modification of pronostic_s
-budget_store_end(pronostic_s)
-```
+## CONTRIBUTION WORKFLOW FOR MESO-NH DEVELOPERS
 
-Valid:
-```
-computation of delta tempo_s
-budget_store_add(delta tempo_s)
-```
+The physics source code is embedded in the Méso-NH source code.
 
-## PULL REQUESTS
-This section deals with the pull request procedure from the developer point of view. The integrator point of view is described in the Intergator documentation.
+The physics source code comes directly from a mesonh specific branch (these branches are named mesonh\_\<commit\_hash\>) which contain code ready for use in the Méso-NH model (array-syntax...).
 
-To contribute to the PHYEX repository, developer must fork the repository, contribute on the main or on the GPU branch and send a pull request. Alternatively, a contribution on a model specific branch is also possible (especially for minor modifications).
+Pull requests concerning the physics continue to follow the same path as before (ie pull requests are submitted to the Meso-NH repository). The Meso-NH integrator will submit a pull request to the PHYEX repository with only the relevant files.
 
-If a modification must be applied to the main and to the GPU branches, the pull request must be made on the main branch (and will be merged into the GPU branch).
+## CONTRIBUTION WORKFLOW FOR OTHER DEVELOPERS
 
+Other developers must work with source code comming directly from the GPU branch. They issue pull requests directly on this branch as usual with git repositories.

docs/Integrator.md

@@ -8,22 +8,86 @@ This document is written using the markdown language. With pandoc, it can be con
 
 ## BRANCHES AND NORMS
 
-Regarding array-syntax, the applicalble norm depends on the branch:
+Regarding array-syntax, the [applicable norm](./CodingNorms.md) depends on the branch:
 
   - The main branch of PHYEX (and all branches based on main) is written using array-syntax
   - The GPU branch is written using array-syntax with mnh\_expand directives
   - arome specific branches based on the GPU branch are written using DO loops
-  - mesonh specific branches based on the GPU branch are written using array-syntax withour mnh\_expand directives
+  - mesonh specific branches based on the GPU branch are written using array-syntax
 
 Pull requests can be received on all these kind of branches and must be merged into the main or the GPU branch with according norm.
 
+## NORMAL WORKFLOW FOR A CONTRIBUTION DEVELOPED IN AROME-HARMONIE
+
+### Until cycle 49t1
+
+![](./AROMEworkflow1.svg)
+
+The pull request comes from the IAL integrator. It must be based on an arome specific branch.
+
+Details for point 6, the PHYEX administrator:
+  - validates (see [below](#tests)) the contribution
+  - integrates the contribution in the arome branch and merges it in the GPU branch
+  - regularly, he tags a new (minor) version of the GPU branch
+  - when asked by the IAL integrator, he builds a new arome specific branch
+  - when an arome specific branch is used in an official cycle, the arome specific branch is tagged accordingly
+
+### After cycle 49t1
+
+![](./AROMEworkflow2.svg)
+
+The pull request comes directly from a developer. It must be based on an arome specific branch.
+
+Details for point 6:
+  - The PHYEX administrator checks the pull requests in the other applications (see [below](#tests))
+  - The IAL integrator integrates the contribution on the arome specific branch
+  - The PHYEX administrator
+    - integrates the modifications in the GPU branch
+    - regularly, tags a new (minor) version of the GPU branch
+    - when asked by the IAL integrator, builds a new arome specific branch (see [below](#code-preparation))
+    - when an arome specific branch is used in an official cycle, the arome specific branch is tagged accordingly
+
+## NORMAL WORKFLOW FOR A CONTRIBUTION DEVELOPED IN MESONH
+
+The developer sends its pull request on the Méso-NH repository (the physics source code is embedded in the model source code).
+
+Integration details:
+  - The Meso-NH integrator extracts, from the different pull requests, what concern the PHYEX repository and send a pull request on PHYEX based on a mesonh specific branch
+  - The PHYEX administrator:
+    - validates (see [below](#tests)) the contribution
+    - integrates the contribution in the arome branch and merges it in the GPU branch
+    - regularly, he tags a new (minor) version of the GPU branch
+    - when asked by the IAL integrator, he builds a new arome specific branch (see [below](#code-preparation))
+    - when an arome specific branch is used in an official cycle, the arome specific branch is tagged accordingly
+ puis teste les différents modèles et l'intègre dans la branche GPU (avec tag réguliers)
+
+## NORMAL WORKFLOW FOR ANOTHER CONTRIBUTION
+
+Pull requests must be based on the GPU branch.
+  - validates (see [below](#tests)) the contribution
+  - integrates the contribution in the arome branch and merges it in the GPU branch
+  - regularly, he tags a new (minor) version of the GPU branch
+
 ## TESTS
 
-The source code must follow strict mnh\_expand directives (described in the [Developer documentation](./Developer.md)). The script verify\_mnh\_expand.py must be used to give an additional check.
+The source code must follow strict mnh\_expand directives (described in the [Coding Norms documentation](./CodingNorms.md)). The script verify\_mnh\_expand.py must be used to give an additional check.
 
-In addition to the scientific validation, the folowing tests must give the same results (with bit-reproducibility) in each of the model:
+In addition to the scientific validation, the folowing tests must give the same results (with bit-reproducibility) in each of the model (arome, mesonh and testprogs):
 
   - compilation transforming the mnh\_expand directives in DO loop
   - compilation keeping the array-syntax
-  - execution with a different umber of processors
+  - execution with a different number of processors
+
+When possible, the new version of PHYEX must reproduce the old results (scientific modifications must be activated with namelist keys).
+
+## CODE PREPARATION
+
+The source code stored in the main and GPU branches must be usable by all the models. But these models can have contradictory constraints. To bypass this difficulty, the source code is preprocessed before being included in the compilation environment of each model.
+
+This preprocessing step can be done on the fly (in this case the preprocessing tools must be available aside of the compilation tools), or the result of the preprocessing can be stored in the PHYEX package (in this case, the preprocessing is done once and can be used by several users).
+This second possibility is usefull to historize the source code really used during the model compilation and enables contributions to the PHYEX package without the need of the preprocessing tools.
+
+The preprocessed versions of the source code are put in branches named \<model\>\_\<commit\> where \<model\> is the name of the model for which the source code have been preprocessed and \<commit\> is the commit hash used as a basis.
+
+The preprocessing tools are described in the [Tools documentation](./Tools.md).
 

docs/PHYEX.md

@@ -7,8 +7,9 @@ This document is a general presentation of the PHYEX package.
 More specific documentation can be found:
 
  - [Developer](./Developer.md): package organisation, how to contribute, coding norms
+ - [Coding norms](./CodingNorms.md): coding norms
  - [Integrator](./Integrator.md): how to merge contributions
- - [Offline](./Offline.md): how to compile the library and the test programs, how to use the library with python, how to use the test programs
+ - [Offline](./Offline.md): how to compile the library and the test programs, how to use the library with python and how to use the test programs
  - [Plugging](./Plugging.md) : how to plug the physics package in a model
  - [Tools](./Tools.md): description of the check\_commit\_\*.sh scripts (to check bit reproducibility between two commits) and of the prep\_code.sh script
 
@@ -16,15 +17,18 @@ This document is written using the markdown language. With pandoc, it can be con
 
 ## HISTORY
 
-The physics was first developed for the Meso-NH model (http://mesonh.aero.obs-mip.fr/).
+The physics was first developed for the [Meso-NH model](http://mesonh.aero.obs-mip.fr/).
+
 Then, a part of the physics have been used to build the AROME model ([Seity et al, 2011](http://dx.doi.org/10.1175/2010MWR3425.1)).
-The last step was to extract the physics to build the independent PHYEX package.
+
+The last evolution was to extract the physics to build the independent PHYEX package.
 
 ## CONTENT
 
 The folowing paramerisations are included in the PHYEX package (see the MesoNH documentation for references):
+
  - turbulence scheme
  - shallow convection scheme
- - microphysics scheme
+ - microphysics schemes
 
 In addition to the parametrisatin source code, test programs and a library for python binding are also provided.