S. Riette 30/06/2022 Add some documentation

README.md

 # PHYEX
 PHYsique EXternalisée
 
+## Doc
+
+Some documentation can be found in the doc directory.
+
 ## Build
 
 The build systems can be found in the `build` directory.

docs/Developer.md

+# PHYEX developer documentation
+
+# ABOUT THIS DOCUMENT
+
+This document is intended for developers who want to contribute to the PHYEX package.
+Developer who is interested in plugging the physics in a new model can refere to the Plugging documentation.
+The topics covered are as follows:
+ - [Package organisation](#PACKAGE-ORGANISATION)
+ - [Code preparation](#CODE-PREPARATION)
+ - [Coding norms](#CODING-NORMS)
+ - [Pull requests](#PULL-REQUESTS)
+
+This document is written using the markdown "language". With pandoc, it can be converted to HTML (pandoc \<filename\>.md -o \<filename\>.html) or PDF (pandoc \<filename\>.md -o \<filename\>.pdf).
+
+# PACKAGE ORGANISATION
+
+The package contains the folowing directories:
+ - docs: for documentation
+ - build: an autonomous build system is included in the package. Its usage is covered in the Offline documentation
+ - src/common: the main source code which is the basis for all models
+ - src/\<model\>: the source code specific to one model that must replace source code found in the common directory
+
+In addition to this organisation, the package uses git branches. The main branches are as follows:
+ - main: source code without rewriting for GPU transformation (used for official Meso-NH source code)
+ - GPU: source code adapted for GPU transformations (used for official AROME source code, starting from the 48t3 cycle)
+ - arome\_\<commit\>: source code ready for inclusion in the AROME compilation environment (the generation of such a branch is described in [Code preparation](#CODE-PREPARATION))
+ - testprogs\_data: modified source code used to generate samples for the test programs (more on this topic in the Offline documentation)
+
+# CODE PREPARATION
+
+The source code stored in the main and GPU branches must be usable by all the models. But these models can have contradictory constraints. To bypass this difficulty, the source code is preprocessed before being included in the compilation environment of each model.
+
+This preprocessing step can be done on the fly (in this case the preprocessing tools must be available aside of the compilation tools), or the result of the preprocessing can be stored in the PHYEX package (in this case, the proeprocessing is done once and can be used by several users).
+This second possibility is usefull to historize the source code really used during the model compilation and enables contributions to the PHYEX package without the need of the preprocessing tools.
+
+The preprocessed versions of the source code are put in branches named \<model\>\_\<commit\> where \<model\> is the name of the model for which the source code have been preprocessed and \<commit\> is the commit hash used as a basis.
+
+During the initial development step, it was found easier to store the preprocessing tools outside of the PHYEX repository in the [PHYEX\_tools](https://github.com/SebastienRietteMTO/PHYEX_tools) repository. In the future, they will be moved in the PHYEX repository.
+
+Installation and usage of the preprocessing tools are described in the PHYEX\_tools package.
+
+# CODING NORMS
+
+## File names
+The fortran file names use a capital F letter (eg: foo.F90) except if working a branch (mesonh\_\<commit\>) or in the folder (src/mesonh) specifci to the Meso-NH model.
+
+Names for the module:
+ - modd\_ for module containing only variable declaration (eg: tuning parameters)
+ - modi\_ for module containing only interface declaration
+ - modn\_ for namelist declaration
+ - mode\_ for module containing executable source code (subroutine or function)
+
+## When using mode\_ or modi\_?
+When writing a new subroutine, should we put it in a module (in a mode\_ file) or should we write the subroutine in a file and write the interface bloc in another file (modi\_ file)?
+
+The answer depends on whether the routine is the 'main' routine of the parameterisation or not. If it is the 'main' routine, the interface bloc is declared apart, if not we can use a module.
+The idea behind is to break compilation dependency at the parameterisation level, and to isolate the interface declaration of the different routines that must be pluged in the hosting model.
+
+## Norm
+Several constraints are imposed:
+ - The code must be written with up to 132 characters per line.
+ - CODE IS IN CAPITAL LETTERS! comments in small letters
+ - All variables must be declared: IMPLICIT NONE
+ - except in rare cases, use automatic arrays, no allocatable
+ - dimensions of dummy argument arrays are explicit (no (:,:))
+ - use parenthesis when manipulating arrays (eg: A(:,:)=B(:,:)+C(:,:) instead of A=B+C)
+
+The variables are named according to the doctor norm:
+|Type / Status | INTEGER    | REAL       | LOGICAL     | CHARACTER      | TYPE             |
+|--------------|------------|------------|-------------|----------------|------------------|
+|Global        | N          | X          | L (not LP)  | C              | T (not TP,TS,TZ) |
+|Dummy argument| K          | P (not PP) | O           | H              | TP               |
+|Local         | I (not IS) | Z (not ZS) | G (not GS)  | Y (not YS, YP) | TZ               |
+|Loop control  | J (not JP) | -          | -           | -              | -                |
+
+Regarding array-syntax, code is written using array-syntax in the main branch and in mesonh specific branches based on the GPU branch, using array-syntax with mnh\_expand directives in the GPU branch, using DO loops in arome specific branches based on the GPU branch. If in doublt, check what is done in other routines in the brach you are working in.
+Be carrefull when using the mnh\_expand directives, code must respect some constraints:
+ - parenthesis after array variables are mandatory (no A=B+C, but A(:,:)=B(:,:)+C(:,:))
+ - no space between array variables and the opening parenthesis (no A (:)=B (:), but A(:)=B(:))
+ - same bounds as declared in the mnh\_expand directive should be used in the array-syntax (A(D%NIB;D%NIE)=...)
+A tool (verify\_mnh\_expand.py) can help at checking the validity of the written code.
+
+For the GPU branch (and branches on GPU, including model specific branches):
+ - except variables declared with the PARAMETER attribute, no variable from modules can be used in the physics. Varaibles must be put in a type received by interface.
+ - subroutines or functions must not be called from within a loop on horizontal or vertical dimensions (see below for exception)
+ - functions returning arrays must be rewritten as subroutine
+
+Call to external subroutine in loop on horizontal or vertical dimensions must be suppressed in the GPU version. If possible, the call must be put outside of the loop (acting on the full array as a whole) or the subroutine must be put in the CONTAINS part but, in this case, the included subroutine cannot use local array. There are 3 cases:
+ - the subroutine does't use local array: subroutine is put in an include file (with the .h extension) and included with the fortran INCLUDE statement.
+ - the subroutine use local arrays but it is called from only one place in the code: the source code of the subroutine is moved (no INCLUDE) in the CONTAINS part and the array declarations are moved in the main subroutine.
+ - the subroutine use local arrays and is called from several places: the previous technique is not recommended. The source code is put in an include file (with the .h extension) and an extra argument is provided to the subroutine and is used as a buffer so there is no more need to declare local arrays in the called subroutine.
+
+# PULL REQUESTS
+This section deals with the pull request procedure from the developer point of view. The integrator point of view is described in the Intergator documentation.
+
+To contribute to the PHYEX repository, developer must fork the repository, contribute on the main or on the GPU branch and send a pull request. Alternatively, a contribution on a model specific branch is also possible (especially for minor modifications).
+
+If a modification must be applied to the main and to the GPU branches, the pull request must be made on the main branch (and will be merged into the GPU branch).
+

docs/Integrator.md

+# PHYEX integrator documentation
+
+# ABOUT THIS DOCUMENT
+
+This document is intended for integrators who are in charge of assembling contributions received through pull requests.
+
+his document is written using the markdown "language". With pandoc, it can be converted to HTML (pandoc \<filename\>.md -o \<filename\>.html) or PDF (pandoc \<filename\>.md -o \<filename\>.pdf).
+
+# BRANCHES AND NORMS
+
+Regarding array-syntax, the applicalble norm depends on the branch:
+ - The main branch of PHYEX (and all branches based on main) is written using array-syntax
+ - The GPU branch is written using array-syntax with mnh\_expand directives
+ - arome specific branches based on the GPU branch are written using DO loops
+ - mesonh specific branches based on the GPU branch are written using array-syntax withour mnh\_expand directives
+
+Pull requests can be received on all these kind of branches and must be merged into the main or the GPU branch with according norm.
+
+# TESTS
+
+The source code must follow strict mnh\_expand directives (described in the Developer documentation). The script verify\_mnh\_expand.py must be used to give an additional check.
+
+In addition to the scientific validation, the folowing tests must give the same results (with bit-reproducibility) in each of the model:
+ - compilation transforming the mnh\_expand directives in DO loop
+ - compilation keeping the array-syntax
+ - execution with a different umber of processors
+

docs/Offline.md

+# OFFLINE documentation
+
+# ABOUT THIS DOCUMENT
+
+This document is intended for persons who want to use PHYEX in an offline mode.
+Some offline test programs are provided with the package and a library suitable for use with python is also available.
+
+# COMPILATION
+
+The build/with\_fcm directory contains a build system.
+This build system has two dependencies (installation is done automatically by the compilation script):
+ - [fcm](https://metomi.github.io/fcm/doc/user_guide/)
+ - [fiat](https://github.com/ecmwf-ifs/fiat)
+
+The script build/with\_fcm/make\_fcm.sh uses a configuration file and build the library and test programs.
+They can be found in the build/bin sudirectory in the architecture specific directory arch\_\<achitecture name\>.
+
+# TEST PROGRAM
+
+## Data generation
+
+The branch testprogs\_data contains modified source code for the AROME model to enable the generation of data samples.
+Using this branch, in the drivers of the different parameterisations (aro\_\* files), output can be enable for the AROME model.
+Running the AROME model with these modifications outputs files in the running directory.
+This must be done once by parametrisation.
+
+These files should be renamed with the folowing command:
+i=0; for file in ????_??_????????.dat; do mv $file `printf %08d $i`.dat; i=$((i+1)); done
+
+## Data storage for check\_commit\_testprogs.sh
+
+To be usable by the check\_commit\_testprogs.sh script, these data must be put in the corresponding directory under tools/testprogs\_data
+
+## Usage
+
+The different main\_\*.exe programs obtained by the compilation can be run. Each of these executables is expecting the presence of a 'data' directory in their working directory containing the different files.
+
+In addition, these data files can be used through the check\_commit\_testprogs.sh script (if installed accordingly).
+
+# PYTHON BINDING
+
+This section must be written. Key ideas are:
+  - ctypesforfortran
+  - example
+- 

docs/PHYEX.md

+# PHYEX
+PHYsique EXternalisée
+
+# ABOUT THIS DOCUMENT
+
+This document is a general presentation of the PHYEX package.
+More specific documentation can be found:
+ - Developer: package organisation, how to contribute, coding norms
+ - Integrator: how to merge contributions
+ - Offline: how to compile the library and the test programs, how to use the library with python, how to use the test programs
+ - Plugging : how to plug the physics package in a model
+
+This document is written using the markdown "language". With pandoc, it can be converted to HTML (pandoc \<filename\>.md -o \<filename\>.html) or PDF (pandoc \<filename\>.md -o \<filename\>.pdf).
+
+# HISTORY
+
+The physics was first developed for the Meso-NH model (http://mesonh.aero.obs-mip.fr/).
+Then, a part of the physics have been used to build the AROME model ([Seity et al, 2011](http://dx.doi.org/10.1175/2010MWR3425.1)).
+The last step was to extract the physics to build the independent PHYEX package.
+
+# CONTENT
+
+The folowing paramerisations are included in the PHYEX package (see the MesoNH documentation for references):
+ - turbulence scheme
+ - shallow convection scheme
+ - microphysics scheme
+
+In addition to the parametrisatin source code, test programs and a library for python binding are also provided.

docs/Interfaces → docs/Plugging.md

+This document must still be written
+
+Some elements to include (or not):
+
 The PHYEX parameterizations can be called from different models (eg. Meso-NH, AROME) and from
 a driver (which will be included in this repository).
 Moreover, PHYEX parameterizations call externals subroutines, which are dependencies.

docs/TODO

@@ -19,15 +19,6 @@ Utilisation des clés:
     - defined(REPRO48) || defined(REPRO55): la version de code qui sera retenue à la fin est nouvelle
 Ces clés devront être supprimées
 
-Ecrire doc sur marche à suivre pour intégrer un nouveau développement:
-  - dev dans MNH à faire en array-syntax
-  - dev dans AROME à faire en boucles do
-  - intégration dans PHYEX: en array-syntax avec directives mnh_expand
-  - les 3 tests suivants doivent donner les mêmes résultats (au bit près) dans chacun des deux modèles:
-    - compilation directe sans activer mnh_expand
-    - compilation en activant mnh_expand
-    - exécution en changeant le nombre de processeurs
-
 Merge pb:
   - rain_ice_old a rebrancher dans Meso-NH
   - rain_ice_red: le cas test MesoNH n'est pas bit repro (diffs > 1% sur rapports de melange)
@@ -75,3 +66,7 @@ Regarder s'il ne serait pas possible/souhaitable de supprimer modd_lunit de PHYE
 Nettoyage des répertoires aux nécessaire
 
 Initialiser dans AROME la variable ldiag_in_run de MODD_DIAG_IN_RUN pour pouvoir phaser le modd
+
+Les outils PHYEX_tools devront être nettoyés et rangés dans PHYEX
+
+Les outils testprogs doivent être étendus pour boucler sur tous les fichiers présents, avec un nom composé sur 8 characteres. Doc Offline doit être modifiées en accord pour expliquer comment créer les data