S. Riette 19 oct 2022: doc update

docs/CodingNorms.md

@@ -46,7 +46,7 @@ The variables are named according to the doctor norm:
 |Local         | I (not IS) | Z (not ZS) | G (not GS)  | Y (not YS, YP) | TZ               |
 |Loop control  | J (not JP) | -          | -           | -              | -                |
 
-Regarding array-syntax, code is written using array-syntax in the main branch and in mesonh specific branches based on the GPU branch, using array-syntax with mnh\_expand directives in the GPU branch, using DO loops in arome specific branches based on the GPU branch. If in doubt, check what is done in other routines in the branch you are working in.
+Regarding array-syntax, code is written using array-syntax in the legacy main branch, using array-syntax with mnh\_expand directives in the master branch and in mesonh specific branches based on the master branch, using DO loops in arome specific branches based on the master branch. If in doubt, check what is done in other routines in the branch you are working in.
 Be carrefull when using the mnh\_expand directives, code must respect some constraints:
 
   - parenthesis after array variables are mandatory (no A=B+C, but A(:,:)=B(:,:)+C(:,:))
@@ -55,13 +55,13 @@ Be carrefull when using the mnh\_expand directives, code must respect some const
 
 A tool (verify\_mnh\_expand.py) can help at checking the validity of the written code.
 
-For the GPU branch (and branches on GPU, including model specific branches):
+For the master branch (and branches on master, including model specific branches):
 
   - except variables declared with the PARAMETER attribute, no variable from modules can be used in the physics. Variables must be put in a type received by interface.
   - subroutines or functions must not be called from within a loop on horizontal or vertical dimensions (see below for exception)
   - functions returning arrays must be rewritten as subroutine
 
-Call to external subroutine in loop on horizontal or vertical dimensions must be suppressed in the GPU version. If possible, the call must be put outside of the loop (acting on the full array as a whole) or the subroutine must be put in the CONTAINS part but, in this case, the included subroutine cannot use local array. There are 3 cases:
+Call to external subroutine in loop on horizontal or vertical dimensions must be suppressed in the master version. If possible, the call must be put outside of the loop (acting on the full array as a whole) or the subroutine must be put in the CONTAINS part but, in this case, the included subroutine cannot use local array. There are 3 cases:
 
   - the subroutine doesn't use local array: subroutine is put in an include file (with the .h extension) and included with the fortran INCLUDE statement.
   - the subroutine use local arrays but it is called from only one place in the code: the source code of the subroutine is moved (no INCLUDE) in the CONTAINS part and the array declarations are moved in the main subroutine.

docs/Developer.md

@@ -7,7 +7,7 @@ Developer who is interested in plugging the physics in a new model can refer to
 The topics covered are as follows:
 
   - [Package organisation](#package-organisation)
-  - [Contribution workflow for AROME-HARMONIE developers](#contribution-workflow-for-arome-harmonie-developers)
+  - [Contribution workflow for AROME and HARMONIE-AROME developers](#contribution-workflow-for-arome-and-harmonie-arome-developers)
   - [Contribution workflow for Méso-NH developers](#contribution-workflow-for-mesonh-developers)
   - [Contribution workflow for other developers](#contribution-workflow-for-other-developers)
 
@@ -17,8 +17,8 @@ This document is written using the markdown language. With pandoc, it can be con
 
 The package contains two kinds of branches:
 
-  - generic branches which contain codes for all the models and applications (eg: main and GPU branches)
-  - model specific branches which are automatically derived from generic branches (eg: arome\_\<commit\_hash\>, mesonh\_\<commit\_hash\>)
+  - generic branches which contain codes for all the models and applications (eg: master branch)
+  - model specific branches which are automatically derived from generic branches (eg: arome\_\<commit\_hash\>, mesonh\_\<commit\_hash\>, testprogs\_\<commit\_hash\>)
 
 The directories found in the package are different depending on the branches (generic or model specific).
 
@@ -34,24 +34,30 @@ The generic branches contains the following directories:
 
 Here is a short description of the different generic branches:
 
-  - main: source code without rewriting for GPU transformation
-  - GPU: source code adapted for GPU transformations
+  - master: source code adapted for GPU transformations
   - testHUGE: modified source code to check the incomplete NPROMA feature
   - testprogs\_data: modified source code used to generate samples for the test programs (more on this topic in the [Offline documentation](./Offline.md))
 
-## CONTRIBUTION WORKFLOW FOR AROME-HARMONIE DEVELOPERS
+## CONTRIBUTION WORKFLOW FOR AROME and HARMONIE-AROME DEVELOPERS
 
-The AROME build systems are evolving.
+The build systems are evolving.
 Until cycle 49t1 (included), the physics source code is directly included in the source code tree.
-After cycle 49t1, the physics source code (as well as other model parts such as ectrans, fiat...) will be available through "bundles".
+After cycle 49t1, the physics source code (as well as other model parts such as ectrans, fiat...) will be available through a bundle component.
 
 This evolution will impact the way to contribute to the PHYEX repository.
 
-Whatever is the cycle, the AROME-HARMONIE developers only see codes coming from arome specific branches (branches named arome\_\<commit\_hash\>). This code is ready for inclusion (array-syntax already transformed into DO loops for instance).
+Whatever is the cycle, the AROME and HARMONIE-AROME developers only see codes coming from arome (or testprogs) specific branches (branches named arome\_\<commit\_hash\> or testprogs\_\<commit\_hash\>). This code is ready for inclusion (array-syntax already transformed into DO loops for instance).
 
 Said differently, developers do not need to manipulate code transformation tools.
 
-## Until cycle 49t1
+The workflow was chosen so that the developers would not have to change their working methods several times:
+
+- Developers who have a scientific contribution will submit their pull request on the IAL repository until the ecbundle mechanism is active (49t2 or 50t1), afterwards they will submit the pull request directly on PHYEX.
+- Developers who work on the refactoring cannot use the IAL repository as a starting point and must use the source codes in the PHYEX repository. They will directly use the PHYEX repository.
+
+### Scientific contributions until cycle 49t1
+
+Who: developers with scientific contributions based on cycles 48t1, 48t2, 48t3 and 49t1
 
 Workflow summary: because the physics source code is still included in the IAL source code, pull requests concerning the physics continue to follow the same path as before (ie pull requests are submitted to the IAL repository). Afterwards, the IAL integrator will submit a pull request to the PHYEX repository with only the relevant files.
 
@@ -60,25 +66,41 @@ Workflow summary: because the physics source code is still included in the IAL s
 Workflow details (getting the source code in blue, pull request in red, integration in green):
 
   - 1: PHYEX administrator sends (pull request) the content of a specific arome branch to the IAL Integrator. The IAL integrator tags a new release of IAL.
-  - 2: AROME-HARMONIE developer forks the IAL repository
-  - 3: AROME-HARMONIE developer compiles, executes, modifies the source code in its environment
-  - 4: AROME-HARMONIE developer sends a pull request to the IAL repository
+  - 2: The AROME or HARMONIE-AROME developer forks the IAL repository
+  - 3: The AROME or HARMONIE-AROME developer compiles, executes, modifies the source code in its environment
+  - 4: The AROME or HARMONIE-AROME developer sends a pull request to the IAL repository
   - 5: The IAL integrator extracts the physics source files and sends a pull request to the PHYEX repository
-  - 6: The PHYEX administrator checks and integrates the modifications in the GPU branch and, eventually, produce a new arome specific branch for future integration in IAL
+  - 6: The PHYEX administrator checks and integrates the modifications in the master branch and, eventually, produce a new arome specific branch for future integration in IAL
+
+### Refactoring contributions from now on, and scientific contributions after cycle 49t1
 
-## After cycle 49t1
+Who: developers with scientific contributions based on cycles 49t2 and following; and developers with GPU-refactoring contributions.
 
-Workflow summary: after the cycle 49t1 (starting from 49t2?), HARMONIE/AROME will become a bundle. Il will be built with source codes coming from various places. One of these places will be the PHYEX repository. Pull requests must be sent to each modified bundles.
+Workflow summary: after the cycle 49t1 (starting from 49t2?), AROME and HARMONIE-AROME will become a bundle. Il will be built with source codes coming from various places. One of these places will be the PHYEX repository. Pull requests must be sent to each modified components of the bundle.
+
+Developer must use a model specific branch (arome\_\<commit\_hash\> when working with the model, or testprogs\_\<commit\_hash\> when working with the offline tools).
+These branches receive tags based on the master branch version. For example the commit, in the master branch, corresponding to the version 1.0.0 of PHYEX will receive the tag "v1.0.0".
+The arome specific commit corresponding to this version will be tagged "v1.0.0\_arome".
 
 ![](./AROMEworkflow2.svg)
 
 Workflow details (getting the source code in blue, pull request in red, integration in green):
 
-  - 1 and 2: AROME-HARMONIE developer forks the different repositories needed to build the model
-  - 3: AROME-HARMONIE developer compiles, executes, modifies the source code in its environment
-  - 4 and 5: AROME-HARMONIE developer sends pull requests to the different repositories where files have been modified
-  - 6: The PHYEX administrator checks the pull requests in the other applications, the IAL integrator integrates on the arome specific branch; then the PHYEX administrator integrates the modifications in the GPU branch and, eventually, produce a new arome specific branch for future integration in IAL
+  - 1 and 2: AROME or HARMONIE-AROME developer forks the different repositories needed to build the model
+  - 3: AROME or HARMONIE-AROME developer compiles, executes, modifies the source code in its environment
+  - 4 and 5: AROME or HARMONIE-AROME developer sends pull requests to the different repositories where files have been modified
+  - 6: The PHYEX administrator checks the pull requests in the other applications, the IAL integrator integrates on the arome specific branch; then the PHYEX administrator integrates the modifications in the master branch and, eventually, produce a new arome specific branch for future integration in IAL
+
+### Special notes for building the model from PHYEX before cycle 49t2
+
+The model version hosted in IAL is incompatible with the content of the PHYEX repository.
+To build the model, you must:
 
+- take the source code of cycle 48t3
+- remove the directories 'mpa/\*/internals' and 'mpa/\*/modules'
+- put the PHYEX directories 'aux', 'conv', 'micro' and 'turb' into a new directory (at the same level as 'mpa') named 'phyex'
+- dispatch the code contained in 'ext' into the subdirectories of 'arpifs'
+- remove the files listed in the PHYEX repository in the 'src/arome/gmkpack\_ignored\_files' file
 
 ## CONTRIBUTION WORKFLOW FOR MESO-NH DEVELOPERS
 
@@ -90,4 +112,4 @@ Pull requests concerning the physics continue to follow the same path as before
 
 ## CONTRIBUTION WORKFLOW FOR OTHER DEVELOPERS
 
-Other developers must work with source code coming directly from the GPU branch. They issue pull requests directly on this branch as usual with git repositories.
+Other developers must work with source code coming directly from the master branch. They issue pull requests directly on this branch as usual with git repositories.

docs/Integrator.md

@@ -10,16 +10,16 @@ This document is written using the markdown language. With pandoc, it can be con
 
 Regarding array-syntax, the [applicable norm](./CodingNorms.md) depends on the branch:
 
-  - The main branch of PHYEX (and all branches based on main) is written using array-syntax
-  - The GPU branch is written using array-syntax with mnh\_expand directives
-  - arome specific branches based on the GPU branch are written using DO loops
-  - mesonh specific branches based on the GPU branch are written using array-syntax
+  - The legacy main branch of PHYEX (and all branches based on main) was written using array-syntax
+  - The master branch is written using array-syntax with mnh\_expand directives
+  - arome specific branches based on the master branch are written using DO loops
+  - mesonh specific branches based on the master branch are written using array-syntax
 
-Pull requests can be received on all these kind of branches and must be merged into the main or the GPU branch with according norm.
+Pull requests can be received on all these kind of branches and must be merged into the master branch with according norm.
 
-## NORMAL WORKFLOW FOR A CONTRIBUTION DEVELOPED IN AROME-HARMONIE
+## NORMAL WORKFLOW FOR A CONTRIBUTION DEVELOPED IN AROME AND HARMONIE-AROME
 
-### Until cycle 49t1
+### Scientific contributions until cycle 49t1
 
 ![](./AROMEworkflow1.svg)
 
@@ -27,12 +27,12 @@ The pull request comes from the IAL integrator. It must be based on an arome spe
 
 Details for point 6, the PHYEX administrator:
   - validates (see [below](#tests)) the contribution
-  - integrates the contribution in the arome branch and merges it in the GPU branch
-  - regularly, he tags a new (minor) version of the GPU branch
+  - integrates the contribution in the arome branch and merges it in the master branch
+  - regularly, he tags a new (minor) version of the master branch
   - when asked by the IAL integrator, he builds a new arome specific branch
   - when an arome specific branch is used in an official cycle, the arome specific branch is tagged accordingly
 
-### After cycle 49t1
+### Refactoring contributions from now on, and scientific contributions after cycle 49t1
 
 ![](./AROMEworkflow2.svg)
 
@@ -42,8 +42,8 @@ Details for point 6:
   - The PHYEX administrator checks the pull requests in the other applications (see [below](#tests))
   - The IAL integrator integrates the contribution on the arome specific branch
   - The PHYEX administrator
-    - integrates the modifications in the GPU branch
-    - regularly, tags a new (minor) version of the GPU branch
+    - integrates the modifications in the master branch
+    - regularly, tags a new (minor) version of the master branch
     - when asked by the IAL integrator, builds a new arome specific branch (see [below](#code-preparation))
     - when an arome specific branch is used in an official cycle, the arome specific branch is tagged accordingly
 
@@ -55,19 +55,19 @@ Integration details:
   - The Meso-NH integrator extracts, from the different pull requests, what concern the PHYEX repository and send a pull request on PHYEX based on a mesonh specific branch
   - The PHYEX administrator:
     - validates (see [below](#tests)) the contribution
-    - integrates the contribution in the mesonh branch and merges it in the GPU branch
-    - regularly, he tags a new (minor) version of the GPU branch
+    - integrates the contribution in the mesonh branch and merges it in the master branch
+    - regularly, he tags a new (minor) version of the master branch
     - when asked by the Meso-NH integrator, he builds a new mesonh specific branch (see [below](#code-preparation))
     - when a mesonh specific branch is used in an official release, the mesonh specific branch is tagged accordingly
 
 ## NORMAL WORKFLOW FOR ANOTHER CONTRIBUTION
 
-Pull requests must be based on the GPU branch.
+Pull requests must be based on the master branch.
 
 The PHYEX administrator:
   - validates (see [below](#tests)) the contribution
-  - integrates the contribution in the GPU branch
-  - regularly, he tags a new (minor) version of the GPU branch
+  - integrates the contribution in the master branch
+  - regularly, he tags a new (minor) version of the master branch
 
 ## TESTS
 
@@ -83,7 +83,7 @@ When possible, the new version of PHYEX must reproduce the old results (scientif
 
 ## CODE PREPARATION
 
-The source code stored in the main and GPU branches must be usable by all the models. But these models can have contradictory constraints. To bypass this difficulty, the source code is preprocessed before being included in the compilation environment of each model.
+The source code stored in the master branches must be usable by all the models. But these models can have contradictory constraints. To bypass this difficulty, the source code is preprocessed before being included in the compilation environment of each model.
 
 This preprocessing step can be done on the fly (in this case the preprocessing tools must be available aside of the compilation tools), or the result of the preprocessing can be stored in the PHYEX package (in this case, the preprocessing is done once and can be used by several users).
 This second possibility is useful to historize the source code really used during the model compilation and enables contributions to the PHYEX package without the need of the preprocessing tools.

docs/Offline.md

@@ -18,7 +18,28 @@ This build system has two dependencies (installation is done automatically by th
 The script build/with\_fcm/make\_fcm.sh uses a configuration file and build the library and test programs.
 They can be found in the build/bin subdirectory in the architecture specific directory arch\_\<architecture name\>.
 
-Some more details can be found in [build/with\_fcm/README.md file](../build/with_fcm/README.md).
+Some more details on the build system can be found in [build/with\_fcm/README.md file](../build/with_fcm/README.md).
+
+### Compilation in the repository
+
+When on a master commit, the build/with\_fcm/make\_fcm.sh script can be used to compile the offline tools.
+
+### Compilation of a testprogs specific commit
+
+When on a master commit, the tools/check\_commit\_testprogs.sh script can be used to compile a testprogs\_\<commit\_hash\> commit already available in the PHYEX repository.
+
+### Modification of a testprogs specific commit
+
+The idea is to use two clones of PHYEX: one to provide the building tools, the another one to provide the source code.
+Something like this can be used:
+
+- cd $HOME; git clone \<PHYEX url\> PHYEXtools
+- cd PHYEXtools; git checkout master
+- cd $HOME; git clone \<PHYEX url\> PHYEX
+- cd PHYEX; git checkout arome\_\<commit\_hash\>; source code moddifications...
+- . PHYEXtools/tools/env.sh; check\_commit\_testprogs.sh $HOME/PHYEX REF
+
+The last step will creates a directory (in $HOME/TESTPROGS) with a copy of your source code and the build system, builds the testprogs and executes them.
 
 ## TEST PROGRAM
 
@@ -36,6 +57,10 @@ i=0; for file in ????_??_????????.dat; do mv $file `printf %08d $i`.dat; i=$((i+
 
 The different main\_\*.exe programs obtained by the compilation can be run. Each of these executables is expecting the presence of a 'data' directory in their working directory containing the different files.
 
+### Usage through the check\_commit\_testprogs.sh script
+
+As described in [COMPILATION](#compilation).
+
 ## PYTHON BINDING
 
 **TODO** This section must be written. Key ideas are: