udo/exoplasim
Private
Public Access
1
0 Fork 0
Code Activity

Added HPC Readme, plus a few other small changes

Browse Source
This commit is contained in:
Adiv Paradise committed 2019-04-14 18:37:47 -04:00
1 parent 2e6fc04a27
commit e40eca44b9
3 files changed
+337 -14
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

hpc/README
+316
View File
@@ -0,0 +1,316 @@
======================
**** INSTALLATION ****
======================
Very little needs to be done to get this set up. The major thing it needs is to know
who you are, along with any extra information your job scheduler might need to know
about you. The most basic installation is to simply run:
python buildhpc.py
This will get your username from the operating system, and write a file called identity.py
which stores that in a variable for later use. Job submission scripts will use this information.
If you use a system like Slurm which also requires an account name and an email address, you
can build the system with that information by running:
python buildhpc.py email=you@mail account=whospayingforthis
Including these fields is optional and harmless, so you can do this even if your system does
*not* require this information: in that case, those variables will simply never be used. If
you do not provide this fields, the variables will still be defined, but they will contain
empty strings.
-------------------------------------------
* Configuring your PlaSim HPC environment *
-------------------------------------------
Once you've set up your identity information with the above steps, there's one more thing
you'll need to do for basic operation: define a template folder for PlaSim runs. You can specify
any directory you wish for the template as a job-specific argument, but if that's not defined,
the default is to check for a folder called 'clean' (as in, a clean PlaSim configuration).
Therefore it's a good idea to create such a folder, in the same directory as launcher.py, and
populate it with all the files you would need for a basic, vanilla, modern Earth PlaSim run.
The files and namelists in this directory will serve as the default configuration, so any
parameter *not* specified will assume the value it has in this template folder.
If there are special run scripts you want to use beyond those provided, you should put them in
the top-level directory (the same directory as launcher.py). If however there are *other* files
your job requires, such as specific restart files or boundary condition files, you should place
those in a folder called 'hopper'. If this folder does not exist, it will be automatically created, even if it is not needed.
=================
***** USAGE *****
=================
The major strength of this program is that you can specify the parameters for a large number of
PlaSim runs in a single file. That file is tasks.crwl. The basic format of this file is a
header, followed by a list of jobs which use that header. Examples are given in the default
tasks.crwl file.
Each header line must begin with a #. After that, the first 4 fields *must* be, in this order,
JOBNAME, STATUS, NCORES, and QUEUE. JOBNAME is the name you want to give your job; every job's
name should be unique. The job will be run in a folder with that name (you don't need to create
it yourself). STATUS is whether or not the job has been initialized--0 if no, 1 if yes. When
you run launcher.py, it will find every job whose status is 0 and either set it up or set it
up and submit it. NCORES is the number of MPI threads the job will use. QUEUE is the name of
the job queue to which you want to eventually or immediately submit the job. After these four
fields, the order of the other fields is mostly irrelevant, though the program will iterate
through them from left to right, so if you put in any conflicting parameters, only the rightmost
will apply.
To run the program and generate all available jobs, run:
python launcher.py
If you want to generate the job folders and configuraiton files but not yet submit the jobs,
you can simply run:
python launcher.py nosub
While you can include any namelist parameter with the header field syntax KEY@namelist, there
are a large number of predefined arguments. These are described in the next section.
---------------------------------------------
* Pre-defined parameters and configurations *
---------------------------------------------
* source
- This should be a string, and it should give the path to the template directory you want
to use. If left undefined, it will default to "clean"
* noutput
- Switch for writing model output. 0=no, 1=yes
* flux
- Incident stellar flux (insolation) at the planet, in W/m^2.
* startemp
- Mostly unused. Effective temeprature of the parent star, in Kelvins
* pCO2
- CO2 partial pressure in ubars (10^-6 bars)
* pN2
- N2 partial pressure in bars
* year
- Number of days per year
* sidyear
- Number of days in a sidereal year
* solyear
- Number of days in a solar year
* sidday
- Number of hours in a sidereal day
* solday
- Number of hours in a solar day
* nlevs
- Number of vertical model levels to use
* vtype
- Which vertical scheme to use. Options are:
-1: User-defined, through the SIGH parameter in plasim_namelist.
0: Linear-like (actually a polynomial, but nonlinear behavior is only at bottom and top)
1: Linear, equidistant
2: Logarithmic, equidistant
3: Pseudo-logarithmic; top is pinned and there's extra resolution near the surface
4: Like 0, but with the top pinned at PTOP.
* rotspd
- Inverse of the planet's period. I.e. 3-day rotation period means ROTSPD=0.3333333.
* lockedyear
- A shortcut for creating a tidally-locked planet. Use the format "DAY/LON0" where DAY is the
length of the sidereal day (in days) (and sidereal year) and LON0 is the substellar
longitude. For example, 15.0/0.0 would mean the orbital and rotational periods are 15 days,
and the substellar point is at 0 degrees longitude.
* restart
- If this is "none", delete any restart file that was copied over from the template folder.
- Otherwise, assume that the value is the name of a restart file, and copy it from 'hopper'
to 'plasim_restart' in the job folder.
* gravity
- The planet's surface gravity, in m/s^2.
* radius
- The planet's radius in Earth radii.
* eccen
- The plane'ts orbital eccentricity.
* obliq
- The planet's axial tilt, in degrees.
* vernlon
- The planet's longitude of vernal equinox (MVELP). This is only relevant if eccen or obliq
are nonzero. Even then you can probably ignore it.
* nfixorb
- (0/1 = no/yes) Treat the orbit as fixed, and ignore any precession effects that would
otherwise be computed (i.e. Milankovitch cycles)
* oroscale
- Multiplicative factor by which to scale topography; also turns on glacier module
* nglacier
- (1/0 = yes/no) turn on/off the glacier module
* glacelim
- Thickness Limit at which glaciers are formed/removed, in meters
* icesheets
- Wipe clean some of the surface boundary conditions, and prescribe an initial ice sheet whose
height is this parameter's argument given in meters.
* lowerANTGRN
- (CURRENTLY BROKEN; requires additional files) Substitute in boundary condition files that
don't assume that Antarctica and Greenland are covered in km-thick ice sheets.
* ncarbon
- (1/0=yes/no) Switch for the carbon evolution model.
* co2evolve
- (1/0=yes/no) Whether or not CO2 should be evolved year-to-year or handled externally by the
user (1 = internally, year-to-year).
* nsupply
- (1/0=yes/no) Institute a supplit limit on the weathering rate, due to erosion.
* wmax
- Maximum weathering rate to be used as the supply limit, in nbars/yr (bars/Gyr)
* volcanCO2
- Volcanic CO2 outgassing rate in nbars/yr (bars/Gyr)
* snowmax
- Maximum surface snow thickness in meters of liquid h2o equivalent (-1 = no limit)
* soilalbedo
- Default albedo to use for land surface (removes albedo boundary condition file)
* wetsoil
- (1/0=yes/no) Make soil albedo dependent on moisture content
* soilh2o
- Soil water capacity in meters of liquid water equivalent
* naqua
- (1/0=yes/no) Initialize the planet as an aquaplanet (flat surface, no land)
* ndesert
- (1/0=yes/no) Initialize the planet as a desert planet (flat surface, no oceans, no soil h2o)
* drycore
- (1/0=yes/no) Turn off evaporation, so the atmosphere has no water vapor
* soilheat
- Heat capacity of soil, in J/m^3/K. 1 J/kg/K is approximately 1 mJ/m^3/K. The ocean is 4.18e6.
* soildepth
- Multiplicative factor by which to scale the depth of the soil model, which defaults to 5
layers of 0.4, 0.8, 1.6, 3.2, and 6.4 meters.
* nwpd
- Number of writes per day.
* months
- Number of months to run each PlaSim instance (each time it loops)
* rhum_c
- Multiplicative factor by which to scale the critical relative humidity for cloud formation.
* rhum_m
- The slope to use for the mostly-linear function that describes how the critical relative
humidity changes throughout the atmosphere
* rcrit
- List with NLEVS elements, giving the critical relative humidity at each model layer.
* days
- Number of days to run each PlaSim instance
* steps
- Number of timesteps to run each PlaSim instance
* o3
- (1/0=yes/no) Whether not to include the effects of ozone (O3)
* ptop
- Pressure in hPa at which to pin the top of the atmosphere. NOTE THIS ENABLES NEQSIG=3. If you
want to use a different vertical scheme, you should specify ptop FIRST and then vtype.
* timestep
- Timestep in minutes. This will scale the number of steps per write and the the number of
steps per snapshot (it will turn snapshots on) so that I/O does not increase with
smaller timesteps.
* rayleigh
- (1/0=yes/no) Switch for layer-by-layer Rayleigh scattering, as opposed to doing it all in the
bottom layer.
* script
- The run script to be called from the job submission script. Must be in the top-level
directory.
* postprocessor
- The burn7 .nl configuration file to use with the postprocessor.
* notify
- Email notification flags. For PBS/Torque, valid arguments are combinations of 'a' (abort),
'b' (begin), and 'e' (end).
* columnmode
- Up to two arguments, separate by |. Arguments can be:
static: Turn off horizontal advection
clear: Turn off cloud diagnostics and feedback
* snapshots
- Number of timesteps per snapshot write.
* extra
- Any additional file in the toplevel directory that should be included.
* runyears
- Number of years to run PlaSim if using most_plasim_run.
* fixedlon
- Fix the sun in the sky at a longitude given in degrees.
------------
* Examples *
------------
1. Modern Earth, with 410 ubars of CO2, run to energy-balance equilibrium, and notifying only on
abort. Run with 8 cores on hpq.
# JOBNAME STATUS NCORES QUEUE pCO2 pN2 script extra notify
modern-earth 0 8 hpq 410.0 1.01067 super-equil.sh super_relax.py a
2. Aquaplanet with 8 bars of N2 with a pinned linear-like scheme, layer-by-layer Rayleigh
scattering, run to energy-balance, notifying on start, abort, or end, run on 16 cores on sandyq.
We'll pin the top at 50 hPa and use a 15-minute timestep. pCO2 is 360 ubars.
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep naqua pN2 pCO2 rayleigh script extra notify
aqua-8.0bar 0 16 sandyq 50.0 4 10 15.0 1 8.5 360.0 1 super-equil.sh super_relax.py abe
3. Tidally-locked aquaplanet, 1.3 Earth radii, 4.5 Earth masses, 12-day orbital period, 2.5 bars of
pN2 and 10 mbar of CO2, with 800 W/m^2 insolation, no axial tilt, and circular orbit; run to
energy-balance equilibrium, and notifying only on abort or end. Using 32 cores on greenq.
30-minute timestep.
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep radius gravity nfixorb eccen obliq lockedyear year naqua flux pCO2 pN2 script extra notify
locked_superearth 0 32 greenq 100.0 4 10 30.0 1.3 20.086447 1 0.0 0.0 12.0/0.0 36 1 800.0 1000.0 2.5 super-equil.sh super_relax.py ae
We had to do a little bit of math here, because we need to know the surface
gravity, not the mass. The gravity is M/R^3*g_earth, where g_earth in PlaSim
is 9.80665 W/m^2. For 4.5 Earth masses and 1.3 Earth radii, we get 20.086447
W/m^2.
hpc/buildplasimjob.py
+10 -3
View File
@@ -180,8 +180,12 @@ def prep(job):
if name=="pCO2":
found=True
gotpress=False
if "pressure" in fields:
p0 = float(job.parameters["pressure"])
if "pN2" in fields:
p0 = float(job.parameters["pN2"])
p0 *= 1.0e6
p0 += float(val)
p0 *= 1.0e-6
edit_namelist(home,"plasim_namelist","PSURF",str(p0*1.0e5))
gotpress=True
if not gotpress:
p0 += float(val)
@@ -191,10 +195,13 @@ def prep(job):
pCO2 = float(val)/(p0*1.0e6)*1.0e6 #ppmv
edit_namelist(home,"radmod_namelist","CO2",str(pCO2))
if name=="pressure":
if name=="pN2":
found=True
p0 = float(val)*1.0e6
if "pCO2" in fields:
p0 += float(job.parameters["pCO2"])
edit_namelist(home,"radmod_namelist","CO2",str(float(job.parameters["pCO2"])/p0*1.0e6))
edit_namelist(home,"plasim_namelist","PSURF",str(p0*0.1))
if name=="alloutput":
hpc/tasks.crwl
+11 -11
View File
@@ -1,11 +1,11 @@
# JOBNAME STATUS NCORES QUEUE timestep nfixorb eccen obliq naqua flux pCO2 pressure source script extra notify
samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
samplerun_02 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 380.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
samplerun_03 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 400.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
samplerun_04 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 420.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
samplerun_05 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 440.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep nfixorb eccen obliq vernlon lockedyear year naqua flux pCO2 pressure source script extra notify
tlocked_01 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1400.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
tlocked_02 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1300.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
tlocked_03 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1200.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae
# JOBNAME STATUS NCORES QUEUE timestep nfixorb eccen obliq naqua flux pCO2 pN2 source script extra notify
samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
samplerun_02 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 380.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
samplerun_03 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 400.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
samplerun_04 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 420.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
samplerun_05 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 440.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep nfixorb eccen obliq vernlon lockedyear year naqua flux pCO2 pN2 source script extra notify
tlocked_01 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1400.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
tlocked_02 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1300.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae
tlocked_03 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1200.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae