diff --git a/hpc/README b/hpc/README new file mode 100644 index 0000000..10a91ff --- /dev/null +++ b/hpc/README @@ -0,0 +1,316 @@ + +====================== +**** INSTALLATION **** +====================== +Very little needs to be done to get this set up. The major thing it needs is to know +who you are, along with any extra information your job scheduler might need to know +about you. The most basic installation is to simply run: + + python buildhpc.py + +This will get your username from the operating system, and write a file called identity.py +which stores that in a variable for later use. Job submission scripts will use this information. +If you use a system like Slurm which also requires an account name and an email address, you +can build the system with that information by running: + + python buildhpc.py email=you@mail account=whospayingforthis + +Including these fields is optional and harmless, so you can do this even if your system does +*not* require this information: in that case, those variables will simply never be used. If +you do not provide this fields, the variables will still be defined, but they will contain +empty strings. + +------------------------------------------- +* Configuring your PlaSim HPC environment * +------------------------------------------- + +Once you've set up your identity information with the above steps, there's one more thing +you'll need to do for basic operation: define a template folder for PlaSim runs. You can specify +any directory you wish for the template as a job-specific argument, but if that's not defined, +the default is to check for a folder called 'clean' (as in, a clean PlaSim configuration). +Therefore it's a good idea to create such a folder, in the same directory as launcher.py, and +populate it with all the files you would need for a basic, vanilla, modern Earth PlaSim run. +The files and namelists in this directory will serve as the default configuration, so any +parameter *not* specified will assume the value it has in this template folder. + +If there are special run scripts you want to use beyond those provided, you should put them in +the top-level directory (the same directory as launcher.py). If however there are *other* files +your job requires, such as specific restart files or boundary condition files, you should place +those in a folder called 'hopper'. If this folder does not exist, it will be automatically created, even if it is not needed. + + +================= +***** USAGE ***** +================= + +The major strength of this program is that you can specify the parameters for a large number of +PlaSim runs in a single file. That file is tasks.crwl. The basic format of this file is a +header, followed by a list of jobs which use that header. Examples are given in the default +tasks.crwl file. + +Each header line must begin with a #. After that, the first 4 fields *must* be, in this order, +JOBNAME, STATUS, NCORES, and QUEUE. JOBNAME is the name you want to give your job; every job's +name should be unique. The job will be run in a folder with that name (you don't need to create +it yourself). STATUS is whether or not the job has been initialized--0 if no, 1 if yes. When +you run launcher.py, it will find every job whose status is 0 and either set it up or set it +up and submit it. NCORES is the number of MPI threads the job will use. QUEUE is the name of +the job queue to which you want to eventually or immediately submit the job. After these four +fields, the order of the other fields is mostly irrelevant, though the program will iterate +through them from left to right, so if you put in any conflicting parameters, only the rightmost +will apply. + +To run the program and generate all available jobs, run: + + python launcher.py + +If you want to generate the job folders and configuraiton files but not yet submit the jobs, +you can simply run: + + python launcher.py nosub + +While you can include any namelist parameter with the header field syntax KEY@namelist, there +are a large number of predefined arguments. These are described in the next section. + +--------------------------------------------- +* Pre-defined parameters and configurations * +--------------------------------------------- + +* source + - This should be a string, and it should give the path to the template directory you want + to use. If left undefined, it will default to "clean" + +* noutput + - Switch for writing model output. 0=no, 1=yes + +* flux + - Incident stellar flux (insolation) at the planet, in W/m^2. + +* startemp + - Mostly unused. Effective temeprature of the parent star, in Kelvins + +* pCO2 + - CO2 partial pressure in ubars (10^-6 bars) + +* pN2 + - N2 partial pressure in bars + +* year + - Number of days per year + +* sidyear + - Number of days in a sidereal year + +* solyear + - Number of days in a solar year + +* sidday + - Number of hours in a sidereal day + +* solday + - Number of hours in a solar day + +* nlevs + - Number of vertical model levels to use + +* vtype + - Which vertical scheme to use. Options are: + -1: User-defined, through the SIGH parameter in plasim_namelist. + 0: Linear-like (actually a polynomial, but nonlinear behavior is only at bottom and top) + 1: Linear, equidistant + 2: Logarithmic, equidistant + 3: Pseudo-logarithmic; top is pinned and there's extra resolution near the surface + 4: Like 0, but with the top pinned at PTOP. + +* rotspd + - Inverse of the planet's period. I.e. 3-day rotation period means ROTSPD=0.3333333. + +* lockedyear + - A shortcut for creating a tidally-locked planet. Use the format "DAY/LON0" where DAY is the + length of the sidereal day (in days) (and sidereal year) and LON0 is the substellar + longitude. For example, 15.0/0.0 would mean the orbital and rotational periods are 15 days, + and the substellar point is at 0 degrees longitude. + +* restart + - If this is "none", delete any restart file that was copied over from the template folder. + - Otherwise, assume that the value is the name of a restart file, and copy it from 'hopper' + to 'plasim_restart' in the job folder. + +* gravity + - The planet's surface gravity, in m/s^2. + +* radius + - The planet's radius in Earth radii. + +* eccen + - The plane'ts orbital eccentricity. + +* obliq + - The planet's axial tilt, in degrees. + +* vernlon + - The planet's longitude of vernal equinox (MVELP). This is only relevant if eccen or obliq + are nonzero. Even then you can probably ignore it. + +* nfixorb + - (0/1 = no/yes) Treat the orbit as fixed, and ignore any precession effects that would + otherwise be computed (i.e. Milankovitch cycles) + +* oroscale + - Multiplicative factor by which to scale topography; also turns on glacier module + +* nglacier + - (1/0 = yes/no) turn on/off the glacier module + +* glacelim + - Thickness Limit at which glaciers are formed/removed, in meters + +* icesheets + - Wipe clean some of the surface boundary conditions, and prescribe an initial ice sheet whose + height is this parameter's argument given in meters. + +* lowerANTGRN + - (CURRENTLY BROKEN; requires additional files) Substitute in boundary condition files that + don't assume that Antarctica and Greenland are covered in km-thick ice sheets. + +* ncarbon + - (1/0=yes/no) Switch for the carbon evolution model. + +* co2evolve + - (1/0=yes/no) Whether or not CO2 should be evolved year-to-year or handled externally by the + user (1 = internally, year-to-year). + +* nsupply + - (1/0=yes/no) Institute a supplit limit on the weathering rate, due to erosion. + +* wmax + - Maximum weathering rate to be used as the supply limit, in nbars/yr (bars/Gyr) + +* volcanCO2 + - Volcanic CO2 outgassing rate in nbars/yr (bars/Gyr) + +* snowmax + - Maximum surface snow thickness in meters of liquid h2o equivalent (-1 = no limit) + +* soilalbedo + - Default albedo to use for land surface (removes albedo boundary condition file) + +* wetsoil + - (1/0=yes/no) Make soil albedo dependent on moisture content + +* soilh2o + - Soil water capacity in meters of liquid water equivalent + +* naqua + - (1/0=yes/no) Initialize the planet as an aquaplanet (flat surface, no land) + +* ndesert + - (1/0=yes/no) Initialize the planet as a desert planet (flat surface, no oceans, no soil h2o) + +* drycore + - (1/0=yes/no) Turn off evaporation, so the atmosphere has no water vapor + +* soilheat + - Heat capacity of soil, in J/m^3/K. 1 J/kg/K is approximately 1 mJ/m^3/K. The ocean is 4.18e6. + +* soildepth + - Multiplicative factor by which to scale the depth of the soil model, which defaults to 5 + layers of 0.4, 0.8, 1.6, 3.2, and 6.4 meters. + +* nwpd + - Number of writes per day. + +* months + - Number of months to run each PlaSim instance (each time it loops) + +* rhum_c + - Multiplicative factor by which to scale the critical relative humidity for cloud formation. + +* rhum_m + - The slope to use for the mostly-linear function that describes how the critical relative + humidity changes throughout the atmosphere + +* rcrit + - List with NLEVS elements, giving the critical relative humidity at each model layer. + +* days + - Number of days to run each PlaSim instance + +* steps + - Number of timesteps to run each PlaSim instance + +* o3 + - (1/0=yes/no) Whether not to include the effects of ozone (O3) + +* ptop + - Pressure in hPa at which to pin the top of the atmosphere. NOTE THIS ENABLES NEQSIG=3. If you + want to use a different vertical scheme, you should specify ptop FIRST and then vtype. + +* timestep + - Timestep in minutes. This will scale the number of steps per write and the the number of + steps per snapshot (it will turn snapshots on) so that I/O does not increase with + smaller timesteps. + +* rayleigh + - (1/0=yes/no) Switch for layer-by-layer Rayleigh scattering, as opposed to doing it all in the + bottom layer. + +* script + - The run script to be called from the job submission script. Must be in the top-level + directory. + +* postprocessor + - The burn7 .nl configuration file to use with the postprocessor. + +* notify + - Email notification flags. For PBS/Torque, valid arguments are combinations of 'a' (abort), + 'b' (begin), and 'e' (end). + +* columnmode + - Up to two arguments, separate by |. Arguments can be: + static: Turn off horizontal advection + clear: Turn off cloud diagnostics and feedback + +* snapshots + - Number of timesteps per snapshot write. + +* extra + - Any additional file in the toplevel directory that should be included. + +* runyears + - Number of years to run PlaSim if using most_plasim_run. + +* fixedlon + - Fix the sun in the sky at a longitude given in degrees. + + +------------ +* Examples * +------------ + +1. Modern Earth, with 410 ubars of CO2, run to energy-balance equilibrium, and notifying only on + abort. Run with 8 cores on hpq. + +# JOBNAME STATUS NCORES QUEUE pCO2 pN2 script extra notify +modern-earth 0 8 hpq 410.0 1.01067 super-equil.sh super_relax.py a + +2. Aquaplanet with 8 bars of N2 with a pinned linear-like scheme, layer-by-layer Rayleigh + scattering, run to energy-balance, notifying on start, abort, or end, run on 16 cores on sandyq. + We'll pin the top at 50 hPa and use a 15-minute timestep. pCO2 is 360 ubars. + +# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep naqua pN2 pCO2 rayleigh script extra notify +aqua-8.0bar 0 16 sandyq 50.0 4 10 15.0 1 8.5 360.0 1 super-equil.sh super_relax.py abe + +3. Tidally-locked aquaplanet, 1.3 Earth radii, 4.5 Earth masses, 12-day orbital period, 2.5 bars of + pN2 and 10 mbar of CO2, with 800 W/m^2 insolation, no axial tilt, and circular orbit; run to + energy-balance equilibrium, and notifying only on abort or end. Using 32 cores on greenq. + 30-minute timestep. + +# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep radius gravity nfixorb eccen obliq lockedyear year naqua flux pCO2 pN2 script extra notify +locked_superearth 0 32 greenq 100.0 4 10 30.0 1.3 20.086447 1 0.0 0.0 12.0/0.0 36 1 800.0 1000.0 2.5 super-equil.sh super_relax.py ae + + We had to do a little bit of math here, because we need to know the surface + gravity, not the mass. The gravity is M/R^3*g_earth, where g_earth in PlaSim + is 9.80665 W/m^2. For 4.5 Earth masses and 1.3 Earth radii, we get 20.086447 + W/m^2. + + \ No newline at end of file diff --git a/hpc/buildplasimjob.py b/hpc/buildplasimjob.py index 98aac96..b5614ab 100644 --- a/hpc/buildplasimjob.py +++ b/hpc/buildplasimjob.py @@ -180,8 +180,12 @@ def prep(job): if name=="pCO2": found=True gotpress=False - if "pressure" in fields: - p0 = float(job.parameters["pressure"]) + if "pN2" in fields: + p0 = float(job.parameters["pN2"]) + p0 *= 1.0e6 + p0 += float(val) + p0 *= 1.0e-6 + edit_namelist(home,"plasim_namelist","PSURF",str(p0*1.0e5)) gotpress=True if not gotpress: p0 += float(val) @@ -191,10 +195,13 @@ def prep(job): pCO2 = float(val)/(p0*1.0e6)*1.0e6 #ppmv edit_namelist(home,"radmod_namelist","CO2",str(pCO2)) - if name=="pressure": + if name=="pN2": found=True p0 = float(val)*1.0e6 + if "pCO2" in fields: + p0 += float(job.parameters["pCO2"]) + edit_namelist(home,"radmod_namelist","CO2",str(float(job.parameters["pCO2"])/p0*1.0e6)) edit_namelist(home,"plasim_namelist","PSURF",str(p0*0.1)) if name=="alloutput": diff --git a/hpc/tasks.crwl b/hpc/tasks.crwl index ff23aea..de31db5 100644 --- a/hpc/tasks.crwl +++ b/hpc/tasks.crwl @@ -1,11 +1,11 @@ -# JOBNAME STATUS NCORES QUEUE timestep nfixorb eccen obliq naqua flux pCO2 pressure source script extra notify -samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -samplerun_02 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 380.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -samplerun_03 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 400.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -samplerun_04 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 420.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -samplerun_05 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 440.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep nfixorb eccen obliq vernlon lockedyear year naqua flux pCO2 pressure source script extra notify -tlocked_01 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1400.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -tlocked_02 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1300.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae -tlocked_03 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1200.000000 360.0 1.0 ../GPlaSim/plasim/run super-equil.sh super_relax.py ae +# JOBNAME STATUS NCORES QUEUE timestep nfixorb eccen obliq naqua flux pCO2 pN2 source script extra notify +samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +samplerun_02 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 380.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +samplerun_03 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 400.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +samplerun_04 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 420.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +samplerun_05 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 440.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +samplerun_01 0 16 sandyq 45.0 1 0.0 0.0 1 1300.0 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep nfixorb eccen obliq vernlon lockedyear year naqua flux pCO2 pN2 source script extra notify +tlocked_01 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1400.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +tlocked_02 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1300.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae +tlocked_03 0 16 sandyq 77.4 4 10 45.0 1 0.0 0.0 90.0 12.0/0.0 36 1 1200.000000 360.0 1.0 ../plasim/run super-equil.sh super_relax.py ae