Fusion Collaboratory Transp Production

How does it work?
How to Monitor Runs?
Available PPPL Tools (NTCC)

How does it work?

Prerequisite for Client:

Globus must be installed.
If using MDSplus

MDSplus must be installed.
Mdsplus Model Tree must exist.
Client (with valid grid proxy)
- Prepare Input data and Namelist
  - Site specific software
  - PPPL:
    - scruncher, scrunch2, xtranspin, nbfile
    - Ufiles via IDL
- Run Options:
  - Standard
    - User creates a MDSplus Tree and sends a jobrequest with hte REQUEST file to PPPL.
    - At PPPL, Namelist is extracted from MDSplus Tree specified in REQUEST file.
    - Input is extracted from MDSplus Tree specified in Namelist.
    - Output can be fetched as files, or is optionally written into MDSplus Tree as specified in REQUEST file.
  - TREES
    - User creates a MDSplus Tree, containing all Input data, and sends a jobrequest with the REQUEST file to PPPL.
    - At PPPL, the tree triplets are copied via mdstcl into a "local temporary tree".
    - Namelist and Input data are extracted from this local MDSplus Tree.
    - Output is written into this local MDSplus Tree.
    - Tree Triplets are returned to client via mdstcl as specified in the REQUEST file.
    - Non-MDSplus output must be fetched.
  - UFILE (not recommended)
    - User prepares tar file containing REQUEST, Namelist, auxiliary files and Ufiles, to be sendt to PPPL.
      No MDSplus Tree is created.
    - At PPPL, REQUEST file and Ufiles are extracted from tar file.
    - PPPL creates mdsPLUS tree at Clien's Server as specified in REQUEST file.
    - Output: same as STANDARD.
  - CREATE
    - User prepares tar file containing REQUEST, Namelist, auxiliary files and Ufiles, to be sendt to PPPL.
      No MDSplus Tree is created.
    - At PPPL, REQUEST file, Namelist and Ufiles are extracted from tar file.
    - PPPL creates a temporary MDSplus Tree.
    - PPPL produces a tar file, containing MDSplus Tree files (input and output) & optional TRANSP output.
    - User fetches the tar file, extracts the Tree files and moves them into the site's MDSplus directory.
  - NOMDSPLUS
    - User prepares tar file (via datprep) containing REQUEST, Namelist, auxiliary files and Ufiles, to be sendt to PPPL.
      No MDSplus Tree is created.
    - At PPPL, REQUEST file, Namelist and Ufiles are extracted from tar file.
    - PPPL creates a tar file, containing TRANSP output.
    - User fetches the tar file.
- Create MDSplus Pulse and fill with Namelist and Input data
  Note: Skip for CREATE and NOMDSPLUS options
  - PPPL: tr_start (xtranspin on unix cluster)
  - MIT / GA: pretransp
- Generate <runid>.REQUEST file
  - PPPL: tr_start (xtranspin on unix cluster)
  - MIT / GA: pretransp
  Contents of <runid>.REQUEST file
- Transfer the <runid>.REQUEST file to PPPL (globus-url-copy)
  - PPPL: tr_send_pppl.pl (xtranspin on unix cluster)
  - MIT / GA : pretransp
- Submit the run (globus-job-submit)
  - PPPL: tr_send_pppl.pl (xtranspin on unix cluster)
  - MIT / GA: pretransp
- Monitor the status of the run on the web
  - JET / PPPL: tr_status
    (globus-job-submit / globus-url-copy)
- Steering a run
- Request to look at intermittent Output (globus-job-submit)
  - PPPL: tr_look
- Halt an active run (globus-job-submit)
  e.g.: for steering
  - PPPL: tr_halt
- Fetch Output Files (globus-url-copy)
  - PPPL: tr_fetch
- Cancel/Cleanup a run (globus-job-submit)
  - PPPL: tr_cleanup
- Runs, that are not fetched or canceled, are deleted after 1 month
Server (with proxy from client)
- Read <runid>.REQUEST file
- Submit job to available host
- Obtain Namelist from MDSplus as specified in <runid>.REQUEST
- Generate scripts for processing
- Run trdat to read/process Input data as specified in Namelist
- Launch TRANSP
- Optional, write TRANSP Output into MDSplus Tree
- Write <tok><year>_<runid>.FILESREADY file

How to Monitor Runs?

Available PPPL Tools

All software is available in the TRANSP svn repository and also as a NTCC module TR_Client.
Setup at PPPL: "module load ntcc"

Production Related Tools

transp_tree.tcl
script to create a Model Tree (for Sys Admins of sites that support their own MDSplus Server)
mdstcl @transp_tree.tcl <Tree Name>
tr_start <runid> [tshare]
script with pre-set definitions to run tr_start.pl
- if you don't want to use mdsplus, enter "NONE" for MDS_TRANSP_SERVER.
- if you don't want an editor to enter comments, put the comments into the file <runid>LogFile.txt
tr_start.pl <server> <tree> <runid> [<mdsid>][silent] [xtar | start_remote | nocreate | nomdsplus | sendtree] [ -nbi_np <np> ] [ -alt <username> | tshare] [ -q <pbs-queue> ]
interactive script that runs various programs
- must be run from directory where Namelist resides
- creates MDSplus Pulse for Input Data (unless option)
- writes Namelist and Input Data into MDSplus (unless option)
- generates <runid>.REQUEST file
- options
  - xtar: generate tar file (via datprep) containing Ufiles and REQUEST (in addition to createing a tree).
  - start_remote: No MDSplus Tree is created; only a tar file (via datprep).
    REQUEST file contains Server/Tree to be created at PPPL
  - nocreate: No MDSplus Tree is created; only a tar file (via datprep).
    REQUEST file contains no server/tree -> TRANSP output will be a tar file containing tree files.
  - sendtrees: MDSplus Tree is created.
    REQUEST file contains user's local tree input and output paths.
  - nomdsplus: runs datprep to produce tar file.
  - tshare: switch to run alternate transp version.
  - -q <pbs-queue>: select a queue (other than default)
    - INFINIBAND : kruskal (restricted)
    - BIG : extra memory
To avoid repeated typing, use these environment variables:
```
  TR_EMAIL		-- Your e-mail address	  	 
  EDITOR		-- Your favourite editor
  MDS_TRANSP_SERVER     -- MDSplus Server for your Input
                           NONE if nomdsplus
  MDS_TRANSP_TREE	-- Tree Name of your Input
	                   not used if "nomdsplus" or at PPPL
```
tr_send <runid>
performs various checks and runs tr_send_pppl.pl
tr_send_pppl.pl <runid> <tok> [ -s ] [trdat] [FAST] [UFILE | CREATE | NOMDSPLUS | TREES]
- must be run from same directory as tr_start
- transfers the <runid>_<TOK>.REQUEST / <runid>_tmp.tar file to PPPL (globus-url-copy)
- submits the run (globus-job-submit)
- options
  - trdat: run trdat only
  - FAST: submit to pbs-queue fielder
  - UFILE: send tar file, PPPL creates tree at client's server
  - CREATE: send tar file (PPPL returns Tree triplets)
  - TREES: send .REQUEST file; PPPL fetches/returns user's trees via mdstcl
  - NOMDSPLUS: send tar file containing Input; fetch tarfile containing Output.
tr_cleanup <runid> <tok>
- cleanup all traces of a run
  if the run is still active, it will be aborted
tr_look <runid> <tok> [ [archive] [nomdsplus] ]
- write interim MDS output into TRLOOK_<tok> on transpgrid.pppl.gov
- option archive: for aborted and active runs
  if the run is still active, it will be aborted
  archive run "as is" as specified in <runid>.REQUEST
- option nomdsplus: generate CDF file only
  non-PPPL: use only with option "archive"
tr_halt <runid> [<tok>] [t <sec>]
- halt an active run (for steering)
- option t <sec>: halt at time = <sec>
  if no time is given, run is halted immediatly
tr_fetch [-nowait][ -s ] [ -d{ebug} ] [-tar] <tok.yr> <runid> [ <target-dir> ]
- Check if already fetched
- Option -tar recommended
  - wait for <tok><year>_<runid>.tar.gz
    globus-url-copy tar file to $CWD or to <target-dir>
- Default
  - wait for <tok><year>_<runid>.FILESREADY
    (contains names of available TRANSP Output files)
  - read <tok><year>_<runid>.FILESREADY
  - globus-url-copy available TRANSP Output Files to $CWD or to <target-dir>
- Return Status to PPPL
  globus-url-copy <tok><year>_<runid>.FILESCOPIED to ~/transp/result/<tok><year>/
rs_newrun
- Interactive program to manipulate a Namelist for Steering.
mdsplot
Program to write TRANSP output from a CDF file into an existing MDSplus pulse.

Tools to Prepare Runs

XTranspin
GUI to prepare a TRANSP Namelist.
scruncher
Program to generate Fourier moment boundary ufiles.
See scruncher.doc
scrunch2
Program to generate ufiles via analysis of EFIT.
nbfile
Program to set up the beam parameters.

Tools to Examine Output

adisplay
Utility for interactively displaying ACFILE contents.
get_fbm
Reads the TRANSP COMMON array fbm from an ACfile
- to de-compressed ACfiles:
  tr_unzip <runid> <tok>
- IDL procedure to plot <runid>_fi_<id>.cdf:
  PLOT_FBM,runid,ps=ps,ecut=ecut,tim_id=tim_id,tok=tok,spc=spc,code=code,fus=fus,outline=outline
  e.g.: plot_fbm,'20100P07',tok='ITER',spc=2,tim_id=2,ps=2
- Examples of IDL output for ASDEX-U
  See: fbm
Ion Source at Birth
to examine Ions deposited:
- IDL procedure to plot <runid>_birth.cdf<id>:
  PLOT_BIRTH,runid,ps=ps,tim_id=tim_id,tok=tok,s=s,code=code,gc=gc
  e.g.: plot_birth,'32621R02',tim_id=2,tok='ITER',ps=1
pboxn0
Program to read beam-in-box neutral density data.
rplot
- To look at interim or aborted runs:
  rplot t s transpgrid.pppl.gov t trlook_<tok> y <tok>.<yy> q <runid>

Home

Questions,comments,suggestions, etc. to: transp@pppl.gov
See also the disclaimer.