trmpi_requestor broadcast (cpu0): NUBEAM_SAVE % nbi_states: fld_states write OK to filename: 78072Z02_fi/78072Z02_nbi_fld_state.cdf %wrstf: transp_rplot_state_backup %trTeqSaveState: writing TEQ savefile 78072Z02_teq_restart.in % RESTART RECORD WRITTEN AT TA= 1.4217600E+01 arrays allocated in pt_diff_data_init arrays allocated in ptsolver_data_init %WRSTF: virtual memory size = 1.088E+03 MB. %TESEND - NAMELIST REQUESTED ABORT, TIME = 0.0000E+00 %ABORTR: TA = 1.42176E+01 ** TRANSP ABORTR SUBROUTINE CALLED ** FTIME_OK = 1.20000E+01 -- normal exit taken by ABORTR. FTIME_OK (ABNORMAL EXIT) CPU TIME USED (hours): 9.40311E+01 %bad_exit: generic f77 error exit call (errset_mpi status 999) %kill_nubeam_server: no server READY file found. ------------ stderr ---------------- -------------------------------------------------------------------------- [[19979,1],2]: A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces: Module: OpenFabrics (openib) Host: dawson002.pppl.gov Another transport will be used instead, although this may result in lower performance. -------------------------------------------------------------------------- (mpi_share_env) process myid= 0 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 1 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 2 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 3 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 4 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 7 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 5 cwd: /local/tr_budny/transp_compute/JET/78072Z02 (mpi_share_env) process myid= 6 cwd: /local/tr_budny/transp_compute/JET/78072Z02 [dawson002.pppl.gov:32663] 7 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics [dawson002.pppl.gov:32663] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link rename failed: Invalid cross-device link ?abortr exit -- errmsg_exit reached. ERRSET_MPI call by process myid= 0 ; istat = 999 **** myid= 0 MPI_ABORT CALL **** -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 999. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec has exited due to process rank 0 with PID 32664 on node dawson002.pppl.gov exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpiexec (as reported here). -------------------------------------------------------------------------- ==>runtrx: check for ftime_ok ==>runtrx: 78072Z02_ABORTOK.TMP present ==>runtrx: 78072Z02TR.EXE processed as normal exit. ==========(runtrx)====================== ==========TRANSP output conversion====== date: Mon May 21 01:33:46 EDT 2012 ( dawson002.pppl.gov ) ==========(runtrx)====================== ---------------> starting: plotcon 78072Z02 2012/05/21:01:33:46 %initcpl: MDS_CACHE enabled. %PoPlot -- reading .PLN files %POPLT2-- PROCESSING RUN 78072Z02 SHOT NO. 78072 EXPECT 574 SCALAR FCNS, 1500 PROFILE FCNS OF TIME "MF" FILE RECORD SIZE = 40 WORDS (FLOATING PT) dmg_datbuf_expand call from dmgini_sized: isize= 0 78072Z02MF.PLN size = 1.1G setenv NF_64BIT %initcpl: MDS_CACHE enabled. [mds_cache_disable: MDS+ cache disabled.] dmg_datbuf_expand call from dmgini_sized: isize= 0 (retry folding filename to lowercase) ...reading TF.PLN header data... cdfcon: NETcdf file datestamp : 21-May-2012 01:44:22 build_date: call sget_env build_date: call ufopen xshare_build.dat cdfcon: Transp common build date : Wed Mar 21 05:53:07 EDT Define Dimensions 11 define Scalar Fct 574 Define Multi Graphs 579 Write Profiles 1500 X 1 1 40 XB 2 2 40 THETA 3 7 80 RMJSYM 4 22 165 RMAJM 5 24 81 MCINDX 6 25 220 ILIM 7 27 116 XGRID_NPHI 8 426 6 ILDEN 9 466 8 IVISB 10 469 2 INTNC 11 471 19 read NF File : 575 574 Write Multigraph: 579 ...readback test of .CDF file... 2655 variables, 13 dimensions 9 att ...header check SUCCESSFUL; now check data. ...check profile data... plotcon: CDF files: /local/tr_budny/transp_compute/JET/78072Z02/78072Z02.CDF /local/tr_budny/transp_compute/JET/78072Z02/78072Z02PH.CDF %targz_pseq: no directory: 78072Z02_replay (normal exit) --------------->plotcon: normal exit. 2012/05/21:01:45:07 ==>runtrx: TRANSP postprocessing OK ==========(runtrx)====================== ==========TRANSP add to MDSplus ======== date: Mon May 21 01:45:07 EDT 2012 ( dawson002.pppl.gov ) ==========(runtrx)====================== %mdsplot: call INITPL %initcpl: MDS_CACHE enabled. %mdsplot: call sget_env %mdsplot: call ufopen xshare_build.dat %mdsplot: MDSplus controls cleared, server set to local. mds_conopn: option = 4 780722602 TRANSP_JET ...connecting to server: TRANSPGRID1.PPPL.GOV ...tcl("EDIT TRANSP_JET/SHOT=780722602") dmg_datbuf_expand call from dmgini_sized: isize= 0 ... reading NetCDF header data ... cdfhrd: size(time) = 4207 cdfhrd: size(time3) = 4207 MDS 1D Put OK MDS 2D Put OK MDS MG Put OK open 78072Z02_nubeam_init.dat add_file: 183 lines - 94 delete node .TRDATA tcl("write") ...mdsplot: normal exit. ==========(runtrx)====================== ==========TRANSP finish and cleanup===== date: Mon May 21 02:49:14 EDT 2012 ( dawson002.pppl.gov ) ==========(runtrx)====================== %finishup: terminating runtime transp with send_tr_elvis CLOSE %finishup -I- pppl.gov production run %finishup: copying TRANSP permanent output files to /u/tr_budny/transp/result/JET.09 mv 78072Z02_birth.cdf1 /u/tr_budny/transp/result/JET.09/78072Z02_birth.cdf1 mv 78072Z02_birth.cdf2 /u/tr_budny/transp/result/JET.09/78072Z02_birth.cdf2 mv 78072Z02_birth.cdf3 /u/tr_budny/transp/result/JET.09/78072Z02_birth.cdf3 mv 78072Z02_birth.cdf4 /u/tr_budny/transp/result/JET.09/78072Z02_birth.cdf4 mv 78072Z02_birth.cdf5 /u/tr_budny/transp/result/JET.09/78072Z02_birth.cdf5 mv 78072Z02.CDF /u/tr_budny/transp/result/JET.09/78072Z02.CDF mv 78072Z02.DATA1 /u/tr_budny/transp/result/JET.09/78072Z02.DATA1 mv 78072Z02.DATA2 /u/tr_budny/transp/result/JET.09/78072Z02.DATA2 mv 78072Z02.DATA3 /u/tr_budny/transp/result/JET.09/78072Z02.DATA3 mv 78072Z02.DATA4 /u/tr_budny/transp/result/JET.09/78072Z02.DATA4 mv 78072Z02.DATA5 /u/tr_budny/transp/result/JET.09/78072Z02.DATA5 mv 78072Z02ex.for /u/tr_budny/transp/result/JET.09/78072Z02ex.for mv 78072Z02_FI_TAR.GZ1 /u/tr_budny/transp/result/JET.09/78072Z02_FI_TAR.GZ1 mv 78072Z02_FI_TAR.GZ2 /u/tr_budny/transp/result/JET.09/78072Z02_FI_TAR.GZ2 mv 78072Z02_FI_TAR.GZ3 /u/tr_budny/transp/result/JET.09/78072Z02_FI_TAR.GZ3 mv 78072Z02_FI_TAR.GZ4 /u/tr_budny/transp/result/JET.09/78072Z02_FI_TAR.GZ4 mv 78072Z02_FI_TAR.GZ5 /u/tr_budny/transp/result/JET.09/78072Z02_FI_TAR.GZ5 mv 78072Z02FPPRF.DATA /u/tr_budny/transp/result/JET.09/78072Z02FPPRF.DATA mv 78072Z02_ICRF_TAR.GZ1 /u/tr_budny/transp/result/JET.09/78072Z02_ICRF_TAR.GZ1 mv 78072Z02_ICRF_TAR.GZ2 /u/tr_budny/transp/result/JET.09/78072Z02_ICRF_TAR.GZ2 mv 78072Z02_ICRF_TAR.GZ3 /u/tr_budny/transp/result/JET.09/78072Z02_ICRF_TAR.GZ3 mv 78072Z02_ICRF_TAR.GZ4 /u/tr_budny/transp/result/JET.09/78072Z02_ICRF_TAR.GZ4 mv 78072Z02_ICRF_TAR.GZ5 /u/tr_budny/transp/result/JET.09/78072Z02_ICRF_TAR.GZ5 mv 78072Z02_nubeam_init.dat /u/tr_budny/transp/result/JET.09/78072Z02_nubeam_init.dat mv 78072Z02PH.CDF /u/tr_budny/transp/result/JET.09/78072Z02PH.CDF mv 78072Z02TR.DAT /u/tr_budny/transp/result/JET.09/78072Z02TR.DAT mv 78072Z02TR.INF /u/tr_budny/transp/result/JET.09/78072Z02TR.INF %finishup: retaining 78072Z02tr.log mv 78072Z02TR.MSG /u/tr_budny/transp/result/JET.09/78072Z02TR.MSG rm: No match. %finishup -I- both the GRID and the PPPL flags are set. ==========(runtrx)====================== ==========>runtrx normal exit<========== date: Mon May 21 02:50:33 EDT 2012 ( dawson002.pppl.gov ) ==========>runtrx runsite = pppl.gov<======