Analysis codes

Automation

Scripts used for the automation of basic data analysis immediately following each shot (at event LTX_EOC).

• /runltx/action_scripts/LTXB_do_auto_scripts.py (on ltx-bender): Launched directly by the run VI. Acts as a wrapper for data analysis scripts, launching them and logging their text output and error messages. Run with no arguments, operates on the current shot, but can be run with a shot number as the sole argument, and will instruct all child processes to operate on that shot number. Logfiles are saved in /runltx/action_scripts/logging/ and copies are archived to /p/ltxdata/auto_script_logging/.
• /runltx/action_scripts/LTXB_validate_calshot_pro.py (on ltx-bender): Launched directly by LTXB_do_auto_scripts.py. Identifies type of current shot, tests match of assigned calibration shot if any exists; if no good calibration shot is assigned, seeks a recent calibration shot for an uncalibrated plasma shot and seeks uncalibrated plasma shots for a calibration shot; updates calibration shot metadata to reflect results.
• /runltx/action_scripts/LTXB_RTD_reader_auto.py (on ltx-bender): Launched directly by LTXB_do_auto_scripts.py. Reads RTD data into the tree. Calibration is not working right, as of Nov. 1st 2019, but raw data should be useable.
• /runltx/action_scripts/LTXB_tdisrupt_auto.py (on ltx-bender): Launched directly by LTXB_do_auto_scripts.py. Finds time of breakdown and disruption, and writes to the tree.
• /runltx/action_scripts/LTXB_RMaj_TA_auto.py (on ltx-bender): Launched directly by LTXB_do_auto_scripts.py. Estimates major radius using the Toroidal Array and writes to the tree.
• /runltx/action_scripts/LTXB_MHD_spectroscopy_TA.py (on ltx-bender): Launched directly by LTXB_do_auto_scripts.py. Analyzes Toroidal Array data to identify up to four coherent toroidal magnetic perturbation structures with n<5, writing amplitudes, frequencies, and Fourier numbers to the tree.
• /p/ltxdata/reconstructions/LTXB_auto_recon_wrapper.py (on portal): Launched directly by LTXB_do_auto_scripts.py. Acts as a wrapper for Chris Hansen's PSI-TRI shell script wrapper, parsing and storing the resulting reconstruction data.
• /runltx/action_scripts/LTXB_email_daily_log.py (on ltx-bender): Launched in the evening by the scheduled task manager. Checks whether the date at run time matches the date of the current shot; if so, looks for a matching automation logfile in /runltx/action_scripts/logging/, and emails it to a list of operators.

Error Flagging

The LTXB_do_auto_scripts.py script contains umbrella error-tagging for the scripts it runs. A list of error keywords is included in the header for every concatenation of the logfile.

The main error handling function is watching for non-zero returncodes (i.e. failing with raise Exception(errortext) rather than exit()) from the child process, and for specific code words to be returned within the error output of the child process (the errortext string). These error code words currently include:

• "WARNING: " Signifies that the code is able to at least partially complete, but either some data was not read, or not stored, or was outside some built-in credibility threshold.
• "ABORTING: " Signifies that a critical error has occurred, forcing the code to abort operation at some point. This could be an unexpected error within the code, failure to read/write a key file, or anything else that prevents the script from completing any of its tasks successfully.

In any of these cases, a corresponding finishcode is set (warned or aborted)

If none of the above failure markers is found in the error output of the child process but a failure appears within LTXB_do_auto_scripts.py (e.g. it is unable to launch the child process for some reason), it sets for itself a finishcode of FAILED. If none of the above cases is true (i.e. the code completes with a 0 returncode), but the code contains the text "WARNING: " (including the colon and space) in its stdout (e.g. in a print() command), the finishcode will be flagged as warned. Finally, if the code completes with a 0 returncode, the finishcode is set to finished.

There is also the case where a script is skipped entirely because of a failure in a prior script (e.g. the automated reconstruction script is not launched if no plasma current is detected), in which case the skipped finishcode is returned.

New Scripts

New automated scripts can be easily added to LTXB_do_auto_scripts.py, due to its modular, expandable design. Starting on line 64 (as of 8/28/2020) is a series of parameters relevant to child scripts. To automate a new script, simply add it to the ~/runltx/action_scripts/ directory, set it to be executable, open LTXB_do_auto_scripts.py, and add a ..._script variable and a ..._args variable. The first should be the full command line string needed to run the script properly, and the second (starting with a space) should contain all arguments required for the script to run properly. If the script cannot be made python 2.6.6 compatible or needs to be run by ssh on another machine, there should also be a ..._pythoncall variable added in the preceding "PYTHON CONFIG" section, which contains the complete prefix required (e.g. Recon_pythoncall='ssh ltxuser@portalc6 python2.7').

The newly added variables must be added to the appropriate following lists, autoscriptlist, autoscriptarglist, and scriptprefices, and a True or False value must be added to the analyzevacuumshots list to specify whether the script should be run for shots flagged as having no plasma current. Important: the last item in each list should belong to the reconstruction script. The wrapper assumes the reconstruction script will be the last thing run, since it typically takes 2-3 minutes to complete. Once the lists have been updated and the new child script has been thoroughly tested, the appropriate index in the list can be added to the optionalscripts list, used so that broken or deprecated scripts can be easily removed from the automation cycle but still documented for later use.

When LTXB_do_auto_scripts.py runs, it will create a child process with a command line constructed of the prefix string, the script string, the current shot number, and any other arguments, in that order. That is to say, it expects all child scripts to take the current shot number as the first (or only) positional argument. For example, when it launches LTXB_auto_recon_wrapper.py for shot 101889, the command line output reads:

ssh ltxuser@portalc6 python2.7 /p/ltxdata/reconstructions/LTXB_auto_recon_wrapper.py 101889 --logstdout.

Studying existing scripts for other rules of thumb and error clarification ideas is recommended but not necessary. Working with the above guidelines should keep any new scripts compatible with the automated error logging, maintaining searchability of the logfiles.

LTX TV

cd /p/ltxdata/ltxcam/Phantom210-10161/2018/
cleanidl
idl
idl> .run /p/ltxdata/ltxcam/cine_time.pro
idl> .com /p/ltxdata/ltxcam/cine_display_ltx.pro
idl> .run /p/ltxdata/ltxcam/cine_display_runner_ltx.pro

Updating plots

Needs .cshrc settings from dboyle

cd /p/ltxdata/idl_analysis/general/
idl
idl> updating_plots_b

Analysis for Published Figures

Scripts used to generate specific figures. Please specify the figure(s) generated, and include paths and any other information needed to re-run the script.

Known MDSplus Errors

Since the MDSplus documentation is not always very helpful, especially with regards to error messages, some errors that have caused confusion and headaches are listed and explained here.

TreeCONNECTFAIL('Error connecting to remote server.')

Observation: In at least one case arose from a script generating and failing to properly close an excessive number of tree connections. Rather than specifying this problem or forcing old connections to close, the server simply refused all incoming tree connections (at least, from the script in question) and threw a generic error.
Solution: Make sure to call MDSplus.Tree.close(<treepointer>) or its non-python equivalent once the <treepointer> connection is no longer needed.

Related Pages

Running LTX - information on LTX operation software
MDSplus trees - description of the trees and their structures
Tree Changes - brief description of the make-tree script
Equilibrium Reconstruction - information about PSI-TRI and associated scripts
TRANSP - information for running LTX TRANSP