Tree Changes

From LTX Wiki
Jump to: navigation, search

MDSplus trees can be edited by scripts (e.g. the maketree script, LabView), from the command line in TCL, or in traverser.

Minor temporary changes and updates to the MDSplus tree will inevitably be needed, but should hold to the established tree architecture conventions to avoid the tree structure descending into chaos.

Temporary Changes

There are some situations where changes from normal operation may require certain features of the tree (e.g. calibration data, trigger timing, etc) to be modified. These changes should be made to the model shot (-1) in traverser, with the understanding that such changes a) should be reverted at the end of the experiment/run which required them, and b) are vulnerable to being eradicated when the tree is rebuilt. All such changes (and the post-run reversion) should be noted explicitly in the logbook.

Tree Command Language (TCL)

Running mdstcl in a terminal opens the TCL command line, providing low level (i.e. basic and powerful but often abstract and esoteric) access to tree elements, e.g. adding and deleting nodes, changing node write parameters, etc. TCL's main function is the modification of tree structures, although it can also insert data and TDI into nodes. Moderately well-organized TCL documentation is available here.

Tree Data Interface (TDI)

Running tdic in a terminal opens the TDI command line. TDI serves as the data-processing scripting language within the MDSplus tree, performing such calculations as applying calibration factors to signals and establishing signal timebases. Quasi-organized TDI documentation is available here, although the summary of TDI built-in functions and index with links are often the most useful documents available.

Error documentation:

  • %TREE-W-NOT_OPEN: Nominally, this means just what it says. However, there are some subtleties. If you write a piece of TDI scripting using TCL's PUT command, it will let you call out relative node names (e.g. PUT RATIO NUMERATOR/DENOMINATOR where NUMERATOR and DENOMINATOR are both child nodes of the current default) and TCL will figure out the correct paths on its own. However, TDI seems unable to do this, and will assume that the unfamiliar name must be some unknown tree, and complain that this nonexistent tree is not open. Another case is that, when writing a block TDI script, especially filled by python's MDSplus.Data.compile function, TDI sometimes is unhappy being given string node paths within the block, and wants instead to have a reference (e.g. $1) within the block and the string of the path passed externally.
  • %TDI-E-TdiRECURSIVE: This can be difficult to debug, because TDI functions somewhat differently depending on whether run line-by-line in tdic or as scripts within tree nodes. In essence, it appears that when one node references another (e.g. a processed signal node pointing to a raw node), those nodes are forced to share a namespace. As a result, if the raw node (or even a node several pointers away such as the CHANNEL node in a SIGNAL->SIGNAL:RAW->DEVICE:CHANNEL reference daisy-chain) contains a variable such as _data and the same variable name appears in the calling node (SIGNAL in this example), one will overwrite the other. In some cases, e.g. where a variable contains a node path, this may lead to inadvertent recursion.
    • Example: Consider the \LTX_B::TOP.OPER_DIAGS.RTDS.BEAM_DUMP:SENSOR_01 node. In the pre-101397 tree build, this node contained the variables _data, _units, and _time, like many TDI constructs. _data draws from \LTX_B::TOP.OPER_DIAGS.RTDS.BEAM_DUMP:SENSOR_01:RAW, which points to \NI_RTD:PXI4357_S2:INPUT_08. However, \NI_RTD:PXI4357_S2:INPUT_08 also contains the variables _data, _units, and _time. Since the compound TDI shares a namespace, when \LTX_B::TOP.OPER_DIAGS.RTDS.BEAM_DUMP:SENSOR_01 is interpreted, _data is defined by a pointer to a node containing _data, and TDI gets upset.
  • %SS-W-SUCCESS: Probably one of the most puzzling MDSplus error messages.
    • Example: One case where this comes up is when using python MDSplus to get the pointer to a node with malformed punctuation: e.g. tree.getNode('.OPER_DIAGS.POWER_SUPS:'+supplyname+':KOSHIN') where supplyname is accidentally left as an empty string. The resulting .OPER_DIAGS.POWER_SUPS::KOSHIN' doesn't quite make sense, so it doesn't get flagged as "can't find node," although why the error is called "SUCCESS" is unclear.
  • %TDI-E-SYNTAX, Bad punctuation or misspelled word or number: Can indicates a number of problems. Most commonly, it just means that a TDI block script contains multiple returned/echoed (i.e. not silenced with a semicolon) lines, and the compiler can be unhappy if any but the last line in a block returns a value.
  • %TREE-E-INVDTPUSG, Attempt to store datatype which conflicts with the designated usage of this node: This means exactly what it says, but can be confusing when triggered by having accidentally prefixed a node name with a period, which indicates a structure-type node that cannot contain data. No specialized error is returned; as far as the failure to put data is concerned, a structure node and a structure node with a period are identical, so it throws the same error.

Useful tricks:

  • GETNCI to read literal node contents: Sometimes, it's useful for one node to pull the string-literal contents of another node, rather than seeing it compiled, executed, or otherwise interpreted. For example, if you want to acquire node-pointer path text rather than node-pointer value (e.g. to clone a pointer rather than daisy-chaining pointers). In this case, the TDI command GETNCI(<sourcenode>,"RECORD") will return the desired string.
  • Integrals and derivatives: There exist somewhat poorly-documented "derivative" and "integral" commands. However, these can be reliably replicated with very straightforward DERIV(<signal/array>) and ACCUMULATE(<signal/array>)*<dt>. Since derivatives are generally noisy, it's wise to also employ the SMOOTH1D(<signal/array>,<width>) boxcar smoothing function. Note that "DERIV" applied to a signal returns the same units as the signal, rather than the units of the signal divided by the units of the signal's DIM.

Traverser

Traverser is essentially a GUI for TCL, and can in principle perform all (or at least most) functions of TCL. However, since there have been issues in the past with traverser failing to change the node structure, the most useful functions of traverser are:

  • Ease of navigation
  • Inspection of tree structure and node contents
  • Directly modifying node data in the model tree (e.g. shot metadata)

Run simply as traverser from the command line, a dialogue box will request tree (e.g. LTX_B) and shot number (e.g. -1). Traverser can also be run with arguments, such as traverser -tree LTX_B -shot -1 and will skip directly to displaying the top nodes.

As in most linux scripts, highlighting a node (by left-clicking) buffers its path in the clipboard, and it can be pasted with a middle-click. Right-clicking and holding provides a context menu to view data, device settings, etc.

Long-Term Changes and Updates

Lasting modifications to the read-level of the tree (i.e. from top nodes up to and including processed data nodes) should be implemented in the dedicated makefile, called maketree.py. Any such changes should not be undertaken without discussion, oversight, and review by whoever is currently responsible for tree maintenance. Maketree runs should not be performed during or immediately before a planned run, to avoid creating unintended problems, and should always be tested with crates-only shots to verify that the tree and shot cycle function as expected. Changes to diagnostic nodes (e.g. gain factors) should be tested as far as possible to ensure that no new bugs have been introduced. Any changes should be noted in the logbook so that the next person to run is aware of them.

Maketree Script

The maketree script lives in /usr/local/ltx/, with its necessary config files stored in /usr/local/ltx/maketree_config

LTXB_maketree.py should not be used to rebuild LTX_B on a run day unless absolutely necessary, and if it is, the run operator must be informed. Even if the maketree script has not been changed, it may undo various temporary changes without warning.

If run with no arguments, LTXB_maketree.py will write to TOYTREE. This is how changes to LTXB_maketree.py should be tested, to ensure that the script finishes running. If the maketree script does not complete, the model tree will be completely broken until the script completes successfully.

If run as LTXB_maketree.py --commit, the script will issue a warning and require confirmation. If confirmed, the script will write to LTX_B. The script will request comments to go into a change log stored in the tree at .METADATA.TREE:CHANGE_NOTES, and also record the date and time of the tree build. For user convenience, a string defined at the top of the maketree script is included in the change note prompt, so that edits can be accumulated.

Including subtree flags will also rebuild the indicated subtrees which are handled by maketree, whether or not --commit is used. The currently existing and/or planned subtrees are:

  • LTX_CAMERAS (--st_cameras): For Filippo Scotti's fast cameras.
  • LTX_NBI (--st_nbi): For the neutral beam system's diagnostics and parameters.
  • LTX_PCS: Not yet implemented, but planned for use with the LTX plasma control system. May or may not be built by the maketree script.

For example, rebuilding LTX_CAMERAS and LTX_NBI would be run as LTXB_maketree.py --commit --st_cameras --st_nbi at the command line.

Note that LTXB_maketree.py uses the MDSplus.py python module and the LTX-specific MDSplus setup, which means it is only happy living in the environment established on Lithos and may fail spectacularly if transplanted to the cluster. There are some peculiar TCL work-arounds and environment variable reconfigurations to watch out for within the code, as a result of previous customizations, but these are mostly confined to the build_dtAcq_with_TCL function.

Changelog

  • Shot 100346 (01/16/2019)
    1. Replaced .METADATA:DATE and .METADATA:T_ZERO with .METADATA:TIMESTAMP
    2. Corrected default T_DTACQ:CHRONOS trigger time from 0s to 0.4s
    3. Added .TIMING.EVENTS and subnodes; added add_device_timing_node constructor function which includes :SIG (signal) and :DT (pulse length) subnodes within each timing node
    4. Added config_J221 function to correct default settings including channel modes (all set to 5) and device clocks (both set to 0:2^24:1)
    5. Replaced .CONTROL.PCS_PARAMS with .CONTROL.PCS subtree.
  • Shot 100475 (02/26/2019)
    1. Added code for building .LTX_NBI subtree, and added link to .OPER_DIAGS node.
    2. Corrected documentation regarding diamagnetic loop and compensation coil.
    3. Changed how command to rebuild subtrees is flagged.
    4. Added tags to .timing.control and .timing.acquisition nodes.
    5. Added .DEVICES.CAMAC.J221_UNITS flag node.
    6. Added E_EMITTED and E_DEPOSITED to .BEAM_DIAGS.PERFORMANCE node.
    7. Removed .CONTROL top node, moved .LTX_PCS subtree into .OPER_DIAGS top node.
    8. Added add_ion_gauge_controller constructor.
    9. Added .OPER_DIAGS.FUELING.IG_CTRLRS nodes for main vessel ion gauge (MAIN), vessel micro ion gauge (MIG), and NBI tank micro ion gauges.
    10. Removed .OPER_DIAGS.FUELING.LFS_PUFF and HFS_PUFF nodes since there is no LFS puff valve.
    11. Removed .TIMING.FUELING.LFS_GAS and SGI nodes since they don't really exist.
    12. Added .OPER_DIAGS.FUELING.PUFF_SIG node for HFS puff. NOTE THAT CAPACITOR VALUE IN MAKETREE IS NOMINAL.
    13. Updated RTD config file to include linear fit parameters for RTD calibration.
    14. Added .OPER_DIAGS.NEUTRAL_BEAM:T_TRIG and :T_INJECT nodes for beam triggering.
  • Shot 100593 (02/28/2019)
    1. Fixed add_ion_gauge_controller constructor to include signal polarity and alternative exponent factors (besides the standard 11).
  • Shot 101092 (04/16/2019)
    1. Fixed add_resistive_thermal_device to do calibration properly.
    2. Fixed TL and BL reentrant mirnov polarities in REmirnovmap.csv config file.
    3. Added polarity accounting to calibrated_loop constructor function.
    4. Swapped calibration exponents on vessel main and micro ion gauges.
    5. Reversed all upper-shell flux loop polarities.
    6. Added polarity accounting to calibrated_mirniov constructor function.
    7. Added OH waveform setting nodes to .TIMING.BANKS.OH_WAVEFORM, and moved OH_POL_SW and OH_SUP_OFF there.
    8. Added OH polarity record node to .OPER_DIAGS.COILS:OH:NORMAL_POL.
    9. Changed Jorway tags from \JORWAY_A and \JORWAY_B to \J221_19 and \J221_21 respectively.
  • Shot 101123 (06/06/2019)
    • MAIN TREE:
      1. Added .RUN_PARAMS structure nodes and its reconstruction timebase-definition numeric nodes T0 and DT to the .RECONSTUCT.PSITRI_STD node.
      2. Added .OUTPUT_FILES structure node and its file path text subnodes to the .RECONSTUCT.PSITRI_STD node.
      3. Added Q_A, Q_0, Q_95, R_ICEN, BETA_TOR, BETA_POL, and VOLUME signal nodes to the .RECONSTUCT.PSITRI_STD node.
      4. Fixed bug, moving STATUS text node from .RECONSTUCT.PSITRI_STD:IP to .RECONSTUCT.PSITRI_STD where it was meant to be.
      5. Updated coil/supply relations so the blue coil points to Robicon IF and the TF points to Robicon T1 (no PEI digitized currently).
      6. Added Greg's modifications (based on his modifyDtacqDevice.sh script) to the build_dtAcq_with_TCL function, and changed it to the more accurate name rebuild_dtAcqs_with_TCL.
    • MAKETREE SCRIPT STRUCTURE:
      1. Moved config file paths to global constants section near top to simplify future changes/archiving.
      2. Altered rebuild_dtAcqs_with_TCL (previously named build_dtAcq_with_TCL) function to use lists and a for loop rather than three separate sets of TCL commands.
    • SUBTREES:
      1. Fixed broken subnode pointers in \LTX_NBI.BEAM_DIAGS.PERFORMANCE:E_DEPOSITED:E_TOTAL TDI scripting."
  • Shot 101396 (09/17/2019)
    • MAIN TREE:
      1. Added real calibration values and hardware nodes for LFS Langmuir probe in config file. Set same calibration values for HFS Langmuir probes, since same supply and readouts may be used.
      2. Reversed polarity of all lower shell flux loops (1 to -1, and -1 to 1).
      3. Added REQUEST nodes to power supply nodes, and referenced REQUEST nodes for each equilibrium coil to the appropriate power supply node.
      4. Broke IP calculations into IP:CALIBRATED for current IP, and COMPENSATED to correct for proportional and derivative pickup from equilibrium coils.
      5. Added Koshin nodes for Hoffman boxes/power supplies T2, T3, T4, DF, and IF to power supply nodes.
      6. Added Koshin nodes for all coils but OH and Yellow. TF coil has its own Koshin, all others automatically refer to the appropriate power supply Koshin.
      7. Added "DIAMAG_LOOP_INT" tag to .DIAGNOSTICS.MAGNETICS:DIAMAG_LOOP node.
      8. Added .DIAGNOSTICS.MAGNETICS:DIAMAG_LOOP2 node for new loop, with "DIAMAG_LOOP_EXT" tag.
      9. Added "V_LOOP" tag to .DIAGNOSTICS.MAGNETICS:V_LOOP node.
      10. Added fiber position subnode to .DIAGNOSTICS.SPEC.CHERS node, and fiber number nodes to subordinate spectrometer nodes.
      11. Added numerous diagnostic subnodes to .DIAGNOSTICS.SPEC.THOMSON node.
      12. Moved .RECONSTRUCT.SIMPLE:R_AXIS_TA into new .RECONSTRUCT.SIMPLE.TOR_ARRAY node as :R_AXIS.
      13. Added R_MINOR, R_LFS, and R_HFS subnodes to .RECONSTRUCT.SIMPLE.TOR_ARRAY node.
    • MAKETREE SCRIPT STRUCTURE:
      1. Changed construction of power supply nodes to resemble construction of equilibrium coil nodes.
      2. Added automated user identification and post-rebuild email dispatch.
    • SUBTREES:
      1. Added .LTX_NBI.METADATA:STNC_TO_SHOT node to record independent verification of shot sync between NBI and LTX.
  • Shot 101401 (09/26/2019)
    • MAIN TREE:
      1. Removed unused .ACTIONS top node.
      2. Implemented .DOWNSAMPLED subnodes for IP, NE, and V_LOOP signals.
      3. Removed extraneous .T_DISRUPT from .DOWNSAMPLED top node.
      4. Implemented possible fix to TdiRECURSIVE errors in .RTDS nodes.
      5. Added pointer from T1:KOSHIN node to I_COIL_TF:KOSHIN node.
      6. Fixed problems in \IP scripting.
    • MAKETREE SCRIPT STRUCTURE: N/C
    • SUBTREES: N/C
  • Shot 101462 (10/30/2019)
    • MAIN TREE:
      1. Fixed NEW problems in \IP scripting.
      2. Updated (provisional) external diamagnetic loop parameters and structure.
      3. Changed \T_DTACQ:HELIOS to point to \T_DTACQ:NEOS rather than being empty on rebuild.
      4. Removed deprecated .CHRONOS top level node.
      5. Added missing write to .METADATA.TREE:REBUILT_BY node.
      6. Removed extraneous :DIVIDED node from internal diamagnetic loop and turned off spuriously on :CAL:DIVIDED node value (since 101092).
      7. Added .MHD_SPECT to .RECONSTRUCT.SIMPLE top node in preparation for new automated script.
      8. Updated NEOS channels used by Koshin current monitors.
    • MAKETREE SCRIPT STRUCTURE:
      1. Moved \IP:COMPENSATED construction to TCL script at end of maketree, where it seems to be happier.
    • SUBTREES: N/C
  • Shot 101564 (01/02/2020)
    • Manually added .META_DATA and subnodes to .DIAGNOSTICS.SPEC.HAL node through mdstcl.
  • Shot 101986 (03/10/2020)
    • MAIN TREE:
      1. Added .META_DATA node and subnodes (FRAME_TRACK, EXP_START, EXP_END) to .HAL (manually added 1/2/2020)
      2. Added TN (last timepoint) and N (number of reconstructions) subnodes to .PSI_STD.RUN_PARAMS
      3. Added .WALL_CONDITN and subnodes (time of deposition, age, thickness) to .OPER_DIAGS
      4. Repaired config_Joerger constructor function to set hardware node conditions (e.g. WriteOnce) on inputs rather than the device node itself
      5. Added .META_DATA node and subnodes (FRAME_TRACK, EXP_START, EXP_END) to .HOLOSPEC and .ISOPLANE
      6. Added .CONCURRENT device nodes
      7. Added .OPER_DIAGS.COILS:OH:VI_MODE node
      8. Attempted new fix for pickup-compensated IP node
    • MAKETREE SCRIPT STRUCTURE:
      1. Added config_CRT constructor function
    • SUBTREES:
      1. Added .LTX_PCS subtree as top node

Related Pages

MDSplus trees - more details about tree structures
Analysis codes - A list of data analysis scripts with a brief description of their function and purpose, including those used for post-shot automation.

Retrieved from "http://ltxwiki.pppl.gov/index.php?title=Tree_Changes&oldid=934"