Overview of Scientific Workflow Automation with Kepler/SPA

Speaker: Bertram Ludaescher (UC Davis) & Ilkay Altintas (San Diego Supercomputing Center) Title: Overview of Scientific Workflow Automation with Kepler/SPA Abstract: The DOE SDM/SPA team and various other groups from the Kepler community are working on an integrated problem-solving environment for scientific workflow management and automation. A common goal of the Kepler collaboration is to support the whole life cycle of scientific workflows (SWFs), from modeling and design, to deployment, execution, monitoring, and SWF exchange, reuse, and archival. We provide an overview of (i) the characteristics and requirements of SWFs that we encountered in a number of existing applications, (ii) the current capabilities of Kepler in support of those requirements, and (iii) upcoming extensions. In the SPA context, the focus is on compute-intensive and data-intensive workflows. Our system demonstrations highlight the use of external job schedulers (such as NIMROD or CONDOR) to manage compute-intensive tasks in SWFs, and the use of the SDSC Storage Resource Broker (SRB) for data-intensive tasks, including efficient parallel data transport, replication, and metadata-based querying of data collections.
Last modified: Wed Jun 1 11:22:58 EDT 2005