All dates, times and locations of tech program events or other scheduled plans are subject to change. Please check back regularly to ensure you view the most up-to-date version of the schedule.
High performance infrastructures enable large-scale scientific applications and workflows to run with increased complexity and improved accuracy. However, as the size of these workflows grow, they are generating massive amounts of data that must be processed and analyzed in order to gain insights into the phenomena being studied. Traditional methods of running these types of workflows involves the applications writing and reading data from the parallel file system (or hard disk), but this approach is not practical for exascale. To solve this problem, the DataSpaces framework provides an in-memory data staging area that applications can utilize in order to communicate and interact during runtime without reading or writing from disk. The approach reduces the read/write access times and allows different workflow stages to overlap when possible, thereby reducing the overall time to completion of the workflow. The goal for the DataSpaces project is to allow scientists and application programmers to run their codes efficiently on a variety of HPC infrastructures, without knowledge of the underlying I/O subsystems or management (including high-speed interconnects). In this tutorial, we will demonstrate the easy-to-use DataSpaces API and show application programmers how to enable data staging by integrating their code with DataSpaces. We will then show users how to run these codes on high-performance machines in order to accelerate the I/O associated with their applications and reduce the overall time-to-completion of the workflow, using TACC Stampede as an example.
