Computational Science Initiative Event

Data-related challenges are quickly dominating computational and data-enabled sciences, and are limiting the potential impact of scientific application workflows enabled by current and emerging extreme scale, high-performance distributed computing environments. These data-intensive application workflows involve dynamic coordination, interactions and data coupling between multiple application processes that run at scale on different resources, and with services for monitoring, analysis and visualization and archiving, and present challenges due to increasing data volumes and complex data-coupling patterns, system energy constraints, increasing failure rates, etc. In this talk I will explore these data challenges in extreme scale science and investigate how solutions based on data sharing abstractions, managed data pipelines, in-memory data-staging, in-situ placement and execution, and in-transit data processing can be used to address these data challenges at extreme scales. This research is part of the DataSpaces project at the Rutgers Discovery Informatics Institute.