Partners and International Organizations
(English)
|
FORWISS (D), Active Knowledge (D), University of Surrey (UK), Numeca (B)
Assoc. Partners: CCLRC (UK), Cineca (I), MPG.IMET (D), DLR (D), IHPC&DB (RU)
|
Abstract
(English)
|
The ESTEDI project aims at establishing a European standard for the storage and retrieval of multi-dimensional high-performance computing (HPC) raster data. It addresses a main technical obstacle, the delivery bottleneck of large HPC results to the users, by augmenting high-volume data generators with a flexible tool for the management and extraction of spatio-temporal raster data. The main motivation stems from the fact that multi-dimensional data, e.g., data that uses the array as data type abstraction, cannot be satisfactorily mapped onto a conventional database management system (DBMS). The need for a 'real' multi-dimensional DBMS is increasing in accordance with the increased usage of numerical simulation tools and production of experimental data. In many fields, e.g., satellite data, computational fluid dynamics (CFD) simulation, statistical data, ect., increasingly larger data sets in the order of Terabytes or even Petabytes are produced and to be able to manage this amount data in an efficient way specialized DBMSs should be used.
A critical mass of large European HPC centres, a CFD package vendor, and two database expert groups are involved in this project and are specifying the common data management platform. A subsequent prototype implementation platform will rely on the multi-dimensional database system RasDaMan which will be optimised towards HPC by enhancing it with intelligent mass storage handling, parallel retrieval, and further enabling technologies. This platform will be operated and evaluated in various environments and for the most important HPC applications in different fields by the HPC centres and all developments will be in response to user requirements. The project outcome will be a field-tested open prototype platform for flexible, standards-based retrieval on multi-Terabyte raster data. It will be based on requirements collected on a trans-European level and demonstrated in real-life applications in the key areas. Additionally, the participating HPC centres will act as multipliers to further centres and the European industry using HPC with the main goal to strengthen their research position and to offer improved services to their users and clients.
CSCS-ETHZ is involved in the project as a HPC centre active in the field of chemical reactors flow modeling and one main task will be to couple the DBMS to the HPC data generator and visualization frontend. The resulting system will then be operated under real-life conditions with focus on the particular application area mentioned, but also possibly exploiting other fields in which CSCS-ETHZ is active, e.g., meteorology and Grid technology/infrastructure.
|
