Speaker
Dr.
Cedric Serfon
Description
Rucio, the next-generation Data Management system in ATLAS
On behalf of the ATLAS Collaboration
Rucio is the next-generation of Distributed Data Management (DDM) system benefiting from recent advances in cloud and "Big Data" computing to address HEP experiments scaling requirements. Rucio is an evolution of the ATLAS DDM system Don Quijote 2 (DQ2), which has demonstrated very large scale data management capabilities with more than 160 petabytes spread worldwide across 130 sites, and accesses from 1,000 active users. However, DQ2 is reaching its limits in terms of scalability, requiring a large number of support staff to operate and being hard to extend with new technologies. Rucio addresses these issues by relying on new technologies to ensure system scalability, cover new user requirements and employ new automation framework to reduce operational overheads.
In this talk, we will present the history of the DDM project and the experience of data management operation in ATLAS computing. Thus, We will show the key concepts of Rucio, including its data organization. The Rucio design, and the technology it employs, will be then described, specifically looking at its architecture and the various software components it uses. We will show also the performance of the system and will conclude by giving a status of the project and its future evolution.
Primary author
Prof.
Farid Ould-Saada
(University of Oslo)
Paper
Paper files:
