rchitecture Architecture review date |
| |
|---|---|---|
Project lead | ||
On this page |
|
...
The XrdCeph plugin is a Storage level plugin implementing an API in the spirit of the POSIX filesystem abstraction [ref]. XrdCeph relies on libradosstriper to split up write data into RADOS objects, and to re-assemble read data from RADOS objects.
When Echo was being built it was noted that as an object store it was better suited to streaming entire objects rather than performing many small reads (AKA Vector reads) on it. It was therefore decided to place XCaches infront of all the storage endpoints.
...
\uD83D\uDC65 Stakeholders
Name | Role |
|---|---|
Architect | |
Architect | |
✏ Software quality attributes
...
The goal is to have a single code base combining libradosStriper and XrdCeph, which could be easily maintained by RAL. We would like to maximise throughput by parallelise parallelising operations / transfers where possible.
...
Project | Description | Estimated effort required | |
|---|---|---|---|
| 1 | Fork libradosStriper | Fork libradosStriper and compile it outside of Ceph to see that we can continue to use data stored in Echo. | 1 Month |
| 2 | Investigate librados Sparse Reads | Implement Assess and implement sparse reading in the forked libradosStriper (https://github.com/ceph/ceph/blob/main/src/include/rados/librados.hpp#L1060-L1062 ) for efficient vector read operation. | 2 - 3 Months |
| 3 | Implement a many_aio_read in libradosstriper | Implement in forked libradosstriper a method to perform multi aio_reads in one call (see A. Peters: 26 Mar 21) for more efficient vector read operations (either/or with “Investigate librados Sparse Reads” project) | |
| 4 | Understand how deletes work | The CERN FTS team measured the performance of deletes (https://codimd.web.cern.ch/UDL3fgqWT1a2HJC7LkWXNg# ) and don’t believe this is sufficient performance for expected data rates. |