Compiling enough of Ceph to build the RADOS Striper

This requirement arises out of the development work to improve XRootD 's read access to ECHO by providing lock-less reads.

We are broadly following the direction suggested by Andreas-Joachim Peters by investigating removing read locks, at least

for the case of Vector Reads.

• THIS MAY BE OBSOLETE, TASK IS WITH JYOTHISH TO FIGURE THAT OUT

• Compiling enough of Ceph to build the RADOS Striper

  • Instructions for Ubuntu

  • Instructions for Scientific Linux

Instructions for Ubuntu

I created a VM in SCD OpenStack from the image ubuntu-focal-20.04-nogui. I specified a c3.large instance (8 CPUs, 16GB RAM, 80GB disk).

In the following instructions, commands run as an ordinary user are prefixed with '$', and commands which need root are prefixed with '#'.

Before starting run an update:

I cloned the Ceph source from Git:

I then checked out the revision we are currently using (as of March 2022, this is v14.2.15):

Next, I installed the suggested dependencies:

The install-deps.sh script didn't quite install everything needed...

The packages and options I supplied to satisfy the requirements were:

Edit the file do_cmake.sh to set "PYBUILD=3" just before the check for the PYBUILD version (line 39):

else
echo Unknown release
exit 1
fi
PYBUILD="3"
if [ "$PYBUILD" = "3" ] ; then
ARGS+=" -DWITH_PYTHON2=OFF -DWITH_PYTHON3=ON -DMGR_PYTHON_VERSION=3"
fi

Try the configuration builder:

If there are no errors, compile the required library:

cd build && make radosstriper

Instructions for Scientific Linux

One method of building Ceph under Scientific Linux requires the configuration scripts to believe that the operating system is "centos", and fixing some Python package names.

Prerequisites for this approach are:

1. An installation of Python 3.8 (e.g. building it from source), and

2. Providing a recent version of the GCC/C++ tools, e.g. version 8 or or newer. On ceph-test-gw691, we have DevToolSet 9 from the Software Collections Library; installation instructions are here.

Install git if not already available:

yum install -y git

Clone the Ceph source code from GitHub:

git clone GitHub - ceph/ceph: Ceph is a distributed object, block, and file storage platform

I then checked out the revision we are currently using (as of October 2021, this is v14.2.15):

Provide access to a recent version of the GCC/c++ development tools:

Enable Python 3.8 as the current version. You might need to build this from source; see the instructions for building Python and installing with "make altinstall" at https://computingforgeeks.com/how-to-install-python-3-on-centos/ ), and set the install directory as /usr/local/bin. Install libffi-devel before compiling python or you might run into a missing ctypes module error.

Edit the script that installs dependencies to set the OS ID as "centos" and prevent attempts to install the Centos version of the Software Collections Library (as we already have the DevToolSet from the Software Collections Library available):

git diff install-deps.sh
diff --git a/install-deps.sh b/install-deps.sh
index 0fccd5b..0b61f82 100755
--- a/install-deps.sh
+++ b/install-deps.sh
@@ -275,6 +275,7 @@ else
for_make_check=false
fi
source /etc/os-release
+ ID=centos
case "$ID" in
debian|ubuntu|devuan)
echo "Using apt-get to install dependencies"
@@ -349,7 +350,7 @@ else
$SUDO $yumdnf install -y python36-devel
case "$ARCH" in
x86_64)

• $SUDO $yumdnf -y install centos-release-scl

+ # $SUDO $yumdnf -y install centos-release-scl
dts_ver=8
;;
aarch64)

Fix the name of the Python scipy package in the Ceph spec file:

git diff ceph.spec.in
diff --git a/ceph.spec.in b/ceph.spec.in
index 8211ae7..7ddfa8a 100644
--- a/ceph.spec.in
+++ b/ceph.spec.in
@@ -296,7 +296,7 @@ BuildRequires: python%{_python_buildid}-pyOpenSSL
BuildRequires: python%{_python_buildid}-cherrypy
BuildRequires: python%{_python_buildid}-jwt
BuildRequires: python%{_python_buildid}-routes
-BuildRequires: python%{_python_buildid}-scipy
+BuildRequires: scipy
BuildRequires: python%{_python_buildid}-werkzeug
%endif
%if 0%{?suse_version}

Convince the CMake invocation script to use Python 3:

git diff do_cmake.sh
diff --git a/do_cmake.sh b/do_cmake.sh
index ab8f5c8..28e260c 100755
--- a/do_cmake.sh
+++ b/do_cmake.sh
@@ -35,7 +35,7 @@ else
echo Unknown release
exit 1
fi
-
+PYBUILD=3
if [ "$PYBUILD" = "3" ] ; then
ARGS+=" -DWITH_PYTHON2=OFF -DWITH_PYTHON3=ON -DMGR_PYTHON_VERSION=3"
fi

Install the dependencies:

You might have to install cython trough pip if it's not detected.

Run CMake, turning off the options we don't need for a Ceph gateway client:

Compile the RADOS Striper library:

This will create the following files in the lib directory:

Of these, we only need the shared object files; we don't need the static library files, those ending in ".a".

Notes:

• symlinking python3.8 to python3 in /usr/bin or usr/local/bin makes it easier to debug when unexpected things happen

• edit [python -version] - Cython to python3-Cython on ceph.spec.in if not detected