XRootD Vector Reads
Setup
Python script: test_edi_xrootd.py
Bash script: runJobJo.sh, wrapper for python script
#!/bin/bash
#export XrdSecGSISRVNAMES=$(hostname)
SERVER=ceph-gw8.gridpp.rl.ac.uk
paths=""
for i in {1..150}
do
paths="$paths root://${SERVER}/dteam:test2TBRandom/${i}.txt"
done
source /cvmfs/sft.cern.ch/lcg/views/LCG_101_ATLAS_11/x86_64-centos7-gcc11-opt/setup.sh
export XrdSecPROTOCOL=gsi,unix; export XRD_LOGLEVEL=Dump; export XRD_LOGMASK=All;
chmod +x ./test_edi_xrootd.py
./test_edi_xrootd.py $paths 2> stderr_other${1}
exit_code=$?
bzip2 --best stderr_other${1}
arc job file: jobSubmitJo.rsl
&( executable = "runJobJo.sh" )
( stdout = "stdout" )
( stderr = "stderr" )
( inputfiles = ( "runJobJo.sh" "" ) ( "test_edi_xrootd.py" "" ) )
( outputfiles = ( "stdout" "" ) ( "stderr" "" ) ( "stderr_other.bz2" "" ) )
( queue = "EL7" )
( wallTime="48 h" )
( jobname = "Custom Python job Jo" )
( memory = 8192 )
Data: 150 copies of a 17GB urandom file in dteam echo space
Test outputs:
Stdout, out.txt - raw outputs
Stderr_other – log dump
*success – success rate (divide by 1000), number of vector reads
*num – time taken for reads
Plot.png - histogram of time taken, in intervals of 0.2 seconds
*stats – summary statistics
Ground test:
From lcgui05, ran as bash script and as an arctool job, with SERVER set to root://xrootd.echo.stfc.ac.uk
Results:
70% vector read failure rate. Fails with socket timeout