XRootD Vector Reads

Setup

Python script: test_edi_xrootd.py

Bash script: runJobJo.sh, wrapper for python script

#!/bin/bash 

#export XrdSecGSISRVNAMES=$(hostname) 

SERVER=ceph-gw8.gridpp.rl.ac.uk 

paths="" 

for i in {1..150} 

do 

  paths="$paths root://${SERVER}/dteam:test2TBRandom/${i}.txt" 

done 

source /cvmfs/sft.cern.ch/lcg/views/LCG_101_ATLAS_11/x86_64-centos7-gcc11-opt/setup.sh 

export XrdSecPROTOCOL=gsi,unix; export XRD_LOGLEVEL=Dump; export XRD_LOGMASK=All; 

chmod +x ./test_edi_xrootd.py 

./test_edi_xrootd.py $paths 2> stderr_other${1} 

exit_code=$? 

bzip2 --best stderr_other${1}

arc job file: jobSubmitJo.rsl

&( executable = "runJobJo.sh" ) 

( stdout = "stdout" ) 

( stderr = "stderr" ) 

( inputfiles = ( "runJobJo.sh" "" ) ( "test_edi_xrootd.py" "" ) ) 

( outputfiles = ( "stdout" "" ) ( "stderr" "" ) ( "stderr_other.bz2" "" ) ) 

( queue = "EL7" ) 

( wallTime="48 h" ) 

( jobname = "Custom Python job Jo" ) 

( memory = 8192 )

Data: 150 copies of a 17GB urandom file in dteam echo space

Test outputs:

Stdout, out.txt - raw outputs
Stderr_other – log dump

*success – success rate (divide by 1000), number of vector reads
*num – time taken for reads
Plot.png - histogram of time taken, in intervals of 0.2 seconds
*stats – summary statistics

Ground test:

From lcgui05, ran as bash script and as an arctool job, with SERVER set to root://xrootd.echo.stfc.ac.uk

Results:

70% vector read failure rate. Fails with socket timeout