nonstandard stripe size read

Standard test:

echo "generating file"
head -n 500000 /dev/urandom > t6
echo "ROOT xrd"
c1=$(md5sum < ./t6)
xrdfs root://${1}:1094 rm dteam:test450.txt
rm -rf t7
echo "write/read test start"
xrdcp -d1 ./t6 root://${1}:1094/dteam:test450.txt
xrdcp -d1 root://${1}:1094/dteam:test450.txt ./t7
c2=$(md5sum < ./t7)
echo "checksum comparison"
echo $c1
echo $c2
echo "stat, removal and confirm"
xrdfs root://${1}:1094 stat dteam:test450.txt
xrdfs root://${1}:1094 rm dteam:test450.txt
xrdfs root://${1}:1094 stat dteam:test450.txt
rm -rf ./t7
echo "ROOT gfal"
c1=$(md5sum < ./t6)
echo "write/read test start"
gfal-copy ./t6 root://${1}:1094/dteam:test450.txt
gfal-copy root://${1}:1094/dteam:test450.txt ./t7
c2=$(md5sum < ./t7)
echo "checksum comparison"
echo $c1
echo $c2
echo "stat, removal and confirm"
gfal-stat root://${1}:1094/dteam:test450.txt
gfal-rm root://${1}:1094/dteam:test450.txt
gfal-stat root://${1}:1094/dteam:test450.txt
rm -rf ./t7

echo "DAVS gfal"
c1=$(md5sum < ./t6)
echo "write/read test start"
gfal-copy ./t6 davs://${1}:1094/dteam:test450.txt
gfal-copy davs://${1}:1094/dteam:test450.txt ./t7
c2=$(md5sum < ./t7)
echo "checksum comparison"
echo $c1
echo $c2
echo "stat, removal and confirm"
gfal-stat davs://${1}:1094/dteam:test450.txt
gfal-rm davs://${1}:1094/dteam:test450.txt
gfal-stat davs://${1}:1094/dteam:test450.txt
rm -rf ./t7

echo "TPC transfer"
xrdcp -v -f --tpc delegate only root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/test450_2.txt root://${1}:1094//dteam:test450_2.txt
xrdcp -v -f --tpc delegate only root://${1}:1094//dteam:test450_2.txt root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/test450_2.txt
echo "stat and confirm"
xrdfs root://${1}:1094 stat dteam:test450_2.txt
xrdfs root://golias100.farm.particle.cz:1094 stat /dpm/farm.particle.cz/home/dteam/test/test450_2.txt

echo "multistream"
xrdcp -d1 -S 2 ./t6 root://${1}:1094/dteam:test450.txt
xrdcp -d1 -S 2 root://${1}:1094/dteam:test450.txt ./t7
c2=$(md5sum < ./t7)
echo "checksum comparison"
echo $c1
echo $c2
gfal-rm davs://${1}:1094/dteam:test450.txt
rm -rf ./t7

 

Test results

-bash-4.2$ xrdcp t6 root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/test450_2.txt --force
[122.1MB/122.1MB][100%][==================================================][61.03MB/s]
-bash-4.2$ bash fntest.sh ceph-svc16.gridpp.rl.ac.uk
generating file
ROOT xrd
rm dteam:test450.txt : [SUCCESS]
write/read test start
[122MB/122MB][100%][==================================================][40.67MB/s]
[122MB/122MB][100%][==================================================][122MB/s]
checksum comparison
e85de1fa42d58f8001e1839050c84bf4 -
e85de1fa42d58f8001e1839050c84bf4 -
stat, removal and confirm

Path: dteam:test450.txt
Id: 4294967297
Size: 127930685
MTime: 2023-12-06 09:46:55
CTime: 2023-12-06 09:46:55
ATime: 2023-12-06 09:46:55
Flags: 48 (IsReadable|IsWritable)
Mode: 0666
Owner: root
Group: root
rm dteam:test450.txt : [SUCCESS]

[ERROR] Server responded with an error: [3011] Unable to locate dteam:test450.txt; no such file or directory

ROOT gfal
write/read test start
Copying file:///home/tier1/fur43467/t6 [DONE] after 2s
Copying root://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt [DONE] after 1s
checksum comparison
e85de1fa42d58f8001e1839050c84bf4 -
e85de1fa42d58f8001e1839050c84bf4 -
stat, removal and confirm
File: 'root://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt'
Size: 127930685 regular file
Access: (0600/-rw-------) Uid: 33504 Gid: 1182
Access: 2023-12-06 09:47:00.000000
Modify: 2023-12-06 09:47:00.000000
Change: 2023-12-06 09:47:00.000000
root://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt DELETED
gfal-stat error: 2 (No such file or directory) - Failed to stat file (No such file or directory)
DAVS gfal
write/read test start
Copying file:///home/tier1/fur43467/t6 [DONE] after 2s
Copying davs://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt [DONE] after 1s
checksum comparison
e85de1fa42d58f8001e1839050c84bf4 -
e85de1fa42d58f8001e1839050c84bf4 -
stat, removal and confirm
File: 'davs://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt'
Size: 127930685 regular file
Access: (0777/-rwxrwxrwx) Uid: 0 Gid: 0
Access: 1970-01-01 01:00:00.000000
Modify: 2023-12-06 09:47:05.000000
Change: 1970-01-01 01:00:00.000000
davs://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt DELETED
gfal-stat error: 2 (No such file or directory) - Result HTTP 404 : File not found after 1 attempts
TPC transfer
[122.1MB/122.1MB][100%][==================================================][15.26MB/s]
[122.1MB/122.1MB][100%][==================================================][20.34MB/s]
stat and confirm

Path: dteam:test450_2.txt
Id: 4294967297
Size: 127983363
MTime: 2023-12-06 09:47:16
CTime: 2023-12-06 09:47:16
ATime: 2023-12-06 09:47:16
Flags: 48 (IsReadable|IsWritable)
Mode: 0666
Owner: root
Group: root

Path: /dpm/farm.particle.cz/home/dteam/test/test450_2.txt
Id: 0
Size: 127983363
MTime: 2023-12-06 09:47:23
Flags: 48 (IsReadable|IsWritable)
multistream
[2023-12-06 09:47:23.478011 +0000][Error ][XRootDTransport ] [ceph-svc16.gridpp.rl.ac.uk:1094.1] kXR_bind request failed
[2023-12-06 09:47:23.478074 +0000][Error ][AsyncSock ] [ceph-svc16.gridpp.rl.ac.uk:1094.1] Socket error while handshaking: [FATAL] Hand shake failed
[2023-12-06 09:47:23.478216 +0000][Error ][XRootDTransport ] [ceph-svc16.gridpp.rl.ac.uk:1094.2] kXR_bind request failed
[2023-12-06 09:47:23.478245 +0000][Error ][AsyncSock ] [ceph-svc16.gridpp.rl.ac.uk:1094.2] Socket error while handshaking: [FATAL] Hand shake failed
[2023-12-06 09:47:23.482350 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.482386 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[8MB/122MB][ 6%][===> ][8MB/s] [2023-12-06 09:47:23.490648 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.490670 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.498347 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.498392 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.504954 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.505512 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.514908 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.514950 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.527679 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.527722 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.797978 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.798034 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:23.818048 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:23.818093 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[64MB/122MB][ 52%][==========================> ][64MB/s] [2023-12-06 09:47:24.031338 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.031443 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.049100 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.049145 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.359544 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.359593 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.373921 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.374071 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.568717 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.568803 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.583585 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.583626 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.798303 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.798455 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[2023-12-06 09:47:24.812400 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:24.812440 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[122MB/122MB][100%][==================================================][61MB/s] [2023-12-06 09:47:25.067251 +0000][Error ][Utility ] The given URL is empty
[2023-12-06 09:47:25.067282 +0000][Error ][PostMaster ] Unable to get transport handler for protocol
[122MB/122MB][100%][==================================================][61MB/s]
[2023-12-06 09:47:25.633699 +0000][Error ][XRootDTransport ] [ceph-svc16.gridpp.rl.ac.uk:1094.2] kXR_bind request failed
[2023-12-06 09:47:25.633760 +0000][Error ][AsyncSock ] [ceph-svc16.gridpp.rl.ac.uk:1094.2] Socket error while handshaking: [FATAL] Hand shake failed
[2023-12-06 09:47:25.633964 +0000][Error ][XRootDTransport ] [ceph-svc16.gridpp.rl.ac.uk:1094.1] kXR_bind request failed
[2023-12-06 09:47:25.633995 +0000][Error ][AsyncSock ] [ceph-svc16.gridpp.rl.ac.uk:1094.1] Socket error while handshaking: [FATAL] Hand shake failed
[122MB/122MB][100%][==================================================][122MB/s]
checksum comparison
e85de1fa42d58f8001e1839050c84bf4 -
e85de1fa42d58f8001e1839050c84bf4 -
davs://ceph-svc16.gridpp.rl.ac.uk:1094/dteam:test450.txt DELETED

 

Non-standard file size read

[arogovsk@lxplus785 arogovsk]$ gfal-copy root://ceph-gw1.gridpp.rl.ac.uk:1094/lhcb:accounting/lhcb/dump_20231120 . Copying root://ceph-gw1.gridpp.rl.ac.uk:1094/lhcb:accounting/lhcb/dump_20231120 [DONE] after 11s [arogovsk@lxplus785