GINSC2007-002: SRB/SRM island interoperabilities

@ Booth: EGEE - Enabling Grids for e-Science (Booth #2523)

A collaboration is sought with a group that has a Java-based version of the SRM interface. We will then seek a port on top of the Java interface to the SRB, JARGON. Such a port will also provide an SRM interface to the iRODS rule-oriented data system.

Update requested on: 2007-10-29 (Alex, Jens) by Morris via Email

Response by Alex: Update planned for this week (2007-10-29)

Contact: Alex Sim, Jens Jensen, Mike Wan

SRM and SRB interoperability - Part I.

Work done by Jens Jensen (coordination), Roger Downing (SRB admin), Derek Ross (SRM admin and BDII), and Matt Hodges (FTS), all from the STFC's Rutherford Appleton and Daresbury laboratories (UK).

Aiming for "low hanging fruit" in this demo, we wish to be able to transfer files between SRM and SRB. There are three related ways to accomplish this:

1. Using a GridFTP client to transfer data from one to the other;
2. Using SRM level data transfer tools such as FTS from gLite;
3. Using higher levels of data replication tools like lcg-utils, also from gLite.

They have one thing in common: SRB must be endowed with a GridFTP interface which is able to accept certificates and map them into the relevant account.

Looking at 1, this works (only) because many SRMs can accept data transfers via the GridFTP interface without first contacting the SRM itself. For dCache, a file can be fetched via a dCache door; for DPM and CASTOR it will also generally work because it is roughly equivalent to accessing the file locally via RFIO. For StoRM it should work (but is untested). The drawback is that you need to know the pool node because you are not asking the SRM for a TURL.

So 1 isn't really a viable option because usually you don't know the pool node in advance. So we need to use SRM tools. The trick here is to pretend that SRB is a Classic SE. Recall that a Classic SE is essentially a GridFTP server with an information publisher. The other half of the trick is that FTS still supports Classic SEs. So it talks to the SRM using the SRM protocol, and "talks" to the SRB using GridFTP, and GridFTP is used to transfer data between them. This is easy, because FTS doesn't depend directly on the information system, the channel just has to be configured to access the right path in SRB - and SRB's GridFTP server has to be configured to map the person/credential doing the transfer into the same account.

Going further, we can publish the Classic SE in the information system. This is not terribly hard, as long as users are all mapped into one single account on SRB (at least one per VO) - because the right path has to be published - but this usage is typical for many of our SRB customers. We found here that it doesn't matter which port number we publish for the GridFTP server: lcg-utils will always attempt to use 2811 (the default port). This is approximately where we are stuck at the moment because most of our SRB GridFTP servers run on non-standard ports.

So in summary, this demo shall aim to demonstrate 2: FTS transferring files between an SRM and an SRB, using GridFTP as the file transfer protocol.

Item 3 has now also been resolved, by running the GridFTP server on port 2811. The lcg commands ignore the SARoot (you'll be happy to learn) and use the SAPath as expected. lcg-cr, lcg-cp, and lcg-del all work, interacting perfectly with the SRB as if it were an SRM!