Present: Alessandra, Elena, Ewan, Jeremy, JohnB, JohnH, Matt, Rob, Robert, Sam, Steve, Wahid, Jens 0. BNL DPM transfer issues There is a fix for the current problem, and tips for how to tune. fasterdata is useful but there may be more to it required. Wahid will put it on our wiki page. 3. WLCG storage interfaces and benchmarking meeting(s); Any issues for wlcg ops mtg; WebDav etc... etc. Any more to do on the WebDAV stuff - I mean specific things we could designate as "deliverables" in the next quarter. Need to transfer multiple files. What is the next step? ATLAS wanted WebDAV, asking for all sites to provide this interface. Seemed like a lot of effort. could be a user friendly interface if we can figure out how to mount it properly. Issues with dpm implementation which "will be fixed". Need to deploy at all sites. Could work in Windows. Wou'ldn't workt with a proxy certificate? Some clients might get confused. Likewise the "small" VOs who might be interested in a no-fuss interface. Should make sure it's clear it is not a production service when it isn't. Could ask for the LFC @ RAL to allow redirects for WebDAV. Takes "a few minutes" - LFC at RAL for "small" VOs. ops meeting (tomorrow): also noted that no one objected to moving away from RFIO. Some things not yet working with xroot. Is GridFTP less efficient (than DPM's RFIO). Testing suggests xroot or https. If applications are all using ROOT, or are prestaging files onto the WN, then the transition should be easy. Log files should reveal this. CMS open files directly with RFIO but are keen to move away from this model. 1. Storage workshops - EGI CF, and a larger, "proper", workshop later this year. We have a sponsor for the latter! "Big data" for science, that kind of stuff. DDN have offered to sponser the data workshop in London. James Coomer used to be at Dell and turned up for the last Lustre group. Also George Jones who was previously with Dell. Lustre is interesting Should we get them to talk in this meeting or are we already talking enough to them. DDN are involved with DIRAC as well. They are also interested in iRODS. We tend to buy cheaper stuff, though. DDN may be a way to the industry. "Big data" - would industry get involved except to sell us stuff? 2. Puppet revisited (from last week) - recipes, upcoming puppet camp. If the CASTOR team is going we could give them stuff to mention or stuff to report back. 4. Usual reminder about blog posts!! 5. GridPP meeting - Neasan and Dave Colling putting it together; first day on clouds, second day on impact. Cloud discussions. 6. BTW did the negative space publishing get solved. Some random person submitted a fix, but it worked. The patch has been submitted through the proper channels. If anybody is interested in GLUE2 (for storage), then we can discuss it at some point. Nobody else is interested, though. 7 Which reminds me I didn't get notes written up for last week. I still have them somewhere. 8 AOB [09:57:26] Wahid Bhimji morning [09:58:07] Elena Korolkova Morning [10:04:10] Alessandra Forti hello [10:08:24] John Bland just because HUGE works, it doesn't mean something a bit less huge wouldn't also work [10:10:52] Ewan Mac Mahon I think the mechanics of applying the settings is easy enough onceyou know what settings to set. [10:11:27] Alessandra Forti @John: matter of testing [10:11:30] Ewan Mac Mahon I think our current recommendation is going to be essnetially fasterdata+the 'liverpool' default window sizes. [10:12:11] Wahid Bhimji https://indico.cern.ch/conferenceDisplay.py?confId=233970 [10:12:15] Alessandra Forti For example I haven't seen any differencce between 8/16/33MB in the max [10:13:10] Alessandra Forti at least when there are parallel transfers [10:13:43] Alessandra Forti with single 9stream transfers it seemed to go faster but I have to do something more systematic than what I've done the other night [10:14:26] Ewan Mac Mahon @Alessandra - thanks for the test results for Oxford, btw - I'm in the process of waiting for cfengine to apply the new settings now, I'll let you know once it's sorted itself out and we're all set for the 'after' test. [10:17:35] Wahid Bhimji https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/WebDAV/ClientTutorial [10:18:02] Wahid Bhimji if these clients (or others) don't work in some way -(Chris et al.) then we should / could mention it to the DPM guys [10:21:14] Christopher Walker https://www.gridpp.ac.uk/wiki/WebDAV [10:21:31] Christopher Walker Has site status and some links [10:22:27] Christopher Walker Modifications welcome - and if sites can update their status it would be good. [10:23:37] Alessandra Forti there is a gridsite htcp [10:23:48] Alessandra Forti it is built around curl [10:24:02] Alessandra Forti but it doesn't work with dpm [10:24:58] John Bland note that on sl6 you have to force it to use libcurl-openssl, otherwise it doesn't work with our x509 certs [10:25:21] Alessandra Forti on the client side? [10:25:24] Wahid Bhimji aha john I noticed that - how to you do that forcing ? [10:25:28] John Bland yes [10:25:37] John Bland wahid: LD_LIBRARY_PATH [10:25:44] Christopher Walker Can you put a link on the webdav page please Alessandra [10:25:47] Alessandra Forti I was still on a sl5 UI [10:26:09] Alessandra Forti I think it got stuck in the redirection phase [10:28:15] Alessandra Forti maybe we should let ricardo know about it [10:28:19] Wahid Bhimji its Wed afternoon BTW the storage / EMI session for those who want to come. [10:28:45] Alessandra Forti /it/htcp/ [10:29:12] Wahid Bhimji what is the point of htcp if you can just use curl [10:29:28] Wahid Bhimji what does it add. Curl works (though thanks john for clairfiying the sl6 solution) [10:30:18] Alessandra Forti it's a wrapper arond curl that makes it easier to handle the transfers [10:30:30] Alessandra Forti you don't have a long command line to use [10:30:37] Alessandra Forti it automatically looks at the proxy [10:30:50] Alessandra Forti it has htrm/htls/ht.. commands too [10:31:13] Alessandra Forti htcp htll htmkdir htmv htping htfind htls htmlview htop htrm [10:31:31] Wahid Bhimji ok -sounds good - I will try - but yes I think Ricardo should know if it doesn't work . [10:31:31] Alessandra Forti of course you can use curl though [10:33:18] Steve Jones Ewan, it sounds like your transmitting from Mercury. [10:33:26] Wahid Bhimji is he a good talker ? [10:33:26] Ewan Mac Mahon Oh. [10:34:00] Steve Jones s/your/you're/ [10:34:38] Ewan Mac Mahon As in I'm just quiet, or something else? [10:35:50] Wahid Bhimji might be worth getting talks from people doing cheap resiliant storage methods (e.g. the hdfs approach ) [10:36:23] Ewan Mac Mahon Just to be clear, I think getting DDN in on the 'big storage' meeting is a really good idea, getting them in on specifically gridpp stuff, not quite so much. [10:36:32] Steve Jones All distorted, like a fuzz box. You're OK now. [10:37:47] Ewan Mac Mahon We should definitely get people like EBI/Sanger involved. [10:38:44] Ewan Mac Mahon I think the trouble with 'industry' is that they don't tend to worry so much about cross boundary interoperability stuff - one company buys one big thing and their done. [10:38:52] Sam Skipsey Indeed. [10:39:09] Wahid Bhimji great - I think its perfect [10:39:17] Ewan Mac Mahon That's becoming less true with 'cloud', but it's still common. [10:39:19] Sam Skipsey Plus, the one company determines their architecture and resources, and spends the money on what they need / can afford. [10:39:29] Ewan Mac Mahon But overall, this meeting sounds like a very, very good thing. [10:39:39] Sam Skipsey We tend not to (we get told how much we have to buy, and are given a budget limit additionally) [10:39:54] Ewan Mac Mahon We should think about Whamcloud/Alces as well. [10:43:37] Wahid Bhimji In the cloud meeting I got an action for a more concrete Cloud Storage plan - I will circulate something to the storage list for feedback [10:44:20] Wahid Bhimji I'm NOT interested