Attending: John B, John H, Sam, Steve, Wahid, Matt, Robert, Ewan, Chris, Gareth, Jens, Pete, Brian Apologies: Elena 1. Notes from the workshop - feedback was that it was useful with lots of interesting things. There will now be a writeup, where the notes are merged and summarised and all that, and perhaps with some sort of diagram thingy which illustrates the issues or something. However, there may not be that much common between the user communities. Most seem relatively happy with what they have already. Maybe climate seem to be among the ones more likely to try new stuff? How are we moving forward? We need to keep the momentum, and it was suggested to form a SIG. There are also a few action items. It would be useful to have examples of how to do stuff. GO is very popular, particularly among users who have never used it - they have great PR... GO is certainly useful for some types of use but for the bigger volumes there may be advantages to using FTS. We could investigate providing FTS services for big data. FTS also has a REST API. Maybe a nice front end is a useful exercise for a student? 2. Cloud storage next week - probably more a case of "we haven't talked about cloud for a while" rather than "there is a need to talk about something that cloud will solve." Cloud storage is probably not relevant in this context (and is also possibly harder to do?) S3 and local-file-access (and the plethora of other protocols) support different types of file access. CERN investigated accessing files via S3, but regarding whether it works and whether it performs well enough, we'd expect something else like NFS would be more appropriate. Or xroot if you are using ROOT. For an IaaS cloud, users have root - so there may be security issues with locally accessible distributed file systems (eg Lustre). Otherwise they lose the benefit of having data in Lustre (locally). GridFTP is also kind-of-a-cloud service in the sense that we can relatively quickly set up a GridFTP endpoint and have users use them - as long as they are small, such as <100TB. There will be an evaluation of the OpenSTack swift setup at IC. 3. How is the DPM collaboration working? There is a slot for it at the GDB next week. It's still early days, but it is not not working. It is expected to work - CERN are still doing lots, and non-CERN people are active on the mailing list. Regarding this, should we now worry about the support for LHC? Larger experiments have moved to using their own catalogues (although ATLAS are also still using LHC). Smaller VOs also sometimes have their own catalogues - but still depend on LHC, and we are not aware of an alternative. We probably need the same kind of planning which we had done for DPM - understand and manage the risks. 4. and 5. were postponed till next week. 6. NOB [10:05:49] Ewan Mac Mahon I think we need to pick out a few easier and smaller things to follow up on, like pitching FTS to people who are already interested in GlobusOnline, for example. [10:06:03] Ewan Mac Mahon Or just gridftp to people using anything else. [10:08:42] Ewan Mac Mahon I think the two may be complimentary. [10:08:49] Ewan Mac Mahon GO for one guy grabbing some files, [10:09:00] Ewan Mac Mahon FTS for routine bulk transfers run by IT support. [10:09:27] Ewan Mac Mahon And in FTS 3 they both run over the same gridftp endpoints. [10:10:29] Jens Jensen The diracs differ in how they are capitalised? Like "DiRAC" vs "Dirac" vs..? [10:11:22] Ewan Mac Mahon I was quite stuck by how much we don't seem to be exploiting the RAL potential - [10:11:42] Ewan Mac Mahon we've got all these people stuck in the middle of nowhere, we should make them talk to each other. [10:11:59] Ewan Mac Mahon We should send the T1 guys on a site wide roadshow. [10:12:12] Ewan Mac Mahon They've sort-of got a captive audience. [10:12:16] Wahid Bhimji chuckle [10:12:25] Ewan Mac Mahon Or Jens, of course. [10:12:27] Ewan Mac Mahon [10:12:32] Ewan Mac Mahon Action item! [10:13:18] Wahid Bhimji rest interface - so should be easy [10:15:24] Jens Jensen http://indico.cern.ch/conferenceDisplay.py?confId=253858 [10:15:39] Ewan Mac Mahon Actually, I've got a better idea for the name - the Wide Area Large Link Organized Mover [10:15:51] Ewan Mac Mahon "I'm sorry, WALLOM's fallen over again....." [10:15:57] Ewan Mac Mahon etc. [10:16:04] Jens Jensen [10:18:55] Wahid Bhimji yes - still do - its on the todo list - slowly being done [10:19:41] Ewan Mac Mahon NFS Chris. [10:20:07] Ewan Mac Mahon It actually shouldn't be too awful if you have the guests mount from each host. [10:20:21] Ewan Mac Mahon As an interim solution before the lustre support comes along. [10:20:29] Ewan Mac Mahon Which was planned, I thought someone said. [10:21:22] Wahid Bhimji xrootd [10:21:25] Wahid Bhimji exactly [10:22:32] Ewan Mac Mahon And really, I'm not kidding about NFS. NFS could really work, and you can certainly re-export lustre over NFS. [10:23:07] Ewan Mac Mahon And indeed, all_squash them. [10:23:23] Ewan Mac Mahon Which is more like you'd want for the guest VM case. [10:26:46] Ewan Mac Mahon Well yes, but that's not S3 then, is it? [10:26:53] Ewan Mac Mahon S3 is whole object get and put. [10:27:23] Jens Jensen Yes indeed. They also said to get to exascale we must move "beyond posix" (at the workshop) [10:30:50] Ewan Mac Mahon Also, does anyone think we should just chuck this Phyics lark in and start selling boxes with DPM and FTS servers as 'worldwide scalable object stores'? There seems to be money in it, judging from what DDN are doing. [10:31:23] Wahid Bhimji haha -indeed - stick DPM on a box and call it WoDPM [10:32:12] Wahid Bhimji I'm up for it . gridpp spin-off [10:32:32] Wahid Bhimji We are in the collaboration [10:32:41] Wahid Bhimji its only been going for 2 months [10:34:51] Wahid Bhimji that would have been my advice anyway to be honest [10:38:14] Wahid Bhimji or evaluate whether we as a community can support it ... given DPM will be supported [10:38:49] Wahid Bhimji Yeah - worth raising - if Chris could make a list of who needs it - lets go from there [10:40:07] Ewan Mac Mahon Not sure you missed much. [10:40:11] Wahid Bhimji is that within irods [10:40:14] Pete Gronbech Is it only small VOs now that have any use for the LFC? [10:40:19] Ewan Mac Mahon EUdat seems to be a massive wheel reinvention excercise. [10:40:51] Ewan Mac Mahon Mostly predicated on the assumption that Particle Physicists are so much smarter than everyone else that normal mortals cannot possibly use our tools. [10:41:03] Wahid Bhimji can we get a talk on irods at one of these meetings [10:41:03] Jens Jensen [10:41:31] Jens Jensen Yes we will have iRODS on the agenda... [10:41:45] Jens Jensen we do today, but there isn't that much time [10:41:59] Matt Doidge Replace normal with sane and I think Ewan is onto something [10:42:40] Ewan Mac Mahon I do think iRODS is interesting; particularly if we can draw the SRB/iRODS world and the SRM worlds a bit closer together. [10:42:52] Ewan Mac Mahon Can we FTS things to/from an iRODS server? [10:43:10] Sam Skipsey If it speaks GridFTP, presumably?? [10:43:17] Wahid Bhimji yes - I might not be here next week ... [10:43:19] Jens Jensen THere is a gridftp interface [10:43:26] Ewan Mac Mahon Well, maybe. Answers for next week, anyway. [10:43:58] Jens Jensen didn't hear that? [10:43:58] Pete Gronbech sorry no headset