Attending: Elena, Gareth, Jens, John B, Matt, Rob, Steve, Tom, Wahid, Jeremy, Adam, Sam, Govind, Duncan, Chris W, David, Matthew Viljoen (from STFC, previously led the CASTOR team), Ewan. 'Pols: Brian's on leave. 0. Operational blog posts? We don't think there are any Shellshock type vulnerabilities in our stuff. But of course people should patch their systems anyway. DPM 1.8.9 in EPEL-Test. Wahid upgrading on the testbed. YAIM is now no longer officially supported, people will need to use puppet which will be based on modules, but improvements to the installation process are still expected (but no further changes to DPM itself are expected, so once it's installed, you should actually have a DPM 1.8.9.) xroot4 being tested at CERN: xroot3 and 4 cannot coexist? Duncan has an xroot4 client on a WN, and is able to download files from CERN. Chris has a server in test, currently with ATLAS stuff in it. Why not support multiple VOs on xroot, including everyone's favourite VO, dteam? Is the xroot server in DPM capable of supporting more than one VO, or would it expose both VOs' files for reading at least? (ATLAS is probably world readable for the selected parts of the world that has certificates, but it seems polite to not unduly expose VOs' data more than necessary.) The standalone xroot server ought to be able to support multiple VOs. Duncan thinks the dCache one can. Other than everybody's favourite VO, also T2K would be good to support on xroot. 1. It's the end of the quarter. Usual end of quarter stuff - things I should report but don't know about, and things to achieve for the coming quarter. We have six blog posts for the quarter which is kind of green-ish as a metric. Also need to report on usual meetings and papers submitted, etc. 2. Amsterdam was totally abuzz with activity yesterweek. Matthew Viljoen from STFC's scientific computing data services group joined to talk about as much as possible data-related. The RDA discussed cost models for data, using relative costs rather than absolute, to avoid exposing sensitive commercial arrangements, for example. Previously, other work had been done by Maciej Brzezniak from Poznan, and there has been cost models presented/discussed in HEPiX, possibly the latter as part of DPHEP, and the former as associated with EUDAT sustainability (maybe). Also PanData progressing (neutron and synchrotron sources, amongst which STFC is participating and also Diamond Light Source). They use Nexus which is a specialisation of HDF5. PanData moving into "federated data policies", and also using iCAT. https://code.google.com/p/icatproject/ http://www.nexusformat.org/ EGI had a theme of big data at least some of the time; Wednesday had the federated cloud meeting and there was also a joint big data meeting looking at ELIXIR and KM3NET (http://www.km3net.org/home.php). Matt, however, was only able to attend half the Friday, and witnessed the discussion of CDMI support for OpenStack (from GRNET) and CCP4 cloud, an ancient protein crystallography project working on web services access to data, syncing data between data repository or your desktop to the cloud. http://www.ccp4.ac.uk/ Apropos big data, there's also this - http://www.isc-events.com/bigdata14/ which is starting tomorrow and you may want to keep the ol' eyeball on it. 3. For DPM workshop, the throttling issue would be useful to raise: Sam is still experimenting with the cgroup IO but its effectiveness may depend on whether xroot is using kernel buffers and it looks like it isn't? Maybe using DPM with Lustre/GPFS/CEPH? Edinburgh using DPM with GPFS. There is in general a possible set of limitations or at least not getting full advantages of the underlying advantages as DPM is expected to talk to , so any underlying optimisations regarding blocks, parallel transfers, and whatnot are potentially lost. In general it may be worth looking at opportunities for object stores (again) and other stuff. Now is it worth doing something together for either CHEP or ISGC? Suggestions include: * How we're already doing cloud-or-latest-buzzword-thing and have been doing it for 10 years or more * How we're a jolly nice group of knowledgeable techies who work well together - more the sociology angle * How we support diverse communities - if we do - probably the small VO angle, or for HEP maybe focus on physics type VOs like CERN@School, NA62, Pheno. * How small VOs are supported and getting started in GridPP on DIRAC and CVMFS Of course it's also OK if people want to do their own stuff instead, or in addition. 4. AOB wahid: (01/10/2014 10:06:43) you can see the changes on the blog https://svnweb.cern.ch/trac/lcgdm/blog ok Jens Jensen: (10:19 AM) Like what we'd call small VO, or cloud? Amsterdam does that to people :-) So SSO is important to them Ewan Mac Mahon: (10:27 AM) I'm sure someone's told be about a storage system with a policy rule engine recently...... Samuel Cadellin Skipsey: (10:31 AM) Is this the crystallography CCP4, or something else? Jens Jensen: (10:31 AM) Yes Ewan Mac Mahon: (10:37 AM) There's an open question about IPv6 gridftp logging that Duncan's been looking at, but he's asked me to look at a thing on my test DPM and I haven't done it yet, but there might be something arising from that. wahid: (10:37 AM) https://indico.cern.ch/event/324705/timetable/#20141009 Ewan Mac Mahon: (10:37 AM) Or there might not if it's already fine, Ewwww! wahid: (10:39 AM) so Ewan you also have a IPV6 DPM testbed ? Ewan Mac Mahon: (10:40 AM) I have a little IPv6-only test DPM. Duncan Rand: (10:41 AM) The ipv6 mgridftp issue is probably solved https://ggus.eu/index.php?mode=ticket_info&ticket_id=106665 Ewan Mac Mahon: (10:45 AM) Same trick tht StoRM does, I think? (as opposed to Lustre) We'd need to actually support diverse research communities first. "The Grid, or How We've Been Doing Cloud This Entire Damn Time" wahid: (10:51 AM) maybe more an ISGC thing very hard to get talks at CHEP -...a poster would be fine Jens Jensen: (10:52 AM) blog posts! ... how to measure productivity :-) Ewan Mac Mahon: (10:52 AM) We need someone to do something like www.pegasus.lse.ac.uk again, but just on the storage group. Jens Jensen: (10:54 AM) It also works because people like you are contriubing Useful Things(tm) and ask quetions! Duncan Rand: (10:58 AM) Except that it is not storage related. Samuel Cadellin Skipsey: (11:00 AM) What isn't storage related, Duncan? Ewan Mac Mahon: (11:00 AM) Any of the stuff using dirac atm. Samuel Cadellin Skipsey: (11:00 AM) DIRAC does do storage. Ewan Mac Mahon: (11:00 AM) It's all jobs submission so far. I think, Samuel Cadellin Skipsey: (11:00 AM) My DIRAC stuff is definitely data :D