Attending: Jens (chair+mins), Brian, Sam, Steve, Winnie, John B, John H, Ewan, Raja, Daniel, Elena, Matt 0. Happy new operational blog year! Glasgow offline with power failure, now need to fix Everything(tm). PSUs died, most other things OK. LHCb restripping before Christmas, test staging @T1. Disk server issues. Brian wrote a blog post! Hooray! 1. Any storagey things for hepsysman next week? And volunteers to present them... Sam: site report @ WLCG early Feb. hepsysman will have tutorial on GANGA day before (ie Thu 14); considered Useful(tm) for "small" VOs, so would be useful for people working with "small" VOs to get some experience with, e.g. with GANGA to DIRAC. 2. GridPP as a "data infrastructure" - (which can include any, all, or none, of what we do for the non-LHC VOs) Following up from this, GridPP as a data infrastructure could be useful for folks who are not processing the same way HEP is (ie embarassingly parallel). However, not having a catalogue is a weakness (all the larger VOs have their own) and a strength; the former for not having something to offer people who don't have catalogues and the latter by being able to use existing catalogues if they have them already (e.g. LIGO). Also, as "the Grid" has been an "object store" long before the term became popular, perhaps this helps explain to users what they can expect because now the term is more widely understood [indeed, SRM1 had no directory functions...] We also have the DiRAC document which now needs updating; there will be information which is specific to DiRAC (scripts to run for backups) but probably most is generic (certificates, GridFTP) DFC is more generic. Provenance metadata in DFC? Feed into Tom's doc/recommendations for new VOs. 3. Future T2 revisited: Specifically, is there a role for CEPH and/or S3 interfaces in a non-negligible number of T2s? Alastair Dewhurst had previously reported on LOFAR finding S3 "easy to use compared to grid stuff" (but then S3 is not an open standard) Regarding CEPH, it does NOT seem like a future option for T2s. It is relatively unproven, and the Tier 1 team working on it has taken up to four people many months (eight?) to get it near production, so not likely to be an option for a T2, even building on the T1 experience. Another issue is the migration, where in the interim we would still need GridFTP and xroot interfaces, and data would need to be copied out of the current storage and into CEPH (and updated in catalogues, etc., e.g. if the filepath changes.) As regards S3, it is considered a more generic interface than GridFTP (say) or xroot but our stuff is not generic; it is optimised for large data volumes and high performance, as with the GridFTP parallel streams. S3 is not an open standard but is popular. And the status of testing at Oxford (with ATLAS^W CMS^W LHCb) http://storage.esc.rl.ac.uk/weekly/20151125-minutes.txt http://storage.esc.rl.ac.uk/weekly/20151209-minutes.txt Ewan is leaving his post at Oxford and joining some other part of the university to do other computeringy things. May focus on upgrading DPM - e.g. leaving it on CentOS or SL7 - to Kashif. Perfsonar likely to keep running but will drop the IPv6 work and monitoring the T2C testing. 4. It's the end of the quarter (ie the end of previous quarter). Could Marcus, Sam, and Brian please send me the Usual Stuff™ that is not in the minutes of the meetings and not in blog posts (if anything) 5. AOB Jens Jensen: (06/01/2016 09:57:57) Morning all. we will start at 10.00 Ewan Mac Mahon: (10:05 AM) I think there's supposed to be vidyo; there are links on the Indico pages. Brian Davies @RAL-LCG2: (10:05 AM) https://indico.cern.ch/event/465560/ Ewan Mac Mahon: (10:06 AM) Though in practice it's probably worth double checking with Alessandra that it's all set up. i think what we call the Dirac thing is a 'case study' 'The Original Object Store' I think we need to be very careful about claiming things are 'general' when they're more 'non-specific'. Raja Nandakumar: (10:19 AM) I understand that the DFC (Dirac File Catalog) is used by ARGUS and CTA at least Ewan Mac Mahon: (10:19 AM) /most people's/ use cases are a little different to everyone else's, so I would expect a lot of communities (and certainly established ones that already have their own thing) will need their own thing to understand their model. The layer at which the probably don't need their own thing is the basic object storage. Samuel Cadellin Skipsey: (10:20 AM) Sure, I mean, I agree, it's just worth noting that there's probably a core of useful stuff in Lydia's documentation, and then a bunch of stuff which is less generally useful Ewan Mac Mahon: (10:22 AM) FFS. I mean, really. Raja Nandakumar: (10:24 AM) Won't we need gridFTP for FTS? Ewan Mac Mahon: (10:24 AM) Strictly, no. FTS should be able to talk other things too. Samuel Cadellin Skipsey: (10:25 AM) FTS *can* talk to HTTP, apparently. Ewan Mac Mahon: (10:25 AM) But we have considered using the Tier 1's gridftp work at Tier 2s as well, and indeed the PMB did see a project proposal for a trial of that a while back. And, yeah. That went well. Raja Nandakumar: (10:30 AM) Apologies - got to go now Brian Davies @RAL-LCG2: (10:32 AM) puppetized? John Bland: (10:37 AM) are people intending to integrate DPM-puppet with site-puppet? I wasn't anyway Ewan Mac Mahon: (10:37 AM) I think we thought it would be the right thing to do in principle, then decided that it would be utterly horrific in practice. So probably not. As well as being busy, this falls into two areas - storage and networking - that have been very much my specialist areas and things that he's spent relatively little time on. He = Kashif. But short version, we can't do this at Oxford. Edinburgh? Samuel Cadellin Skipsey: (10:43 AM) Yeah, I need to talk to Marcus and Andy about that directly. Ewan Mac Mahon: (10:44 AM) There may be a point that we just can't do this at all, of course. John Bland: (10:48 AM) there's no requirement for ipv6 is there? Ewan Mac Mahon: (10:48 AM) There's not. Ideally someone also needs to pick up some of the IPv6 stuff as well, but it's a whole separate issue.