Brian, Jens, Henry, John B, Matt, Winnie, John H, Steve, Daniel, Tom, Marcus, Raul, Pete, Elena, Govind Apologies: Sam 0. Operational blog posts Issues reported by MICE: problems recovering "large" files (problem arising somewhere between 1.2 and 5.7 GB if there is a threshold); previously uploaded large files cannot now be downloaded, whether via lcg-utils (old UI) or web UI (which uses http). A related problem is the upgrade - since the repository has changed, package names have also changed, which means they cannot upgrade in place but will probably need uninstalling and the new packages installed. Could be some compatibility between DPM and the old client? Worth doing some more testing with other tools, or as Steve suggests try a different UI and see if it makes a difference, or try other protocols (although the above seem to have been for both http and GridFTP...?!) Brunel have DPM 1.8.11, also give that a go specifically. 1. There is a GDB today (= Wednesday): http://indico.cern.ch/event/394784/ As you can see Oliver has a slot on the proposed data workshop in September. Also accounting and information systems may be tangentially of interest to this group. Given that August is likely (hopefully?) to be quiet (see also item 3), now is not too soon to start thinking about making a good GridPP (re)presentation at the September pre-GDB. Suggested topics include data caching, future evolution of a T2, accounting, and information systems(??) 2. The long awaited round-table update, to give everyone around the, er, table a chance to say what they are working on at the moment (storage-and-data-management-wise) (we were supposed to have done an update in May...) Daniel - benchmarking HP Apollo, 24 disks in 2U, iozone, optimising Elena - "nothing interesting" (which is also a good thing...), working on http ticket. John B - "nothing exciting" (also a good thing), YAIM->puppet John H - generally stable; head node lost db table Marcus - enabling other VOs, LSST database. also contiuing ZFS testing with xple VOs, xrootd, setting up testbed Matt - "boring", upgrading SL5. Draining. Annoyingly saw no performance increase in database during upgrade. also talked to T2K about storing data online outside of RAL (ie @Lancs) - Brian points out these are ~100TBs Winnie - ticketed, GridFTP fw on servers - do blocks that stop evildoers also block legitimate tests? Also HDFS problem with Luke on hols; files placed in /tmp fill up /, only seen on newest WNs. GGUS ticket is 122713, from 8 June. Pete - stable Raul - MySQL->MariaDB upgrade. Procurement - infiniband vs omnipath. Maybe we should have a networking technology discussion at some point Steve - same as John Tom - nothing to report Brian - comparing avg # input files per job, CMS 8 vs ATLAs 16, for direct IO. Also dashboard, failed xfer when src file missing or corrupt; ATLAS dashb more informative than FTS's. Also looking at input and deletions; ATLAS del'd 600TB over past ~month. Jens - at Daresbury open day which had LHC exhibit; also data placement on tape/archive. 3. If ops are moving to fortnightly over the summer, should we, too? There was a general consensus that it was a good idea, or at least not a bad one. Here's the proposed schedule: 20 Jul -> 27 Jul -> Cancelled 03 Aug -> 10 Aug -> Cancelled 17 Aug -> 24 Aug -> Cancelled 31 Aug -> 4. AOB NOB Chat Log (thanks, Matt!) Marcus Ebert: (13/07/2016 10:05) 1.8.10 should be the latest production version. 1.8.11 I have installed for the MW readiness test right now John Hill: (10:06 AM) I think 1.8.11 is latest production version - I got it from epel Marcus Ebert: (10:07 AM) interesting. I was asked by Andrea Manzi to have 1.8.11 installed for MW tests John Bland: (10:09 AM) 1.8.11 was announced as production on 1 June. Centos 7 needs testing though. raul: (10:09 AM) I have DPM 1.8.11 on head node and all servers of Brunel's grid storage John Hill: (10:09 AM) Thanks John - I was checking my emails for the announcement! Marcus Ebert: (10:10 AM) Thanks! I missed that announcement. John Bland: (10:10 AM) it's on their twitter (not that I use such things) Matt Doidge: (10:10 AM) Does an upgrade require a re-yaim/puppet? John Hill: (10:10 AM) For reference, I have 1.8.11 everywhere There are some puppet updates. Whether you *must* re-puppet I don't know John Bland: (10:12 AM) matt: 1.8.10 - 1.8.11, at least on the pool nodes needed a re-yaim for me, but I updated everything including puppet modules re-yaim=re-puppet raul: (10:13 AM) I upgrade 1.8.10-1.8.11 without yaaim and without puppet. I's a light update Matt Doidge: (10:14 AM) Thanks guys, I'll upgrade when I have a downtime in a few weeks. raul: (10:14 AM) and btw i don't like puppet. I use ansible foor everything else I had to remove emi-dpm-disk and them 'yum update' John Bland: (10:15 AM) with the new release style it's very hard to know which 'version' you're on. dpm, puppet modules, xrootd etc all get new versions out of sync. raul: (10:19 AM) yes, it's sensible.O major updates coming No major Matt Doidge: (10:31 AM) Bristol Ticket: https://ggus.eu/?mode=ticket_info&ticket_id=122713 Kashif might be the chap to talk to about this - I think these are the new argo servers. Daniel Peter Traynor: (10:33 AM) sussex? I know sussex were going to upgrade there infiniband network and were looking at edr(?) or omnipath. I can't remember what they chose raul: (10:35 AM) Great! Who's the admin? Daniel Peter Traynor: (10:36 AM) Jeremy maris j.maris@sussex.ac.uk Matt Doidge: (10:37 AM) From their hepsysman talk they're upgrading to "Omni -Path half bandwidth tree" this summer raul: (10:38 AM) Good! I'll send a message. Omnipath seems to be ominously good. I want to know if I am reading right. Tom Whyntie: (10:41 AM) Thanks, bye!