If Globus Online just supported SRM then... Chris W went to a Cloud Show - IBM were showing off Atlassian project (which looks a lot like FTS but over UDP). GO is lovely for not scaring people, but does it have the features we want? If we moved away from SRM, would we be moving to gridFTP endpoints anyway (or xrootd/WebDAV)? ATLAS Problem of jobs transferring from UK sites to the US [ these are jobs from US Cloud running on UK T2s, and then staging back to USA w/ FTS]. "Fixed" by stopping such jobs going ot the sites (the problem was transfer speeds transatlantically). Ewan noted that this is kind of not a problem in that it's expected. [The problem was not simply with non-T2D sites - RHUL and Cambridge had an issue.] Brian: seemed to work with the FTS2 but not the FTS3 when some tests were done. Single file transfer monitoring page for ATLAS for large files between the particular sites and BNL, they are lower than for (say) Man->BNL, but not devastatingly so. Chris noted that it would be worth investigating number of threads used by gridftp in FTS across sites. Brian has also asked for bandwidth targets to be added as an option for FTS3 links. - discussion of nlinks=-1 problem Daniela/Govind noticed. This is an historically generated issue from old bugs now fixed in DPM. There is a tool to fix the database appropriately. - discussion of remote deletion, storage management over Davix. [Brian will test.] - "As T2 Site Admins, what is your plan for keeping disk servers in production?" (run them 'til they die, or you run out of rack space and need to remove some to make room) Discussion about if it is sensible for multiple sites to decommission disks at once. Tape archive availability at RAL for small VOs (as T2s are *not* set up for high-resiliency storage). - Chat Log 10:07:51] Rob Fay joined [10:07:51] John Bland joined [10:07:52] Steve Jones joined [10:08:28] Matt Doidge joined [10:08:31] Sam Skipsey Hello everyone. [10:08:38] Rob Fay Morning [10:08:49] Sam Skipsey Apologies for late start, some connectivity issues to start the day. [10:08:56] Elena Korolkova joined [10:09:00] Matt Doidge Good Moaning. [10:09:09] Christopher Walker joined [10:09:20] Elena Korolkova Morning [10:12:01] Ewan Mac Mahon joined [10:12:13] Gang Qin joined [10:12:46] Tom Whyntie joined 10:15:23] Ewan Mac Mahon I think part of the hope is that it might replace scp for the non-grid folks rather than replacing anything in the already-doing-it-right community. [10:15:30] John Bland does it support checksums? [10:17:45] Sam Skipsey https://support.globus.org/entries/23583857-Sign-Up-and-Transfer-Files-with-Globus-Online [10:18:39] Ewan Mac Mahon Possibly also worth noting this: http://www.lrz.de/services/compute/grid_en/software_en/gsisshterm_en/#globusonline [10:18:55] Ewan Mac Mahon A GO client built into the gsi-sshterm thing that I think everyone should be using. [10:23:50] Alessandra Forti joined [10:26:33] Alessandra Forti lxrootd/http [10:27:28] Ewan Mac Mahon WebDavOnline [10:27:31] Alessandra Forti FTS3 does that [10:27:34] Ewan Mac Mahon Or WebWebDav [10:28:42] Christopher Walker WebDAV should now work at QMUL [10:29:07] Ewan Mac Mahon Update on DPM 1.8.8? 10:30:04] Brian Davies joined [10:30:06] Sam Skipsey DPM 1.8.8 in Epel-test [10:30:21] Ewan Mac Mahon And likely to be in EPEL proper soon? [10:30:21] Sam Skipsey Wahid is testing it, and has found a few puppet config issues so far. [10:31:01] Matt Doidge Is 1.8.8 the one that does away with pool accounts on the disks servers? [10:31:06] Sam Skipsey Yes. [10:31:10] Ewan Mac Mahon Ah, right. I'm really hoping for a smoothly puppet-able thing, so that sounds worth holding off until Wahid's done. [10:31:13] Elena Korolkova And Liverpool is OK [10:31:40] Ewan Mac Mahon And the no pool accounts thing is a key part of being easily puppetable. [10:31:42] Elena Korolkova And Cambridge is T2D as well but there were problems [10:32:31] Matt Doidge I'm tempted to give it a go on some new servers. [10:33:08] Matt Doidge Are pool accounts still needed on the headnode? [10:33:19] Sam Skipsey No, shouldn't be. 10:33:26] Ewan Mac Mahon All my DPM servers are yaim-controlled SL5 at the moment, I'm hoping to do the new ones with puppet/SL6. [10:33:59] Ewan Mac Mahon I think there might be a couple of other less-than-polished bits in the puppet scripts for the head node still though, aren't there (something mysql related?) [10:34:23] Sam Skipsey The issue Wahid had seems to be with the mysql config in puppet. [10:34:37] Ewan Mac Mahon But pool servers should be fairly simple now. [10:34:43] Matt Doidge This could feasibly make argus integration easier for us - a configuration gaff a few years ago split the uid/gid paradigms for both clusters and the DPM. [10:34:59] Alessandra Forti I agree [10:35:23] Alessandra Forti also I suspect the files were small as this was group production but I should check that [10:37:07] Ewan Mac Mahon IIRC RHUL don't have a lot of bandwidth. [10:37:25] Alessandra Forti maybe they shouldn't be a T2D then [10:37:48] Ewan Mac Mahon Indeed. [10:38:32] Ewan Mac Mahon Presumably there are clear criteria for being a T2D though, and it may be that those need revision, and then everyone should be assessed to see if they meet the new ones. [10:39:18] Alessandra Forti there are but they are still based on single file transfers [10:39:33] Alessandra Forti and a site may look good even if 1GB [10:39:46] Alessandra Forti but when the site is loaded it doesn't work anymore [10:50:11] Christopher Walker System Message to user Christopher Walker: SeeVogh has detected significant packet loss in your incoming video stream. [10:50:12] Christopher Walker System Message to user Christopher Walker: SeeVogh has reduced the quality of your incoming video to optimize your performance. [10:51:25] Ewan Mac Mahon Who's stiring coffee with a headset, btw? [10:51:28] Steve Jones Run them until they are fit for the scrap yard. [10:51:37] Sam Skipsey As with Steve. [10:51:57] Steve Jones And canibalise them then, too. [10:52:05] Ewan Mac Mahon We're seriously considering decomissioning our 2007 generation Viglens, but not for a while yet. [10:52:09] John Bland well, until it's no longer economically viable to keep them running [10:52:10] Matt Doidge I have 6 year old servers that run without a peep. [:52:11] Ewan Mac Mahon Later this year, probably. [10:52:40] Ewan Mac Mahon They're a bit power hungry - ~350W for 8TB, cf 200W for 40TB on the new kit. [10:52:50] Ewan Mac Mahon And they're taking up rack space. [10:53:16] Matt Doidge That's the kicker for us - rack space. [10:53:59] Ewan Mac Mahon ATLAS need to start doing erasure coding across sites. [10:55:14] Ewan Mac Mahon Also, there's the notional cost of ditching the old kit - our 2007 gen adds up to about 80TB, which is two new disk servers, is about £10k. [10:55:47] Ewan Mac Mahon And having just bought fifteen, 'paying' two servers to decomission the old kit is not too horrible. [10:56:18] Ewan Mac Mahon That's a 2:13 ratio of 'decomissioning' to 'expansion' [10:58:25] Matt Doidge I agree, but that's not the view of our local PMB overseer - who's of the opinion the only way is up (for compute as well as disks). There's a view of decommissioning is a thing you do to broken hardware. [10:58:38] John Bland t2k is our No2, after atlas [10:58:52] Alessandra Forti and cms [11:01:42] Ewan Mac Mahon We should all buy Glasgow Brand 'battle tested' servers. [:02:07] Ewan Mac Mahon It's sortof an evolutionary algorithm. [11:05:08] Ewan Mac Mahon So we need a dpm-toolkit tool for finding idle files and automatically copying them to tape, and leaving a link in the dpm db, so instant distributed HFS. [11:05:33] Tom Whyntie Yup [11:07:33] Alessandra Forti HPC? [11:07:43] Alessandra Forti what mailing list is that? [11:08:07] Steve Jones Punched cards? [11:08:10] Ewan Mac Mahon The one associated with this group: http://www.hpc-sig.org/ [11:08:34] Ewan Mac Mahon It's mostly university cluster folks and dirac and archer/hector type people, plus me, and Chris. [11:09:31] Alessandra Forti I see [11:09:32] Ewan Mac Mahon Institutions have to be members, then your university rep can get you added to the list if you're interested. You're not missing a whole lot to be honest, but it's occasionally interesting to see the way some other people think about things. [11:09:56] Alessandra Forti ok I'll stick to your reports then [1:10:16] Ewan Mac Mahon And Dave Britton's on it too, it's a lot more managementy than technical. It's no tb-support. [11:10:43] Elena Korolkova left [11:10:47] Brian Davies left [11:10:52] John Bland left [11:10:53] Alessandra Forti left [11:10:54] Rob Fay left [11:10:55] Christopher Walker left [11:10:55] Ewan Mac Mahon left