Present: Adam, Brian, Elena, Gareth, Jeremy C, John H, Matt, Robert, Sam, Steve, Wahid, Rob A, Mark N, Pete, Ewan, Chris, Alessandra, Jens Rob Appleyard gave a presentation of the puppet camp. Puppet is used at T1, in conjunction with Quattor (Quattor does the low level stuff, Puppet the other, mostly placing config files. Probably both could do everything, the mix is mostly for hysterical raisins.) Attendance at the camp seemed to be mostly business rather than academia. No LHC, but then this was UK camp; there will be a major conference in San Francisco in July. There is a Puppet 3 coming out, but deployment is premature (see also comments in chat.) Razor - tool for figuring out hardware, currently in 0.9 so not production ready. Some people are using EC2 with puppet (and without a master). Discussion of "pets" vs "cattle", T1 is very much pets; EC2 machines'd be cattle. Also note the forge as a resource. RAL's use of puppet has "atrophied" a bit since Chris Kruk left. In GridPP, Mark S (Bham) is running YAIM via puppet. DPM may move to puppet? RAL has seen memleaks with 2.7, need regular restarts. There is a 3.0 coming with better speed - apparently not fully backward compat (see chat). How are people managing the deployment databases? Oxford using cobbler; Imperial using Foreman. Hiera is supposed to be fully integrated with 3.0, including ENC (external node classifier) Suggestions for hepsysman and/or other techie topics we might want to pick up at hepsysman or elsewhere? * Update from Chris re Lustre metadata, or using WebDAV (StoRM) * 100+ TB disk servers - servers with 3 TB disks (or more) - Chris already has 3TB disks, also Ewan. Does what's on it affect the rebuild speed - eg if you rebuild your RAID array "locally" or need to replicate copies from across the LAN as with HDFS. * Future hardware - also @hepsysman, or maybe the storage workshop to be organised at IC, 27-28 June (sponsored by DDN). [09:58:13] Wahid Bhimji room with fish [10:01:03] Wahid Bhimji just wait to see if anyone else e.g. Jens joins [10:08:12] Brian Davies going off headset during rib's talk, if asking questions of me , poke [10:08:37] Jens Jensen Here's what we got: https://www.gridpp.ac.uk/wiki/Puppet [10:10:57] Adam Huffman That "pets vs. cattle" metaphor is compulsory for all talks now, it seems... [10:11:01] Rob Appleyard heh [10:13:58] Adam Huffman There were several HE people at a Puppetcamp I attended in Edinburgh last year [10:14:09] Adam Huffman though it was tacked on the end of another conference [10:15:38] Ewan Mac Mahon I suspect the general answer to 'how are we using it?' is somewhere around the area of 'we don't know yet' - [10:15:51] Ewan Mac Mahon it's all a bit new, so it's mostly a matter of feeling our way. [10:16:00] Adam Huffman I've been planning for ages to complete our current partial Puppet setup of dCache [10:16:09] Adam Huffman Pretty much everything else here is Puppet-based [10:16:45] Alessandra Forti 3 is not really stable and has some significant differences [10:16:52] Alessandra Forti not sure it si worth doing it now [10:17:02] Ewan Mac Mahon And so far we (=Oxford) are on 2.6.8, aka the one in EPEL 6 as standard. And we're staying there unless there's a really good reason not to. [10:17:23] Adam Huffman Yes, I had some trouble with cloud nodes running 3, because the modules didn't work, so reverted to 2.x [10:17:47] Ewan Mac Mahon Deviating from stock versions of anything is a pain. Sometimes worth it, but always a pain. [10:18:08] Adam Huffman Isn't there a question over how far back they go in supporting older versions? [10:20:21] Robert Frank not in all cases [10:20:33] Robert Frank I had to modify some 2.6 modules to get them to work in 3 [10:22:09] Adam Huffman Of course. I ask because it came up on the EPEL list and wasn't quite resolved [10:22:44] Ewan Mac Mahon We've been doing a sort of cobbler/puppet thing at Oxford. [10:22:48] Ewan Mac Mahon But it's all a bit new. [10:22:50] Adam Huffman Foreman is used a lot for that now. [10:25:44] Steve Jones We're using puppet 0.24.6! [10:25:59] Alessandra Forti [10:26:01] Steve Jones We keep meaning to update it. [10:26:23] Jens Jensen https://indico.cern.ch/conferenceDisplay.py?confId=242423 [10:26:48] Rob Appleyard That's probably the client version - we have something similar on the clients and 2.7.* on the master [10:27:06] Adam Huffman Thanks Rob [10:27:44] Pete Gronbech sorry mike not working [10:27:54] Pete Gronbech agenda is on the indico page [10:28:03] Pete Gronbech there is a slot for storage [10:28:14] Pete Gronbech all places allocated [10:28:31] Alessandra Forti at the end [10:29:24] Alessandra Forti Even if it is all puppet it's ok. Better to hammer on it [10:29:56] Alessandra Forti decide few questions [10:30:15] Pete Gronbech https://indico.cern.ch/conferenceDisplay.py?confId=242423 [10:30:37] Pete Gronbech http://hepwww.rl.ac.uk/sysman/June2013/main.html [10:30:43] Ewan Mac Mahon Well, yes - maybe discussing the DPM puppet modules is a good thing to look at in the storage section. [10:32:22] Wahid Bhimji https://indico.cern.ch/conferenceDisplay.py?confId=246453 [10:34:37] Adam Huffman Is there a link with details of this big data workshop? I only know about it because I saw it in our local calendar. [10:34:51] Adam Huffman Ah - thanks Wahid [10:37:15] Ewan Mac Mahon It was James Coomer that came last time. [10:39:31] Ewan Mac Mahon I guess the other obvious thing to find out is whether Imperial could host the Lustre meeting too. [10:40:08] Wahid Bhimji (well you could have it at QM- the lustre thing - not a million miles away... !) [10:40:57] Ewan Mac Mahon True. To an extent as long as it's in London it'll work fine. I suppose there's also an argument for having them at different places just to keep a degree of separation. [10:41:23] Ewan Mac Mahon So that they're clearly two meetings handily next to each other, not one meeting all blurring together. [10:42:48] Wahid Bhimji I think thats definately ought to be a hepsysman topic - though the slot is just before the viglen sponsor talk [10:43:30] Alessandra Forti indeed [10:44:13] Wahid Bhimji DDN have super big boxes... I think potentially it makes management easier - but the ddn ones are expensive [10:46:13] Ewan Mac Mahon DDNs big boxes are sort-of more like clusters in a way though. [10:47:24] Ewan Mac Mahon We might have (say) a single disk server die; I wouldn't expect to deploy several DDN systems to cover against one dying; I'd have one and let it deal with component failures. [10:48:14] Alessandra Forti not sure it does [10:48:17] Brian Davies yes [10:48:21] Ewan Mac Mahon I'm not sure either. [10:48:51] Wahid Bhimji it does... (I know nothing about it ) [10:49:04] Ewan Mac Mahon I agree we should probably find out a bit more though, under the circumstances. [10:49:21] Wahid Bhimji thanks - bye [10:49:22] Alessandra Forti bye [10:49:24] Adam Huffman Bye