Attending: Elena John B Matt Rob Sam Steve Gareth David Robert John H Jeremy Brian Ewan Duncan Mark N Apologies: Wahid 0. The blog posts...?! Sam and Wahid are doing exciting things - or about to do exciting things. Maybe everybody should be able to blog. 1. There are still spaces at hepsysman: https://eventbooking.stfc.ac.uk/news-events/hep-sysman--with-security-workshop-limited-numbers-for-workshop 2. There is also the security event - does it contain the stuff you need to respond to SE security challenges? Is it "low level" security, like iptables, but we also need grid security and how to respond to security challenges. Rob put in a link to a previous course (see chat). 3a. Whatever happened to the cloud stuff? How do we make "progress" with it? (What would convince us we are making progress?) Would some sort of storage workshop help? What we already have is cloud storage...? dCache and DPM support S3 - which is "cloud" to some people. dCache and DPM both have Web interfaces. We are elastic for small users, particularly if they are already in a VO. If customer is going to use multiple sources, so federation is needed. This is what the grid does... We are then already "CloudPP." But cloud is old hat, boring, all the others are now going on doing more There is the Mesos. Google call their project the borg. Linux containerisation to avoid the overhead of the virtualisation. Version of the kernel we need are not supported. 3b. Whatever happened to the big data stuff? How do we make "progress" with it? (What would convince us we are making progress?) Would some sort of storage workshop help? What is big data? MapReduce? At the site level we do data aware processing, can do file level processing. Can't do block level. Location aware processing is part of big data. We can't do MapReduce in the same way because ROOT is hierarchical. The original paradigm is that you do Map (which are embarrasingly parallel operations, storing the results), and then Reduce, which, er, reduces the result in some way (as in statistics, or something.) And in the sense of the Vs (three? five?) we probably have 2.5 out of the three. Find out what Wahid's plan/stuff is... 4. How is the FAX/AAA stuff working? We should be gaining practical experience with it now... Monitoring from ATLAS is available - monitoring appears to be working. From ATLAS, appears to be going well (Wahid would know more). Federated stuff is not really being used yet; failover only being tested. Jobs may not be using federation in a way which uses redirect at all? Files will need accessing from the global namespace rather than a specific local copy (what used to be called LFNs and SURLs, respectively.) Some thought is required to plan the testing: eg. you may end up doing random access to a remote file. There is a general transfers page where you can select xroot. 5. AOB [10:00:12] John Bland I can even see you as well [10:07:31] Rob Fay http://www.first.org/events/colloquia/lisbon2013/program/neic-security-training-lisbo2013 -- I asked Pete what was in it previously, this was the link he sent about a similar course run previously [10:10:50] Ewan Mac Mahon 'Cloud' in GridPP at the moment seems to be defined as 'What Dave Colling does'. [10:17:23] Gareth Roy http://incubator.apache.org/mesos/ [10:17:26] Steve Jones Until defined "cloud" protocols are universally adopted, it is not possible to measure the compliance of any specific cloud implementation. [10:17:43] Steve Jones It's a matter of "standards". [10:17:45] Ewan Mac Mahon Thanks, I googled 'misos' and mostly got soup and noodles. [10:17:52] Sam Skipsey mesos, I think [10:18:06] Sam Skipsey https://github.com/apache/mesos/blob/trunk/docs/Home.md [10:20:11] Ewan Mac Mahon 'Big data' and 'cloud' are essentially opposed ideas; one is about fitting the processing around things like data location, the other is about glossing over those sorts of details and treating everything as uniform and amorphous. [10:21:21] Ewan Mac Mahon On a purely PR front though, I have no problem claiming that we do big data. We have data, it's big. [10:21:24] David Crooks [Mesos aside] Interesting article about Borg and Mesos in Wired: http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/all/ [10:24:28] Duncan Rand define:The term Big data from software engineering and computer science describes datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage , search, sharing, analytics , and visualizing. ... [10:25:27] Duncan Rand http://en.wikipedia.org/wiki/Big_data#Big_science [10:26:00] Ewan Mac Mahon Does it list GridPP by name? [10:26:13] Ewan Mac Mahon Because it probably wouldn't hurt if it did. [10:26:56] Sam Skipsey It lists the LHC, really. I think we're a subset of the Big Science bit from the Wikipedia perspective. Of course, you can add edits, Ewan... [10:27:20] Steve Jones You can implement Map Reduce in almost any language. We did it in C, for example. [10:30:10] Duncan Rand http://dashb-atlas-xrootd-transfers.cern.ch/ui/# [10:30:44] Ewan Mac Mahon Indeed, we are. [10:36:23] Brian Davies yes