Minutes of Storage EVO meeting 3 Nov 2010 Present: Glasgow: Sam, David Liverpool: John, Stephen Lancaster: Matt Edinburgh: Wahid QMUL: Chris IC: Duncan Oxford: Ewan Sheffield: Elena Bristol: Winnie Manchester: Alessandra RHUL: Govind RAL T1: Brian, James, Jens (chair+mins) 1. Jens gave a OGF summary, focusing on the data area groups here, first to a muted mike then to the group. 1.1 The DAIS-WG (Database access and integration) had a core spec and were working on different backends, relational is done (DAIR), XML in progress (DAIX), and RDF being worked on. 1.2. DFDL, Data Format Description Language, is based on XML schema and is actually quite an elegant and standards-based way of describing data format, whether they be binary or text or something else. Collaboration between IBM and NCSA (who have the two implementations) and others. Looking for more applications and implementations. 1.3 GSM-CG, the group for SRM, had a session where, as it happens, no one attending had much prior experience with SRM protocol or implementations. So after an introduction, the group ended up discussing large scale data management, eg replication, the use of tape vs disk, the rationale behind the design of SRM (eg latency, and the role of retention policy). A very useful session even if it was more dissemination than anything else. 1.4 GFS is the Grid File System, really an exercise in naming with the standard, RNS (Resource Naming Scheme), being a way of naming things which then maps onto WS-Addressing (which as I understand is more like a SURL, possibly.) There should be one full implementation of RNS and another one on the way - maybe something to look into at some point. 1.5 THere was a BOF on data placement policies; chaired by Reagan Moore from RENCI so clearly with iRODS in mind, but the session aimed at locating gaps not covered by other standards organisations such as IETF, OASIS, W3C, etc. 1.6 Session on testing cloud performance. Jens presented (improvised) talk on GridPP work on testing file systems and worker nodes and suchlike (showing slides from the CHEP presentations) and hammercloud. Some people run real tests in public (ie commercial) clouds. Generally found to be performant but not cost effective for long sustained work. The session noted that clouds, like grids, and unlike traditional distributed computing on a cluster, can be heterogenous, but you have less control over the heterogeneity than in grids (in the grid you can schedule to known CEs with known WNs). 2. CHEP summary, Q&A for Sam and Wahid (see also last week's summary) Brian points out that Stephen Burke's experience talk is worth reading: http://117.103.105.177/MaKaC/materialDisplay.py?contribId=52&sessionId=121&materialId=slides&confId=3 This session contained a secret (not advertised) but useful discussion of CEPH and clouds: http://117.103.105.177/MaKaC/contributionDisplay.py?contribId=268&sessionId=73&confId=3 There was some discussion of clouds vs grids vs Condor, and on running virtual images. E.g. clouds = deployment, grids = federating. As for the feedback on "our" sessions, Wahid's generated a fair bit of buzz and was reported in the summary. Sam's caused some shock, consternation, and uproar, because he presented a result which people didn't expect. It was suggested that he look at raided SSDs which he will next. Chris points out that this will make the IO/s limits relevant. In general themes: * StoRM - underrepresented, just some stuff from CNAF. * Lustre - mentioned by almost no one, only on posters. * xroot - working-ish for DPM? * NFS4 for DPM - said to be working but was not demonstrated. NFS4 is known to beat DCAP, yet the picture may change with TreeCache. * SRM for iRODS - couldn't attend the session (slides available) * SAML for AuZ - couldn't attend the session (slides available) * GLUE2 roadmap - presentation by Stephen Burke; working group led by Oliver Keeble should have contacted providers. A most Learned Discussion ensued on the pronounciation of CEPH, and of Greek in general, and Latin. N. AOB Jens out next week Brian will be deleting 200K+ orphaned ATLAS files from RHUL. [09:58:35] John Bland joined [09:58:38] Matthew Doidge joined [09:59:10] Wahid Bhimji joined [09:59:19] Queen Mary, U London London, U.K. joined [09:59:35] Duncan Rand joined [10:00:06] Stephen Jones left [10:02:00] Stephen Jones joined [10:02:12] Elena Korolkova joined [10:02:30] Wahid Bhimji oh no [10:02:31] Brian Davies [10:04:32] Winnie Lacesso joined [10:04:51] Ewan Mac Mahon joined [10:05:50] David Crooks joined [10:08:58] Alessandra Forti joined [10:10:19] James Thorne joined [10:13:40] Govind Songara joined [10:15:14] Govind Songara left [10:15:51] Alessandra Forti cloud concept is just a reshuffle [10:16:05] Alessandra Forti cloud word is the new buzz word to attract funding [10:16:12] Alessandra Forti once it was grid [10:18:03] Alessandra Forti politics! [10:19:15] Elena Korolkova could you put a link to the talk [10:19:15] Duncan Rand link please [10:20:19] Govind Songara joined [10:21:18] Jens Jensen http://117.103.105.177/MaKaC/materialDisplay.py?contribId=52&sessionId=121&materialId=slides&confId=3 [10:22:38] Alessandra Forti jens there is a missing confId number [10:24:28] Queen Mary, U London London, U.K. cvmfs would make the atlas software relatively easily [10:26:49] Jens Jensen http://117.103.105.177/MaKaC/getFile.py/access?contribId=205&sessionId=58&resId=0&materialId=slides&confId=3 [10:26:53] Jens Jensen confid = 3 [10:27:10] Jens Jensen Wait, sorry that's the wrong one [10:27:34] Jens Jensen http://117.103.105.177/MaKaC/materialDisplay.py?contribId=52&sessionId=121&materialId=slides&confId=3 [10:27:49] Jens Jensen This one works for me [10:30:17] Wahid Bhimji I didn't go either [10:34:12] Duncan Rand link please! [10:34:35] Duncan Rand kef? oh ceph? [10:34:42] Wahid Bhimji yes ceph [10:34:48] Ewan Mac Mahon I think that's what Yes Minister would call a 'courageous' decision. [10:35:13] Duncan Rand ceph is a test FS [10:35:22] Alessandra Forti leeps on not working for me from the chat [10:35:24] Duncan Rand as Ewan implies [10:35:41] Duncan Rand leeps? [10:36:55] Duncan Rand surprise surpise [10:37:05] Duncan Rand me too [10:37:15] Alessandra Forti /leeps/keeps/ [10:37:25] Duncan Rand let me ask my greek wife [10:37:53] Winnie Lacesso Yes, and you can see how Caesar became Kaiser... [10:38:01] Duncan Rand sam is correct [10:38:26] Alessandra Forti this is all very interesting but... [10:38:26] James Thorne They were "kephalopods" when I was studying geology at uni [10:38:39] Alessandra Forti there still are [10:39:35] Wahid Bhimji SAM - on the NFS v DCAP [10:39:40] Govind Songara ok [10:39:41] Wahid Bhimji in particks talk [10:39:41] Sam Skipsey YES WAHID? [10:39:44] Govind Songara thanks [10:39:50] Elena Korolkova left [10:39:53] Alessandra Forti bye bye [10:39:55] Wahid Bhimji http://indico.cern.ch/contributionDisplay.py?contribId=12&confId=92498 [10:39:56] James Thorne left [10:39:58] John Bland left [10:40:00] Brian Davies left