Minutes of the storage EVO meeting, 15 Sep 2010 Present: Glasgow: David, Sam Edinburgh: Wahid Liverpool: John, Stephen Lancaster: Matt Oxford: Pete RAL: James, Jens (chair+mins) 0. Storage, so far, at All Hands UKQCD switching to SRM. Now UKQCD have their own storage, they should make it available. Using supercomputers to generate datasets then copy it to datagrid, and do analysis on the desktops. Interesting plenary from Alex Szalay - see chat. SRB used for a project, CARMEN, replicating data between sites. 1. Any items for EGI? "Data management" looks very high level. There is a "heavy user" session chaired by Jamie Shiers. Also m'ware security and interoperation. Enough people going for us to get good reports. There is also a security training which has a slot on DPM. 2. Review of upgrade process Glasgow - not stuck, but waiting for hardware. SL5, ext4. No need to worry too much about xfs and SL5 as we used to. Maybe not needed to have as big as Manchester's. Tests at Edinburgh. Not easy to find a single optimal size, put information in wiki. Liverpool - new hardware, cabling, decommissioned old 32 bit cluster. CE is virtual, maxing out the connection. Will be using XFS. Lancaster - upgraded yesterday to 1.7.4-7, but things not working as well as it should be. Database problems, or with shift.conf? VOMS issue, apparently memory leak. Not clear if it is fixed, may be fixed in -7. Follow up. Fixed in source seven days ago, so maybe not in production. Edinburgh - not stuck! new storage, playing with it. I mean testing it, before putting it into production. Oxford - 200TB orederd, put into production ~8 weeks. SL5 with RFIO tweaks, using XFS. RHUL - 3. StoRM tests at IC? More generally, StoRM 1.5 status? 4. Outcome of syncat for T2K - next steps - offer to T2K at other sites, other VOs (to whom?) 5. Review of actions, see below. 6. AOB [09:57:25] Jens Jensen Actually the current talk is highly relevant to us [09:57:47] John Bland joined [09:57:47] Jens Jensen Alex Szalay from Johns Hopkins is talking about the cost of high end storage [09:58:31] Jens Jensen building it ourselves is cheaper than outsourcing to the cloud - no surprise there [09:59:03] Jens Jensen As an experiment they build an array of SSDs so got high IO and low power (at the cost of a higher equipment cost) [10:00:18] Sam Skipsey A much higher equipment cost, one assumes. [10:00:25] David Crooks joined [10:00:43] Jens Jensen Indeed [10:00:59] Jens Jensen He is talking about tradeoffs required for extreme end storage [10:01:08] Wahid Bhimji are we having a silent meeting today [10:01:17] Sam Skipsey Was this still cheaper than Cloud storage? [10:01:20] Jens Jensen You can talk if you like [10:01:38] Sam Skipsey I suppose you have to amortize the initial cost over the operation period. [10:01:46] Jens Jensen It depends on how long you operate it - cloud was _much_ more expensive [10:02:11] David Crooks left [10:02:23] David Crooks joined [10:02:28] Jens Jensen The tradeoff was 1. lower redundancy - he was talking about tiered storage like we do [10:02:44] Jens Jensen 2. tradeoff was cost of storage [10:02:53] Jens Jensen more data means need to store at lower costs [10:03:03] Jens Jensen larger systems more efficient than samller ones [10:03:31] Jens Jensen 3. Need higher seq iO - how long does it take to move a dataset - datasets are typically 10-100 TB [10:03:37] Jens Jensen Even across a single uni [10:04:00] Govind Songara joined [10:04:02] James Thorne joined [10:04:06] Jens Jensen 4. Need faster processing - he is talking about CUDA [10:04:21] Jens Jensen 5. Need for low power storage [10:04:36] Wahid Bhimji can you hear us jens - i guess not [10:04:43] Wahid Bhimji we kind of started the agenda [10:04:53] Wahid Bhimji are you going to join vocally [10:06:51] Govind Songara left [10:07:35] Pete Gronbech joined [10:09:29] Jens Jensen https://www.egi.eu/indico/conferenceTimeTable.py?confId=48&showDate=all&showSession=all&detailLevel=session&viewMode=room [10:18:23] Govind Songara joined [10:25:29] Matthew Doidge  User's echo suppression automatically activated [10:26:56] Matthew Doidge We're now on 1.7.4-7 [10:27:03] Wahid Bhimji https://savannah.cern.ch/bugs/?71041 [10:31:27] Jens Jensen Govind can you hear us? [10:31:49] Govind Songara i had mic problem [10:32:32] Wahid Bhimji yeah - no probs