Attending: Brian, David, Gareth, Jens (C+M), John B, Lukasz, Matt R-B, Tom
Byrne, Wahid, Robert, Elena, Steve, Sam, Ewan, Alastair Dewhurst, Chris B,
Raul, Adam, Matt D

Apologies: Tom W, Raja


0. Operational blog posts - we still have 0 posts for this quarter!

1. Event programmes:

    1a. MEW programme out: Tue 2 Dec, Lustre, Hartree; Wed 3 OpenStack,
STFC scientific computing

In particular the Lustre singalong would be of interest to many people.  Chris
W and Matt R-B believed to be attending.

    1b. http://indico.cern.ch/event/272779/  GDB includes a DPM workshop
summary. Plus some cloud stuff. May be of interest.

Wahid mentioned a presentation on federated-HTTP as well.

2. Alastair Dewhurst (ATLAS/RALT1) and Tom Byrne: Ceph and Xrootd.

Tom Byrne who is currently working on CEPH for the T1 and data services
groups at RAL gave a presentation on CEPH with xrootd (presentation
available here: http://storage.esc.rl.ac.uk/weekly/Ceph20141112.pptx

Looked at S3 gateway as well, but odd filenames do not help with other
plugins.  There is an xroot-HTTP plugin which may be useful to explore.
Working with Andreas Peters from CERN, and Sam of course.

Using both cache and erasure coded storage; can tune the cache cleanup
parameters, but probably not recode the algorithm as with CASTOR.  Lots of
parameters for tuning, though.

Erasure code works best with reading whole or partial files, and writing full
files, as it updates chunks when a part of a file is updated.  Default chunk
size is 128MB; but apparently the client can request a chunk size, and even
get a smaller chunk size?!  Erasure is done in chunks, using the jerasure
library.

Unhelpfully the file system is known as EOS Diamond, bearing no relation
whatsoever to either EOS or to Diamond.

Tested with local xrdcp, and can send data near line speed.  Future
improvement will include talking to the cluster from WNs instead of to a
gateway.

As others have testified, building is not quite straightforward as it needs to
have proxies and other networking juju right or it won't work [from this
description, presumably it's python?]

3. AOB


BTW for your NVM fans, you might be interested in the following SNIA
webcast, Thursday 7pm.
https://www.brighttalk.com/webcast/663/132761
(will be recorded so you can watch later)


Matt Raso-Barnett: (12/11/2014 10:03:31)
I'm going mostly just for that Lustre user group
Samuel Cadellin Skipsey: (10:06 AM)
I can fill in some other things, as I'm also on Alastair's email list for this :D
(This gateway is also supported by the packages that Tom is now mentioning)
That's Andreas Peters for those who know him.
(The cache tiers default to using bloom filters for their popularity tracking, but as Alastair notes, there are other mechanisms available)
Duncan Rand: (10:36 AM)
Is there a URL for the slides?
dewhurst: (10:38 AM)
I emailed them to the storage group
Ewan Mac Mahon: (10:39 AM)
I havent seen any sign of thm - they might be bogged down in jiscmail.
Jens Jensen: (10:40 AM)
We will make the slides available...
dewhurst: (10:40 AM)
I only did it a minute or two ago
Ewan Mac Mahon: (10:44 AM)
So, where is everyone that's built these RPMs putting the resulting packages?