David, Duncan, George V (RALT1), Ian Collier (RALT1), Jens (chair+mins) John H, Winnie, Sam, Brian, Elena, John B, Rob, Matt D, Govind, Steve, Ewan, Tom, Jeremy


Slightly permuting the canonical order so Ian can leave after finishing
his presentation.

1. Indigo data cloud special presentation - Ian Collier, RALT1

Ian gave a presentation of the Indigo Datacloud - an EU funded project
(Horizon 2020), which despite the name is more about XaaS than a "datacloud"
specifically.  It is a 11 MEUR project with 26 partners and 30 months'
duration.  Aiming to bring together a unified cloud experience, kind of like
grid but easier, building on existing work - can't develop new stuff - even if
they wanted to, there wouldn't be time.

STFC is involved doing PaaS and deployment, aims to deploy in the pilot.  Also
involved in security and accounting via APEL.  Ian himself is managing the
project for STCF and will be interested in the deployment of the pilot cloud.

Also of interest will be DESY's involvement with resource virtualisation,
basically federated storage.  Will be able to develop accounting for dCache.
Will also look at interfacing with EUDAT and similar projects, and there is a
role with EGI Engage on storage and data.

0. Operational blog posts... still only three posts for the quarter?!

Including some, or none, of the following topics
 - DIRAC storage issues
 - GLUE 2 for storage revisited?
 - VO (storage) support for sites, and quotas (cf dteam and follow up
from last week's point on shared storage)

GridFTP for DiRAC turned out to use UDP for data transport which was blocked
at Durham - has always been possible but apparently now it is the default.
As a data transfer tool supports both third party copying (with GridFTP for
example; what would it look like for xroot?) and copying from one flavour of
endpoint to another.  SRM to HTTPS "has been tested" - maybe worth us doing
some work with it.  We also now have xroot4 endpoints.

One point was on the DPM collaboration, specifically the postwahidian
contributions - do we need to worry or change our commitments?  Sam "is
getting access" to Wahid's relevant stuff at Edinburgh.  At the recent meeting
it was thought that a reduction of effort would be ok at this stage; still
around 1 FTE so no worries.

As regards quotas and allocations to non-LHC VOs at T2s, enforcing quotas is
not easy.  Sam reports that the HTTP deployment task force is looking at
storage accounting but by subdirectories, so like du or ls -lR rather than df
- as does CASTOR at RAL, sort of, but it is different from the general use of
accounting which is more by filesystem and filesystems are generally fairly
large for performance reasons.

2. File transfer methods feedback/recommendations/discussion?  Which
reminds me we haven't really progressed the 'generic slides'

It is considered useful to have a glossy "look how good we are at moving data"
thing as well as something technical which shows that we know (hopefully) what
we are talking about.  Previously we have also talked about having some
generally reusable slides.  Sam's presentation shifted slightly in focus, and
was presented to the project directors group (in which GridPP is represented
by our very own Dave Colling).

WebFTS as a tool is reappearing at RAL: "nearly there" - the "data portal"
approach is useful; EUDAT is also doing it.  Error messages should be
helpful... compared to GO WebFTS seems less featureful but it also is a much
younger project.  There are commercial data transfer services which are very
slick but they are also expensive.

3. AOB

Probably next week we should get the feedback on the digital preservation
course at CERN last week.




Brian Davies @RAL-LCG2: (24/06/2015 10:02:45)
I can hear you, but my mic does not work 
Samuel Cadellin Skipsey: (10:05 AM)
Has Iain's share gone black for everyone else?
Govind: (10:06 AM)
yes 
John Hill: (10:06 AM)
No, but all I can see is the first page
so I guess it's frozen
Paige Winslowe Lacesso: (10:07 AM)
It's not black but it's "?stuck" on the Intro page.
Jens Jensen: (10:08 AM)
We're not seeing the slides updating
Quite a few paternity leaves in the accounting business?!
Brian Davies @RAL-LCG2: (10:16 AM)
have a working gfal-copy wotjh castopr
versin 2.9.3-1
wear and tear
Ewan Mac Mahon: (10:17 AM)
But new USB headsets are cheap.
Brian Davies @RAL-LCG2: (10:17 AM)
ned to rollout rpms to WNs to 
yes
udp i think is some of th ecomms traffic
Matt Doidge: (10:18 AM)
Slightly related - I'm building a new WN tarball that will have the gfal tools in it at last.
Brian Davies @RAL-LCG2: (10:18 AM)
ie i think it is some of th e2811 traffic
Issue, appeasr durham gsi sever checksums are md5 not adler32
so trying to chcekcsum these vo.dirac.ac.uk trnafsres is not easy
i thought 3rd party copy fo rxroot was a v4 not v3 functionality?
developement system, not visible outside RAL
only setup fo rcms and dirac vos at the m,pomet
moment
John Bland: (10:27 AM)
separate pools are fine for big VOs, but are limited to a minimum size of your filesystem (eg 10TB in our case) which may be way more than you want to give. It's very inflexible
Stephen Jones: (10:28 AM)
Perhaps it doesn't have to be real time? 

We can count what each user has written each night, and stop them writing at all if them are over their limit.

Ewan Mac Mahon: (10:29 AM)
In DPM, wouldn't "recursing" be essentially a SQL query? It's dentries aren't split all over the show like a normal block device filesystem, they're all in the database.
Samuel Cadellin Skipsey: (10:30 AM)
Fabrizio also doesn't want to rely on trusting the historical MySQL db record, apparently.
Ewan Mac Mahon: (10:35 AM)
Do you know Morse code, Duncan?
John Bland: (10:35 AM)
Dubstep Duncan
Brian Davies @RAL-LCG2: (10:38 AM)
WLCG VOs wanted to control retry policy so asked ( and got ) dfault set from 3 tries to 1.
better PR