Present: Brian, Jens, John B, John H, Winnie, Sam, Elena, Steve, Daniel, Matt,
David, Ewan, Gareth


0. Operational blog posts

Including Matt's Deteriorating Problematic Machine (DPM)  Would it be
useful if we could get a mysqloptimisingologist to talk to us or do
people know what to do?

There are (as arguably there should be) different levels of tuning tips;
some of the more dramatic ones relax the ACID properties of the database
- request tables are usually the busiest tables.  Previously we had lots
of old requests clogging up the tables but modern DPMs come with tools to
clean these up.

DMlite also comes with a thread model which can be tweaked; problems are not
necessarily with the database.  Some time ago there was a heavyweight
monitoring system which would frequently run heavy queries against the
database, but that is now a thing of the past.

1. We have a lot of actions from yesterday:

* ATLAS instructions? Elena, Alastair. Alastair is on his way to Russia,

These refer, it turns out, to the ongoing discussion of namespace dumps for
the purpose of data catalogue/SE synchronisation.  The whole thing is still
under discussion - Brian was a bit worried about the relative names (RUCIO
path) but at least they are relatively predictable from the namespace dump.
Sam thinks the same tool should be able to generate the namespace dump and
"upload" it into the SE to the predetermined location (like we discussed last
week) as you'd run the tool privileged anyway.  How frequently we should run
the tool depends among other things on how expensive it is to run, ie how long
it takes to run and how much load it puts on the system while it's running.
Perhaps weekly is a reasonable compromise - Sam volunteers to try it out with
Glasgow's relatively large SE, so we can get a feel for the impact.

rucio_dump is the corresponding dump of RUCIO (not a dump _for_ RUCIO); need
to ensure no communcations problems like the ones that led to inadvertently
deleting data at another T1.

* Shared space (Matt, Steve) - we've talked about this before...

Oxford took the approach to not let VOs in unless they use a space token - ie
have a dedicated space allocated to them.  Is it easier to resize the space
tokens these days than it used to be?  Being stricter about VO data policies
may help.

In the other protocols that do not use space tokens, some sort of quotaing
system would be required to maintain the allocations to VOs.

* Data centric view for E2E workshop?

While the workshop itself looks very networkological, it is possible that
input from storage-and-data-management will be sought from/by GridPP.  We
should check.

* Apps views of data transfers

Can't remember what this was about (an API?) and we were running a bit late as
usual so skipped this step.

* The famous DiRAC Document?

Seen by people on the DIRAC-USERS list.  May need to be updated following the
recommendations to ignore VOMS extensions (which will only work for DiRAC
because neither one of their endpoints requires VOMS, and FTS3 will happily
work without it albeit without sticking the right sticker on the accounting
log).  It is very specific to DiRAC but maybe an updated shareable version
would be useful.  It seems very heavyweight on the "DiRAC end" (as opposed to
the T1 end) installation side - could we not have done something simpler?

2. AOB

Wednesday 26th Jens will be in a department meeting and will ask Sam or Brian
to chair.


Samuel Cadellin Skipsey: (19/08/2015 10:11:59)
Relevant repository for the scripts that ATLAS provide for this: https://svnweb.cern.ch/cern/wsvn/adcops/ddmscripts/dark_data_cleanup/?rev=2843
Ewan Mac Mahon: (10:14 AM)
Why does any of this need to be anything other than globally visible anyway? Who are we worried about stealing our DPM dumping secret sauce here?
That said, I can see it with my very basic CERN external account, so Jens actually getting denied is quite an aceivement.
John Bland: (10:16 AM)
I'm in but I'm in ATLAS.
Samuel Cadellin Skipsey: (10:16 AM)
(It works for me, and I'm not in ATLAS)
Ewan Mac Mahon: (10:18 AM)
So basically it's an "anyone but Jens" ACL?
Jens Jensen: (10:19 AM)
I used a google id to log in :-)
Elena Korolkova: (10:20 AM)
This procedure is under discussion.
Ewan Mac Mahon: (10:20 AM)
Ha, no.
Is anyone?
John Bland: (10:21 AM)
I downloaded the dpm_dump.py script and looked at it. That's it.
Matt Doidge: (10:28 AM)
I like that plan of action.
Ewan Mac Mahon: (10:30 AM)
There's an extent to which my 'enforcement' of this is entirely virtual - new VOs are all getting spacetoken or nothing in principle, but in practice they're all getting nothing.
And we don't support biomed :-P
Jens Jensen: (10:31 AM)
that will prbably take a while...
Daniel Peter Traynor: (10:31 AM)
using quotas in lustre imposes a 5 to 10% performance impact, so we don't. rely on space tokens in storm. 
sorry we DO use space tokens in storm
Steve Jones: (10:33 AM)
Yes, and the brakes on my car cost me 10% in extra fuel!
Everything seems to cost something.
Daniel Peter Traynor: (10:33 AM)
never had an isue of runnig out of space yet
Steve Jones: (10:34 AM)
Yes. We support 30 VOs or there abouts. That might be why it occurs.
Not all are current, oif course.
Daniel Peter Traynor: (10:34 AM)
non atlas data at qm = 100Tb
Ewan Mac Mahon: (10:40 AM)
I think there is a significant extent to which this should be partly technical and partly a credibility waving excercise - we want to show off that we're awesome first.
But that "we should have some slides" thing?
We should. But it sounds like Jens is probably best places to make some.
Samuel Cadellin Skipsey: (10:42 AM)
Sure, although sites could probably donate local monitoring slides for network use?
Ewan Mac Mahon: (10:44 AM)
Or if they are going to install a bajillion packages, start with the emi-ui metapackage.
Which you might not want to do on a machine you're using fot anything else, but is a sensible approach if you've got a dedicated (virtual) machine you're using.
Matt Doidge: (10:46 AM)
*ahem* tarball *ahem*