Attending: Jens (chair+mins), Sam, Gareth, John B, John H, Winnie, Steve, Stuart, Raja, Brian, Pete, Robert, Tom, Matt D, Govind, Ewan, Elena, Raul. 0. OPeraional blug posts. The UKT0 was neither UK nor T0 specifically, so has now been renamed to an amazing ETLA, namely MSDC. Jens should still write a blog post. Regarding the DPM 1.8.10 upgrade discussed last week, there is a JIRA ticket and sites should perhaps hold off waiting for 1.8.11. Tickets are known to be processed quickly these days. Speaking of DPM, Sam will be attending the workshop 8-9 Decimal, and will be asking for issues to raise. Brian points out the issue that Sam has raised at all the other workshops in the past few years, namely the ability to limit the connections per protocol per disk server. 1. updates on storag e accounting by Stuart Stuart gave a presentation (which can be found in the entry for today here: http://storage.esc.rl.ac.uk/weekly/ There is a portal view of the accounting, and accounting data is timestamped with, er, timestamps. However, the initial timestamp is the creation time which is the time when the information was gathered and the end time is the time when the accounting record was created(?) or it is set arbitrarily to make the lifetime 1hr, 1day, etc. (see presentation) The Italians developed a method to populate accounting records (hereinafter ARs) from the BDII, but since the BDIIs have to comply with InstalledCapacity, the result may be fairly meaningless. For example InstalledCapacity requires that free+used=total, which doesn't actually make sense in many contexts, for example, when ephemeral file copies are created or when a file is deleted from tape. Also there are some other things in InstalledCapacity which do not make sense, such as accounting for disk cache in front of tape. Plan to move to OGF accounting records. If WLCG is moving away from BDII, could we use the ARs instead? Might be a means of ignoring the conventions of InstalledCapacity and publish useful information; however, some of the problem of publishing accounting data is intrisically difficult, e.g. when storage areas are shared between VOs. 23. VBrian feedback form the E2e workshp Brian gave a quick summary of the E2E workshop wherein he did give a presentation unto the audience. The presentation was well received but there were questions about the apparently high failure rates of FTS (20%). No specific interest in perfsonar as JANET have their own tool, but theirs allows only one account per installation. How much would we be interacting, if at all, with JANET networking folks? Would we be interacting mainly with the site IT or would it make sense to interact directly with JANET for some things, eg future planning? The intention is not to bypass site IT as such, but more to showcase GridPP's achievements (which depend on the protocols used, such as GridFTP). 3. If ther eis time updae on someof the other random isuses 4. AOB Connectinon detils (should be tpyo free bevuassd ethey are pasted from the bworser) https://vidyoportal.cern.ch/flex.html?roomdirect.html&key=QBhf7AIRnfR7 Ewan Mac Mahon: (04/11/2015 10:21:40) Does the OGF usage record also have the start time/end time concept? Jens Jensen: (10:22 AM) https://www.ogf.org/documents/GFD.204.pdf Samuel Cadellin Skipsey: (10:23 AM) Ewan: yes, but the wording is odd. The StartTime is the "time at which the measured resource consumption started" and the EndTime is the "time at which the measured resource consumption ended" Ewan Mac Mahon: (10:26 AM) Hmm. I can't see that making much sense unless it was done for each file creation and deletion. Samuel Cadellin Skipsey: (10:26 AM) which sounds like it's trying to conceive of accounting as a charging model for discrete resource usage incidents Ewan Mac Mahon: (10:26 AM) Presumably that would work though, you'd just have a lot of entries to total up. Samuel Cadellin Skipsey: (10:26 AM) "We stored these files from then to now." Ewan Mac Mahon: (10:27 AM) Ah, no, that wouldn't work - if this is a single message (rather than a 'start' message and an 'end' message) you'd only be able to do it on deletion. On a slightly separate point, who is the community for which this is useful? Samuel Cadellin Skipsey: (10:28 AM) Yeah, it works for compute as jobs have start and end dates built into them (In fact, the whole OGF usage records thing looks like it's intended for purely opportunistic "job" use - "this job ran from X to Y, and used [CPURECORD] CPU, [STORAGERECORD] Storage, etc" Steve Jones: (10:39 AM) I missed the first part of the audio. Please can you put in a link to the slides? Ewan Mac Mahon: (10:40 AM) That sounds more like something you'd do to account scratch space usage. Samuel Cadellin Skipsey: (10:41 AM) Yep. Ewan Mac Mahon: (10:41 AM) Which you might want to do on a non-traditional resource, but that we tend to ignore on our dedicated hardware. We just assume that (e.g. the recent T2C discussions) you can stage in a few tens of gigabytes of stuff for each job without limit. There's an interesting question there about whether we ought to be accounting scratch usage; perhaps we should. Samuel Cadellin Skipsey: (10:44 AM) I mean, I think it can be used for separate Storage as well, but it does look like its kinda intended for "You will be charged for this usage over this timescale". Ewan Mac Mahon: (10:46 AM) IIRC S3 charges in what are effectively gigabyte hours. Samuel Cadellin Skipsey: (10:46 AM) yeah, basically Tom Whyntie: (10:46 AM) Thanks, that was very interesting Ewan Mac Mahon: (10:48 AM) I remain slightly hazy on quite what problem this is intended to solve - if it's a billing type problem feeding into the allocation of funds, maybe that would be a better model. If it's to answer "How much data do I have, and how much free space is left" then it wouldn't. Samuel Cadellin Skipsey: (10:49 AM) If it's a billing style system, then I still have the more-detailed-accounting scripts for DPM I wrote ages ago. :) Stuart Pullinger (APEL Accounting): (10:50 AM) I would hope it would be useful for billing and the "How much data..." question. EGI certainly want to be able to use this for billing Sorry. I have to leave. Thank you for all your comments.