Per-Entity Metadata Working Group

[Rhys Smith <>]

> On 29 Oct 2016, at 16:07, Tom Scavo 
> <>
>  wrote:
> 
> On Sat, Oct 29, 2016 at 6:42 AM, Rhys Smith 
> <>
>  wrote:
>> 
>> Turned on serving .gz versions of the UKf aggregates 3 or 4 days ago now
> 
> So you enabled compression on ALL of your aggregates simultaneously?
> That's a bold move. Obviously you didn't see the need to start with a
> single aggregate (such as the preview aggregate) but may I ask why?

Yep. Well, my view is that the prod/back/test aggregates are for testing 
changes to the SAML MD, *not* the distribution settings. I don’t want to 
conflate the those two things.

I have a separate http://metadata-test.ukfederationorg.uk/ server that serves 
the same set of aggregates, but with different apache settings, for that kind 
of thing. I enabled compression on that vhost a few weeks ago and tested 
various test entities against it for a while and everything seemed happy. 

That, plus the experience of the other federations of having no problems, 
meant we decided to just go for it. Along with the fact, of course, that if 
anyone out there is using software that specifically sets the request such 
that it says it can handle a gzip encoding, and it can’t, then it’s their own 
damn fault (Don’t ask if you can’t handle it, man!)

Ultimately, you can always switch it back off again in pretty short order if 
any problems are reported.


>> and not a single reported problem yet.
> 
> Thanks, that's encouraging. You may have answered this elsewhere but I
> just wanted to confirm: Are you compressing on the fly or
> pre-computing compressed aggregates? I suspect you're doing the
> latter...would care to speculate on how on-the-fly compression would
> affect your stats?

Pre-computed .gz versions of each, not on the fly. I’ve no real desire to 
find out how well mod_deflate works when compressing a 28MB XML file on the 
fly.

From what I remember, gzip tends to be better supported in clients than 
deflate (i.e. zlib) encoding. And given the file sizes involved, I’d be 
surprised if there was any major difference in compression efficacy between 
the two.


> 
>> A few data points, just in case you’re interested!…
> 
> Yes, very interesting, thanks for sharing.
> 
> One final question: Do you support compression on per-entity metadata?
> If so, I'm sure it's pre-computed, but my question is: Why compress
> per-entity metadata at all? Seems like that could actually work
> against you.

Yes. And yes, pre-computed. As for why… my question in return is why not?

It still ends up serving smaller files, so reducing overall traffic and 
making the transfer faster, for clients that support it. And given the files 
are already pre-compressed, it shouldn’t affect response latency on our end. 
There would be a bit of extra work on the client end of uncompressing it, but 
that’d be negligible really (and of course is only done once until the MD 
expires in the cache).

Rhys.
--
Dr Rhys Smith
Chief Technical Architect, Trust & Identity
Jisc

T: +44 (0) 1235 822145
M: +44 (0) 7968 087821
Skype: rhys-smith
GPG: 0x4638C985
Lumen House, Library Avenue, Harwell Oxford, Didcot, OX11 0SG

jisc.ac.uk

Jisc is a registered charity (number 1149740) and a company limited by 
guarantee which is registered in England under Company No. 5747339, VAT No. 
GB 197 0632 86. Jisc’s registered office is: One Castlepark, Tower Hill, 
Bristol, BS2 0JA. T 0203 697 5800.

Attachment: smime.p7s
Description: S/MIME cryptographic signature