Metadata Distribution Subcommittee of TAC

[Ian Young <>]

On 6 Dec 2013, at 11:49, Tom Scavo 
<>
 wrote:

> You may have heard that UKf is now publishing eduGAIN metadata in
> production, which of course increases the size of the UKf metadata
> aggregate. (I'll let Ian report on the specifics.)

We're accepting most of the eduGAIN entities, which adds around 190 to our 
aggregate. The size went up by 12%. For comparison, that's about 18 months of 
"normal" growth for us.

We do get a few customers a year who hit memory limits anyway with the steady 
growth; we've had a couple report that this particular change put them over, 
but as I understand it they both just dialled something up and went on their 
way. I imagine that there are more that will just do that without mentioning 
it to us.

> Related to this,
> Dick Visser reported on the simpleSAMLphp mailing list this morning
> that his UKf metarefresh process failed, the root cause being not
> enough memory allocated to the metarefresh process. You can read his
> account of what happened towards the end of this mailing list thread:
> 
> https://groups.google.com/forum/?hl=en#!topic/simplesamlphp/b6fTf53iq4w

I guess it's worth mentioning that his memory limit was set pretty low as 
things stood. In fact, it sounds like the required increase (to 384MB) is 
something I could run on a Raspberry Pi. Maybe I should try that ;-)

> We've been talking a lot about metadata aggregates and per-entity
> metadata in this WG, so I thought you might find this story
> interesting. Personally, I'm taking it as a wake-up call. Metadata
> aggregates can't continue to grow without bound. We need to find a
> better way.

That's obviously true, but I think we're still some distance from needing 
scary amounts of memory in practice.

        -- Ian

Attachment: smime.p7s
Description: S/MIME cryptographic signature