InCommon metadata support

[Ian Young <>]

> On 23 Mar 2015, at 18:21, Tom Scavo 
> <>
>  wrote:
> 
> I've always thought the optimal validUntil value should be around 4
> days but we're a long way from achieving that goal.

The validUntil window dictates how long a client can go without being able to 
access new metadata. If there is ever a hardware outage longer than that, 
your federation stops functioning. Smaller values make your federation more 
brittle in the face of hardware failures, while the only benefit of a smaller 
value is that it narrows the window after a key compromise is detected during 
which entities can be spoofed into accepting older valid metadata to be 
replayed to *extend* the life of the vulnerability represented by the key 
compromise.

This is a hard attack to mount, and unnecessary prior to the detection of the 
compromise, so I personally don't regard it as something that should weigh 
very highly in the balance. I've therefore never seen a benefit in pushing 
towards smaller values in the UK federation. Our usual setting is 14d and we 
actually extend it over the Xmas/New Year vacations to 21d, to allow for 
things like it being harder to get staff and hardware engineers on site 
during vacations.

I think four days would be, let's say, a very courageous choice.

> On the Metadata
> Consumption wiki page [1], we recommend that clients attempt a refresh
> every hour (which is cacheDuration, right?).

As Scott says, more or less. Current Shibboleth clients actually try for new 
metadata sooner than cacheDuration and have algorithms to adjust the rate to 
back off if failures occur.

cacheDuration gives an indication of how quickly clients will pick up changes 
after they happen. Smaller values cost more for all parties, while larger 
values mean that changes take longer to propagate. In that sense, 
cacheDuration is very similar to TTL in DNS. You should set cacheDuration to 
the highest value you can consistent with your propagation goals, but of 
course to a value significantly shorter than your validUntil window size.

In the UKf we say:

> A daily refresh operation should be regarded as normal.
> 
> Entities should not refresh metadata more than four times a day unless they 
> make use of conditional GET


We don't supply cacheDuration in UKf aggregates because there are (or were) 
implementations which fall over if they see both cacheDuration and 
validUntil. However, if we did supply cacheDuration we'd probably go for six 
hours as being consistent with those guidelines.

Going to more frequent refresh would, for us, require identifying a use case 
which requires more rapid propagation of changes. I (obviously) haven't seen 
one I've been convinced by.

> Since metadata is
> typically signed once a day and the server supports HTTP Conditional
> GET, I'm not sure that is optimal either (but I'm not too concerned
> about it).

"Optimal" is probably not the best word choice here, because you're balancing 
a lot of factors that you can't really assign precise values to. I think that 
in practice there are a wide range of acceptable solutions depending on local 
circumstances and, frankly, gut feel.

> On the beta server mdq-beta.incommon.org (which serves per-entity
> metadata), we have the following attributes on the root element:
> 
> /md:EntityDescriptor/@validUntil? (currently 14d)
> /md:EntityDescriptor/@cacheDuration? (currently 6h)
> 
> I doubt these values are optimal.

I set those values, so you can deduce that I think they are within the 
acceptable range. In practice, because of some aspects of the underlying 
metadata sources as well as the current implementation, queries even an hour 
later will report updated metadata but that doesn't mean that there's a need 
to query that frequently.

> It's a fact that per-entity
> metadata doesn't change near as often as aggregate metadata. Should
> this fact be factored into the parameters somehow?

I don't think that affects validUntil at all.

With cacheDuration, again, the issue at hand is not how often the underlying 
metadata changes but how soon changes MUST be propagated to clients. I think 
there's an expectation that propagation won't take multiple days, but I think 
it's reasonable to expect that it might take a significant fraction of a day 
for existing entities. For new entities, there's probably less tolerance for 
a delay in getting the initial metadata out, but in per-entity world that's 
not actually a cacheDuration issue, as the client just sees a 404.

    -- Ian

Attachment: smime.p7s
Description: S/MIME cryptographic signature