Skip to Content.
Sympa Menu

metadata-support - Re: [Metadata-Support] optimizing metadata refresh

Subject: InCommon metadata support

List archive

Re: [Metadata-Support] optimizing metadata refresh

Chronological Thread 
  • From: Ian Young <>
  • To:
  • Subject: Re: [Metadata-Support] optimizing metadata refresh
  • Date: Tue, 24 Mar 2015 12:27:15 +0000

> On 23 Mar 2015, at 18:21, Tom Scavo
> <>
> wrote:
> I've always thought the optimal validUntil value should be around 4
> days but we're a long way from achieving that goal.

The validUntil window dictates how long a client can go without being able to
access new metadata. If there is ever a hardware outage longer than that,
your federation stops functioning. Smaller values make your federation more
brittle in the face of hardware failures, while the only benefit of a smaller
value is that it narrows the window after a key compromise is detected during
which entities can be spoofed into accepting older valid metadata to be
replayed to *extend* the life of the vulnerability represented by the key

This is a hard attack to mount, and unnecessary prior to the detection of the
compromise, so I personally don't regard it as something that should weigh
very highly in the balance. I've therefore never seen a benefit in pushing
towards smaller values in the UK federation. Our usual setting is 14d and we
actually extend it over the Xmas/New Year vacations to 21d, to allow for
things like it being harder to get staff and hardware engineers on site
during vacations.

I think four days would be, let's say, a very courageous choice.

> On the Metadata
> Consumption wiki page [1], we recommend that clients attempt a refresh
> every hour (which is cacheDuration, right?).

As Scott says, more or less. Current Shibboleth clients actually try for new
metadata sooner than cacheDuration and have algorithms to adjust the rate to
back off if failures occur.

cacheDuration gives an indication of how quickly clients will pick up changes
after they happen. Smaller values cost more for all parties, while larger
values mean that changes take longer to propagate. In that sense,
cacheDuration is very similar to TTL in DNS. You should set cacheDuration to
the highest value you can consistent with your propagation goals, but of
course to a value significantly shorter than your validUntil window size.

In the UKf we say:

> A daily refresh operation should be regarded as normal.
> Entities should not refresh metadata more than four times a day unless they
> make use of conditional GET

We don't supply cacheDuration in UKf aggregates because there are (or were)
implementations which fall over if they see both cacheDuration and
validUntil. However, if we did supply cacheDuration we'd probably go for six
hours as being consistent with those guidelines.

Going to more frequent refresh would, for us, require identifying a use case
which requires more rapid propagation of changes. I (obviously) haven't seen
one I've been convinced by.

> Since metadata is
> typically signed once a day and the server supports HTTP Conditional
> GET, I'm not sure that is optimal either (but I'm not too concerned
> about it).

"Optimal" is probably not the best word choice here, because you're balancing
a lot of factors that you can't really assign precise values to. I think that
in practice there are a wide range of acceptable solutions depending on local
circumstances and, frankly, gut feel.

> On the beta server (which serves per-entity
> metadata), we have the following attributes on the root element:
> /md:EntityDescriptor/@validUntil? (currently 14d)
> /md:EntityDescriptor/@cacheDuration? (currently 6h)
> I doubt these values are optimal.

I set those values, so you can deduce that I think they are within the
acceptable range. In practice, because of some aspects of the underlying
metadata sources as well as the current implementation, queries even an hour
later will report updated metadata but that doesn't mean that there's a need
to query that frequently.

> It's a fact that per-entity
> metadata doesn't change near as often as aggregate metadata. Should
> this fact be factored into the parameters somehow?

I don't think that affects validUntil at all.

With cacheDuration, again, the issue at hand is not how often the underlying
metadata changes but how soon changes MUST be propagated to clients. I think
there's an expectation that propagation won't take multiple days, but I think
it's reasonable to expect that it might take a significant fraction of a day
for existing entities. For new entities, there's probably less tolerance for
a delay in getting the initial metadata out, but in per-entity world that's
not actually a cacheDuration issue, as the client just sees a 404.

-- Ian

Attachment: smime.p7s
Description: S/MIME cryptographic signature

Archive powered by MHonArc 2.6.16.

Top of Page