Per-Entity Metadata Working Group

[Tom Scavo <>]

Can we merge this (important) content into the wiki?

MDQ Server Software https://spaces.internet2.edu/x/iYAABg

MDQ via Apache https://spaces.internet2.edu/x/JIMABg

A new wiki page could be created if necessary.

Thanks,

Tom


On Fri, Oct 14, 2016 at 12:23 PM, Rhys Smith 
<>
 wrote:
> Hi all,
>
> Promised Nick I’d pass along the stuff I’d set up for the UKf MDQ server, 
> for your info. It’s pretty simple.
>
> So the Shib MDA creates a set of signed .xml files with naming being 
> {%-encoded entityid}.xml
>
> This winds it way through Git (at the moment), and gets pushed onto the 
> metadata distribution servers. There’s a git post-commit hook that creates 
> a gzipped version, and creates symlinks to {sha1 hashed version of the 
> entityid}.xml (and the gzipped version). Some of that post-receive script 
> is pretty custom to us and our servers (rsyncing from the git repo to the 
> apache distribution directory, etc), so i’ve removed those below and just 
> have the main creation of the gzipped files and symlinks for illustration 
> (this is all bash):
>
> (Also note this was the first go at this; it seems to work properly for all 
> the tests I’ve done, but I may tidy some bits up before it becomes prod 
> ready - especially the apache config).
>
>
>
> =====
> # Make a gzipped version of each per-entity fragment; also create symlink
> # to the XML file and its .gz version named from the SHA1 hash of the 
> entityId
> cd $gitdir/mdqcache
> for f in $gitdir/mdqcache/*.xml
> do
>     # Create the gzipped version of the file
>     gzip -9 < $f > $f.gz
>
>     # Convert the /full/path/and/filename.xml to just filename.xml
>     filename=${f##*/}
>
>     # And then filename.xml to just filename (i.e. the % encoded entityId)
>     entityidpercentencoded=${filename%.*}
>
>     # Un-%encode the entityId
>     entityid=$(echo $entityidpercentencoded | sed "s@+@ @g;s@%@\\\\x@g" | 
> xargs -0 printf "%b")
>
>     # Calculate the sha1 hash of the entityId
>     entityidsha1=$(echo -n $entityid | openssl sha1 | awk '{print $2}')
>
>     # Create the symlinks to the XML file and the gzipped version
>     ln -s $filename {sha1}$entityidsha1.xml
>     ln -s $filename.gz {sha1}$entityidsha1.xml.gz
> done
> =====
>
>
>
>
> Then on the apache end, in the vhost, there’s just this:
>
> =====
> <VirtualHost *:80>
>     ServerName mdq-test.ukfederation.org.uk:80
>     UseCanonicalName On
>     DocumentRoot /var/www/html/mdq-test.uou
>     FileETag MTime Size
>
>     #
>     # %2F in URLs should come through literally
>     #
>     AllowEncodedSlashes NoDecode
>
>     #
>     # Start your (rewrite) engines, we're going do us some rewriting...
>     #
>     RewriteEngine On
>
>     #
>     # Requests for /entities should return the whole set of entities
>     # (Note: all.xml is a copy of ukfederation-metadata.xml)
>     #
>     RewriteCond %{REQUEST_URI} ^/entities$
>     RewriteRule ^(.*)$ /entities/all.xml
>
>     <Directory /var/www/html/mdq-test.uou>
>         Require all granted
>     </Directory>
>
>     <Directory /var/www/html/mdq-test.uou/entities>
>         Require all granted
>
>         #
>         # We want MDQ responses to be cached for 6 hours after access
>         #
>         ExpiresActive on
>         ExpiresByType application/samlmetadata+xml A21600
>
>         #
>         # Any extensionless request should have .xml appended
>         # (but only if the {request}.xml does actually exist)
>         #
>         RewriteCond %{REQUEST_URI} !^.*\.xml$
>         RewriteCond %{REQUEST_FILENAME} !-f
>         RewriteCond %{REQUEST_FILENAME}.xml -f
>         RewriteRule ^(.*)$ $1.xml
>
>         #
>         # Also, if requests come when the client has indicated support
>         # for gzip encoding, then send them the gzipped version
>         #
>         RewriteCond %{HTTP:Accept-Encoding} gzip
>         RewriteRule ^(.*)\.xml$ $1\.xml\.gz [L]
>
>         #
>         # And make Apache send the right encoding flag if we're sending the 
> gzip
>         #
>         AddEncoding gzip .gz
>
>         #
>         # Apply appropriate settings to individual files.
>         #
>         <FilesMatch ".+\.xml$">
>             AddType application/samlmetadata+xml .xml
>         </FilesMatch>
>         <FilesMatch ".+\.xml\.gz$">
>             RemoveType .gz
>             AddType application/samlmetadata+xml .xml
>         </FilesMatch>
>
>     </Directory>
>
>     ErrorLog  "logs/mdq-test.uou-error_log"
>     CustomLog "logs/mdq-test.uou-access_log" combined
>
> </VirtualHost>
> =====
>
>
> Rhys.
> --
> Dr Rhys Smith
> Chief Technical Architect, Trust & Identity
> Jisc
>
> T: +44 (0) 1235 822145
> M: +44 (0) 7968 087821
> Skype: rhys-smith
> GPG: 0x4638C985
> Lumen House, Library Avenue, Harwell Oxford, Didcot, OX11 0SG
>
> jisc.ac.uk
>
> Jisc is a registered charity (number 1149740) and a company limited by 
> guarantee which is registered in England under Company No. 5747339, VAT No. 
> GB 197 0632 86. Jisc’s registered office is: One Castlepark, Tower Hill, 
> Bristol, BS2 0JA. T 0203 697 5800.
>