inc-ops-notifications - [InCommon NOTICE] Follow-up to June 27 Federation Manager outage
Subject: InCommon Operations Notifications
List archive
- From: David Shafer <>
- To: "" <>, "" <>
- Subject: [InCommon NOTICE] Follow-up to June 27 Federation Manager outage
- Date: Fri, 27 Jul 2018 15:37:17 +0000
- Accept-language: en-US
- Authentication-results: spf=none (sender IP is ) ;
- Ironport-phdr: 9a23:d++XpRBrQNIroF5D+l5eUyQJP3N1i/DPJgcQr6AfoPdwSP36osuwAkXT6L1XgUPTWs2DsrQY07SQ6/iocFdDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXdrXKo8DEdBAj0OxZrKeTpAI7SiNm82/yv95HJbAhEmDuwbaluIBmqsA7cqtQYjYx+J6gr1xDHuGFIe+NYxWNpIVKcgRPx7dqu8ZBg7ipdpesv+9ZPXqvmcas4S6dYDCk9PGAu+MLrrxjDQhCR6XYaT24bjwBHAwnB7BH9Q5fxri73vfdz1SWGIcH7S60/VDK/5KlpVRDokj8KOT4n/m/Klsx+gqFVrxygpxNjzIDbb5qYNOZlcaPYYd8WWXBMU8RXWidcAo28dYwPD+8ZMOhWtYb9uVoOogajDgexBOPg0DFIiWHs3aYn1OkuDBrG3Aw6ENIArnvUts74NKYUUeCp0qbI1zPDYO1K2Tvj8oTHbA0uoeyVUL92bMHfx04vFwbfgVWRr4zoJzKV1uIRs2eF9epgU/ygi3I5pw1rvDeg29osh4/UjYwW0lDJ7Sp0zJovKdGlVEJ2Y8SoHZpTui2AKod6X8cvT3lptSs00LEKpJ62cDYQxJg62RLSaOaLf5aU7h/hTOqRLyl3iXF5dL+6ghu96kytx+zyVsSxy1lGsytIncXJu30MyxDf9NSIR/5+80u/2zuC2Abe4fxeL08uj6rUMZshz6YwlpUNtUTDGTf7ll3qgaGRakkp9fGk5vn/bLjov5OcMJR7hR/kPqQph8y/Hf83Mg8TX2id5OuwzqXj/Vf+QLVWkPI5jrXZsJHdJcQdvKK5BBJV0pon6xa4CDem084UkmUALFJAYB6Hjo7pNE/SIP3gAvqzn06gnCp3y/3EJLHsA5bAImLenLv8ebtx80tcxxAyzdBb6ZJUELYBIPfrV0Dsr9zYDgU2MwKqzOb9E9h914UeWX6RDa+DNqPdr0OI6vw1L+aSeYAVpS7xK+I56P72kX85hVgdcLGo3ZsRdH+4GfFmI0CeYXrqmNsBFn0KvgUnQOzyklKCSyJTZ3euUKI9/jE0EoSmDZvYRo22hLyB3Ty7HoFNZmxYEFyMEHHod5maVPcWbiKdPNNhniIeWbe/VoAhyAmktBXmxLp/MurU5ioYuIri1Nhz++LTkhQy9TlzD8SbyW2NSHt0nmwRSz832qBwvVVxylCb0ahjmvBYE9tT5+9IUgc0NJ7T0/Z1C8zzWgLGeNeJU02rQtGnATEqUtIx2MECb1x8G9WkkhDMwTCqA7kLmLyXGpA46LzT0GXsJ5U193GTnrEolUUnScpXPGu6nYZ+8QPUAovOlQOejangPfAE1TTR9WyC1WOIoFpwUQh7VqDAWnZZYVHZ+5CxrELERPqpEa9iNAxdyMCDIaJWa9rvln1HQvzkPdHZZSS2gWj6TUKJwLjJa5LxPmMawCTTCUMNjwEU+22uNA41ASKkpGSYCyZhQwHBeUTppMV3tnK3BnQ0xg+HdQU126G18xAYn9SdTe8exLQJpH1npjlpSgXul+nKAsaN8lIyNJ5XZskwtQ8djzDQqhB9M5q8Lqtrml8ZdUFtsljz0wlsVNkSitAk+XUtygc6abmV1l9MbXu55djxIfWWTwu65xWzc+jT01Da3syR//ID8/QxpVj5lACvCkc49Xh7iZ9Y32bPrpg=
- Spamdiagnosticmetadata: NSPM
- Spamdiagnosticoutput: 1:99
On June 27, 2018, a problem during a routine upgrade of the InCommon
Federation Manager led to a partial service outage. Metadata signing,
publishing, and distribution were not affected, but the Federation Manager
interface was unavailable or in a degraded state for 19 hours.
After analyzing the incident, InCommon staff have identified and corrected
the underlying problem. We've also expanded our change management processes
to protect against similar incidents in the future, and we're working to
build a continuous deployment pipeline to support automated deployment
testing.
For more information, see the incident report summary:
https://spaces.internet2.edu/display/InCFederation/Incident+Handling?preview=/107022229/129632546/20180627%20InCommon%20Federation%20Manager%20Incident%20Summary.pdf
If you have any questions or comments, please contact InCommon Support at
.
Thanks,
Dave Shafer
DevOps Manager,
Internet2 Trust & Identity
- [InCommon NOTICE] Follow-up to June 27 Federation Manager outage, David Shafer, 07/27/2018
Archive powered by MHonArc 2.6.19.