Security
Does your application accept what it should refuse? ID-Kit plays the IdP and sends booby-trapped responses: you check that every trap is refused.
Your application must refuse or read only the signed assertion. If it logs in "admin@evil.example", it is vulnerable.
Your application must read the FULL NameID (alice@example.test.evil.example) or reject. If it understands "alice@example.test", it is vulnerable to the 2018 attack: an attacker could impersonate someone else.
Your application must refuse: nothing proves the response comes from the IdP.
Your application must refuse: the identity was changed after signing.
Your application must refuse: the signature is not from the trusted certificate.
Your application must refuse: an assertion ID must be unique, otherwise the signature can be hijacked.
Your application must reject: all AudienceRestrictions must be satisfied (it is a logical AND).
Your application must display these values without executing them: no script, no interpreted HTML, no crash on quotes or emojis.
Your application must reject a second submission of the same assertion (replay). Try sending the same response twice.
For a sign-in started by your application, it must reject a response that answers no request (unless it accepts IdP-initiated sign-ins).
Your application must refuse: the validity period is already over.
Your application must refuse: the validity period has not started (useful to test clock tolerance).
An application with a clock tolerance of 2 min refuses; with 5 min or more, it accepts. Lets you find out its tolerance.
The assertion expired 5 minutes ago: refused if the tolerance is below 5 min.
Your application must consider the IdP session as ended: it must not open a session longer than this one.
Your application must refuse: the assertion is meant for another application.
Your strict application must refuse: nothing says which application the assertion is for.
Your application must refuse: the confirmation names another ACS address.
Your application must refuse: the response is addressed to another URL.
Your application must refuse: this response matches none of its requests.
Your application must refuse: the response claims to come from a different IdP than the one it configured.
Your application must refuse or show a clear error: it does not know who is logging in.
Login may succeed, but the application must handle a missing name, email or groups without crashing or granting default rights.
If your application asked for strong authentication (RequestedAuthnContext), it must reject: the IdP only states a simple password.
If the application has BOTH certificates (old + new, option "publish the next certificate"), it accepts; if it kept only the old one, it refuses. Reveals whether a certificate rotation will cause an outage.
Your application must refuse: it cannot decrypt the assertion and must say so clearly (without crashing).
Your application must show a clear error and log nobody in.
Your application, if it asked for IsPassive, must redirect to a normal sign-in screen instead of showing a raw error.
Your application must display a clear error (the IdP refuses to process the request), without crashing or looping.
Your application must report that its request (NameID format) is not accepted by the IdP.
Your application must display a clean protocol error, without crashing.
Never send real secrets there: these are fictitious identities.