notification/webhook: all webhooks are signed by a new set of signing keys

[EvanMerlock]

• Identified the issue which this PR solves.
• Read the CONTRIBUTING document.
• Code builds clean without any errors or warnings.
• Added appropriate tests for any new functionality.
• All new and existing tests passed.
• Added comments in the code, where necessary.
• Ran make check to catch common errors. Fixed any that came up.

Description:
This PR:

• initializes a new set of signing keys for the webhook delivery method
• ensures that every outgoing webhook is signed
• ensures that the signature is included in X-Webhook-Signature, converting the raw binary signature to base64

Further cryptographic details:

• The message is hashed via SHA512/224. Then the signature is generated via ECDSA P256 and is encased in DER-encoded ASN.1 (A sequence of r, then s).
• The signature is calculated against the complete webhook payload being sent to the external endpoint. This means the whole message is covered.
• To verify manually, one would have to have the public key and then:
  • Decode X-Webhook-Signature from Base64 to bytes
  • Decode r/s from the DER-encoded ASN.1 payload
  • Hash the webhook body with SHA512/224
  • Verify the signature against the digest using ECDSA P256.

Which issue(s) this PR fixes:
Part of #4224

Out of Scope:

• API changes to expose the public signing key: this will be addressed in a stacked PR.
• API changes to allow for the rotation of the signing key (if desired)
• UI changes to allow for getting the signing key of the instance

Screenshots:

Describe any introduced user-facing changes:
Users will notice a new X-Webhook-Signature header on any outgoing webhook.

They will not be able to validate the webhook signature with just this PR as the public key will not have been exposed.

Describe any introduced API changes:
N/a

Additional Info:

• Additional PRs for API/UI changes to come with confirmation this is the approach we want to take for resolving Webhook Signatures and Validation #4224

[mastercactapus]

Can we get some info (PR description is fine):

• how the signature is calculated
• what is covered by the signature
• what is the signing algorithm
• how would to verify it manually

I know this is currently using our in-app code; but since this is going to be an external API, we'll need to talk through that before introducing it.

If we change anything about in-app signatures, this will need to remain as-is; so let's flush it out a bit.

Also, as mentioned in a comment, I think we need to include the GraphQL API for at least the public key to start so we can validate before enabling this.

Alternatively, we could put it behind an experimental flag until it's 100%.

[mastercactapus]

Adding the GraphQL/API bit (but not the UI changes) to grab the public key might be worthwhile so that the smoke test can demonstrate getting the key and verifying.

It's also essential we do NOT use any in-app code to verify the signature -- if the in-app keyring ever changes, the test won't detect the compatibility break; we need to insulate the two. This test should use it's own validation code, which should match whatever we'd document users use.

This will also help us understand how we expect verification to work externally.

Our existing keyring works well for internal GoAlert stuff. Still, we also need to do a little bit of digging into libraries of other languages to ensure validation isn't going to be painful if we use that too.

[EvanMerlock]

I think the best method of ensuring external use-cases would be to allow the keyring to execute SignASN1 and MAYBE VerifyASN1 operations rather than sending r/s along side the signature itself; most libraries support ASN1 verification:

• Java via BouncyCastle (unclear to me if it supports ECDSA with SHA512/224 digest as we currently use without some extra work)
• Python via Cryptography (allows for precomputed hashes)
• Golang via crypto/ecdsa (requires digest input)
• Rust through RustCrypto or OpenSSL bindings (requires digest input)
• Erlang/Elixir through :crypto (can accept digest input)

[EvanMerlock]

I think if we want to flesh this out a bit we should move to ECDSA P256 with SHA512 hashing rather than 512/224; it looks to me like BouncyCastle (Java, C#) supports 512 more than 512/224 (and I'd imagine 512 alone is more common than 512/224 for support in other languages that can take digests).

Another strong consideration would be moving to Ed25519 for this; it would remove a large amount of complexity here (ASN1 vs. manual r/s, selection of hashing algorithm, etc) but would add additional complexity to the codebase (we would likely need to implement another keyring.Keyring for Ed25519, although the original interface defn would work)

[mastercactapus]

I think using a more established combo is a good idea. We use the 224 mostly just for size since we sign and verify our own tokens internally.

Since implementation may change a bit based on this convo, let's continue in the issue so it's easier to find/reference later: #4224 (comment)