HTTP Signatures (and mastodon) continue to confound

Programming Python
1

I’m trying to use the Mastodon blog posts as a guide for a dead simple federated ActivityPub server. Like many others on this forum, I am running into HTTP Signature problems that I can’t quite figure out despite lots of debugging efforts. Lots of print statements in mastodon later, I’ve figured out quite a bit but am still not succeeding.

The following is the HTTP Request I’m making:

{'_body_position': None,
 '_cookies': <RequestsCookieJar[]>,
 'body': '{"@context": "https://www.w3.org/ns/activitystreams", "id": '
         '"http://9462-75-164-4-199.ngrok-free.app/my-first-follow", "type": '
         '"Follow", "actor": '
         '"http://9462-75-164-4-199.ngrok-free.app/@ddsdafdSAF", "object": '
         '"http://127.0.0.1:3000/@railsadmin"}',
 'headers': {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Digest': 'sha-256=0t+aD34XYYFOwkI9gqNPVDAJUafDUS0UyMS7GJFoEtU=', 'Host': '127.0.0.1:3000', 'Date': 'Mon, 01 Jan 2024 22:05:36 GMT', 'Signature': 'keyId="http://9462-75-164-4-199.ngrok-free.app/@ddsdafdSAF#main-key",headers="(request-target) digest host date",signature="dvFPKwwL9aGZ3uKHvY2j4z0qgpLn/QohoTm3NNzJSxyZnRCSNgIIwH2jJDwpw96vfVQ49lsYBZNN1b5qqSwqsnVu/FsAZhFpj1+kpQtUNtzcmSDkmDDnaKmiGwFWUYrp0IC0QkZrOB73QBIIo4Ovyfd+9veVMGTTP0+AsCl8QwD7Mjw+HIM7M5d8Dclvj0kQ+XAwJrGsbFxGOQBdayMetxYw1KKvjyaQz71LOCfU4C1SqD9NmB9I96dWz6N0PIkyPwI4UD8ZBF6mLCpdk9AqNIZI8mgAYV8TQu2t6FUdHa0SaQ4OgOpfuAU61HhKXy7ocVjn0z8II83xOlleADMkL2dtNKHZLblIoDdq0Xyu7KW0zvL36UlAGbPTsqyn351y7/xajztJ6dM1eemoAmB3GQs590aK6xfCBieYXivxQvm7fGhVwcNn4NjpAlnrS23/BfBb7Vl6pg85XzRvP7hN2pST9QcTk8DNT+AyL1WkCHVCvJaHaZsseIQ6w9gII7ABjOThRRsE2+G2zO5sx5J+mXeysh9HdxfdubDLUW9mNrB5vxLQD1vzY+GC3uTMNq+gWBOkgi4fwID09ZlcJj6+iw1vOr9cPgTzYBjn3JQOAwzqVERJ7ZClI72Qz6IIwPLSMs/NCYeETbHudEUXpFFhheEoeZALdVeXSp32ILxP3BQ="', 'Content-Length': '247'},
 'hooks': {'response': []},
 'method': 'POST',
 'url': 'http://127.0.0.1:3000/inbox'}

Back from Mastodon, I get

>>> pprint.pprint(_)
(b'{"error":"Verification failed for ddsdafdSAF@9462-75-164-4-199.ngrok-free.ap'
 b'p http://9462-75-164-4-199.ngrok-free.app/@ddsdafdSAF using rsa-sha256 (RSAS'
 b'SA-PKCS1-v1_5 with SHA-256)","signed_string":"(request-target): post /in'
 b'box\\ndigest: sha-256=0t+aD34XYYFOwkI9gqNPVDAJUafDUS0UyMS7GJFoEtU=\\nhost:'
 b' 127.0.0.1:3000\\ndate: Mon, 01 Jan 2024 22:05:36 GMT","signature":"dvFPK'
 b'wwL9aGZ3uKHvY2j4z0qgpLn/QohoTm3NNzJSxyZnRCSNgIIwH2jJDwpw96vfVQ49lsYBZNN1b5qq'
 b'SwqsnVu/FsAZhFpj1+kpQtUNtzcmSDkmDDnaKmiGwFWUYrp0IC0QkZrOB73QBIIo4Ovyfd+9veVM'
 b'GTTP0+AsCl8QwD7Mjw+HIM7M5d8Dclvj0kQ+XAwJrGsbFxGOQBdayMetxYw1KKvjyaQz71LOCfU4'
 b'C1SqD9NmB9I96dWz6N0PIkyPwI4UD8ZBF6mLCpdk9AqNIZI8mgAYV8TQu2t6FUdHa0SaQ4OgOpfu'
 b'AU61HhKXy7ocVjn0z8II83xOlleADMkL2dtNKHZLblIoDdq0Xyu7KW0zvL36UlAGbPTsqyn351y7'
 b'/xajztJ6dM1eemoAmB3GQs590aK6xfCBieYXivxQvm7fGhVwcNn4NjpAlnrS23/BfBb7Vl6pg85X'
 b'zRvP7hN2pST9QcTk8DNT+AyL1WkCHVCvJaHaZsseIQ6w9gII7ABjOThRRsE2+G2zO5sx5J+mXeys'
 b'h9HdxfdubDLUW9mNrB5vxLQD1vzY+GC3uTMNq+gWBOkgi4fwID09ZlcJj6+iw1vOr9cPgTzYBjn3'
 b'JQOAwzqVERJ7ZClI72Qz6IIwPLSMs/NCYeETbHudEUXpFFhheEoeZALdVeXSp32ILxP3BQ="}')

However, when I paste the private key and public key into Online HTTP Signature tool, I get the exact same signature as I’m sending from python.

When I add print statements to mastodon, I can confirm my digest is alright and it’s truly failing at verification. When I print out compare_signed_string, and paste those results into the httpsig tool, I get the same signature that my python code is calculating.

I see mastodon correctly downloading my actor/doing a Webfinger and those are succeeding.

What am I missing? What is Mastodon expecting in this request that I am not providing?

2

This is the value of compare_signed_string when I print it out during verification

22:05:38 web.1     | verifying before refreshlooking for:
22:05:38 web.1     | 
22:05:38 web.1     | (request-target): post /inbox
22:05:38 web.1     | digest: sha-256=0t+aD34XYYFOwkI9gqNPVDAJUafDUS0UyMS7GJFoEtU=
22:05:38 web.1     | host: 127.0.0.1:3000
22:05:38 web.1     | date: Mon, 01 Jan 2024 22:05:36 GMT

Which looks exactly like what I’m signing…
The signing code, with debugging statements, looks like

def sign_and_send_request(url, document, pem_key_material, public_pem_material, key_id):
    document = json.dumps(document)
    keypair = serialization.load_pem_private_key(
        pem_key_material.encode(), password=None
    )

    parsed_url = urlparse(url)
    host = parsed_url.netloc
    target = parsed_url.path
    if parsed_url.query:
        target += "?" + parsed_url.query

    hasher = hashlib.sha256()
    hasher.update(document.encode())
    digest = base64.b64encode(hasher.digest()).decode("utf-8")

    date = datetime.datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S GMT")
    signed_string = f"(request-target): post {target}\ndigest: sha-256={digest}\nhost: {host}\ndate: {date}"
    print("XXX MARK \n")
    print(signed_string)

    signature = base64.b64encode(
        keypair.sign(signed_string.encode(), padding.PKCS1v15(), hashes.SHA256())
    ).decode("utf-8")

    if verify_signature(signed_string, signature, public_pem_material):
        print("Signature verified successfully.")
    else:
        print("Signature verification failed.")
        return None


    header = (
        f'keyId="{key_id}",headers="(request-target) digest host date",signature="{signature}"'
    )


    response = requests.post(
        url, headers={"Digest": f"sha-256={digest}","Host": host, "Date": date, "Signature": header}, data=document
    )

    return response
3

I had two important missing problems: first, the Signature header needed to specify the algorithm. Second, my print debugging in the rails app needed to not cause a function to return nil. I.e. PEBKAC

1 Like
4

This online tool likely uses a different version of HTTP signatures.

In Fediverse we use this draft: draft-cavage-http-signatures-12

I’d also try this actor validator: https://verify.funfedi.dev/

1 Like
5

I found server-to-server signatures to be the jankiest and fussiest part of implementing AP. I wrote up my (polite, but bad-attitude) notes in a comment here: squidcity/signatures.ts at main - squidcity - Cinnamon Rangers

Aside from none of it being documented anywhere, the biggest gotchas for me were:

  • You must use RSA-2048 and SHA-256, nothing else.
  • Specific headers are required to be in the signature. You just have to know them.

I would love to see consensus around migrating to something better, but until then, maybe those notes and sample working code will help future travelers.

6 Likes