Switching AP server software on the same domain

And that’s part of the problem. This problem seems so ingrained in ActivityPub that I don’t see any way to fix it in a backwards-compatible way. It seems like to fix this problem, a breaking protocol change or a new protocol (same thing I guess) would be required. Just another reason to eschew JSON-LD (see also Desired changes for a future revision of ActivityPub and ActivityStreams).