I imagine it more like servers being agnostic to the type so you can authenticate with and post to your preferred (for whatever reason) server from any client. I see the server as a simple storage device. It’s then other clients which present the data back to you. Clients may be read only (like a feed reader) or may be integrated read-write clients (like you’d expect from a fully fledged system like Mastodon or Pixelfed) but they’re fetching data from your outbox on the server you’ve authenticated with. The client can then choose to display the activity/object types they know how to handle. Basically I’d like to see UI/presentation for both reading and writing completely separate from the backend/storage.
None of this is new - remoteStorage and Solid operate this way.
I totally see that this gets more complicated where media is concerned, and special endpoints (endpoints, rather than servers) for things like photos and videos may need to be defined. Which was totally a thing in ActivityPub but proper handling of it got postponed due to time constraints.