There is actually nothing stopping you from sending stuff to any actor’s inbox. The simple thing is: You should not. In this way ActivityPub is exactly like email. It’s even easier to find somebody’s inbox with ActivityPub.
I would expect that most servers have some kind of filtering on what is shown to the user. They need to implement it anyway in order to support the shared inbox.
Somewhat realistic use case: In order to maximize the amount of people getting to enjoy my musings, I post all my public posts to every public shared inbox. As far as I can tell, the spec is ok with it. I’m not sure how well Mastodon would play along with making my posts available in the federated timeline and in hashtag searches.
Thank you very much for your answer. Now I sure that I haven’t overlooked anything.
I think it will work itself out. If an ActivityPub/Fediverse service doesn’t follow the rules and unwantedly fills other inboxes with content, it certainly won’t be welcome in many other communities and end in the list of moderated servers.