The wiki below contains a summary of possible changes for a future version of AS/AP specifications.
This is a wiki. Anyone can update the summary, by clicking ‘Edit’.
Discussion goes here: Desired changes for a future revision of ActivityPub and ActivityStreams
I can give it a shot. I’ll try to split it into the categories that @trwnh suggested at the top:
Spoiler: The last category is the longest one… 
.
But I’m not sure in all cases and sometimes it’s just a guess so take it with a grain of salt. Otherwise not in any particular order (mostly reading order as I’m going through the thread).
1.1 Support Dislike and the dislikes collection on the same level as Like and the likes collection. From point 3 here.
1.2 “Get rid of all types that are not unambiguously defined”. From here.
1.3 Formally specify that “updated” is intended to be a machine-readable field that’s useful for “most recent wins” conflict resolution. From point 1 here. Maybe consider simply mandating a timestamp field for all activities for when they were sent/produced.
1.4 Formally specify how URLs with fragment IDs should be resolved. From point 2 here.
1.5 A new version of the spec needs to somehow address common misconceptions such as “ActivityPub requires JSON-LD” and “identities are attached to domain names”. From point 1 here.
2.1 Support emoji reactions in the base specification. From point 4 here.
2.2 Provide better semantics/support for forum/reddit-like social media (AP as it is today is mainly designed for microblogging it seems). From point 7 here
2.3 Consolidate Article, Document, Note and Page into one type, as there is no meaningful difference between them. From here.
2.4 Fix sharedInbox, possibly by reviving multibox From point 3 here and the first heading here.
2.5 Make groups make sense (take inspiration from other vocabularies). From the second heading here.
2.6 Better support for access control and not just delivery. From the fourth heading here.
2.7 Formally standardize the FEP process. Mentioned here.
2.8 Distinguish between persistent messages and ephemeral ones like presence and status updates. From point 7 here.
2.9 A robust test suite. “if you don’t have a robust test suite then you don’t have a protocol”. From point 6 here.
3.1 Remove JSON-LD and use plain JSON instead. This is mentioned by several users in this thread and this is what most of the thread has been discussing at length but I hope it is fair to say that most people argue against JSON-LD and the implementations that exist today certainly do so as well (in that they don’t use JSON-LD). If nothing else, it is controversial and it’s also not great to have such a controversial thing in the protocol. ~95% of this thread is about this topic so please don’t feel like you missed anything.
3.2 Provide a way to make it possible to switch from one AP server implementation to another without having to support old IDs. From point 1 here.
3.3 Remove HTTP Signatures and instead put signatures on objects themselves. From point 5 here.
3.4 Provide a way to send multiple activities with 1 request, instead of only 1 (probably requires 3.3 first). From point 6 here
3.5 Provide better semantics/support for forum/reddit-like social media (AP as it is today is mainly designed for microblogging it seems). From point 7 here.
3.6 Provide a way to specify roles and/or permissions for actors. From point 8 here. Also similarly mentioned in point 4 here.
3.7 Don’t use inheritance for types defined in the specification. From here.
3.8 Resolve the as:Public issue. From the third heading here.
3.9 Redefine Mention in a way that doesn’t depend on microsyntax. From the fifth header here.
3.10 A protocol-defined federated authentication mechanism (not sure exactly what this means, wasn’t further explained). From point 2 here.
3.11 No self-authenticating/forwardable messages (personally not clear on how you avoid that). From point 3 here.
3.12 An alternative extensions mechanism once JSON-LD is out, requires 3.1. From point 5 here and maybe elsewhere too. Point 5 on this wishlist points out that it is important to have the ability to define extensions that don’t necessarily live in the payload.
3.13 Specifications for managing distributed ownership of shared resources. From point 6 here.
3.14 The protocol should facilitate the construction of general libraries and toolkits. Synthesized from point 10 here.