There seems to be a lot of confusing discussion here. If an instance wants to discourage scraping and index-building, then they probably should NOT put a NC-ND CC license on content. That only prevents commercial activity, and search can be done for reasons other than commercial. Note that socialhub.activitypub.rocks has search, but isn’t commercial. archive.org has search on billions of pages, but it isn’t commercial.
Let’s not confuse people who have different intents. I can think of several reasons why people don’t want content indexed.
- Maybe an author doesn’t want their post to be discoverable via search at all.
- Maybe an author only wants their post to be discoverable for a period of weeks, but they want it forgotten after that.
- Maybe an author doesn’t want their post to be used for commercial purposes (e.g, search results with ads).
- Maybe an instance operator wants to provide a blanket policy for all posts emanating from their server.
I can also think of reasons why people DO want their content indexed.
- Maybe an author wants their content to have as widely accessible as as possible.
- Maybe an author wants to be able to find their own content in the future.
- Maybe an author wants to create a permanent record of their post.
- Maybe an instance wants users on their server to be bound by transparency.
These are all social constructions, and in my opinion it’s not the job of a standard to say which is “correct”. It’s only useful to describe a mechanism for people to express their intents, and we should guard against limited imagination when it comes to recognizing the intents of others.