FEP-5feb: Search indexing consent for actors

[Claire]

@stevebate

What is the scope of the directive? Does it include the actor’s local instance in addition to remote instances?

I would say yes, this should include the local instance as well, although to be honest this FEP is written with Mastodon in mind, where only S2S is provided, so the actor is only represented for other servers.

@stevebate

The FEP seems to conflate “indexing” (an object retrieval optimization technique) with “availability for search” (search authorization). The text implies that objects may be available for search by the author but “MUST NOT be made available for search to other users”. To be available for search, even only for the author, the objects would typically have been indexed. I’m wondering if indexing and search authorization should be considered separately. Of course, the name of the property implies the object will not be indexed, which would usually mean it’s not available for search by anyone, including the author.

I agree the name of the attribute is not ideal. The intent is really to provide consent for the purpose of generalized search.

@stevebate

There are references to robots.txt and noindex, but this is a simpler mechanism. Was there any discussion about a rule-based approach (like search opt-in/opt-out for specific instances, for example)?

There was not. There was a discussion of a per-object flag, but that was ruled out as unpractical if a user were to change their mind.

@nutomic

Whats the reason for this? In case of Lemmy the assumption is definitely that all content is public and searchable. In fact there is already a search engine for Lemmy , and people are working on better SEO so Lemmy shows up on Google etc.

The reason, is, again, rooted in Mastodon’s history: from the moment search features were introduced back in 2017, they were purposefully limited to not have generalized search. This means that (many) Mastodon users have that expectation, and treating it otherwise would suddenly make their existing posts searchable by default without their explicit consent. We understand that different projects have different expectations, and that is why this is a SHOULD and not a MUST.