Optimize activity delivery for large audiences

[dahlia]

Current design

Currently, when Context.sendActivity() is called, Fedify creates separate queue entries for each recipient's inbox. This design has been intentional to achieve the following benefits:

• Independent delivery: Each delivery attempt to a recipient is independent, allowing fast-responding servers to receive messages promptly without being delayed by slow servers.
• Isolated retries: Failed deliveries can be retried independently without affecting successful deliveries.
• Granular error handling: Errors can be tracked and managed per recipient.

The core principle is to prioritize delivery reliability and speed in the face of the fediverse's unpredictable server behaviors and response times.

Current limitations

While this approach works well for activities with a small to moderate number of recipients, it shows significant performance issues when delivering to large audiences:

1. Web request latency: When an activity is sent to many recipients (e.g., a user with thousands of followers), the sendActivity() method takes a long time to return because it must enqueue separate messages for each recipient.

2. Memory consumption: Each queue message contains a copy of the entire activity payload, leading to significant data duplication when delivering to large audiences.

3. Queue overhead: Managing thousands of individual queue entries for a single activity creates unnecessary overhead for the message queue system.

4. UI responsiveness: The long processing time affects user experience, as the UI might appear frozen while enqueueing all the individual messages.

Proposed design

To address these limitations while preserving the benefits of the current design, we propose a two-stage queuing approach:

1. Fast initial enqueueing:

   • When sendActivity() is called, create a single queue entry containing:
     • The complete activity payload
     • The full list of recipient inboxes
   • This operation completes quickly, allowing sendActivity() to return promptly

2. Background fan-out processing:

   • A background task worker processes this consolidated queue entry
   • The worker splits the recipients and re-enqueues individual delivery tasks into a second-stage queue
   • The second-stage queue works as our current queue does: one message per recipient, with independent retry logic

This approach effectively moves the fan-out process from the web request handler to a background worker.

Expected benefits

1. Improved response times: sendActivity() will return much faster, particularly for high-volume deliveries.

2. Better resource utilization: Memory consumption will be significantly reduced by avoiding unnecessary duplication of the activity payload.

3. Enhanced user experience: Web UI responsiveness will improve as federated publishing actions complete more quickly.

4. Preserved reliability: We maintain the independent delivery and retry mechanisms that make the current system robust.

5. Scalability: This design better accommodates accounts with large follower bases or posts with extensive distribution lists.

Implementation considerations

Backward compatibility: Ensure all existing consumer code continues to work with this implementation change.

---

Would appreciate feedback on this approach before we begin implementation. Our goal is to make this optimization transparent to users while significantly improving performance for high-volume scenarios.

[ThisIsMissEm]

This sounds reasonable to me. Though, folks will possibly still face issues if they need to vary payload per recipient.

[ThisIsMissEm]

Perhaps instead of modifying sendActivity we introduce a enqueueActivity method that does the fan out logic automatically?

So like if the number recipients is low, it just directly enqueues the sends to each, if it's past X threshold, it chunks and enqueues in batches

[dahlia]

@ThisIsMissEm Thank you for the feedback! You raise a valid concern about payloads that need to vary per recipient.

Building on your suggestion, what if we add a fanout option to the sendActivity() method? Something like:

sendActivity(
  sender, 
  recipients, 
  activity, 
  { 
    fanout: "auto" | "skip" | "force",
    // other existing options...
  }
)

Where:

• "auto" (default): Automatically choose based on recipient count (direct enqueueing for small counts, fan-out for large counts)
• "skip": Skip the fan-out queue and always enqueue individual messages (for per-recipient payload customization)
• "force": Always use the fan-out queue regardless of recipient count (for testing)

This would preserve backward compatibility while giving developers explicit control when needed. The system would optimize for the common case by default, but provide an escape hatch for scenarios requiring per-recipient customization.

What do you think?

[ThisIsMissEm]

I'm not a fan of overloading the sendActivity method like this, I think it'd be better to create a dedicated API for this, and then just document "hey, don't use sendActivity for large recipients, you want to use enqueueActivity instead"

[ThisIsMissEm]

Arguably, sendActivity should always be the synchronous API then, and enqueueActivity or something would be the queue-based API.

[dahlia]

I appreciate the API design perspective! Your point about method naming clearly reflecting behavior is excellent.

For Fedify 1.5.0, we need to balance immediate performance improvements with backward compatibility. The sendActivity() method already has an immediate option and changing its fundamental behavior would be a breaking change.

Here's what we're thinking:

1. For Fedify 1.5.0: Implement the fan-out queue with options to control the behavior without breaking existing code:

   sendActivity(
     sender,
     recipients,
     activity,
     { 
       fanout?: "auto" | "skip" | "force",
       // existing options
       immediate?: boolean,
       // ...
     }
   )

2. For Fedify 2.0.0: We could introduce a clearer API separation with dedicated methods that better reflect their behavior:

   • sendActivity() for truly synchronous sending (or small recipient sets)
   • enqueueActivity() for explicitly queue-based operations

This gives us the performance gains we need now while setting up for a cleaner API design in the future.

[ThisIsMissEm]

I think you could retain, but mark as deprecated the immediate option, and go straight with introducing a enqueueActivity method, because even if you added this fanout option to sendActivity developers would still need to update their code to make use of it, unless it defaults to auto which might be fine?

[dahlia]

Yeah, we're making it default to fanout: "auto".

Another question I've been considering: If we were to introduce an enqueueActivity() method in the future, how should it behave when no queue is configured?

Currently, sendActivity() gracefully falls back to synchronous operation (essentially immediate: true) when no queue is configured. For a dedicated queueing method, we have a few options:

1. Throw an error: Makes it explicit that a queue is required for this operation
2. Fall back to synchronous: Like current sendActivity() behavior
3. Warn but continue synchronously: Middle ground with developer feedback

Option 1 is cleaner from an API perspective (the method does what its name implies or fails), but option 2 might be more practical for development environments where queues aren't always configured.

[ThisIsMissEm]

Ah, okay. I'd probably suggest not adding the option & just making it always use auto mode in that case. For testing you probably just want to mock sendActivity entirely and just assert things about the arguments passed to it

For a new API, I think error when no queue configured is the right way to go