Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Max number clients reached using subscribe_with_timeout #1259

Closed
gap777 opened this issue Apr 1, 2024 · 2 comments · Fixed by #1260
Closed

Max number clients reached using subscribe_with_timeout #1259

gap777 opened this issue Apr 1, 2024 · 2 comments · Fixed by #1260

Comments

@gap777
Copy link

gap777 commented Apr 1, 2024

This is a Rails 7 app using REDIS for pubsub, hosted on Heroku ("Heroku Data for Redis" add on). This also happens locally (when I set the maxclients down to 20, like is set for the Heroku instance)

Our code sets up REDIS like this:

redis = Redis.new(url: ENV.fetch('REDIS_URL', nil), ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE })
pool = ConnectionPool.new(size: 5, timeout: 5) { redis }

We listen for pubsub like this:

  trap(:TERM) { shutdown }
  pool.with do |conn|
     conn.subscribe(@stream) do |listener|
        register_subscription_callback(listener)
        register_message_callback(listener)
        register_unsubscription_callback(listener)
     end
 end

I want to allow the caller to abort the listening, so I'm trying to change it like this:

  pool.with do |conn|
     until abort_signal.set?
        @logger.debug("listening for pubsub events for 5 seconds")
        conn.subscribe_with_timeout(5, @stream) do |listener|
          register_subscription_callback(listener)
          register_message_callback(listener)
          register_unsubscription_callback(listener)
        end
     end
  end

However, after 20 loops (on the 21st attempt), I get an error

RedisClient::CommandError: ERR max number of clients reached (RedisClient::CommandError)

Alternatively, I've tried putting my while loop above the withdrawal from the connection pool:

    
 until abort_signal.set?
    pool.with do |conn|
        @logger.debug("listening for pubsub events for 5 seconds")
        conn.subscribe_with_timeout(5, @stream) do |listener|
          register_subscription_callback(listener)
          register_message_callback(listener)
          register_unsubscription_callback(listener)
        end
    end
 end

I've also tried removing the connection pool altogether, but still experience the issue.
I've also tried just using the recipe from your README

redis = Redis.new(reconnect_attempts: 0)
while true
  redis.subscribe_with_timeout(5, "news") do |on|
    on.message do |channel, message|
      # ...
    end
  end
end

and still experienced the issue.

Can you help me to get an interruptable subscribe for PUBSUB events?
If via subscribe_with_timeout, what is the underlying reason for my error?

Thank you

casperisfine pushed a commit to casperisfine/redis-rb that referenced this issue Apr 2, 2024
Fix: redis#1259

Otherwise we need to wait for GC to close it.
casperisfine pushed a commit to casperisfine/redis-rb that referenced this issue Apr 2, 2024
Fix: redis#1259

Otherwise we need to wait for GC to close it.
casperisfine pushed a commit to casperisfine/redis-rb that referenced this issue Apr 2, 2024
Fix: redis#1259

Otherwise we need to wait for GC to close it.
@byroot
Copy link
Collaborator

byroot commented Apr 2, 2024

This was a bug fixed in #1260. You can point your gemfile at the repo and your code should now work as intended.

@gap777
Copy link
Author

gap777 commented Apr 2, 2024

Super! Thanks for the quick turnaround. Will you be releasing a new version of the gem anytime soon?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants