Work Queue

Overview

A work queue distributes a stream of tasks across a pool of workers so each task is handled exactly once, instead of every worker doing every task — the pattern you reach for whenever you need to parallelize processing (image resizing, batch jobs, background work) without coordinating which worker owns which item. The queue itself does that coordination: workers just keep polling, and the broker load-balances whatever is next in line across whichever workers happen to be asking.

receive_queue_messages pulls a batch bounded by max_messages and blocks up to wait_timeout_seconds if the queue is empty, so a worker long-polls instead of busy-looping or hanging forever. Delivery is competing-consumer: once one worker's call returns a message, no other worker gets it — the messages returned to msgs1 and msgs2 here never overlap.

Gotchas: a worker that pulls a full max_messages batch and then crashes mid-processing can lose the remainder of that batch if the SDK's default acknowledgement already marked it delivered, so size batches to what you can safely redo. A short wait_timeout_seconds turns polling into a busy-loop that hammers the broker for empty results; too long delays workers noticing new work. And running workers as sequential calls (as this sample does for clarity) isn't the real pattern — production workers poll concurrently, each in its own process or thread, against the same channel.

Prerequisites

• KubeMQ server running on localhost:50000
• Ruby SDK installed (gem install kubemq)

Code

require 'kubemq'

address = ENV.fetch('KUBEMQ_ADDRESS', 'localhost:50000')
channel = 'ruby-patterns.work-queue'

begin
  client = KubeMQ::QueuesClient.new(address: address, client_id: 'work-queue-example')
  puts "Connected to #{address}"

  6.times do |i|
    msg = KubeMQ::Queues::QueueMessage.new(channel: channel, metadata: "task-#{i}", body: "data-#{i}")
    client.send_queue_message(msg)
  end
  puts 'Enqueued 6 tasks'

  msgs1 = client.receive_queue_messages(channel: channel, max_messages: 3, wait_timeout_seconds: 5)
  msgs1.each { |m| puts "Worker-1 processed: #{m.metadata}" }

  msgs2 = client.receive_queue_messages(channel: channel, max_messages: 3, wait_timeout_seconds: 5)
  msgs2.each { |m| puts "Worker-2 processed: #{m.metadata}" }
rescue KubeMQ::Error => e
  puts "KubeMQ error: #{e.message}"
ensure
  client&.close
  puts 'Done'
end

How It Works

• Multiple workers pull from the same queue — each message is delivered to exactly one worker.
• This pattern enables horizontal scaling of task processing.
• Workers can run in separate processes or threads.
• Review timeouts, channel names, and client IDs before running against shared environments.

Related

• Ruby SDK Reference
• Send & Receive