Rabbit behind the scenes

Queuing in the background – getting started with RabbitMQ message broker

In PHP business logic is usually put right in action’s method or just behind it. Hence, every little piece of delaying and long-running code will be processed with a request. The problem is almost undetectable if a user sends an e-mail but with more complex actions it may take a little bit longer than preferred.

An example with handling e-mails is the first to come to mind. But let’s imagine that an application needs to compress an image or to compress batch import images. It will take a way too much time until the page will return any response. Probably, most developers will create a simple “queue” table and use SELECT * FROM queue WHERE done = 0 query. Unfortunately, databases are not designed to handle huge traffic so this solution might work only in case of tiny transfer rate, where – to be honest – using queues is unnecessary. In any other situation it will only cause deadlock or stuck and lead to further problems related to traffic growth. Of course, it is possible to write a custom solution to queue the import bulk and render images one by one and call it in the background right after the request, but it is unreasonable effort considering availability of ready made products.

Besides, would custom made solutions be prepared for scalability, cloud systems or speed performance out of the box? I believe they might be, but certainly not in a couple of hours – after this time off the shelf solutions generally will be inefficient and poorly prepared for development and reusability in the future. Thus, we can claim that this method is not a perfect problem fixer, so we should find another way to cope with the queuing.

As you presume, there is an effective solution that mitigates the impact of above mentioned problem – usage of a ready made message broker. Basically, the message broker receives the message with a new database record with every request but it will not cause a deadlock, because the server asks the message broker for data only when has resources and capacity.

In this article I would like to make an attempt to present a solution to the very annoying everyday problem that probably many programmers came across in their organisations – deadlocks in databases caused by a vast number of requests in relatively short time. The main aim of this text is to introduce RabbitMQ, which I value as a very functional and practical message broker, to help you solve the queuing problems and decrease the amount of work you would otherwise have to spend on it.

Why do we need a message broker?

Queuing brokers are useful for several reasons. Firstly, a queue between an application and the web service will reduce coupling, because both need queue’s configuration parameters and its name. It is more likely and convenient to move one system to different server or environment than to move whole queue management system. Secondly, queues work as a buffer – if the main application receives a lot of requests coming in in a short time it certainly will get stuck. The accounting system is able to process them all anyway, but using brokers prevents deadlocks – the broker will handle all requests which cannot be processed on time by the main application. Moreover, queuing is a smart bridge between applications written in different technologies and languages. A software does not need to know anything about receiver, it only sends a message to broker which handles the reports and distributes them to external programs.

How to choose the perfect broker?

Likewise choosing a framework or even a programming language, a few tools purposed for queuing tasks in the background are available. Similarly, according to the t-shape principle (suggesting to gain a wide spectrum of skills with perfectly mastered one) you should have basic knowledge about most of the queuing tools but dig into only one. Each one is different and handles various problems and it is really important to be aware of pros and cons of every alternative.

Because of this great number of choices, to narrow users’ options there is a website http://queues.io/ that collects and compares queues libraries, provides short descriptions of advantages and weaknesses and as a result helps to select the right tool.

I advise to make the choice between one of these: RabbitMQ (http://www.rabbitmq.com), Gearman (http://gearman.org/), ActiveMQ (http://activemq.apache.org/), Apollo (http://activemq.apache.org/apollo/), laravel (

Truncated by Planet PHP, read more at the original (another 51439 bytes)

more