In large organizations, Magento Commerce can be one of several systems that comprise an end-to-end e-commerce operation. This means that fast, reliable and comprehensive integration between Magento and these other systems is fundamental for increased merchant capability and consistency.
Magento 2 comes with vastly improved API capabilities. Recently, Magento released a few new features into the Open Source version (former Community Edition) that were previously only available in the Commerce version (former Enterprise Edition). These include:
For more information, please refer to our developer documentation:
What is a queueing system?
A queueing system is a technology that can be used to improve the speed of Magento by giving the developer a built-in ability to delay operations that currently may be happening in the main rendering process and which may slow the frontend response. It is possible now to execute these operations asynchronously by adding them to a queue and setting up a worker process to read from this queue. The same results are achieved, while the frontend is freed to focus on rendering the pages for customers.
Examples of these operations are: API calls to external systems, bulk operations, or other expensive computations (like data propagation, or large file generations).
Why is this now available in Magento 2 Open Source?
The message queue was originally included only in Magento Commerce. The feature worked well, but modules developed primarily for Magento Open Source would often not take the message queue into account in their overall design. This could sometimes lead to issues when these modules were used with Magento Commerce. A similar challenge occurred with the Full-Page Cache feature that was only available in Enterprise Edition on Magento 1. Magento resolved that issue by moving its Full-Page Cache implementation (now also supporting Varnish) to the Magento Open Source version in Magento 2 to provide a unified approach for all extension developers that support it.
Support for AMQP/RabbitMQ implementation has been added to Magento Open Source to help standardize development and to incentivize extension and integration developers to follow consistent approaches to asynchronous operations. In other words, there is now a single “Magento way” of doing queueing.
Further discussion on reasons for such move
Currently, some integration extensions end up building their own not-so-robust queuing system. When you install multiple extensions that provide integration capabilities within Magento, each may create its own “synchronization” table. The codebase can end up with several approaches to queueing within the same application. Now, with Magento having one system for asynchronous tasks across all versions, extensions may simply use that, reducing the need for developers to learn integration intricacies that are specific for each extension.
Moreover, the lack of standardization for asynchronous operations across versions also made it very expensive for extension developers to do an integration “right”. This meant that they would sometimes fall back to ill-advised practices just to have a module quickly published, such as doing integrations with live/direct API calls hooked onto controller actions. This could mean, for an example, pushing an order event to another system during the “place order” action. In several cases, this could have been done asynchronously in the background a few moments later to save time on the request processing, without impacting customer response time.
The final issue was that many developers were ignoring AMQP message queuing opportunities in favor of approaches that would work across both Magento Open Source and Magento Commerce to simplify development and code maintenance. This meant that the Magento Commerce was not being used to its full potential and limited the value of using extensions with the platform. By adding messaging queuing support to Magento Open Source, we are ensuring that all Magento users can benefit from extensions and enhanced performance.
How did this happen? Hint: The Magento Community Engineering team is great!
I had this idea while working as a Senior Manager for Software development at Netstarter, an Enterprise Magento Partner in Australia that specializes in large integrations with Magento 2 Commerce. I also wrote about it earlier this year.
I then brought the above arguments to Eugene Tulika, a Software Architect at Magento, via the Community Engineering Slack. He kindly took up this discussion with the Magento Product team, and a decision was made to add support for the AMQP implementation to the open source version. The code change was done by Volodymyr Kublytskyi, also a Software Architect at Magento.
I believe it strongly shows how Magento is a continuously evolving the platform, how welcoming the Magento Community Engineering group is, and how easy it is to engage with them. It’s not by accident that Magento was the most contributed to PHP repository during the last year. If you have suggestions on how Magento can be even better, don't hesitate in letting them know!
