This post is first in a series talking about planned improvements to the persistence layer in Magento 2. It provides an overview of the scope of work planned. This work is still reasonably early in the research cycle and we are looking forward to community engagement as we proceed.
What? Another ORM?
Starting from a traditional database access perspective, Magento 2 business logic has SQL queries directly embedded into the code. One of the areas of investigation is to improve the separation of business logic from persistence logic. This has a number of potential advantages, including exploring alternative database technologies for improved performance. This separation of business logic from database access code is a traditional benefit of ORM technologies.
The persistence API we envision for Magento however is not a simple ORM like Doctrine. Traditional ORMs deal with mapping rows from relational database queries into in-memory objects. Good ones also deal with schema changes. The Magento persistence layer will go beyond such ORMs. The Magento persistence layer understands Magento concepts such as entities. The Magento Entity Attribute Value (EAV) model will be managed by the persistence layer (business logic gets a consistent view of all entity attributes without care of the physical storage representation).
Going further, the importance of software patterns such as the Command Query Responsibility Segregation (CQRS) pattern are becoming more widely acknowledged in modern software systems. CQRS separates "commands" (which result in modification to master data) from "queries" (retrieval of data which may have been transformed / aggregated in some way for faster retrieval). This has direct alignment with the Magento EAV master data model (modified by commands) and indexes (generated for supporting more efficient queries). Indexing is also managed by the persistence layer (business logic does not have to care about what indexing approach is used and how to trigger it).
So, while the persistence layer in Magento does include traditional ORM functionality, it goes a lot further.
Goals
There are multiple goals for this separation, but a high level the overall intent is to open up the possibility for more experimentation in use of different persistence technologies. Do you want to use Oracle instead of MySQL? Sure. Do you want to try MongoDB or ElasticSearch for your catalog index? Go for it. Do you want to use an in-memory product catalog? Fine. Separating business logic from persistence more cleanly will make such cases easier to do. (Not “easy”, just “easier”!)
More specific examples:
- Improved CQRS alignment - separation of "commands" from "queries", recognizing the data model for each can be different.
- Allow different indexing strategies to be plugged in without impact to business logic.
- Allow different persistence technologies to be used for different tables (MySQL, ElasticSearch, etc.).
- Align service contracts with persistence data structures providing a standardized query (and most likely update) API across all entities, and to improve performance by allowing the same data structure to propagate from the persistence layer all the way up to the presentation tier without data copying.
- Introduce a higher-level abstraction of the persistence model than the raw database tables – e.g. “Customer” as one entity rather than talking about all the EAV tables behind it.
- The higher-level abstraction of data model includes separation of “master” from “query” data model views.
- Database schema upgrades are automated – schemas are declarative and schema differencing is used to automatically generate schema upgrade commands.'
- Flat tables can be used for master data attributes for improved performance. Knowledge of the richer Magento entity data model is built into the persistence layer.
- Allow higher performance bulk import of data, either through the persistence layer or by providing sufficient metadata for such tools to use.
Conclusions
The persistence layer work is a significant fundamental platform work item. It is not going to be rushed. For myself, I am excited by the new opportunities it will open for Magento and the community to push the boundaries of performance by allowing special purpose technologies (like in-memory databases) without change to business logic. It is this standardization and decoupling that excites me the most about this work stream.
Future posts in this series will dive deeper into specific concepts to share plans and gain community feedback as plans proceed. If you are interested in this area and want to be involved, please keep your eyes open for future posts in this blog.
