You can buy extensions from the online Magento Marketplace (marketplace.magento.com) and they appear in your Composer repository for download (repo.magento.com), but how does this work behind the scenes? Why do I need to go through Marketplace to get free extensions, but not to get Magento Community Edition (CE)? And why does Magento Enterprise Edition (EE) appear only after I pay? This post describes the relationship between the Marketplace online store, the Composer repository, and the entitlements that link the two.
Composer Repository Structure
Composer is now the de facto way to download 3rd party PHP code. There is a public packagist.org repository that provides links to a wide range of public domain libraries. Each separately downloadable library of code is shipped in a separate “package”.
Magento 2 has adopted Composer as the standard way to download code and patches for the CE and EE products. It is also used to download extensions. Having all the code in our own repository makes it easier to do things like quality and security scans across the code – something we will continually monitor and improve upon.
(Quick side note, we still make ZIP files available for CE and EE for convenience, but these ZIP files are built from Composer. We just did a few steps for you. So the ZIP files you can consider the same as using Composer – they behave in the same way and have the same structure.)
Magento has the concept of “products” and “extensions”. Put simply, products refer to CE and EE and extensions are everything you add to a product. For example, each project has a “composer.json” file that will always contain a reference to CE or EE plus zero or more additional extensions. (For simplicity in this post I am going to include themes and language packs in with extensions – they behave exactly the same way.)
Composer however does not have the concepts of products and extensions – it only knows about “packages”. If you explore the Composer repository (e.g. use ‘curl’ to fetch https://repo.magento.com/packages.json using your repository keys for authentication) you will see a flat list of packages. Magento has taken the approach to create a Composer “metapackage” per product and per extension which references all the other packages to include. (Extensions with a single package can skip the metapackage.) So a project’s composer.json file points at a single metapackge for each product and extension to include, and that metapackage draws in all the individual packages that form the entire product or extension.
So how does the Magento Composer repository know which packages to make available to each user? It does so by authenticating users, a database of “entitlement” records, and product/extension metdata. There are several steps in this process.
So it is the combination of authentication, entitlement records, and product/extension metadata (listing all packages in the product/extension) that drive what the Composer repository makes available for download.
There are different ways in which entitlements are created that align with different user experiences.
So there are several different ways in which entitlements are created behind the scenes. This is the linkage between the online store and the Composer repository.
So what about Magento extensions? They will follow the same rules as above.
We may introduce other options in the future, but the internal mechanics of entitlement records will be the same for Magento extensions as community extensions.
Internally we are discussing ideas about possible changes related to the Composer repository. These are just ideas – they have not been committed to (there is certainly no planned release date).
Currently everyone shares the same Composer repository URL. The repository authentication credentials are used to determine what the user can download. These credentials are typically shared by all developers on a project. There is support for creating multiple keys if the project owner wants to, allowing them to invalidate individual keys.
One possible change would be to create a different URL per project – like GitHub does for GitHub projects. Users would be then granted access to projects and entitlements would similarly be changed to be associated with projects. What this would allow is developers at a solution partner or agency to move more easily between projects without changing their identity. (In practical terms, the "~/.composer/auth.json" file for a user would not have to change when they switch between projects.)
Further, that in turn may allow users to user their normal Magento account name to authenticate with and control their own download password, eliminating the need for the current long hex sequences we have today (the “public and private keys”). It would still be a separate password from their normal Magento account as Composer stores this password clear text on disk, but it would potentially take some of the pain around key management away.
There are other similar options being considered. They may be incremental steps where keys are kept but the project structure is introduced. We are currently investing more into extension quality than improving the download experience for priority reasons, but community feedback on whether such changes to the Composer repository would help (or other ideas) are always welcome.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.