Bluefoot Integration Vision

akent99 · ‎05-15-2017

Magento purchased the Bluefoot CMS extension from Gene Commerce (thanks guys!) and are merging it into Magento 2.3 Enterprise Edition as a part of the standard core product. This extension provides a drag-and-drop interface for creating rich content with embedded widgets. This blog post describes the current integration vision (this vision is subject to change) and the capabilities that both theme developers and Admin users will have. Feedback is welcome!

This post is intended for developers familiar with internal Magento terminology such as blocks and layout files.

Warning: this is a long post due to the complexity of the topic.

Today

In Magento today you can write modules/themes containing layout files that control the positioning of blocks on a page. I call these layout files “code” in that developers code and test them on a laptop, commit them to a Git repository, and then push the files to production.

Admin users (typically merchandisers) can also create widgets and associate them with pages. Admin users enter the content via the Admin interface and it is saved in the database. I call this “data” as it is developed and stored directly in the production database.

Magento has additional, native CMS features including:

You can create a new empty page and add content and widgets to it. Pages like the 404 page (displayed when the URL is not known by Magento) use CMS Pages.
Widgets can be assigned to pages using different rules. You can specify whether a widget is to be displayed for a specific CMS page (identified by name), a product page for a particular product, a category, and so on. A drop-down list of supported scopes is provided in the Admin.

This functionality is powerful, but a bit complicated to use.

The Bluefoot extension provides some very cool, easy to use drag-and-drop functionality for authoring rich content, including widgets. That is, the easiest way to think of the Bluefoot extension is as an HTML editor that can also drop in Magento widgets. It is not a fundamental change in the way that Magento CMS works.

Peeking Inside Bluefoot

Before getting too far into this post, it is worth understanding that the Bluefoot 1.0 extension uses an internal, private format (not HTML) to store its rich content. It can read an attribute holding HTML, but if you start editing that attribute, it converts the content into a format that only the internal Bluefoot module understands. (Additional tables are also used behind the scenes, referenced from the internal Bluefoot format.) This can cause problems if you want to share that content with external systems, such as a PIM.

As a part of integrating Bluefoot into Magento Enterprise Edition 2.3, we are considering encoding content as HTML, so that the Bluefoot functionality can interoperate with other modules and extensions. Widget references would be embedded using HTML5 elements with special CSS class names and data attributes, as script tags, or similar. (The proposed format has not been finalized yet.) Eliminating usage of the separate table behind the scenes by including that data inside the HTML content would allow attributes such as “product description” to be fully self-contained, making tasks such as import/export easier. Using HTML reduces the need to learn a radically different format when dealing with Bluefoot content. It also simplifies upgrades from HTML – no change is required.

A negative of this approach is there would be “special” HTML markup in attributes that would only be understood by Bluefoot-enhanced Magento, as the example below illustrates. This is still under consideration – should custom Bluefoot markup be allowed in any WYSIWYG (HTML) attribute (flexible), or should it be restricted to specific use cases (to ensure product descriptions can be understood by external systems)?

In this blog post, “rich content” means simple HTML as currently supported in Magento and “advanced content” means HTML that can contain advanced Bluefoot content such as widget references.

The following is a possible example of advanced content:

<div class="magento-row">
    <div class="magento-slider" data-role="magento-slider">
        <div class="magento-slide" data-role="magento-slide-item">
            <img src="slide-1.jpg" />
            <span>Featured Products</span>
            <script type="magento-widget">
               1,43,55,3,5,23,slide-1
            </script>
        </div>
        <div class="magento-slide" data-role="magento-slide-item">
            <img src="slide-2.jpg" />
            <span>Our new collection</span>
            <script type="magento-widget">
               2,43,55,4,5,23,slide-2
            </script>
        </div>
    </div>
</div>

Note that the content may include HTML placeholder content as a hint for humans reading the HTML markup. When Magento renders the content, it would look for the “magento-*” CSS class names (or similar) and replace such <div> elements with the output of the widget. The exact proposed encoding has not been finalized – this example is illustrative only. One consideration is the performance of parsing such content at render time.

This possible change in format is one reason that Magento currently is not guaranteeing an upgrade path from the old Bluefoot 1.0 extension format to the new Magento Enterprise Edition 2.3 format. Until the new format is locked down, the feasibility of upgrading old content is unknown. We certainly understand the desire to upgrade existing content.

To summarize, Bluefoot is fundamentally a (nice) HTML editor. It edits HTML stored in an attribute, such as the description attribute of products. Editing always replaces the complete value of an attribute with a new version of the HTML. Bluefoot is not yet a Magento layout file editor. It cannot adjust parts of an existing page container tree like layout files can. It can only edit HTML (the HTML can contain embedded Bluefoot widgets).

Assuming that we do proceed with the richer form of HTML, how is Magento going to merge Bluefoot into Enterprise Edition? We want to preserve the Bluefoot ease of use, but expand the use cases. A few options were considered.

Go Wild!

Drag and drop is cool! So why not turn all the blocks on product pages (e.g. blocks that display product images, product details, ratings and reviews, etc.) into widgets and then let Admin users use Bluefoot to define exactly which widgets to display and where to put them on the page per product! Every product can then be defined with rich, unique structure.

Cool, right? Customized sites can be built just using drag and drop. Each product has super-rich flexibility.

Yes, but you need a degree of control and structure.

First, let's explore in a bit more depth how this would work in practice. What would happen is the product page layout file would become somewhat empty – it would contain a single “stage” block for the product attribute holding the content and widgets. (“Stage” is the term currently used by Bluefoot – this may be renamed to avoid confusion with “staging” support introduced in Enterprise Edition 2.1. My favorite name this week is “drop zones”.) The page would not contain blocks for displaying product title, product images, add to cart buttons, and so on.

One strategy would be to have an enriched product description attribute that contained widgets for displaying product images and so forth. The negative however is it would make maintaining a consistent site harder. Each product description may over time end up having its own, unique visual structure – subject to the whims of the last person who edited the product description. If you want to globally change how products are displayed, you would need to edit the description of every product. More structure and controls are required so that the layout of the product page is not redefined for every product. Some information does need to be defined per product (such as the product description), but other information should be sharable across products (such as positioning of widgets and blocks on the page).

The challenge is therefore to find the right balance between the rich Bluefoot functionality and control over the consistency of your site.

Master Data Management

There are a few other implications to keep in mind as well:

If your master product data is sourced from an external Product Information Manager (PIM), you need to think about whether you can externally create the richer markup including widget references. For example, do you want to use a PIM to manage product descriptions that can contain Bluefoot advanced content, such as widgets?
Similarly, if you are using Magento as your master data for product information and export it to other systems, the more richly marked up content may not be understood by external systems. They won’t know how to interpret or process the content.

So you need to think carefully about what attributes, such as product description, should contain.

Bluefoot Pages

So let’s leave the product name and description in product attributes, but have a template for displaying a product that is defined by using advanced content (HTML with widgets). The product layout file would be the same as above – very sparse. There would be a database record that holds the template contents. That content would include widget(s) to fetch product attributes (such as name and description) for display.

This would result in

The outer page would be defined by a frontend developer using layout files. The page would contain a stage block for the main content of the page.
The stage would fetch the template contents from a database record, based on the current page name / handle. This advanced content would include “display product attribute” widgets to fetch the product name and description etc.
The product description attribute would also be advanced content. This content would be merged into the stage via the widget mentioned in the previous point.

Then when you deploy Magento for the first time you would get an initial set of preconfigured templates as a starting point. (We may allow themes to define such initial template content.)

This approach is much better, but there are still issues to think through:

Your online look-and-feel is a part of your brand. You should think carefully about how much control you want to give Admin users writing product description content. How “different” should an individual product be allowed to go?
Frontend developers would not get the same fine-grained presentation control as they do with layout files. Bluefoot content has more layout capabilities than Magento 2 today, but it is not as rich as what a designer with full HTML and CSS control can achieve.
This lack of control over the HTML, especially when it comes to nesting, makes it harder to do a good responsive site when the drop zone is a large portion of the page. Custom HTML gives the web developer better control over responsive break points.
The layout of product pages would be “data”, not “code”. Much of your site structure would not be under Git version control with other layout files and themes.

So it feels like frontend developers should decide when they should control the HTML and CSS for the structure of a page versus what an Admin user should have control over. This would differ from site to site.

Multiple Stages Per Page

To recap slightly, frontend developers use layout files to add stage blocks to a page using layout files. The content for these stages would be stored in a separate database table. Note: there is no reason to limit a page to a single stage – a page might contain a stage immediately below the product title and another in the footer. The frontend developer decides how many stages should be made available to Admin users per page.

In this case when you edit the stages on a page, you don’t edit one stage at a time. Instead, you edit all stages at the same time. This makes it easier to drag content (including widgets) from one stage to another.

Ideally when editing a page, information for a real product would be displayed on the page close to how it will appear on the store front, with highlighted editable regions (stages). This is to provide closer to an inline editing experience. Full inline editing is tricky as the Admin has different CSS styles than the store front. We are still thinking this one through, but it will get easier as future themes begin to standardize on component-based CSS approaches.

Scopes

In addition to the above, several levels of context fallback are planned for templates, based on the page type. For example, a product page will most likely support content to be defined:

per product id
per attribute set
per category id
as a global default

When editing a product page template (as distinct from the product itself), the scope must be specified – “for product X”, “for category Y”, “for any bundled product”, “for a CMS page”, and so on. This is similar to website/store view scopes currently in the Admin for product and other entities. This stage content will be stored in a separate table, capturing the content and the scope in which the content is defined.

What this allows is default global advanced content that defines the default presentation, but it can be overridden for particular categories or even individual products.

Page Type Templates

Further, Magento is considering introduction of a new concept of page templates, implemented using layout files, that can be defined per page type (the final name is still to be determined – maybe “content templates”). This gives Admin users some control over the presentation outside of a scope as well as inside a scope.

There would be product page templates, category page templates, and so on. This page template is separate from the theme which is responsible for providing the visual styling. The idea is to allow designers to have multiple approved page structures that an Admin user can choose between, such as “wide text” vs “2 columns of narrow text under a wide image”. This allows some variety per product, while maintaining high quality responsive HTML and CSS markup. Again, the theme developer decides how much control they retain vs. how much control they give to Admin users.

Magento would then store for each stage:

The stage name/id (defined by page templates)
Store view (store_id) – with store_id set to zero for the default
The page type (product page, category page, etc)
The page type template to use
The Bluefoot advanced content

The inclusion of store views allows for localization of content.

Conclusions

This blog post has shared the plans for the Bluefoot CMS extension integration into Magento Enterprise Edition. Some of the above functionality is being considered for release in Magento Enterprise Edition 2.3 with further functionality coming in future releases as time permits. Exact functionality to be included in each release has not yet been determined – this post is a very early preview into what is being considered to enable community feedback.

For Community Edition users, the existing capability to add widgets to specified containers on a page will remain, although there may be some changes if topics such as store views are tackled. A serious consideration here (for another blog post!) is how themes fit into the above. We want to make sure that Community Edition themes and Enterprise Edition themes work the same, or nearly the same.