Introducing Magento PWA Studio

jzetlen · ‎09-06-2017

Magento is becoming a Progressive Web Application platform. That is to say, we at Magento are making a suite of tools for building online stores as Progressive Web Applications. These tools will help developers learn PWA techniques, build lightning-fast PWA frontends, and create PWA components and extensions for reuse or sale on the Magento Marketplace. Taken together, we’re tentatively calling this suite of tools the Magento PWA Studio.

Reader, this blog post aims to answer some of the questions that you’ve already thought of upon reading that opening paragraph. I’m excited to tell you about what we’re up to.

A Progressive Web Application is a software application, written in the Web platform and running in the browser, that behaves like a cloud-delivered native application. It’s an

application because it installs and runs code on the shopper’s device or computer, with more speed and capability than the “single-page JavaScript apps” of the past. It’s web because it’s written in the languages of the Web—HTML, CSS, and JavaScript—instead of in some domain-specific language or in a native framework captive to one platform. And it’s progressive because it lazy-loads itself, along with any relevant data and assets, as the user navigates around your store. All this results in a developer experience with the ease, portability, global reach, and open standards of the Web, and an end-user experience with all of the speed, usability, offline operation, device integration, and delight of a native application.

At Magento, we firmly believe that Progressive Web Apps (hereafter PWAs) are the future of the web, and that someday soon, the techniques that we call PWA today will become as mandatory for a good website as responsive design and image compression. And so, we entered into a collaboration with Google to make PWAs a part of the Magento story.

PWAT?

The term “Progressive Web Application” itself was invented by Google, though Google doesn’t have any intellectual property ownership over PWAs in the way that (for example) Adobe owns Flash. Instead, PWAs are built from open standards, which Google often promulgates and implements, but which are never proprietary. Google maintains a portal full of research and documentation to explain these things, which you’ll find yourself leaving open in a browser tab as you learn and implement PWAs. And Google makes the canonical tool for evaluating PWA performance, Lighthouse.

What distinguishes a PWA from any reasonably fast website?

Minimizes network traffic, so that it can work well on a slow or nonexistent connection.
- First pageload serves a very small shell document with inlined resources to get to First Meaningful Paint
- Web traffic is served over HTTPS and heavily edge cached
- Installs a ServiceWorker that does intelligent caching on subsequent requests
- No postback or page refresh: single-page JS app routes on the client and interacts with data via slim API requests
- App code itself loads new functionality progressively as-needed, just like data
Uses non-blocking techniques to keep UI responsiveness at 60fps, so that it feels like a native app instead of an uncannily slow simulation

Doesn’t touch DOM elements directly, instead rendering through a view component library such as Polymer, React, or Vue
Defers complex calculations to avoid layout thrash, potentially even sending them to the ServiceWorker, to achieve true concurrency and avoid locking the one JavaScript thread owned by UI
Inlined CSS from the “Critical Render Path” establishes dimensions for all elements immediately, so that the page does not “jump around” as content loads

Installs a ServiceWorker to expand the capabilities of the client-side single-page app so it can act more like a “smart client” that doesn’t require network traffic for every interaction
- Intercepts all network requests and dispatches them via route matching to “middlewares” registered by modules or themes
- Caches deserialized requests/response pairs using in-JavaScript data structures
- Detects offline mode, notifies user, and begins serving the traversed portion of the store in response to AJAX requests
- Can request Push Notifications permission and run a JavaScript function in the ServiceWorker when a notification is received…even when the browser is closed!
On supported OSes (first Android, now Windows Phone, and soon more), registers itself as an app that could live on your home screen alongside native apps, rather than launching in a Web browser tab
- Circumvents the restrictive app stores of various vendors
- Write once, run anywhere
- Still runs in a browser process (WebView), but OS treats it as a separate app and removes the browser UI from around it
- Installs instantly—you’ve downloaded it already!—much faster than installing a native binary app from an app store

There will be quite a bit of new stuff in this! I know how that feels to read. We are keenly aware that some of our tools have been opaque or difficult to master, especially in the frontend. And furthermore, we know that PWAs represent a slight change to the skillset of the frontend developer. That’s why almost all of our early-stage design thinking is concerned with developer experience and developer discoverability. We aren’t just building and releasing a “PWA theme”. We’re focused on the end-to-end experience of self-education, feedback cycle, quality assurance, deployment, and extension.

We’ve also kept ourselves attentive to developer experience by involving members of the Magento community in our early meetings and decision-making processes. We’re not stopping there, either. As more of this project solidifies, we’re bringing in a wide swath of community frontend developers, so we can stay in touch with the experience of making Magento work for your own clients. We’re talking to JavaScript people, CSS people, trainers, business leaders, full-stack developers, and neophytes. If you want to talk to us about this, we’d love to hear from you. The strength and success of the Magento PWA Studio will be due to the diversity of voices giving us guidance from our awesome community, more than from grand designs by me or anyone else.

My personal definition of success for this project is that Magento 2 itself should come with a suite of independent, but interoperating, tools—the aforementioned Studio—that can guide a novice developer to hacking on a new Magento PWA in minutes, and then help that same developer get to a reliable and highly optimized production build of that PWA.

Every person who starts learning and making Magento PWAs with the Studio will be one more Magento developer making faster stores in less time, with less frustration and more confidence in the work. And every Magento PWA rolling off the line will be just one more instant shopping channel in your customer’s pocket, loading faster than the competition and giving you more powerful tools to manage your relationship with your customer.

(Also, these tools will be open source.)

The Magento PWA Studio and the Apps We'll Make With It

The core concepts and technologies in the Magento PWA Studio are currently:

API Layer

Magento REST API coverage is already impressive, but the PWA Studio will require some new API methods to work some of its magic—and the PWAs may also need different or more efficient API endpoints at runtime.

We’re also having preliminary internal discussions about adding another style of API that can provide slimmer, more customized responses than REST designs sometimes do; we’re doing some experimenting with GraphQL, a popular choice for building web services that support PWAs and other bandwidth-sensitive things.

Application Shell

The Application Shell is a simple PHTML file, made by hand early in the project and edited rarely. It’s the basis for the HTML response from every route handle, and is responsible for:

Inlined critical CSS and global data
Minimal branded UI during load (header bar, at least)
Indexable and socially linkable metadata for the page
Seed JavaScript file for bootstrapping the PWA runtime and loading data

At runtime, the Application Shell template should receive an Application block instance, that can supply it with inlined asset strings and metadata listed above.

Application Framework

The Magento PWA Studio will want to work with a Magento store running in “headless mode”, using a tiny subset of existing storefront page rendering functionality. The PWA is still hosted by Magento at the same origin, but once it is running, it only interacts with Magento data via API calls. Our PWA implementations will likely be single-page JavaScript applications, running on a new core JavaScript framework which:

Implements the “unidirectional dataflow” pattern, using pure components, render methods and virtual DOM reconciliation instead of dirty state checking, string templates and two-way data-binding
Uses ReactJS to do all rendering and event handling inside the App Shell, and a core library of simple view components and higher-order behavioral components that enable progressive loading, extensible by third parties
Uses Redux to manage all shared state, as well as a core library of Redux reducer functions for common tasks, that can be composed together to make business logic
Adds a Redux store middleware for ServiceWorker-mediated requests, which can rely on the ServiceWorker for batched updates or push updates, or fall back where ServiceWorkers are not available
Uses React-Redux to bind state transformations to views and optimize renders
Swaps out React with Preact in the final production bundle, for maximum performance
Uses CSS Modules to isolate and scope stylesheets to components, with some potential customizations to expose a familiar stylesheet paradigm to third parties
Uses Jest and Storybook to help developers test components, reducers, and actions at build time and in developer mode
Builds, bundles and minifies with Webpack, as part of its standard 12-factor deploy, enhanced with custom Webpack loaders and plugins to pre-compile store configuration into the frontend

The App Framework build will emit static assets that can be read by the Application block, and then injected into the App Shell. It is core functionality, and is distinct from the Studio Starter Theme.

Developer experience

The App Framework will not only be a sturdy foundation for a Progressive Web App, it will also be a self-documenting teaching tool. Developer experience is priority one in the design of the Application Framework’s patterns and behaviors. Developer-mode error messages will be verbose and will deep-link to documentation. Compile-time AST analysis will try to catch common errors and explain step-by-step how they might be fixed in console messages. If any concept novel to Magento (such as CSS Modules or Webpack plugins) has any common “gotchas”, then in subsequent releases, we’ll remove those misleading parts of the development experience if we can. And if we can’t, we’ll document the heck out of it.

For readers who follow Web frontend trends, the big headline here is probably that we chose the React ecosystem for our JavaScript architecture, and the Magento PWA Studio will make “React apps”. We are really excited about this choice; I can’t wait to show our work to all you patient frontend folks in the Magento community who have been wishing aloud for a smoother developer experience.

React

React has the strongest and broadest developer happiness track record of any modern view system. There are tradeoffs with React and React-alike libraries, and there are always intriguing alternatives. But React has three things that are absolute top priorities for us:

A top-notch developer experience, with nearly no state management to worry about and near-instantaneous feedback when a developer makes a change
A large, supportive, and famously nice community of users, maintainers, open-source add-ons, and advice outside the Magento ecosystem
A structured format for view changes (the Virtual DOM tree) that can be statically analyzed and/or transformed more easily than a string template with delimiters. Future optimizations will therefore be easier and more composable, since React components are written in standard JavaScript (after the simple JSX transform) and can therefore be parsed into interoperable ECMAScript syntax trees.

The Magento PWA Studio may one day support alternate application frameworks and build pipelines; we’re certainly building it to be extensible. But our journey starts with a firm commitment to ReactJS, its developer experience, and the great architectures that prevail in its community.

The Application Framework is pretty dense with technologies and patterns that are new to Magento frontends, and there are still more basic features of the Studio to discuss. In some upcoming posts, I’ll talk about:

The Starter Theme, a reference implementation of a full Magento Theme in the Application Framework
ServiceWorkers, the modern browser feature that adds the speed and capabilities of a native app to the Web platform, and the Magento PWA Studio’s tools for creating them
The Build Pipeline, a custom-made build-test-deploy system in NodeJS, using that thriving ecosystem’s industry-leading tools for optimizing and bundling JavaScript and other assets

Seriously, I can hardly wait to share more. Until next time!

Afterword: Why PWA Now?

It’s worth reiterating, if just from a technical point of view, why Progressive Web Applications are the future of eCommerce in general and Magento in particular. In short, PWAs are not some special flavor of Web app; instead, they represent the modern-day best practices for making any interactive website. That’s analogous to responsive web design: when RWD emerged a few years ago and rapidly took over the web production world, the tone was less “here’s a fun new thing”, and more “here’s the right way to do things in general”. And like responsive design, there are a number of different technologies that contribute to a PWA. There are ways that a website can officially declare itself as a PWA, but that’s not a magic wand: A PWA is made by following the architecture and performance guidelines that make your site actually load progressively, and be usable and immersive in the middle of network operations or other work.

The list of core PWA features from the “Distinguishing Features” section above looks suspiciously like the requirements for a Smart Client (or its antecedent, the Fat Client architecture; I’ll stick with the lesser-known “Smart Client”, because I feel like it). It’s true: the community of Web developers tried to make smarter, more app-like website platforms already a couple of times, because the expectations of Web users kept growing, and with them, the need for sophisticated in-browser behavior. Java Applets didn’t stick, and neither did Silverlight or Flex. But this time is quite different. In an upcoming post, I’ll tell you why I think so.

We’ll have FAQs for you soon. Meanwhile, please do start the conversation at the Magento 2 Theming forum!