cancel
Showing results for 
Search instead for 
Did you mean: 

Why GraphQL Is Interesting for Magento 2

akent99
Regular Contributor

GraphQL is an API style that Facebook has released as open source that is getting traction on other projects, including GitHub. This blog post explores reasons why GraphQL could be interesting for Magento 2 to add alongside REST and SOAP as web API protocols.

 

API “Graphs”

 

The name GraphQL came from Facebook where it is used as the API to query the Facebook social graph. GraphQL is part way between query languages such as SQL with JOIN and aggregation functions, and REST or SOAP, where the server decides on the structure of data it makes available. For example, a GraphQL request always specifies what data to be returned (similar to the SELECT line of an SQL query), but GraphQL provides no query operators such as GROUP BY or SUM. While there is no JOIN operator, it is normal for a server to make fields available to navigate to other related data structures. Thus, GraphQL allows a client to navigate through a type graph defined by a server. More on this later.

 

Types

 

GraphQL is strongly typed – you define the exact structure of what can be returned. For example,

 

 

type Product {
    id: Int
    sku: String
    title: String
    description: String
    . . .
}

 

In the case of Magento, types would be generated automatically from the set of available attributes for entities such as “Product”, or from the return type of service contracts.

 

Interoperability

 

The simplicity of the GraphQL type system makes it easier to integrate with a wider range of programming languages. It also aligns well with existing Magento 2 service contracts. This makes it straightforward to add GraphQL support to Magento 2 alongside REST and SOAP.

 

There are several client-side JavaScript libraries available for integrating GraphQL with JavaScript rendering frameworks. For example, Facebook backs Relay for use with React. There is also Apollo. These libraries have various features such as automatic client side caching of data.

 

Data Retrieval

 

GraphQL supports concepts of both read request (queries to read data) and write requests (mutations allow inserting/updating/deleting database contents).

 

When you specify a query to retrieve data, you must also specify what fields you want to return. This aligns well with the new proposed Magento query API (a new persistence layer for Magento to separate database access from business logic), where we want to minimize data to retrieve and transfer.

 

 

{
    product(sku: “MJ01”) {
        sku
        title
    }
}

 

 

Data is returned in JSON format.

 

 

{
    “data”: {
        “product”: {
            “sku”: “MJ01”,
            “title”: “Mens Jacket in leather”
        }
    }
}

 

 

Note that fields map on to function calls. For example, the “sku” field above may be a call to a simple getter function of a product record fetched from the database. But, fields can also invoke any defined code, such as a “relatedProducts” field returning an array of product information about related products. This allows a query to fetch data from multiple nodes in a graph of information in a single query. It is this sort of graph navigation that makes GraphQL more powerful than REST.

 

 

{
    product(sku: “MJ01”) {
        title
        relatedProducts {
            sku
            title
        }
    }
}

 

 

Asking for specific data fields may feel burdensome initially, but there are multiple benefits:

 

  • It reduces network transfer, which can be important for mobile devices.
  • It allows for performance improvements, as the server knows exactly what fields are wanted, enabling query optimizations to be made. (In the case of Magento 2, SQL joins can be avoided for EAV attributes that the client does not need returned.)
  • It allows for schema evolution. New fields (including new linkages between types) can be added without breaking existing applications. Old fields can be removed when no old clients request that field. (GraphQL also supports the concept of marking fields as deprecated for documentation purposes.)

 

Request Batching for Performance

 

GraphQL makes multiple resources or services available from a single endpoint (URL), unlike REST, which uses a separate URL per resource. One request can include multiple queries, if desired, with an aliasing scheme to avoid name collisions. This is useful to share the per HTTP request overhead with several requests.

 

 

{
    p1: product(sku: “MJ01”) {
        sku
        title
    }
    p2: product(sku: “MJ02”) {
        description
    }
}

 

 

In the case of Magento, this would avoid the HTTP and PHP startup overhead per REST call that is currently incurred.

 

Introspection

 

Having a well-defined type system has another benefit. GraphQL endpoints publish their schema allowing interactive clients such as GraphiQL (pronounced “graphical”) to query what a site supports. This includes documentation of all data structures and methods that are available. GraphIQL is available as a Chrome extension, making it straightforward to install and use. With Magento, the GraphQL schema would most likely be dynamically generated from the available service contracts published by loaded modules and entity attribute definitions, allowing developers to determine exactly what a specific Magento site supports.

 

Other Considerations

 

GraphQL has some issues that need further exploration.

 

  • With the performance improvements of HTTP/2, the benefits of merging HTTP requests may be less pronounced.
  • GraphQL supports both GET and POST requests, however merging multiple requests or using POST for sending queries can make caching harder with technologies such as Varnish.
  • Web Application Firewall (WAF) rules can be harder to protect against attacks as it may be necessary to parse the contents of a POST request rather than blocking particular REST URLs.

 

These issues are not necessarily blockers, but need further investigation.

 

Conclusion

 

To wrap up this post, it is not yet definite that Magento 2 will adopt GraphQL. However, there are a number of benefits that make it worthwhile considering, especially considering it can be glued on top of existing service contracts. GraphQL is being reviewed at present to see if it would help the Progressive Web App (PWA) initiative currently underway (which includes using JavaScript frameworks such as React, Angular, Vue, or Polymer to build Magento store fronts). If you have used GraphQL or have an opinion on whether you think GraphQL is a good fit for Magento, please leave a comment!

 

 

11 Comments
lars_roettig
Core Maintainer

Hi,  

GraphQL is really powerful API Language and has for me a good market share.

It will help to implement or the use API faster than before.

I think it is good to ask the Community in some survey which part API part is used most time.

And start with a basic support for CE and EE.

It can b a good project for the Community Engineering Team or Magento Core Contribution Weekend.

 

Good practical use cases:

  • Backend Magento -> Frontend App (iOS, Android) 
  • 3D Planner for a Living Room -> use Magento Checkout and Backend for the Products 

 

best regards 

 

Lars Roettig

 

kalenjordan
M1 Certified

Sounds suuuper interesting! Having a little bit of a hard time wrapping my head around it but I'll chalk that up to me just not being very smart! Smiley Happy

 

One thing I'll say is from a quick read of the github api announcement (https://githubengineering.com/the-github-graphql-api/) it was really easy to grok from looking at the examples they gave. You can see how that would be a real world use case that would require lots of different REST calls and how combining it into one GraphQL call is very simple and elegant.

 

I wonder if you could throw out a few example use cases maybe a bit more fleshed out than the product sku one that this would be good for?

lars_roettig
Core Maintainer

Hi Kalen,

GraphQL is not a unicorn but can be help full for:

  • Implement fast API GraphQL uses always the smallest possible request.
  • GraphQL is already best practice for fetching data with Javascript Frameworks (Implementation cost can be lower) 
  • Bulk API can be improved with GraphQL query language for more flexibilities  

More Information you can here:

https://apihandyman.io/and-graphql-for-all-a-few-things-to-think-about-before-blindly-dumping-rest-f...

https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/

 

I looking forward to discusses the use cases ;-)

 

best regards

 

Lars  

akent99
Regular Contributor

Thanks Kalen. The "relatedProducts" query was one example above where it would have taken multiple REST calls. The "relatedProducts()" call would trigger a database query to fetch the products - you can just fetch any attribute of those other products. You can also fetch cross sells, AI merchandised products, best sellers, the customer's profile (if logged in) - whatever the developer decides to display on the product page. The interesting thing about the JavaScript libraries is (as I understand it) they can look at the React component tree and form the GraphQL query automatically based on what is going to be displayed - so one query that aligns exactly with what is displayed on the page. And it will only ask for the attributes needed by each component.

 

{
    product(query:{ ...search criteria encoded in JSON with AND/OR clauses ...}) {
        title
        description
        fullsizeImageUrls(width: 300) {
           url
           metadata
        }
        relatedProducts {
            sku
            title
            thumbnailUrl(width: 50)
        }
    }
    recentlyViewed {
        sku
        title
        thumbnailUrl(width: 30)
    }
    cart(first: 5) {
        sku
        quantity
    }
    user {
        name
    }
}

But I will try to put some more concrete examples together - hopefully with working prototype code (not full schema, but enough for people to play with).

Tom Robertshaw
Moderator

Like Kalen, I'm new to GraphQL so I have used this as an opportunity to read up. I can definitely see the benefits that this provides, most notably:

 

  • Reducing requests
  • Provides greater flexibility in requests

I think the second is the reason GraphQL could be very useful for our ecosystem because it facilitates headless Magento (official or otherwise) as well as being primed for further customisation. 

 

+1 for the Phil Sturgeon post that Lars mentioned: https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/. He also had an interesting tweet thread recently picking up on some bad comparisons that were being made: https://twitter.com/philsturgeon/status/891839638800916480

 

From what I understand, server-side caching gets more complicated as we lose the benefits of REST being upon HTTP and being able to use Varnish effectively. Because we now have greater request flexibility, we need to ensure that we have tools that are caching as many of the repeated objects between requests. As we're moving towards headless as well it means there's a greater responsibility on careful client side caching than we've had in the past. Phil has another good post on this topic (https://philsturgeon.uk/api/2017/01/26/graphql-vs-rest-caching/). 

 

The reigning in the benefits of reducing the number of HTTP calls for a secondary reason that Phil mentions is that typically it's the database that is the slow part as opposed to the HTTP part. This may be where the eav optimistions come in that Alan mentioned.

 

It's probably too early to consider how this might be implemented. By my reckoning, one way of doing this would still to use the REST APIs as service calls for the GraphQL server (hey, does that mean we get some HTTP caching back for free again?). I would imagine this would lead to less duplicated code. 

 

It's probably too early to consider edge cases but I think there will be times when client-side applications need to fall back to rest, e.g. if their are image uploads needed for customised products. 

 

A general piece of feedback on the APIs at the moment is that they could do with more human-friendly documentation and guides. This is going to be especially important if people need to declare each field that wish to receive. The current REST API docs make it very difficult to browse and understand how you might find products, add them to the basket and checkout (Referring to http://devdocs.magento.com/guides/v2.0/rest/list.html and http://devdocs.magento.com/swagger/index.html).  It's certainly comprehensive as a reference but not easy to get started. 

 

In summary, I think GraphQL would be incredibly useful for the ecosystem to have. Like anything, it'll bring it's own challenges. Some of which we can predict, some of which we won't, some of which will be due to the technology, and some of which will be due to the implementation 😉 

 

Oh and credit where credit is due: Shopify are doing this well: https://help.shopify.com/api/storefront-api. It's interesting to note that there GraphQL documentation is still much more difficult for the uninitiated to read than their traditional rest documentation though. 

 

Thanks for sharing, Alan.

davidumoh
Senior Member

I find these discussions really interesting. I have had discussions with my team members about experimenting with building a Magento store front that is a PWA, built with ReactJs and GraphQL. To see that the community is already thinking along these lines convinces me that, that is the way to go. I have done my research and have read some of the articles by Phil Sturgeon.

 

One of the things to be clear about is that GraphQL is neither a silver bullet nor a replacement for REST.

With that in mind, I think it should exist as an alternative to the REST and SOAP Apis, using a similar approach of taking advantage of Service contracts. 

 

I really think playing around implementing this will make things clearer.

Great work Alan.

pierreocci
New Contributor

Note: I am short on time today to write a full response but have been postponing this for weeks, so here is a first draft that I will edit in the next few days

This thread has very interesting thougts and resources to get started and learn what GraphQL is and how it could be useful! I would add https://edgecoders.com/graphql-deep-dive-the-cost-of-flexibility-ee50f131a83d to the list of must read articles.

 

I have been working on a project (Front-Commerce) for more than one year with the vision on bringing a better frontend stack to Magento, and after having started using Falcor for data fetching we ended up using GraphQL. So in short: yes it could bring a lot to Magento and must be a new way to access data since REST could still be useful for integration between systems.
Being near to a full production release, I would like to provide feedbacks from our experience building Front-Commerce hoping that it could help this choice in Magento.

 

First of all, I would recommend to make schema description separate from data resolution. This is something we will have to refactor in Front-Commerce because when we first started working on the implementation tooling was not mature enough. It will allow you to leverage community tooling (great for frontend developers) for instance to provide autocompletion in IDEs, or contract tests and might become part of the spec sooner or later. I am not sure about the `extend` keyword status but in our case it would also make trivial for plugins to patch the default schema (see Apollo GraphQL-Tools docs).

 

Secondly, I do not think generating them "almost automatically" from service API would make sense. Services are for different purposes and in REST these responsibilities are spread over different endpoints. In GraphQL they would be resolved at different locations in the Graph or could be accessed from the same resolver depending the context. Having a totally separated way to declare the schema definition, with convenient methods to implement resolvers would be much more useful and finally solve the flaws available in the REST API. Introducing GraphQL should (in my opinion) be a way to create an API easy to use by people who do not know the internals of Magento and thus be as simple as possible. 
Some examples: having one "cart" part of the graph with everything related to the user's current Cart (no more differences between `guest_carts` and `carts`), having an homogeneous way to represent prices expressively (to always know what the currency is, whether it includes taxes or not...)

 

Third, it would be great to start since the beginning to consider the use of DataLoaders (see dataloader-php implementation) and make it easy to add new ones. Maybe that could be taken into consideration as part of the new Database Layer API. DataLoaders make things very easy to reason about without wondering about premature optimization, and keeps data fetching encapsulated in a few locations. 

 

Fourth, in a "microservice" era I believe that if Magento wants to keep being relevant as a frontend system (and not being consumed only through its REST API) the GraphQL implementation must be designed in a way that makes it easy to acts as a middleware to other services. For instance allowing developers to create an extension exposing a Wordpress blog as part of the Graph (e.g replace "CmsPages" queries entirely, add a "relatedPosts" field to "ProductType"...). It could also be useful to make it easier to integrate third party solutions within Magento. That was our vision when building Front-Commerce and it seems to be shared by many, however I could understand it might be out of scope with the Magento product vision.

 

Finally, regarding caching one must indeed be aware that application caching is the only solution. For this, DataLoaders are a solution but we are also thinking that with a proper caching on the underlying data source (either ResourceModels, Service API or REST APIs) it would be sufficient enough to provide good performances. Also, since the GraphQL endpoint would be consumed by a client who will fetch much more less data from the server and with technologies making it easier to communicate/animate the "loading state" to the end user, it might result in a better UX overall.

We might also investigate persisted queries for fine grained performance optimisation, but I am not sure about the status of this in the PHP ecosystem.

 

If it can help people to start taking a look at what it could be to query Magento using GraphQL, I would like to point to the FC public demo with an instance of GraphiQL to query our GraphQL endpoint. Here is an example of query involving *lots of* different M2 endpoints: https://t.co/Q2BnLbzQWH

aleron75
Magento Master

I'm new to GraphQL and I approached it just because of this thread.

 

As always happens, innovation brings issues; Phil Sturgeon has a biased point of view, he wrote books on REST APIs (that I love) and he can't help being "against" the change. 

 

But to me there is no change, GraphQL won't kill REST just as functional programming won't kill object-oriented.

 

As far as I've seen, the key difference of GraphQL compared to REST is the approach: GraphQL is declarative ("I want a final result structured like the one I'm showing you with this JSON template; up to you, system, how to retrieve it") whereas REST is imperative ("hello system, give me this resource and then this and this to obtain the final result I want").

 

Each solution has pros and cons and I think that the discussion should not be about GraphQL being generally better or worse than REST but about the possible benefits that Magento would have adopting GraphQL.

 

Cheers,

 

-Alessandro

akent99
Regular Contributor

Thanks @pierreocci, especially for the demo endpoint. Being able to explore the endpoint is very useful, as illustrated by your site.

 

For others reading this, try clicking the "Docs" button in the top right corner of the page.

 

With respect to caching, one aspect I wanted to call out is there is client caching and server side caching. The client view is to avoid asking for data it has already retrieved. This is frequently talked about in the context of Relay, Apollo, etc (these provide client side caching).

 

My concern however is more on the server side of caching (e.g. Varnish). The more multiple queries are bundled into a single request (e.g. if a single GraphQL requests retrieves products 1 and 2 and 3), the lower the probability another client will ask for exactly the same request (someone else may ask for products 1 and 2 which would be a cache miss if the query is used as the cache key). That results in a varnish cache miss. The greater the number of cache misses, the more CPU power would be required server side to support the same load.  (Obviously cache misses are slower than cache hits - but caching is not only about performance - it is also about system load.)

 

This needs experimentation. It may be that by more intelligent grouping (and not grouping) of GraphQL requests is required to achieve the best possible blend of performance and server load.

 

akent99
Regular Contributor

Oh, @kalenjordan, I came across some more use cases. For example, you might want to return the product titles for all products in the current user's wishlist.

 

Customer -> Wishlist -> WishlistItem -> Product -> description

 

With REST this would require multiple requests, but with GraphQL you can navigate between the different objects in one request. Same is true for the current cart (Customer -> Quote -> QuoteItem -> Product).

 

And some little vidoes if anyone missed them on twitter:

 

https://mobile.twitter.com/akent99/status/907499904523681792/photo/1

 

https://mobile.twitter.com/akent99/status/907499713519169536/photo/1

 

kalenjordan
M1 Certified

Nice thanks @akent99! Perfect example.