Update Dawn to use the Cart Ajax API instead of undocumented endpoints

[bakura10]

Hi,

For a few months, we have been experiencing a ton of issues and regressions, happening at the Cart Ajax API level, sometimes on super basic features like "removing a line item from cart" which fails. We are have found today such a regression on a merchant store. The Cart Ajax API has been extremely unreliable to use lately.

Those issues are extremely frustrating to handle on our end because despite being Shopify issues, merchants say that they reached Shopify, that Shopify support tried on Dawn, and because it works on Dawn, then it must be a theme issue (and, of course, merchants will always tend to believe Shopify rather than us).

One thing that exhibits the problem is that Dawn itself uses undocumented APIs that appears to follow different paths on Shopify backend (and that seems to work while the documented API does not). If Dawn theme would use the documented API, it would help a lot to demonstrate to merchants that the issue is indeed due to Shopify and not the theme. Shopify eating its own food would also ensure that such cart issues are spotted and fixed faster.

For instance, when removing a product from the cart will do a POST Ajax request to the endpoint /cart/change, with a JSON payload:

However, this API is undocumented. The only documented one is /cart/change.js (notice the .js at the end): https://shopify.dev/docs/api/ajax/reference/cart#post-locale-cart-change-js

Same thing when changing line item quantity, where Dawn uses undocumented /cart/change with JSON payload instead of cart/change.js:

When adding to the cart, Dawn also uses an Ajax call to /cart/add instead of the documented cart/add.js endpoint (although, contrary to the line item removal, for this one Dawn uses a form data so it would be the same as submitting the form without Ajax ;but still, as soon as you're doing Ajax call Dawn should use the documented APIs for that work).

[resistorsoftware]

Did you ever ask yourself what the dotjs means at the end of the url? Are you sure there is a difference between a POST to /cart/change and /cart/change.js?

As an example of basic web services since a long time, it is entirely possible to add an extension to an endpoint, and that just tells the server how to format the output. In other words, if you add .js to the endpoint, the server formats your response knowing you're interested in JS responses. If you were to tack on change.html the server would know you're interested in HTML as your response.

So instead of thinking there is some deep-state crazy going on here, you might want to check. Does a POST to /cart/change with date XYZ produce the exact same response as one to /cart/change.js? If not, why not? Perhaps in one case your data payload is formatted correctly, perhaps not in the other?

As responsible web developers we go with these things. As a user of Shopify cart JS since inception, I do not think you are pointing out anything "undocumented" that would be directly causing you problems. Maybe you can point out something, but your current "effort" is not complete. It can be frustrating calling endpoints and not seeing the results you expect, but often this is due to minor technicalities like forgetting something, or formatting something slightly wrong.

[bakura10]

Yes, unfortunately the two endpoints are not alias. I remember in the past that I did some experiments and they are not an exact 1:1 for some very legacy reasons (unless it has changed compared to my tests in the past, of course). For instance for the add to cart API, the way errors are returned differ (the Ajax API the one with .js does block if you try to add more items to the cart than the inventory, while the non .js URL will not).

For the non .js API there are also some very odd quirks where you have to add a X-Requested-With header.

They are definitely differences between the two (even if super minor), that justify enough for dawn to use the documented endpoints! The simple fact that Shopify support uses Dawn as a reference to check if « it is a theme issue » and blame our theme if it works on Dawn, is an enough justification to align Dawn to only supports the documented URL. I mean, even in the case there would not have any difference, dont you feel Shopify should follow the documentation for their own code ?

Regarding the URL I don’t know in rails, but all the backend stacks I have worked (Zend, Expressive…) you specify the full name of the routes so if you define a route « cart/add » it won’t match « cart/add.js ». This might be different here but I don’t have access to their backend code to confirm :)

[bakura10]

the Ajax API for products are also not the same obviously (/product/handle returns the HTML while product/handle.js returns the data in JSON)

In the case someone at Shopify can confirm the behaviour is EXACTLY the same now for the Cart API and they are effectively alias for every possible combinations, then the doc should at least be updated to state that both url can work.

I honestly would prefer to use the non .js URL, but for having faced issues by using undocumented features in the past that suddenly broke, I will never take the risk unless this is written somewhere :/

[bakura10]

@resistorsoftware , I did some more testing and I can confirm the endpoints are indeed not exactly the same.

If you use /cart/change, you have to add two headers to be able to use JSON:

fetch(`/cart/change`, {
 headers: {
   'Content-Type': 'application/json',
   'Accept': 'application/json'
 }
});

Note that the documentation only refers to adding Content-Type header. While you might argue this is logical (the Accept header being what the client can understand as response), the simple fact that I had to check Dawn implementation to figure it out is not a good developer user experience.

However, when using the /cart/change.js, only the Content-Type header is required (as documenting), the endpoint returning JSON response all the time.

This therefore confirms that those are not 100% alias and that there might have other subtle differences. This is enough of a justification for me to align Dawn code to use the correct endpoint.

Additionally, during my funny experimentation with the cart API, I also figured out another bug when using key instead of line.

When using line, Shopify will return the items added or removed as part of the response, which is very useful to exactly knows what has changed in the cart:

However, when using the key, those hash are always empty:

This is yet another example that Shopify really did not properly tested the cart Ajax API with all those latest additions.

[resistorsoftware]

I agree you've discovered some facets of networking and making correct HTTP calls, but fundamentally, what I pointed out to you, is that indeed, the endpoint is the same. Same data in, same data out. What you are carefully documenting and discovering, is that there are certain conditions and criteria present when using these HTTP calls.

I will leave you with your discoveries, but for the record, there is no underlying actual differences in these calls. The service provided is consistent. The black box of the update endpoint does not function or behave different. It does require slightly altered calling signature setup work from the developer, but that is not the same issue.

[bakura10]

This does not remove the fact that this is NOT documented anywhere. So while you might be correct, this then needs to be clarified in the documentation. The fact that a different route name match another endpoint might be intuitive and logical for you; it was not for me (all the backend technologies I have used never did this assumption that adding a suffix to a route name would match to the exact same endpoint).

This does not remove neither all the bugs I have found in the API and that should be considered as critical issues, as the documented API does not work properly in a lot of situations.

[TRooDeveloper]

It gives null response of custom given element in /cart/change Payloads.
Please check SS