New Backend integration

Hi All,
Thanks for this great framework.
I am trying to integrate VSF to an python based backend platform. For that I have follow the https://github.com/DivanteLtd/vue-storefront-integration-boilerplate , and I using this python package https://github.com/elastic/elasticsearch-py to insert data into the elasticsearch engine.I works fine for the products but I am still struggling to manage to integrate categories and attributes.
Here is a sample of what I have integrated to eleasticsearch for attribute.

"hits": [
      {
        "_index": "vue_storefront_1",
        "_type": "attribute",
        "_id": "10",
        "_score": 1,
        "_source": {
          "id": 10,
          "attribute_id": 10,
          "options": {
            "value": "10*10*50"
          }
        }
      },

But it doesn’t get shown in the product detail.
Here is my product

"_index": "vue_storefront_1",
        "_type": "product",
        "_id": "4",
        "_score": 1,
        "_source": {
          "id": 4,
          "sku": "ambiance-mer-4",
          "name": "Ambiance mer",
          "price": 1570,
          "priceTax": 1570,
          "priceInclTax": 1570,
          "status": 1,
          "visibility": 4,
          "type_id": "configurable",
          "created_at": "2019-03-12T20:41:00.357121",
          "updated_at": "2019-03-15T16:09:30.351645",
          "category": [
            {
              "category_id": 3,
              "name": "Ile-d'Yeu"
            }
          ],
          "description": "<p>Cet étoile nommé Ambiance mer est .....</p>",
          "image": "/website/image/product.template/4/image",
          "small_image": "/website/image/product.template/4/image",
          "thumbnail": "/website/image/product.template/4/image",
          "media_gallery": [
            {
              "image": "/website/image/product.template/4/image"
            },
            {
              "image": "/website/image/product.image/1/image"
            }
          ],
          "options_container": "container2",
          "required_options": 0,
          "has_options": true,
          "url_key": "ambiance-mer-4",
          "tax_class_id": 0,
          "category_ids": [
            3
          ],
          "special_price": 0,
          "stock": {
            "is_in_stock": true
          },
          "sgn": ""
        }
      },

The response is missing some fields that should match magento fields more. Attaching some working result from ES under /catalog/vue_storefront_catalog/attribute/_search:

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 3,
        "max_score": 0,
        "hits": [
            {
                "_index": "vue_storefront_catalog_52",
                "_type": "attribute",
                "_id": "900",
                "_score": 0,
                "_source": {
                    "entity_type_id": 4,
                    "attribute_code": "size",
                    "frontend_input": "select",
                    "frontend_label": "Size",
                    "attribute_id": 900,
                    "id": 900,
                    "is_comparable": true,
                    "is_filterable": true,
                    "is_filterable_in_search": true,
                    "is_searchable": true,
                    "options": [
                        {
                            "value": 1,
                            "label": "39"
                        },
                        {
                            "value": 2,
                            "label": "40"
                        },
                        {
                            "value": 3,
                            "label": "41"
                        },
                        {
                            "value": 4,
                            "label": "42"
                        },
                        {
                            "value": 5,
                            "label": "43"
                        },
                        {
                            "value": 6,
                            "label": "44"
                        },
                        {
                            "value": 7,
                            "label": "45"
                        }
                    ]
                }
            },
            {
                "_index": "vue_storefront_catalog_52",
                "_type": "attribute",
                "_id": "901",
                "_score": 0,
                "_source": {
                    "entity_type_id": 4,
                    "attribute_code": "color",
                    "frontend_input": "select",
                    "frontend_label": "Color",
                    "attribute_id": 901,
                    "id": 901,
                    "is_comparable": true,
                    "is_filterable": true,
                    "is_filterable_in_search": true,
                    "is_searchable": true,
                    "options": [
                        {
                            "value": 7,
                            "label": "Pink"
                        },
                        {
                            "value": 8,
                            "label": "Gold"
                        },
                        {
                            "value": 35,
                            "label": "Silver"
                        },
                        {
                            "value": 36,
                            "label": "Bronze"
                        },
                        {
                            "value": 37,
                            "label": "Taupe"
                        }
                    ]
                }
            }
        ]
    }
}

Thanks , Where can I find the correct request format of data ?
And how to affect a particular attribute for a product ?

this is what I get after following Required API specification
Once my app exposes the api, I’m using the vue-storefront-api to connect to that exposed API, and vue-storefront-api pushes the data into elasticsearch in the correct format.
once the ES has the data in the correct format, the /catalog endpoint (also descrbed in the API specification) fetches the data from the vue-storefront-api ES instance and returns the output that vue-storefront required and uses.

It is the vue-storefront-api which pushes the data into elasticsearch and not the node-app here https://github.com/DivanteLtd/vue-storefront-integration-boilerplate/tree/master/2.%20Use%20node-app%20to%20import%20the%20data/node-app .?

You’re right! if we want to follow the boilerplate repository to the letter, node-app is the way to go.

I kind of got used to say vue-storefront-api about node-app, because it’s also node-app/vue-storefront-api that converts API data (in our case exposed API data) into the format vue-storefront requires :slight_smile:

Yes, In my particular case , I don’t use node-app , because , my backend app make an update into the ES each time any record has been updated.

In the demo , I am wandering how to add these attributes (14 ) on the ES part ?

your attributes ES index should contain ALL the attributes you’re going to use with products.
then, your products ES index should have those as normal key, like this:

{
    "took": 5,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 5886,
        "max_score": 1,
        "hits": [
            {
                "_index": "vue_storefront_catalog_52",
                "_type": "product",
                "_id": "29",
                "_score": 1,
                "_source": {
                    "entity_type_id": 4,
                    "attribute_set_id": 0,
                    "type_id": "configurable",
                    "sku": "029",
                    "has_options": true,
                    "color": 9,
                    "size": null,
                    "status": 1,
                    "visibility": 4,
                    "tax_class_id": 1,
                    "description": "Außergewöhnlicher Detailreichtum, damit alles in Erinnerung bleibt Die Kamera bietet außergewöhnliche Bildqualität mit hohem Detailreichtum. Ihre 20 Megapixel ermöglichen eine flexible Motivwahl, das Zuschneiden der Aufnahmen und brillante Drucke im Posterformat. Dank des optischen Bildstabilisators werden Kamerawackler, die eine Aufnahme ruinieren könnten, ausgeglichen. Diese Kamera kompensiert derartige Verwacklungen und ermöglicht immer gestochen scharfe und detailreiche Aufnahmen. Großartige Movies aufnehmen macht Spaß und ist ein Kinderspiel. Auf Knopfdruck zeichnen Sie ganz einfach HD-Movies (720p) auf – selbst aus der Bewegung und bei vollem Zoom.",
                    "meta_keyword": "Sony,Entertainment Electronics",
                    "short_description": "Außergewöhnlicher Detailreichtum, damit alles in Erinnerung bleibt Die Kamera bietet außergewöhnliche Bildqualität mit hohem Detailreichtum. Ihre 20 Megapixel ermöglichen eine flexible Motivwahl, das Zuschneiden der Aufnahmen und brillante Drucke im Posterformat. Dank des optischen Bildstabilisators werden Kamerawackler, die eine Aufnahme ruinieren könnten, ausgeglichen. Diese Kamera kompensiert derartige Verwacklungen und ermöglicht immer gestochen scharfe und detailreiche Aufnahmen. Großartige Movies aufnehmen macht Spaß und ist ein Kinderspiel. Auf Knopfdruck zeichnen Sie ganz einfach HD-Movies (720p) auf – selbst aus der Bewegung und bei vollem Zoom.",
                    "name": null,
                    "image": "//images.icecat.biz/img/norm/high/26976109-Sony.jpg",
                    "small_image": "//images.icecat.biz/img/norm/medium/26976109-Sony.jpg",
                    "thumbnail": "//images.icecat.biz/img/norm/medium/26976109-Sony.jpg",
                    "url_key": "sony-cyber-shot-dsc-wx500-029",
                    "url_path": "sony-cyber-shot-dsc-wx500-029",
                    "weight": 1,
                    "price": 38640,
                    "final_price": 38640,
                    "price_incl_tax": 38640,
                    "special_price": null,
                    "special_price_incl_tax": null,
                    "product_links": [],
                    "stock": {
                        "is_in_stock": true
                    },
                    "msrp": null,
                    "is_saleable": true,
                    "id": 29,
                    "category": [
                        {
                            "category_id": 4,
                            "name": "Digitale Kameras"
                        },
                        {
                            "category_id": 2,
                            "name": "Kameras & Camcorders"
                        }
                    ],
                    "is_in_stock": true,
                    "created_at": "2019-03-19 18:50:48",
                    "updated_at": "2019-03-19 18:50:48",
                    "media_gallery": [
                        {
                            "id": 29,
                            "media_type": "image",
                            "label": null,
                            "disabled": false,
                            "position": 1,
                            "types": [
                                "image"
                            ],
                            "file": "",
                            "image": "//images.icecat.biz/img/norm/high/26976109-Sony.jpg"
                        }
                    ],
                    "material": "Hemp, Organic Cotton, Spandex",
                    "configurable_children": [
                        {
                            "entity_type_id": 4,
                            "attribute_set_id": 0,
                            "sku": "029_26976109",
                            "has_options": false,
                            "color": 9,
                            "size": null,
                            "gender": null,
                            "status": 1,
                            "visibility": 4,
                            "tax_class_id": 0,
                            "name": null,
                            "image": "//images.icecat.biz/img/norm/high/26976109-Sony.jpg",
                            "small_image": "//images.icecat.biz/img/norm/medium/26976109-Sony.jpg",
                            "thumbnail": "//images.icecat.biz/img/norm/medium/26976109-Sony.jpg",
                            "url_key": "/de/sony-cyber-shot-dsc-wx500-029",
                            "url_path": "/de/sony-cyber-shot-dsc-wx500-029",
                            "weight": 1,
                            "price": 38640,
                            "price_incl_tax": 38640,
                            "special_price": null,
                            "special_price_incl_tax": null,
                            "msrp": null,
                            "is_saleable": true,
                            "id": 29,
                            "is_in_stock": true,
                            "created_at": "2019-03-19 18:50:48",
                            "updated_at": "2019-03-19 18:50:48",
                        }
                    ],
                    "configurable_options": [
                        {
                            "id": 90,
                            "label": "Size",
                            "use_default": 0,
                            "position": 0,
                            "attribute_id": 900,
                            "attribute_code": "size",
                            "frontend_label": "Size",
                            "store_label": "Size",
                            "values": []
                        },
                        {
                            "id": 91,
                            "label": "Color",
                            "use_default": 0,
                            "position": 0,
                            "attribute_id": 901,
                            "attribute_code": "color",
                            "frontend_label": "Color",
                            "store_label": "Color",
                            "values": [
                                {
                                    "product_super_attribute_id": 91,
                                    "value_index": 9,
                                    "label": "WeiĂź",
                                    "default_label": "WeiĂź",
                                    "store_label": "WeiĂź",
                                    "is_percent": 0,
                                    "pricing_value": null,
                                    "use_default_value": true,
                                    "order": 0
                                }
                            ]
                        }
                    ],
                    "size_options": [],
                    "color_options": [
                        9
                    ],
                    "category_ids": [
                        4,
                        2
                    ]
                }
            }
        ]
    }
}
1 Like

I saw here that the category is set twice is that ok ?
Could you please give an example of categories ES response.
Thanks

I don’t understand your concern about categories being set twice, cannot see it anywhere.

As for the elasticsearch responses Please setup:


and query the API yourself to get the correct response format.

Hi =, I see here that for a product , we set the categories fields twice

I believe category ids is simply meant for faster category lookups (without traversing through category set to find out if it matches the id).

Thanks for your reply .
For my category I have this , and it doesn’t work for me

"_index": "vue_storefront_1",
        "_type": "category",
        "_id": "10",
        "_score": 1,
        "_source": {
          "entity_type_id": 3,
          "attribute_set_id": 0,
          "id": 10,
          "url_key": "huile-sur-toile-10",
          "url_path": "c/huile-sur-toile-10",
          "slug": "huile-sur-toile-10",
          "parent_id": 0,
          "name": "Huile sur toile",
          "is_active": true,
          "position": 2,
          "level": 2,
          "product_count": 0,
          "children_data": [
            {
              "url_key": "huile-sur-toile-a-l-etranger-6",
              "slug": "huile-sur-toile-a-l-etranger-6",
              "id": 6,
              "parent_id": 10,
              "name": "A l'Ă©tranger",
              "is_active": true,
              "position": 1,
              "level": 3,
              "product_count": 1,
              "children_count": 0,
              "created_at": "2019-03-03T09:37:29.022375",
              "updated_at": "2019-03-13T09:29:21.097097"
            },
            {
              "url_key": "huile-sur-toile-eglise-11",
              "slug": "huile-sur-toile-eglise-11",
              "id": 11,
              "parent_id": 10,
              "name": "Eglise",
              "is_active": true,
              "position": 2,
              "level": 3,
              "product_count": 1,
              "children_count": 0,
              "created_at": "2019-03-13T09:14:44.652401",
              "updated_at": "2019-03-13T09:29:16.543941"
            },

@khchine5 I’m afraid I first need to point you at:


Please post precise questions when you have them. Unfortunately, I cannot read your mind, nor I have direct access to your code.

Sorry @jan for my bad formulation of my question.
My backend (is not magneto ) insert in elasticsearch the category entity in this format

"_index": "vue_storefront_1",
        "_type": "category",
        "_id": "10",
        "_score": 1,
        "_source": {
          "entity_type_id": 3,
          "attribute_set_id": 0,
          "id": 10,
          "url_key": "huile-sur-toile-10",
          "url_path": "c/huile-sur-toile-10",
          "slug": "huile-sur-toile-10",
          "parent_id": 0,
          "name": "Huile sur toile",
          "is_active": true,
          "position": 2,
          "level": 2,
          "product_count": 0,
          "children_data": [
            {
              "url_key": "huile-sur-toile-a-l-etranger-6",
              "slug": "huile-sur-toile-a-l-etranger-6",
              "id": 6,
              "parent_id": 10,
              "name": "A l'Ă©tranger",
              "is_active": true,
              "position": 1,
              "level": 3,
              "product_count": 1,
              "children_count": 0,
              "created_at": "2019-03-03T09:37:29.022375",
              "updated_at": "2019-03-13T09:29:21.097097"
            },
            {
              "url_key": "huile-sur-toile-eglise-11",
              "slug": "huile-sur-toile-eglise-11",
              "id": 11,
              "parent_id": 10,
              "name": "Eglise",
              "is_active": true,
              "position": 2,
              "level": 3,
              "product_count": 1,
              "children_count": 0,
              "created_at": "2019-03-13T09:14:44.652401",
              "updated_at": "2019-03-13T09:29:16.543941"
            },

With this data even it is correct regarding the required spec here , my storefont don’t show me the categories in the menu nor in the product detail view.
Have you any idea why I am getting this error ? Is my format not correct ?

@khchine5 spec you are referring to (integration boilerplate) explains how to build Magento API equivalent. Integration boilerplate explains how to build API source which vue-storefront-api can consume.

From what you write is seems like you’re building the vue-storefront-api equivalent (because you’re inserting data into ES, which is what the VSAPI does). The only difference is you also make your program read the data directly from DB. This seems like joining vue-storefront-api with magento API together and I’m afraid there’s not much we can do here just by analyzing inserted data.

What you should do:

  • setup demo vuestorefront instance
  • install demo data using yarn installer
  • compare YOUR custom elasticsearch index against demo data in elasticsearch
  • fix your structure accordingly

There’s no doc on how to build vue-storefront-api equivalent, so there’s nothing I can guide you through.

1 Like

Thanks for your replay @jan.
I am referring to the integration boilerplate , but I will not use the node-app to make the import to ES.
VSAPI will use ES to consomme data in a particular format regardless how this data has been inserted in ES (using node-app or other tools). My question is that about the format of categories entites that VSAPI requires from ES.
Thanks again.

There’s no integration boilerplate for writing vue-storefront-api equivalent I’m afraid and your safest bet is just running an official vue-storefront instance somewhere and compare with yours.
Here’s my ES result from vue_storefront_catalog/category/_search:

{
    "took": 4,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 33,
        "max_score": 1,
        "hits": [
            {
                "_index": "vue_storefront_catalog_52",
                "_type": "category",
                "_id": "22",
                "_score": 1,
                "_source": {
                    "parent_id": 19,
                    "position": 0,
                    "level": 5,
                    "include_in_menu": 1,
                    "name": "Category Name",
                    "is_active": true,
                    "url_key": "category_name",
                    "url_path": "category_name",
                    "id": 22,
                    "created_at": "2019-03-28 17:48:28",
                    "updated_at": "2019-03-28 17:48:28",
                    "related_gender": "",
                    "children_data": [],
                    "children_count": 0
                }
            },
            (...more categories...)
       ]
    }
}

hi, @khchine5, actually i want to do the same - to integrate vue storefront with python backend, is it possible to see your work? Maybe it’s open source? Thanks.