Probably mor ework will be needed but I guess you can have it with two approaches.
I guess the way vendors will upload the prodcuts will be relevant to choose the approach.
You will allow those vendors to use the Backend or how do you think that process will work?
If products are being uploaded always by you you can use a single tree and disable categories per website. But if vendores will be uploading products you will need some advanced permissions/restriction to allow only those vendors to upload products only into the assigned websites (and per wesbite, category tree too).
Maybe if you can answer the first question we can help to get a better answer.
Also, maybe checking the documentation again can help you to be sure about how to build and configure your store and catalog: https://docs.magento.com/m2/ce/user_guide/stores/websites-stores-views.html
(my 2 cents)