Is there a way to remove a product from sitemap.xml via xml code?
If I set Visibility to Not Visible individually product is not added to sitemap.xml but the product is not visible on frontend.
I do not believe there's a built in way to hide a product from the sitemap without also removing it from the site. You will probably need some custom code to modify sitemap generation to exclude products specifically or through some shared attribute value. There may be an extension to help you if you don't mind purchasing and experimenting with one of those. For example: https://www.scommerce-mage.com/magento-google-site-map-exclusion.html (I haven't used this extension vendor so this isn't a recommendation).
There are a few options on the marketplace that look they might be able to do it: https://marketplace.magento.com/catalogsearch/result/?q=sitemap%20exclude