Magento2 Product Collection Observer only changes one page at a time

[RichardPK]

In ListProduct block Magento is using "initializeProductCollection()" to initialize the collection. 

While initializing it dispatches event "catalog_block_product_list_collection" as follows. 

$this->_eventManager->dispatch(
            'catalog_block_product_list_collection',
            ['collection' => $collection]
        );

Using the event catalog_block_product_list_collection I am trying to filter a list of products:

$_collection = $observer->getCollection();
$_collection->addAttributeToFilter('sku', ['nin' => $invalidSkus]);

however instead of limiting the category from 50 products to 49 products, it is limiting the page results of 12 products to 11 products. If I have a list of 12 invalid skus that all coincide to be on page 1, then page 1 returns no products.

How do I filter the entire product list not just the single page?

[RichardPK · Accepted solution]

The product IDs for this one category are coming from elasticsearch based on the page size defined in "Store > Settings > Configuration > Catalog > Catalog > Storefront > Products per Page on Grid Default Value". If you navigate to the next page, Magento requests the next 12 product IDs from Elasticsearch and these will be included in the SQL query that creates the collection.

This is how Magento has been designed, otherwise, if the query contains all the product that has been assigned to the category, there will be a performance impact.

What this means, is that the event `catalog_block_product_list_collection` is far too late to be adding or removing products to the list. The logic to do this must be placed where Elasticsearch is query is being built.

The solution is to create a plugin for the elasticsearch client and edit the query there:

`vendor/module/etc/di.xml`

<type name="Magento\Elasticsearch7\Model\Client\Elasticsearch">
    <plugin name="extend_elasticsearch" type="Vendor\Module\Plugin\Elasticsearch" sortOrder="1"/>
</type>

`vendor/module/Plugin/Elasticsearch.php`

namespace Vendor\Module\Plugin;
class Elasticsearch
{
    public function beforeQuery($subject,$query) {
        //Do not include product ids of the invalid skus in search result.
        $productIdsToExclude = [1,2,3];
        $query['body']['query']['bool']['must_not'] = [
            'ids' => [ 'values' => $productIdsToExclude]
        ];
        return [$query];
    }
}

[zexperto]

@RichardPK  you solution is correct and helped me, I will post here a solution to exclude two  skus in case someone need more help

in the below query we exclude two SKUs (simple-26 and simple-27)

 $query['body']['query'] = [
            "bool"=> [
                "must_not"=> [
                    [
                        "bool"=> [
                            "should" => [
                                [ "match" => [ "sku" => "simple-26" ] ],
                                [ "match" => [ "sku" => "simple-27" ] ]
                            ]
                        ]
                    ],
                ]
            ]
        ];

[monasasanic2b1]

---

@epayitonline wrote:

In ListProduct block Magento is using "initializeProductCollection()" to initialize the collection. 

While initializing it dispatches event "catalog_block_product_list_collection" as follows. 

$this->_eventManager->dispatch(
            'catalog_block_product_list_collection',
            ['collection' => $collection]
        );

Using the event catalog_block_product_list_collection I am trying to filter a list of products:

$_collection = $observer->getCollection();
$_collection->addAttributeToFilter('sku', ['nin' => $invalidSkus]);

however instead of limiting the category from 50 products to 49 products, it is limiting the page results of 12 products to 11 products. If I have a list of 12 invalid skus that all coincide to be on page 1, then page 1 returns no products.

How do I filter the entire product list not just the single page?

---

I had this problem when trying to build an extension that affected the layered nav and I was curious if it's the same for you. Unfortunately I don't have a solution. Apparently the search module rewrites something in the catalog module but I don't know what. 

[imaphpdeveloper]

It helps. Furthermore, I would like to exclude some categories products not to list. How to replace the 'ids' with the category Id's?

$query['body']['query']['bool']['must_not'] = [
            'ids' => [ 'values' => $productIdsToExclude]
        ];

[senthil247a04e]

Your provided solution was successful in restricting the given products in product search and list pages. However, after enabling the "Out of Stock" option, it was observed that product restriction was not functioning as intended. Instead, all products, including those restricted, were displayed.

The desired outcome is to display both out of stock and non-restricted products in the product search and list pages.