2.4.5 Import Issues

[salesgreat5d35]

Hi,

I'm trying to import Products. I'm using a valid file - same structure as previous and the products imported then - in November. I have CloudFlare in the mix, but I've got it on bypass for my IP, and I know that's working because CloudFlare is showing the activity.

There are over 500+ product attributes, but I'm not importing them in this file - I use additional files to do that, so it's only touching 15 custom product attributes.

The file is about 385K and has 2,500 records.

I'm hosted on Nexcess and they're examining every parameter, but seems like they're not seeing a conflict/issue.

Whenever I run an import (and I'd like to run 55,000 products, but have just taken a subset as mentioned above) to test, I get a 524 Timeout. This is a CloudFlare code for >100 seconds, however my understanding of Import is that it does things one record at a time so a timeout of 100 seconds shouldn't ever be reached. The file doesn't get flagged - i.e. there are no errors that the Import function shows before I kick it off.

Apparently, Nexcess sees no logged errors and nothing ever shows up on-screen to give me an idea of where to look - other than the 524 error.

Any ideas? Has anyone seen this?

I can't take CloudFlare out because we filter out all but US traffic - we only do business in the US, and for malicious access attempt safety as well.

Thanks in advance.

[jameswebdede01]

Hello @salesgreat5d35 

WebDesk Solution would like to give you a solution to support you.

It looks like you have configured CloudFlare, and because CloudFlare considered your large request a bot or dummy traffic, it’s blocking your IP. To fix this issue, you can allow your IP address in CloudFlare, and once it’s allowed, you can easily perform the large request, and your issues will be resolved.

[salesgreat5d35]

Hi James,

Thanks for the response. In this case, as mentioned below, I've set CloudFlare up to bypass my IP - I've checked their Security->Events screen and continue to see my IP show up in their log as bypassed, so I believe the solution must lie in another direction.

Even if they had sensed my traffic as something, it would create a Ray ID that would be viewable and specific to the nature of the problem, but this isn't the case. 

A Ray ID is created, but it's not viewable in their system when I search for that ID. Since it does pass the 100-second limit, for some reason, this is the problem. Magento is not processing the import properly - either it's chunking more than one record for import - presumably MANY - and this is causing a timeout, or there's some internal issue with Magento not processing each record properly - possibly due to the product attributes (again, only 15 customer attributes in this import job), although possibly for other reasons.

I'm not sure how else I can "turn off" CloudFlare - i've tried their developer mode, the bypass of the IP, etc. 

Thanks.