Reindex catalog_category_product stalls

[GoatHater]

We are running Magento Enterprise 1.14 and have been running it live for a couple weeks.  A couple days ago, the catalog_category_product didn't complete.  Upon checking it, it seems as though its not even trying to run.  There is no error, the reindexer just seems to stall with no cpu or mem usage showing in top, but it is an active process in ps -aux. We have tried deleting the lock file, giving the lock folder 777, restarting the server, restarting the database server, all to no avail.  We have made no code changes (outside of themes)  and are at a loss as to where to go with this.

[GoatHater]

Upon further investigation, it seems as though our catalog is too large (750k products) and the indexer's sql queries eventually end up in a timeout and the whole thing goes silent.  Any recommendations?

[GoatHater · Accepted solution]

Our issue actually had nothing to do with Magento or the indexer; it was a mysql configuration issue.  The character_set_system was utf8 but all other char sets (client/connection etc) were latin1.  Setting everything as utf8 solved the issue.