I have Simple products that have multiple images associated to the Configurable product. Everything is working well, except for the gallery: whenever I click on the swatch the gallery doesn't update, staying same.
As seen above, I picked purple(which is the associated simple product), yet it shows Configurable product's thumbnails.
I wanted to know if it is a bug, or a feature of CE 1.9.3(if it is I guess i should get an extension.)
Yes, this is functionality that should be there in 1.9.
Here's some documentation that I found on the topic: http://docs.magento.com/m1/ce/user_guide/catalog/product-configurable-swatch-dynamic-image.html is that exactly how you've configured the images?
Are you using the RWD theme or a theme based off of the official RWD theme where this feature is implemented? If you aren't, that could be one reason why it doesn't function that way.
If you are, then it sounds like a bug that's hard to give general guidance on. It would need a developer to investigate the cause of it.