The recently recently technical guidelines include this statement:
2.7. All non-public properties and methods SHOULD be private.
Can somebody from Magento clarify why this is the case?
I'm concerned about the implications if this ends up being applied consistently. While plugins and other forms of overrides are ideal when possible, there are cases where core functionality needs to be overridden, and there is no way to do it other than to overload the class using a di.xml <preference> (for example, when the functionality is in a non-public method). In these cases, it seems the best approach is to inherit from the original class so that you can avoid repeating parts of the code you aren't aiming to change. By making class members private, however, it is impossible to do this.
If the Magento core team sticks to this "private" rule, what is the recommendation for overriding classes in these cases? Copypasta seems to be the only solution, but that seems like a regression from Magento 1, not a step forward.
Inheriting from a class in general is a bad pattern (yes, there are always exceptions). We are actively working to eliminate that pattern as much as possible. It causes all sorts of upgrade and backwards compatibility problems later. The general goal is to have well defined APIs that we guarantee compatibility on, obeying Sem Ver rules. Otherwise extensions and sites can break during patches. The only way to have patches safe to apply is to have such restrictions.
If there are particular classes you need plugins on feel free to submit a request to make things in that area more public. We would rather consciously expose APIs and hook in points rather than have people put plugins in places that can result in patches breaking the site.