This behaviour is very strange in Magento 2. I can actually add only a Dropdown type attribute to a configurable product. Is this a bug or what? If there are Yes/No attributes available, I think it should be possible.
I have reported it in the issue tracker: https://github.com/magento/magento2/issues/3308