Agreed with the other posters. He can drive it, even be the only driver, but the primary use of the car is for your benefit. My husband is the main (pretty much only) driver of our motability car, he’s also my carer. The agreement that you make with Motability is that the car is provided for the use of the disabled person and the majority of the use is for that. Therefore your husband would be able to use it for your use, also for bits and pieces that are not primarily for your use (eg my husband uses the car now and then to visit a friend or to go to a specific shop that isn’t for my benefit) but he wouldn’t be able to use it to go to work.
It’s a brilliant scheme, I still can’t get my head around how it’s possible to exchange your DLA/PIP for a car with insurance, servicing etc included.