Whether drug side effects are “acceptable” is not a fixed yes or no answer, but depends very much on the severity of the problem they are supposed to treat.
Assuming you are not exaggerating, and you literally couldn’t stay awake for 45 minutes for a VEP, or even for five minutes watching TV, this is severely interfering with everyday life, and (in my opinion) worth risking a few side-effects for.
I’ve not heard of ANY really severe or life-threatening side effects with the two main anti-fatigue drugs, Amantadine and Modafinil. The latter is getting quite hard to get hold of for new prescriptions, although most people who were already on it have been allowed to stay on it. But Amantadine is still on the approved list for MS fatigue.
So the issue is not simply whether it has side effects, but whether they outweigh the benefits. Some people get few or no side effects. But if they’re unlikely to be life-changing or life-threatening if you did, and would simply go away again if you stopped the drug, it seems excessively cautious to say it’s not even worth trying.
Understandably, most anti-fatigue drugs are some sort of stimulant, and therefore contra-indicated if you suffer from anxiety - which I do, and have therefore never been prescribed them. But if there’s nothing like that that makes them unsuitable for you personally, I don’t see why you shouldn’t be allowed to try.