I think it’s safe to say, whatever symptoms you have at any point after contracting ms could be ms related. It’s also very difficult to admit when you can’t do something such as driving safely, since it is such a huge benefit to your independence and freedom.
I gave up driving June 2011, I had to sell my beloved car and at the time it was a hard and painful decision. I loved that car and I loved driving, but it came to a point where my ability to drive wasn’t safe enough to be on the road anymore.
If you are having trouble seeing, either through focus, depth perception or foreign objects obscuring/distracting your vision, reporting this to a doctor would likely have you suspended from driving. It could be tiredness, it could be optic neuritis. It’s impossible to tell but an eye test is a good start at least.
It’s your decision and I can’t advise you based purely on what you’ve said here, but you should be aware that you won’t want to give up driving when you should. It’s also worth remembering if you leave it too late to make such a decision, the risk of having an accident and it impacting the life of other road users increases every time you are on the road.
Speak to a doctor, discuss your symptoms, your neurologist would be even better. If you don’t want to discuss it with them for fear of them stopping you driving, that would be an indication that you know what you should do. If you are confident that your symptoms do not make it unsafe for you to drive, the doctor or neuro should share that opinion.