I would have stuck to my guns and left him behind but maybe the long ride in the car will be good as maybe he will open up to you finally.
When I was in Jr High and High School and was a dirt bag when it came to girls but I always made sure to introduce myself to the parents especially the Fathers. Hell most of the parents loved me until I dumped their daughters a few weeks later but I learned early that if the parents liked you it made things easier especially being left alone with their daughters.

This is why even before we tried having kids I knew I would have a daughter first and can guarantee that the next one will be a girl too. "Karma works in strange ways

" Hell, I remember one girl I dated that I had to get rid of her because she was a complete psycho but it was tough because I actually enjoyed hanging out with her dad as we would often hang out in the garage and have a beer together and talk cars and actually stuck around too long because of it. Hopefully if he is as bad as you think it will be a short lived relationship. If not hopefully the ride will help open up the lines of communication between the two of you. Good luck.