This appears to be your first experience dealing with a narcissist. This is how they roll. And, yea, it probably does eat him alive to think someone would not love his company and, by extension, him.
Hopefully he's the run-of-the-mill type. Because if he's a malignant narcissist he may not stop until he feels you have been properly punished for your lack of faith.
(I know this territory well; I've worked for several of them and had a malignant narcissist parent.)
@jackwilliambell sorry but no, not my first time dealing with a Vlad and wasn't even totally shocked about this Vlad doing this