It's not a strange question at all and, I suspect, one that has been hotly debated over the years.
I don't personally consider Barnabas sexy, but I can understand why some people do. First of all, he has more than his share of class and his voice is deep, his way of speaking elegant in an almost British, clipped sort of way. Add to that his brooding, tortured gaze and you have all the elements of an intriguing, mysterious man. As for looks, Barnabas isn't handsome the modern sense, but with his hair a bit softer and the dark circles from under his eyes removed, he is certainly above average in looks. Like many gentlemen I find attractive, his face has character.
And I would call Barn's eyes deep-set, not bug-eyed.
By contrast, someone like Joe Haskell (and maybe Jeff Clark and Tom Jennings, too) are handsome, but I find their mugs rather bland and unintriguing compared to Barnabas'. (My apologies to all Crothers and Briscoe fans!)