Ah, but we do see SOME precedences of Barnabas having some affection for Angelique pre 1840---in 1795 he admits to her when he albeit reluctantly agrees to marry her that he "did care (for Angelique) long before he realized he loved Josette." Barnabas actually admits this. Also, while reluctant to marry Angelique, he did keep his promise and even defied his father to do so, and if you can see in the early marriage scenes before he finds out Angelique is a witch, Barnabas is quite solicitious of his wife and I think was willing to make an honest go of the marriage, though of course Angelique put the kibosh on that by her irrational jealousy of Vicki and her trying to frame her for witchcraft, which gradually leads to the end of her hard won marriage.
Barnabas also goes to Angelique for help time and time again, so he is drawn to her in some way, and I don't think it is always because she is the last resort, if he totally and irrecocably despised her, he would have done everything he could to avoid her. I think Barnabas feelings for Angelique wax and wane between loathing and some affection, Angelique's from sincere love to obsessive jealousy and anger.
He certainly comes to Angelique for help during the Leviathans, and is solicitous of her when she leaves Sky, but then almost immediately has to run out to warn Maggie about Nicholas, not a very gallant thing to do...
I don't maintain that Barnabas always loved Angelique, but rather that he GREW to love her in 1840 once she reclaimed her humanity, as perhaps he again saw the beautiful young girl that he had first been attracted to and by his own admission cared for on some level when he met her on Martinique..