[shilala]

It certainly wasn't a term of endearment during the Civil War. After the war, Yankee soldiers earned the respect of southern soldiers for the most part, and the word changed it's meaning dramatically. Before the fighting began, they literally thought that one southern boy could take 20 yanks because they were all girls.
Southerners strongly disliked (or downright hated) northerners at that time because of their lifestyles, industry, and supposed crudeness and lack of culture, and the term they used to express that was "Yankees" or "Yanks".
Granted, that's been watered way down over time, and who really cares about getting called a name? I sure don't. I like John's (neoflex) response. That's a good way to put it away if it needs putting away. A person can certainly tell if it's an insult.
But between friends, it's a harmless tease. I don't see anything wrong with it at all.