First couple of Google results mention romance and romantic relationships. But I have heard women refer to a man as a sweetheart and they didn't mean a romantic relationship. They meant a man who is generally nice, generous, respectful, appreciative.
I have become more and more aware of some sorts of feminine behavior and comment that are sweet. I mean it is sweet in the sense that I like it and want to be around it. Sure, I notice and appreciate nubile silhouettes, what is often referred to as a "good shape". However, I have been out of the baby-making phase for quite a while now. But I have learned that voices, word choice, speech emphasis, comment timing - all contribute to finding that a woman is a sweetheart.
Another aspect of my reaction to women is gesture and movement. I am often surprised by motions of the hand, head, body and legs that I immediately like and deeply know I would myself have not made. I don't see men do them. Voice, gesture, body language can all create a miasma of pleasant glow that makes female company a treasure