Yahoo is considered a spam IP?!?

I've had Yahoo as a mail service for a couple years now. Occasionally, and recently VERY FREQUENTLY, my emails have triggered the spam filters of people that are on my contacts list, and I get a Mailer-Daemon notice - In my Spam box!!! - indicating the send was blocked because the other IP has it on a watched list. These people are on my contacts list, and I am on theirs. Does anyone have an idea about what's going on, and how can I fix it...other than leaving Yahoo's mail service?