It is like anything else
Anything done in moderation is OK. I like my Big Mac Fries, or White Castle Murder Burgers, but not every day. Twice a month I indulge in fast/junk food. The rest of time I eat a healthy diet. I know people who eat fast food 7 days a week. I would not dare tell them that is why they are over weight, or are sick every month.
Not Pointing Fingers The danger in hobbying is when you go over the top. Examples - You fall in love with a provider, you believe that you can change her. Start painting a picture in your mind of having a long term one to one relationship with her - making plans in your head of marriage, children. I have read many guys who fell in deep love with a provider, to each there own, but that can not be healthy.
It is also easy to go over the top with our morals, crossing the line into homo-sexuality. If that has been your thing, G@d bless you, but to suddenly start seeing TV's, getting sensual messages from men, I believe that is also not healthy.
You can also start over top with SW's, who are for the most part are not clean and drug addicts. I know guys who pick them up daily, make out, go down on them, bbs, take them to hotel rooms and even back to there house. This behavior will only lead to illness. To take it one step further, many guys themselves started doing drugs with the girls, who never did it before. Hanging out with SW's on a steady basis will only lead to no good.
Then of course, how Kinky do you want to go? Having fists in your butt? Bondage? Getting pissed or shitted on? All very unhealthy behavior IMHO.