I wouldn't go as far to say those who hide it are all the degenerate type. Seems like the guys that are always talking about it are the ones who have the real problem... parlaying and teasing left and right, always betting the sucker bet, etc...
Simply put, there's a stigma attached to gambling and gamblers in general because of the actions of those truly addicted to it. Are there those that can gamble and do so within their means? Certainly... But the ones most people see are those that are living paycheck to paycheck, throwing all of it on the big game.
I know personally that when my girlfriend of the time didn't think it was that big a deal that she mentioned to her father that I gambled on sports that I was extremely embarassed. Do I see a problem in it? No. But I'd go out on a limb and say that most people do.