Is changing cat litter bad for you? I mean the dust ...could it pose a problem if inhaled...?

Also I'm thinking it could be bad for the eyes too. I am always wary of changing the litter (I know I have to!!!) but when I do I wonder how much of the dust actually flies up that the eye cannot see....