Your profile isn't set to private for CM users, just in case you thought it was. I can see it. As for non CM users, they can still see the first box on your profile, with your profile picture. This means they can send you a message but not see anything beyond that box. At least I think so. Try logging out and looking at your profile.
I just had this happen to me, too. When I clicked the person's name it came up that they were no longer a member. The message was from a woman saying she was "interested" in me and gave me a yahoo email addy to contact her at. I deleted it.
Also, setting your profile to private, it will still show to your friends. Even so anyone who can see your screen name can send you a private message. You should send anything offensive to cafemom so they can track the IP information.