Well, there's two main reasons:
1-They're part of the aldmeri dominion, so Nords have some resentment against them
2-What they say about them is true.
Don't misunderstand me, I'm not dissing them, i'm their beat defender here(seriously read around and you'll see). But Khajits have a greatly osculating personality and, actually, they don't even have a solid personality. So they may be really friendly and cut you a good deal, but when you then around they change their mind and pickpocket you.
Most of their decisions don't involve killing or stealing, but people don't want to risk it.
And for the fact that there's so many thieves, assassins, skooma dealers and trafficants that are Khajits is also very simple:
ASSASINS: they have a gift for being sneaky, so when they need money, this is always a viable and easy option
THIEVES: same as above
SKOOMA DEALERS: To Khajits, skooma isn't bad, so they have no problems selling it.
TRAFFICANTS: Khajit caravans can't enter cities and are constantly robbed. Imagine how hard it is to them to make a living. So they work as trafficants.
So you see, people have their motives to diss Khajits, although they are way too hard.
Sent from my XT1032 using Tapatalk