You don't even need to have patience when dealing with people with special needs. More often, it is even more difficult to hang out with arrogant people who seem normal at first but will turn out more harmful. Verbal abuse is just as painful as physical pain and it gives about the same amount of damage. You can get sick for real. It's better to just keep your life simple. I don't think a problematic life and being around people with disabilities fits in the same picture. More likely they can find peace on their own and they won't really bother you unless you keep asking them to do things.
Blair Hill is a writer with Connect Autism.Org and provides autism education from the inside out.