GETTING IT WRONG
If you're covered in dirt we might call you casual, you also don't want to be a prematurely embalmed person, so there's a slightly worn middle ground.​​​​​​​
Wear jeans when you feel like roughing it about town, even if they're so called "dress jeans", you can live raw in a city with some transition into the jean-wearing country but it's a delicate balance. Casual shoes like trainers and explorer types shouldn't be worn in a setting where if what is stuck to them becomes dislodged would offend the other freshly showered guests. You can look too lazy, nobody will know if you are nor not, so if you're going to dress casually do your best to avoid being constantly in synthetics.​​​​​​​
DOING IT RIGHT
The people that dress causally the best are those in motion, runners and athletes, the rest of the casual crowd are doing it for leisure which is best done around similar company. What I mean is if you're on a boat and dressed casually then it's a perfect fit, but not when at a party because I've never known anyone to relax at a party. If you like dressing "casually" then you're doing it right if you're minimizing the level of plastic that you're wearing. That includes stretchy garb. Someone in fine linens is as tough as the people in cow hide but the stretchy folks might as well be naked.
THE CRUCIALS
Compliment your casual pieces with shiny things.
Wear a shoe with a sole and heel that is worth a damn. If it's a pathetic piece of nothing then you'll be standing deep in shit. Try cottons and other natural fibers. If you like "dressing down" then do it, but I don't know anyone who absolutely loves hairy skin, so be polite. Show off, if you're casual but loud I'll respect it, but if you're casual and invisible then it's not fashion it's pajamas.