For me, it was the way I answered the questions. I would answer in almost a shameful way, and that would produce a similar response.
After painting for this long, I tell people I'm an artist. If they ask what I do, then I tell them I paint miniatures and models for collectors, gamers, and sculptors.
If it that point, they ask further, I show them pictures that are stored on my phone or direct them to my website.
I tend to take it in steps so they know exactly what they ask for.
If they get to the picture part it's 50/50: either they don't know what to make of it or they're impressed by the size of the minis.
My boss was the last to make fun of me for painting little mens but I pointed out that I'm getting more steady work with my customers than he is with his own business... which is why I've been building the painting side of my life in the first place.
My writing comes off as egotistical and I don't mean it that way... I just try and keep the explanation simple and clear.
Confidence goes a long way.