There is, admittedly, a part of him that bristles at the notion that magic is for children - but he doesn't act on it or let it show. It's an ingrained reaction from his time in Skyrim, where the locals tend to distrust magic and mages, to denounce it as trickery and something only the weak resort to- but he knows that isn't what Rosco meant by his comment.

"Yes, I suppose it does."