Not only that, you can improve it by hitting Treasure Chests, Bottles, Beer Jugs, Pots, Tables; Anything that can receive damage is basically experience for his fist!
It is intended however, yes; You can grind away at his skills with out being in combat areas, if you have the patience!