I would only recommend one or two players, honestly. It gets pretty crowded even at three.
If you decide to go to three or four players, you might consider running a two player game and splitting up the heroes and companions among the players to control. That keeps the model count manageable and the combats quicker.
If you do decide to do a full game with 3+ players, I would recommend adding at least one more dungeon tile to generated rooms, too, just to give a little elbow room.