In our case our main problem was where the colliders for tiles meet. The ball hits one collider and not the other and considers it as hitting a corner rather than a continuous flat wall.
Also realistic drag/friction doesn't give the player enough time to properly mess around with the traps and tilt as the ball gains too much speed. We have some code adjusting friction along manually specified curves to get around this.