Physic, need help

I am not using gravity, maze is static. Force is applied directly to the ball with

 GetComponent<ConstantForce>().force = new Vector3 (speed*Input.GetAxis("Mouse X"), 0f, speed*Input.GetAxis("Mouse Y"));

Let's say i want carefully roll the ball into passage like in "A". The problem is that it actually rolls in, easily. While i want it to perform movement like in "B", preventing player to roll ball so easily from a static position.

To motivate player to do that:

Any idea how to change my code, so that ball rolls like in "B" when you try carefully roll it right into passage?