Cricket Ball Physics?

Keep in mind that physics engines in games are not scientifically accurate simulations of reality (some exist, but not in games). They are based on the classic physics formulas, but fine details are summarized into parameters like "bounciness" (the name should give you a clue that this is not scientific), and since the math is processed in floats there is a limitation on scale.

That said, bouncing, rolling, spinning are among the things a game physics engine does best.

Scale is your first issue. Make sure the scale matches the expected units of Unity. To Unity 1.0 is one meter, so size your objects accordingly if you want the 'speed' of physics objects to look correct.

The default unity of mass is Kg, so establish mass appropriately. There's a catch. The range of mass is a bit limited, so you can't expect simulations to work well when comparing objects of fractions of a Kg with others of thousands of Kg's. I don't sense you have that issue, but it comes up in many games.

Next, pay attention to physics materials. Create a material for the ball and adjust for the kind of bounce and overall response you prefer. Second, pay attention to the physics materials for the surfaces and objects the ball must realistically bounce upon. You will find yourself working on adjustments for quite a while to get things right.

I built a simulation/game of a real world robotics contest (used for a club to practice strategy in preparation for competitions). Among the objectives is to shoot a 3 inch ball of 2 ounces in a 'cannon' like device on the robots. Getting the bounce right was critical. This ball is something like a 3 inch ping pong ball (bounces forever on a hard surface). The game is played on a mat which absorbes considerable amount of 'bounce energy', but the enclosure for the game is made of either metal or plastic parts with a very 'active' bounce, and a net that nearly catches the ball with no real bounce at all. Getting that to work well was a 3 day task of adjustments and comparisons. In that case I took video of the actual ball dropped from a known height onto various surfaces, as well as tossing the ball on surfaces of various types at known velocities. I used that to adjust the physics parameters.

What I find is that if you get the ball's physics material right for bouncing on the floor, it might be too active bouncing off of walls. Another issue in my situation is restitution. Most real time physics engines tend to oscillate, which means a group of balls on a hard surface would constantly jitter (small, fast vibrations) and never stop moving. The engine uses a restitution value that globally inhibits this motion at an arbitrary value, but for my simulation of a 3 inch ball that was too high (the balls tended to roll, slowing down correctly, but stopping suddenly in an odd looking way). Raising this restitution value (a threshold) caused the balls to be too jittery in some situations, so I had to accept compromises which were subsequently smoothed out in scripts attached to the ball.

I chose to allow the engine a very small restitution value, making things generally quite jittery on their own, but attached scripts to sense velocity, and establish an artificial restitution appropriately. Further, I found that bounce was not like the real objects - it was too mathematically pure in the simulation. In the real world, light bounces on the mat were not as well damped as larger bounces. This means there is not a single bounce setting, but multiple values - a lighter bounce setting on heavy strikes, and a stronger bounce setting on light strikes on the mat (this is related to the complex construction of the mat material in the real world). I simulated this with code attached to the ball, which selected a context based on material, speed and the energy of the impact on the floor.

This is a somewhat common means of dealing with the fact that the game engine physics is something of a puppet show, and you'll end up making things "look right", even if they're fudged. It is not going to be all that simple if you're particular (and I was being too particular because I needed the game to follow actual observations of physical behavior of objects the player would use in the real world later.