Weird collision behavior?

So I'm making a game based completely off physics. The player is using a custom controller and the jump is run on physics. And also I'm not using anything like OnTriggerEnter() or anything like that. So what's happening is if I run into anything, nothing really acts like it should, like no momentum/velocity will carry over. Like lets say I hit my ball with my player ( a cube) all that will happen is I'll push the ball along with me and once I move ahead of the ball it should be rolling at the same speed as I was pushing it at. but instead it'll roll as if something tapped it. This all goes as well for all the other objects in my world. I can't imagine why this would be happening. Can anyone help?