Unity 2D - Why does velocity causes the ball to change direction everytime it hits something?

One of the purposes of a Rigidbody is to add realistic movement and automatic collision detection to your objects without having to do all the tedious work it requires to program it yourself.

Once you give an object a velocity (which is a combination of direction and speed), it will keep moving in that direction until something stops it. It could be drag, friction from moving along the ground/wall or an obstacle (another object that has a collider).

As you can tell, the velocity in the code you posted is only set once: in Start()

The fact that you've added a Rigidbody and set the object in motion using Rigidbody.velocity is enough to make the object move and bounce off of other colliders.

In case you're wondering why it says "speed" twice in the code you posted: If the speed is 10, for instance, then you've told it to move 10 units per second on the X-axis and 10 units per second on the Y-axis. Which means diagonally. I say "units" because you can call it whatever you want (inches, cm, meters, etc.).

The Physic Material was added to make sure that the object does not lose speed from touching the ground and bouncing off the wall/paddle. If you don't create a Physic Material and attach it to your object's collider then it will use the default Physic Material which doesn't bounce as much and it also absorbs a bit of speed with every collision.