Collision in fast speed

Rigidbodies move according to their current velocity each frame. That movement should not be understood as movement in a realistic physical sense - it rather changes its position, like a teleport. After doing that, a rigidbody checks whether it teleported into another object and if it did, it moves out and sends collision events.

As a result, at high speeds, rigidbodies are well able to teleport from one side of the object to the other without noticing it.

There are two solutions to this. Which one to use depends on the situation.

The first solution is to reconsider using a rigidbody. Especially for bullets, rigidbodies tend to introduce more problems than they solve. Unless we're taking rockets or Super Mario bullets, bullets are better implemented using Raycasts. Raycasts go from one point into one direction and notice when something is in the way. You can chain multiple Raycasts to implement bullet drop. You can put them all in one loop to make the bullet speed "instant" as in "hits as soon as it's shot" or perform one Raycast each FixedUpdate to get bullet flight time. A SphereCast would be an example of a Raycast with a diameter greater than zero.

I recommend this solution because in most cases, when the bullet is fast enough to port through walls, a raycast is better anyway.

The second solution is to add a script to the rigidbody that performs a so called SweepTest every FixedUpdate. A SweepTest technically does what a raycast does, except for the complex collider-based form of a rigidbody (rather than a point or sphere). I recommend not using SweepTests unless Raycasts are definietly no option. SweepTests are pretty slow and not available in 2D physics.

And, to be honest: Which bullet needs to be physically calculated with a complex form?