Point, Spot lights not working in HDRP?

Try increasing the light Intensity to a really high number, like a 99999999 lumens. Depending on how far your light is from the scene geometry it just might be too dim to see the effect. A household light bulb is around 600 lumen, so if your light is only set to 10 or so lumens the lighting contribution to the scene will probably be negligible.

It worked to crank my lights up to extreme values. 2e+07! Is this the way that this is supposed to work? Is there an HDRP global setting that I have not set up properly?

I was running into the same problem I found my fix by going into the spotlight, setting the intensity really high (for testing) after, I cleared all previously baked lighting data and made sure my Spot Lights settings Intensity value was set to lumens and my Mode was Mixed instead of Realtime. Afterwards I baked lighting and my entire scene lit up Railroad light red after the bake I was able to play around with my lights values in Editor without having to constantly bake.

This is because the reflector is not enabled. Go to Emission section in the spot or point light and click on the three vertical dots. Click Show Additional Properties. Find and enable the reflector. Hope it helps.