Strange Behavior with RTS-Style Selection Box and Moving Camera

I'm currently working on an RTS-style game and am coming across a few quirks with the moving camera that I'm hoping to iron out. I wanted the selection box to be "anchored" to an actual location in the world so it doesn't just float across the screen as the camera moves, so I've turned the initial screen click point into a world point which is then turned back into a screen point for displaying the box/calculating logic.

This works well for the most part, other than the box extends slightly in the direction of the camera movement. That is, when the camera moves right, the original anchor point still remains, but the box will extend to the right. Same for up, down, left, etc. I have the camera moving on LateUpdate, so I don't think this would be impacting it. Other than that, I really have no clue what would be causing this, and would appreciate any ideas!

And then for something that I don't really think can be fixed easily: due to having a perspective camera, as the camera moves around, some objects that were previously on the borders of the selection box sometimes get pushed out of it, despite that border not moving in world-space. I have a feeling this is due to slight calculation differences, but still feels finicky. Any advice towards mitigating this would be great, and thank you for taking the time to read my small novel of a question lol