Wrong colors from ReadPixels()

When using Texture2D.ReadPixels() to read from a RenderTexture, I get the wrong colours when the texture I'm reading in to is ARGB32 and the right colours when the texture is RGB24. Also, when using ARGB32 I can get the right colors if I EncodeToJPG, but the wrong colours if I use EncodeToPNG.

When the colour is wrong, it looks far too light (on a white background) and the pixels have varying levels of transparency. Getting rid of the transparency doesn't seem to fix up the colours, they are still too light.

Has anyone come across this before? Is it possible to get an image that can handle transparency that also contains the right colors?