Strange texture background using Unlit Transparent Cutoff

The "strange background" you are talking about, which probably looks like the edge of your image has duplicated its pixel either up or down in the image to the border, happens because you did not specify pixel data for the areas where the strange background occurs.

It is important when creating alpha transparent or alpha cutout images to create your transparency on an alpha channel of your image. Do not just leave the background layer in photoshop as blank (checker pattern).

To avoid the white border on your images, fill the background of your image with the same color as the border of your transparent image. By border I mean the "outline" of your image. For example, if I were creating leaves I would fill my background with the same color as the edge of the leaf.

Pro tip: If you fill your background with a darker color than the image edge, when combined with the correct alpha channel, you can create a semi-outline. This will make your transparent textures "pop", though it is not necessary

After youve made sure you have an alpha channel, you can export/save it as a regular psd file. I like doing it this way so I can edit textures much faster by just double clicking inside unity. Inside unity select your texture. In the texture import properties choose "Advanced" from the texture type drop down. Check the "Alpha is transparency" check box.

And what is even stranger, when the alpha-cutoff is changed just slightly below 1, the background disappears and thats when I get the white edges.

This happens because unity is creating the transparency for you, which limits your control. It takes the image and wherever you have not specified pixel data, it will make the alpha channel black. Wherever there is an image it will be white. When you move the slider it is raising/lowering the effect the alpha has on the transparency. Since the alpha channel created jumps from black to white, there is no "in between" and the background will pop in. Creating an alpha channel in your image fixes this and you can control the falloff by gradating the alpha colors from white to black. 50% grey = 50% transparent. This does not work with cutout shaders and the "jump" happens because cutout shaders have no gradating. It is either visible or transparent, nothing in between.

So to walk you through fixing your situation. In photoshop select your image (ctrl + click the layer), click channels, fill the selection with the alpha channel selected with white. go back to layers, fill your background with the same red color as your image. Save to your unity project folder. Find the texture in unity and change its properties to "Advanced" and check "Alpha is transparency"