- Home /
Receive Shadows with Custom Vertex Shader & Shadow Passes Functioning Incorrectly
Hello Unity community!
I'm having some serious issues with creating a shader which modifies incoming vertex information to scale them before outputting them to my surface shader.
To give a bit of background, the setup we're using is a deferred render path for Unity. We have a deferred decal system which draws onto our level geometry. Unfortunately due to issues with Unity's deferred renderer, we cannot control the render queue order for our decals (the decals always draw last regardless of our chosen render queue). This meant that if we wanted to have decals draw under a specific piece of geometry we had to force them to be forward rendered.
To solve this, we wrote surface shaders which force rendering into the forward pass using the 'exclude_path: prepass' pragma, as shown in 'Writing Surface Shaders' from Unity's documentation.
This approach functioned perfectly for us until we decided to perform operations in the vertex shader before passing it to the surface shader. What we were attempting to accomplish was to scale the model in the vertex shader. The model has many attached scripts and physics components (such as wheel colliders) which we do not want to scale with the geometry so the logical approach was to scale the vertices at the vertex shader level.
This is pretty trivial to achieve however, there appears to be something Unity is doing with its shadow receivers (casting is fine) when you modify vertices in a surface shader which don't translate well when forward rendering objects in the deferred pathway. Pictures speak a 1000 words, so here is what happens using the shader when I scale the vertex versus just scaling the transform:
The shader which I'm using is the same as below (game specific colour math and textures removed for brevity). You can see that I have ensured that I include 'addshadow' into my pragma to ensure that the vertex shader gets applied to the shadow passes as well (the nofardwardadd makes no difference in the resulting rendered output).
Shader "Vertex Scale Example" {
Properties {
_MainTex ("Diffuse (RGB)", 2D) = "white" {}
_Scale ("Scale", Float) = 1.0
}
SubShader {
Tags {"Queue"="Geometry+50" "RenderType"="Opaque" }
CGPROGRAM
#pragma target 3.0
//This sucks... but force units to be rendered in forward rendering to avoid render queue issues with deferred decals (for now)
#pragma surface surf BlinnPhong exclude_path:prepass vertex:vert noforwardadd addshadow
uniform sampler2D _MainTex;
uniform float _Scale;
struct Input {
float2 uv_MainTex;
};
void vert (inout appdata_full v) {
//According to Unity docs, this should allow us to scale the vertices for all passes by using the
//'addshadow' pragma at the top of our shader however, this seems to completely breaks our shadow collectors despite
//The unity community stating otherwise:
//http://forum.unity3d.com/threads/i-put-a-shadowcollector-pass-in-my-shader-but-why-does-it-cant-recieves-shadow-why.160010/
//http://forum.unity3d.com/threads/simplest-possible-fragment-shader-that-has-shadows.145856/
//http://forum.unity3d.com/threads/fallback-off-turns-off-shadows-in-surface-shader.257430/
//http://forum.unity3d.com/threads/shadows-on-geometry-displaced-by-vertex-shader.40879/
//
v.vertex.xyz *= _Scale;
}
void surf (Input IN, inout SurfaceOutput o) {
fixed4 c = tex2D(_MainTex, IN.uv_MainTex); //Define Diffuse
o.Albedo = c.rgb;
}
ENDCG
}
Fallback Off
}
I've also tried to explicitly implement the shadow passes myself, instead of letting Unity auto-generate them for me with the 'addshadow' #pragma. I have also looked high and wide in the Unity community to see if anyone else has encountered this problem and solved it. There have been many similar issues, however, I haven't seen a solution which resolves the problem for me.
Here's what my explicit shadow passes look like:
Shader "Vertex Scale Example Explicit Shadows" {
Properties {
_MainTex ("Diffuse (RGB)", 2D) = "white" {}
_Scale ("Scale", Float) = 1.0
}
SubShader {
Tags {"Queue"="Geometry+50" "RenderType"="Opaque" }
Fog {Mode Off}
CGPROGRAM
#pragma target 3.0
//This sucks... but force units to be rendered in forward rendering to avoid render queue issues with deferred decals (for now)
#pragma surface surf BlinnPhong exclude_path:prepass vertex:vert noforwardadd
uniform sampler2D _MainTex;
uniform float _Scale;
struct Input {
float2 uv_MainTex;
};
void vert (inout appdata_full v) {
//According to Unity docs, this should allow us to scale the vertices for all passes by using the
//'addshadow' pragma at the top of our shader however, this seems to completely breaks our shadow collectors despite
//The unity community stating otherwise:
//http://forum.unity3d.com/threads/i-put-a-shadowcollector-pass-in-my-shader-but-why-does-it-cant-recieves-shadow-why.160010/
//http://forum.unity3d.com/threads/simplest-possible-fragment-shader-that-has-shadows.145856/
//http://forum.unity3d.com/threads/fallback-off-turns-off-shadows-in-surface-shader.257430/
//http://forum.unity3d.com/threads/shadows-on-geometry-displaced-by-vertex-shader.40879/
//
v.vertex.xyz *= _Scale;
}
void surf (Input IN, inout SurfaceOutput o) {
fixed4 c = tex2D(_MainTex, IN.uv_MainTex); //Define Diffuse
o.Albedo = c.rgb;
}
ENDCG
// Pass to render object as a shadow caster
Pass
{
Name "ShadowCaster"
Tags { "LightMode" = "ShadowCaster" }
Fog {Mode Off}
ZWrite On ZTest Less Cull Off
Offset [_ShadowBias], [_ShadowBiasSlope]
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma multi_compile_shadowcaster
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"
uniform float _Scale;
struct v2f
{
V2F_SHADOW_CASTER;
};
v2f vert( appdata_base v )
{
v2f o;
v.vertex.xyz *= _Scale;
TRANSFER_SHADOW_CASTER(o)
return o;
}
float4 frag( v2f i ) : COLOR
{
SHADOW_CASTER_FRAGMENT(i)
}
ENDCG
}
// Pass to render object as a shadow collector
Pass
{
Name "ShadowCollector"
Tags { "LightMode" = "ShadowCollector" }
Fog {Mode Off}
ZWrite On ZTest Less
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma fragmentoption ARB_precision_hint_fastest
#pragma multi_compile_shadowcollector
#define SHADOW_COLLECTOR_PASS
#include "UnityCG.cginc"
uniform float _Scale;
struct appdata
{
float4 vertex : POSITION;
};
struct v2f
{
V2F_SHADOW_COLLECTOR;
};
v2f vert (appdata v)
{
v2f o;
v.vertex.xyz *= _Scale;
TRANSFER_SHADOW_COLLECTOR(o)
return o;
}
fixed4 frag (v2f i) : COLOR
{
SHADOW_COLLECTOR_FRAGMENT(i)
}
ENDCG
}
}
Fallback Off
}
It almost looks like the deferred pre-pass is still running, rendering the geometry into the depth buffer with the default clear colour of the camera yet, I've excluded this pass? Very confuse, and need your help!
I've checked out the following links but haven't had any success yet:
I put a shadowCollector pass in my shader,but why does it cant recieves shadow,why?
Modifying Wireframe Vertex Shader to Include Lighting & Shadows
Finally, I'm starting to think that this is a bug in the co-operation of the forward and deferred rendering pathways. Any insight would be appreciated!
Answer by GeraldOBBI · Jan 19, 2015 at 10:17 PM
This problem was solved by going back to using deferred rendering and assigning renderer sort layers which force things to be drawn in deferred AFTER the decals. This worked great.
The result of this also removed the need for explicitly written shadow collector and caster passes!
Could you explain what you did, I have the same problem with my very simple shader the shadows work to some extent but it looks like the light position is wrong for them. I tried the addshadow system and the adding shadow passes both with no success. I don't quite understand what you mean when you say you assigned renderer sort layers.
It sounds like your problem is different from the one which I was talking about. What I meant when I said sort layers is described here: http://unity3d.com/learn/tutorials/modules/beginner/2d/sorting-layers
Surprisingly, this can also be applied to 3D (in Unity 5 this is accessible via the inspector for all renderers). You can gain access to the sort layers and sort IDs with a custom inspector here: http://forum.unity3d.com/threads/sort-layer-renderer-extension.219799/