Using shadow texture to recieve shadows (on grass)

Question

Hi everyone!

I've created some grass on a custom mesh terrain (NOT Unity terrain) and I want it to receive shadows from surrounding trees. Grass is created from a point cloud where each vertex becomes a billboard in geometry shader. Trees successfully cast shadows on terrain.

I've tried sampling Unity's shadow textures for light (since they have all that fancy good looking cascading stuff) to render shadows on grass but I have failed. The closest question to what I'm asking is this one: http://answers.unity3d.com/questions/442592/accessing-shadow-texture-in-shader.html For some reason neither of the answers given there helped me. I've tried both with SHADOW_COORDS and LIGHTING_COORDS (and other appropriate functions) but it didn't work. I always receive shadow attenuation equal to 1.

I've never created a multiple pass shader so if an answer is somewhere in there I am kindly begging for detailed info.

At the bottom is shortened grass billboard shader with a screenshot of a scene.(I deliberately exaggerated with light intensity and shadow strength so that it would be obvious). I'm hoping to achieve different attenuation for my grass in shadow and lit areas. As commented in a shader sample, red color means that attenuation is equal to 1 and every bit of my grass is red. (I reduced the contrast of the picture so it doesn't hurt your eyes)

How can I use shadow textures to sample light/shadows? Can it be done even though I have a geometry shader? Can I sample shadow texture in vertex shader and pass its value to fragment shader trough geometry shader?

Thank you

Shader and screenshot:

alt text

 Shader "Custom/GrassBillboardShortened" 
 {
     Properties 
     {
         /** properties **/
     }
 
     SubShader 
     {
         Tags { "Queue" = "Geometry" "RenderType"="Opaque" }
 
         Pass
         {
             Tags { "LightMode" = "ForwardBase"}
 
             CGPROGRAM
                 #pragma vertex vertexShader
                 #pragma fragment fragmentShader
                 #pragma geometry geometryShader
                 #pragma multi_compile_fwdbase
 
                 #include "UnityCG.cginc"
                 #include "AutoLight.cginc"
 
                 struct VS_INPUT
                 {
                     float4 vertex : POSITION;
                     /* .. and other variables used for creating billboard ... */
                 };
 
                 // I put shadow stuff in separate struct because of naming in some macros
                 struct VS_SHADOW
                 {
                     float4 pos : POSITION;
                     SHADOW_COORDS(0)
                 };
 
                 struct GS_INPUT
                 {
                     float4 worldPosition : POSITION;
                     /* .. and other variables like billboard size etc... */
                 };
 
                 struct FS_INPUT
                 {
                     float4 position    : SV_POSITION;
                     fixed2 uv_MainTexture : TEXCOORD0;
                     /* I used variable name _ShadowCoord since SHADOW_ATTENUATION uses it (according to "AutoLight.cginc") */
                     float4 _ShadowCoord : TEXCOORD1;
                 };
 
                 /* ... declarations of variables ... */
 
                 // Vertex Shader ------------------------------------------------
                 GS_INPUT vertexShader(VS_INPUT v)
                 {
                     GS_INPUT vOut;
                     
                     vOut.worldPosition =  mul(_Object2World, v.vertex);
 
                     // I put shadow stuff in separate struct
                     VS_SHADOW shadow;
                     shadow.pos = mul(UNITY_MATRIX_MVP, v.vertex);
 
                     /* I used this instead of TRANSFER_SHADOW(shadow) since I had to store shadow coords in a variable to pass them to
                      * geometry shader. When I use TRANSFER_SHADOW, I guess shadows coords are stored somewhere in _ShadowCoord variable
                      * according to "AutoLight.cginc" but for some reason I can't reference it with shadow._ShadowCoord */
                     /* It's not that I wanted to have it this way but every other attempt (like sampling attenuation here in vertex shader)
                      * also failed */
                     vOut.shadowCoords = ComputeScreenPos(shadow.pos); 
 
                     return vOut;
                 }
 
 
                 // Geometry Shader -----------------------------------------------------
                 [maxvertexcount(4)]
                 void geometryShader(point GS_INPUT p[1], inout TriangleStream<FS_INPUT> triStream)
                 {
                     /* ... bunch of code for creating billboard geometry - that works ... */
                     float4 v[4];    // 4 vertices for billboard are created 
                     float4x4 vpMatrix = mul(UNITY_MATRIX_MVP, _World2Object);
                     
                     FS_INPUT fIn;
                     fIn.position = mul(vpMatrix, v[0]);
                     fIn.uv_MainTexture = float2(1.0f, 0.0f);
                     fIn._ShadowCoord = p[0].shadowCoords;
                     
                     triStream.Append(fIn);
 
                     fIn.position =  mul(vpMatrix, v[1]);
                     fIn.uv_MainTexture = float2(1.0f, 1.0f);
                     fIn._ShadowCoord = p[0].shadowCoords;
 
                     triStream.Append(fIn);
 
                     fIn.position =  mul(vpMatrix, v[2]);
                     fIn.uv_MainTexture = float2(0.0f, 0.0f);
                     fIn._ShadowCoord = p[0].shadowCoords;
 
                     triStream.Append(fIn);
 
                     fIn.position =  mul(vpMatrix, v[3]);
                     fIn.uv_MainTexture = float2(0.0f, 1.0f);
                     fIn._ShadowCoord = p[0].shadowCoords;
 
                     triStream.Append(fIn);
                 }
 
 
                 // Fragment Shader -----------------------------------------------
                 float4 fragmentShader(FS_INPUT fIn) : COLOR
                 {
                     fixed4 color = tex2D(_MainTex, fIn.uv_MainTexture);
                     
                     if (color.a < _Cutoff) 
                         discard;
 
                     float atten = SHADOW_ATTENUATION(fIn);
                     if (atten > 0.99)
                         color = fixed4(1.0,0.0f,0.0f,1.0f);
                     else
                         color = fixed4(0.0,1.0,0.0,1.0f);
 
                     // I always get fully red color which means atten is always > 0.99
                     return color;
                 }
 
             ENDCG
         }
     } 
 
     FallBack "Diffuse"
 }

Answer 1

Best Answer

Answer by nikomikulicic · Sep 19, 2015 at 03:28 AM

Yes, it can be done with geometry shader!

I've managed to solve this problem with the help of this post: http://forum.unity3d.com/threads/is-it-possible-to-use-geometry-shader-with-surface-shaders.164905/#post-1658095

There are couple of things you need to do to make shadows right and it's quite easy once you find out what Unity expects you to do.

BASE PASS

At the end of your vertex shader you need to call TRANSFER_VERTEX_TO_FRAGMENT(f), where f is your fragment shader input structure. If you're creating geometry in geometry shader, then you need to do it in geometry shader for every vertex created there. You also need to include "AutoLight.cginc" which contains that macro. NOTE: That macro will use f.pos so you have to provide SV_POSITION variable named pos. Example:

         struct FS_INPUT  // fragment shader input structure
         {
                float4 pos : SV_POSITION;  // has to be called this way because of 
                                           // TRANSFER_VERTEX_TO_FRAGMENT macro
                LIGHTING_COORDS(0,1)       // data needed to sample light is declared on 
                                           // TEXCOORDS 0 and 1
         };
             
         FS_INPUT vertexShader(VS_INPUT vIn)
         {
             FS_INPUT f;
             f.pos = mul(UNITY_MATRIX_MVP, vIn.position); // classic model-view-projection
             TRANSFER_VERTEX_TO_FRAGMENT(f);      // initialization of data needed for sampling light
             
             return f;
         };

In the fragment shader you'll sample light with:

  float atten = LIGHT_ATTENUATION(fIn); // fIn is input fragment.

I did my own lighting calculation based on (ambient + atten * lambert) * _LightColor0.rgb, but I guess it depends on what you want.
If you are not creating geometry in geometry shader, then adding Fallback "Diffuse" should add proper shadows to your model (because it has ShadowCaster and ShadowCollector passes).

SHADOWCASTER PASS

If you're creating your geometry in geometry shader, then you need to create your own ShadowCaster pass. For some reason ShadowCollector pass is not needed? It works for me perfectly without shadow collector. In "UnityCG.cginc" there are "deprecated shadow collector pass helpers". Is ShadowCollector deprecated and not needed anymore?

Things you need to do:

Add tag: Tags { "LightMode" = "ShadowCaster" }

Instead of TRANSFER_VERTEX_TO_FRAGMENT(f) use TRANSFER_SHADOW_CASTER(f). Example:

     struct SHADOW_VERTEX // helper struct because TRANSFER_SHADOW_CASTER uses v.vertex
     {
         float4 vertex : POSITION; // local position of vertex
     };
     struct FS_INPUT
     {
         V2F_SHADOW_CASTER;
     };
 
     //somewhere in geometry shader:
 
     FS_INPUT f;
     SHADOW_VERTEX v;
     v.vertex = // local position of new vertex;
     TRANFER_SHADOW_CASTER(f)        // uses "v.vertex"

In fragment shader use SHADOW_CASTER_FRAGMENT(f):

     fixed4 fragmentShader (FS_INPUT fIn) : COLOR
     {
         SHADOW_CASTER_FRAGMENT(fIn)
     }

And that should be it! I hope this will be helpful to someone. Here's a screenshot of this working:

alt text

SOURCE:

Here's my shader's source. You can use it wherever you want. You don't have to give me any credits, however they will be warmly welcomed. Enjoy! GrassBillboard.txt

grassbillboard.txt (9.8 kB)

forest-deciduous-9-sd400.jpg (162.7 kB)

Add comment · Show 9 · Share

Danothom · Sep 16, 2015 at 10:20 AM 0

Share

@sejton, Dude, this is a fantastic effort and is something I've been attempting to resolve for a while now. Could you please do a huge favour and post the shader in it's entirety so that I can peruse both the base and caster passes. I'm still having a few issues getting things to work 100%!

Thanks a bunch!

nikomikulicic Danothom · Sep 16, 2015 at 03:20 PM 0

Share

Hey, @Danothom! I'm glad you're interested. I've now included the shader's source at the bottom of the answer. I'm currently on a faraway island so I can't guarantee you that this is the last working shader (can't test on this poor laptop, but I think it is), however it seems like everything you want to know is in there. Enjoy it! I'm glad I'm not the only one who'll make use out of it.

Cya!

Danothom · Sep 16, 2015 at 04:18 PM 0

Share

What a champ, much appreciated! By the way, is this shader still predicated on making changes to the actual AutoLight and UnityCG .cginc files?

I hope not because when I last tried to make the changes based on this post i couldn't get it to work on Unity5 due to all the shader macros and helper files etc.

In fact, when I think about it seems silly that this issue still hasn't been rectified by the Unity $$anonymous$$m.

nikomikulicic Danothom · Sep 16, 2015 at 04:37 PM 0

Share

You're very welcome! And no, you don't have to make any changes to AutoLight, UnityCG or any other .cginc file. This is a one shader solution.

I was also surprised when I found out how Unity is poorly documented when you scratch beneath the surface and reach greater depths. There is no real documentation on how to write your own shadowcaster pass, how to use shadow texture etc. I guess most users don't go to these depths and usually use only what Unity provides them. However, Unity does provide a free download of all their shaders.

Nevertheless, I agree that this topic should have an official solution at least on forums.

Danothom · Sep 16, 2015 at 05:32 PM 0

Share

O$$anonymous$$ well as far as I'm concerned that's great news as I didn't relish having to fiddle with the built-in shaders everytime I upgraded Unity :)

The issue I've found is that the information concerning Geometry shaders in Unity is nearly non-existent. Apart from a few pioneers, such as yourself, who take the time to post solutions, I'd never figure this stuff out without many days of hassle and hair $$anonymous$$ring!

$$anonymous$$any thanks again. Time to get back into shader coding mode.

tengel · Oct 20, 2016 at 07:13 PM 0

Share

Hi.

Great job!

Anyone managed to get it to work with Graphics.DrawProcedural?

Janeator · Apr 27, 2019 at 05:38 AM 0

Share

Hey there; sorry but the txt source link broke. Would you $$anonymous$$d posting it again?

reev4eg Janeator · Aug 20, 2019 at 05:53 PM 0

Share

Just remove .txt like this https://answers.unity.com/storage/temp/54417-grassbillboard

Show more comments

Using shadow texture to recieve shadows (on grass)

1 Reply

Your answer

Follow this Question

Related Questions