koobas.hobune.stream
May I have some help with an outline diffuse shader script?
I seem to be having some problems with a shader script I (mangled) put together for a project I'm working on. Though the problem may be more complicated than just a simple question, so I'll do my best to explain, hopefully not too verbose, my situation.
I've been trying to get a basic toon like outline working with Unity 5 (that can use my beautiful normal maps) for a while, and I've ran into all kinds of hiccups along the way. This seemed to be the perfect solution... until I loaded in my own fbx models. I'm more of an artist and animator than a coder (I cobbled together a pretty awesome edge climb by faking it with animations and turning off gravity- ssh, don't tell anyone). I'm probably a lot more knowledgeable (at least in the javascript world) than I put myself off to be, but I'm totally stumped with shaders. Keep in mind I'm not taking credit for the original script, I just fixed (messed) it up for my own uses.
I just want to know why it's only working with obj files and unity objects, not any other file type. There would be no way I could use it for objects that have boned animations.
Shader "Outlined Diffuse"
{
Properties
{
_Color ("Main Color", Color) = (.5,.5,.5,1)
_OutlineColor ("Outline Color", Color) = (0,1,0,1)
_Outline ("Outline width", Range (0.002, 0.03)) = 0.01
_MainTex ("Base (RGB)", 2D) = "white" { }
//_ToonShade ("ToonShader Cubemap(RGB)", CUBE) = "" { Texgen CubeNormal }
}
SubShader
{
Tags { "RenderType"="Opaque" }
//UsePass "Toon/Basic/BASE"
UsePass "Diffuse/BASE"
Pass
{
Name "OUTLINE"
Tags { "LightMode" = "Always" }
CGPROGRAM
#pragma vertex vert
struct appdata {
float4 vertex;
float3 normal;
};
struct v2f {
float4 pos : POSITION;
float4 color : COLOR;
float fog : FOGC;
};
uniform float _Outline;
uniform float4 _OutlineColor;
v2f vert(appdata v) {
v2f o;
o.pos = mul(glstate.matrix.mvp, v.vertex);
float3 norm = mul ((float3x3)glstate.matrix.modelview[0], v.normal);
norm.x *= glstate.matrix.projection[0][0];
norm.y *= glstate.matrix.projection[1][1];
o.pos.xy += norm.xy * o.pos.z * _Outline;
o.fog = o.pos.z;
o.color = _OutlineColor;
return o;
}
ENDCG
Cull Front
ZWrite On
ColorMask RGB
Blend SrcAlpha OneMinusSrcAlpha
SetTexture [_MainTex] { combine primary }
}
}
Fallback "Diffuse"
}
This is by no means a final project in any way, so if it comes to it that I'll have to abandon the outline effect entirely, I will. I've only been working with Unity for about a year now, so I'm still a learner. I just really, really, really, don't want to get stumped by something as seemingly simple as this.
Also, if you want to list a few different ways I could achieve an effect with an outline in Unity 5, I'd be happy to hear them, even if I've tried them already!
Also, also, no, I'm not asking you to code for me, I just want to know why I'm having this unusual problem. I understand a lot of people come to these boards to ask for 'help' and are really looking for someone to do free work for them. I've used these boards many times already to read questions that other people have asked. If the problem is staring me right in the face, and I'm just not getting it, feel free to shake your finger at me though!
Nothing? Is noone sure? Have I run into something unfixable? :<
Shader warning in 'Outlined Diffuse': Both vertex and fragment programs must be present in a CGPROGRAM. Excluding it from compilation
You lack a fragment program.
// Upgrade NOTE: replaced 'glstate.matrix.modelview[0]' with 'UNITY_MATRIX_MV'
// Upgrade NOTE: replaced 'glstate.matrix.mvp' with 'UNITY_MATRIX_MVP'
// Upgrade NOTE: replaced 'glstate.matrix.projection' with 'UNITY_MATRIX_P'
Unity also updated your code to use unity matrices.
// Upgrade NOTE: excluded shader from DX11 and Xbox360; has structs without semantics (struct appdata members vertex,normal)
And automatically excluded your shader on certain platforms.
I hacked away, commenting stuff out until it worked with a basic vertex/fragment shader, then reenabled various parts and adjusted as I hit blockers. For example Diffuse/BASE doesn't appear to work, but Diffuse/FORWARD does. I didn't include code for fog, because I didn't really understand what I was going to do with it for the outline. It didn't appear to make sense to me, but feel free to modify the code to include it if you need it. I also slimmed down the vertex output to a single position and eliminated arguments to the fragment shader since none were required. I don't know if this is the effect you are looking for, so I guess, have a look at it and adjust where needed.
Shader "Outlined Diffuse"
{
Properties
{
_Color("Main Color", Color) = (.5,.5,.5,1)
_OutlineColor("Outline Color", Color) = (0,1,0,1)
_Outline("Outline width", Range(0.002, 0.03)) = 0.01
_MainTex("Base (RGB)", 2D) = "white" { }
}
SubShader
{
Tags{ "RenderType" = "Opaque" }
UsePass "Diffuse/FORWARD"
Pass
{
Name "OUTLINE"
Tags{ "LightMode" = "Always" }
Cull Front
ZWrite On
ColorMask RGB
Blend SrcAlpha OneMinusSrcAlpha
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
uniform float _Outline;
uniform float4 _OutlineColor;
struct appdata {
float4 vertex : POSITION;
float3 normal : NORMAL;
};
float4 vert(appdata v) : SV_POSITION
{
float4 pos = mul(UNITY_MATRIX_MVP, v.vertex);
float3 norm = mul((float3x3)UNITY_MATRIX_MV, v.normal);
norm.x *= UNITY_MATRIX_P[0][0];
norm.y *= UNITY_MATRIX_P[1][1];
pos.xy += norm.xy * pos.z * _Outline;
return pos;
}
float4 frag() : SV_TARGET
{
return _OutlineColor;
}
ENDCG
}
}
Fallback "Diffuse"
}
As for other ways, well, you could render the geometry you wish to have an outline for on a render texture. If you want to get pixel perfect outline you could process that texture as a second image pass to "fatten" the outline so it's not just a perfect match of the geometry by sampling neighboring texels and mixing them together (if you output white or black, a mask if you want, you can just add left, right, up, down to make a very basic 1 pixel outline. Of course you can do other things, like scale the objects a certain amount when you render the mask. Then you can apply the mask to your final composition.
Okay, this is almost exactly what I tried the first time around when I got the see$$anonymous$$gly bright idea to tackle this view, but it really didn't work well with objects that had a lot of edges (like a character with very complex armor pieces or a horse with crazy hair). To explain it as best as I can remember it, it basically 'puffed' out all around where the edges came together (it's been a long day so that's probably not the best way to express it). I think I saved the project file where I did this, but if not, I can try to duplicate what I did and figure out why it didn't work right. I'll make that a separate question later and tag you in it once I do that.
Another way that kinda-but-its-not-the-same-thing-but-could-work is to do highlight as some rimlight option where you take the dot between the normal and viewing direction (so it looks like you shine a light from the camera) and invert it, you'll get a gradient mask on the object where edges light up. This means you'd either do an additive pass or perhaps more preferably write a fragment shader that does something like color = tex2D(tex, uv) * color + pow(1 - dot(normal, viewdir), 4) * highlightColor;
It'll make it look as if it was "lit from behind" at all times, so it lifts up edges.
Okay, you did take out a few things I wanted to keep, like my bump map stuff, but I can throw that back in with no problem, the only thing is, before I even try to figure out what happened here, I have to show you what it did. I'm genuinely laughing about it, not scolding your abilities. It's pretty funny in my opinion. :P
The rock on the left is an .fbx, the rock on the right is an .obj, and the cube (for my faux terrain), and the sphere are Unity objects. The first image is with my original shader, the second is with the one you wrote. I just threw it on to see it in action while I went over what you did.
I forgot to add that the rocks, and the sphere have the same exact material in both instances. Just so we're on the same page.
If you want to use bumpmaps etc, consider using a different shader UsePass that render with bumps..
UsePass "Bumped Diffuse/FORWARD"
For example.
Bumped Diffuse uses a texture called _Bump$$anonymous$$ap so we should add that property too
_Bump$$anonymous$$ap("Normalmap", 2D) = "bump" { }
And that should be it.
Aaah, okay, I get what you're saying there. That's a lot cleaner to put together for the bump maps.
I'm just still trying to figure out why the edges on the .fbx files aren't displaying right no matter what I do.
To me it sounds like it's a problem with the normals (but on the other hand, that would be a little strange because your first pass render fine with normals). I mean, all the shader get is position & normal to work with. It seems logical to think it's either of the vertex or normal that is different between the models.
You could replace the shader with one that output normals and see if they are the same. The tutorials have a shader that output normals for example. See if they are the same. If it comes out just the same, perhaps it's something with the vertex position data, so you could try displaying those (note, position will probably be outside of color space, so you may need to scale the position down in the shader). If it's not that, then I'd assume there's something different with the matrices or scale (perhaps your model is scaled, or offset from origin, or something?)
I dunno, but I have to go sleep. Good luck trying to find the cause. And for what it was worth, I couldn't display your shader at all on my end when I copied the source into a new shader file. That's why I hacked it to pieces so I could get anything rendering at all.
