Objects not drawn with custom vertex/fragment shader when in any other layer than the default layer

I need some specific objects to be in another layer than the default layer so that instances of these objects don't collide with each other, but still do so with everything else.

These objects are rendered by a custom vertex lit shader which needs to know the location of a point light source as well as the light's intensity. So when i select any other layer than the default layer, the objects just disappear. I googled the problem and checked that the culling mask of my main camera is set to everything, but it's still not working.

As you can see in the screenshot, the two wireframes on the left are not drawn. They should look like the rest of the green cones.

I also tried different shaders, and all the built in shaders render the cones (almost) perfectly regardless of the specified layer. But i also noticed that when i used the built in shaders, and put the objects in any other layer than default, the objects ignored the point light. That is a big deal for me since i depend on using that point light.

Why is rendering behaviour tied to physics collision groups anyways? Can i somehow avoid this whole problem by not changing the layers at all and have the cones not collide with each other in another way?

Here's my shader:

 Shader "Cone Shader"
  {
      Properties{
          _Color("Color", Color) = (1,0,0,0)
          _Offset("Offset", Range(0,5))  = 0.2
          _DistExp("Distance Exponent",Range(0,4)) = 1
      }
      SubShader
      {
          Pass
          {
              Tags {"LightMode" = "ForwardAdd"}
              
              CGPROGRAM
              #pragma vertex vert
              #pragma fragment frag
              #pragma multi_compile_fwdadd_fullshadows
              // shadow helper functions and macros
              #include "UnityCG.cginc"
              #include "AutoLight.cginc"
  
              struct v2f
              {
                  float4 pos : SV_POSITION;
                  float3 normal : NORMAL;
                  float light : BLENDWEIGHT0;
                  SHADOW_COORDS(1)
              };
              float _Offset;
              float _DistExp;
              v2f vert(appdata_base v)
              {
                  v2f o;
                  o.pos = UnityObjectToClipPos(v.vertex);
                  // get vertex normal in world space
                  o.normal = UnityObjectToWorldNormal(v.normal);
                  //_WorldSpaceLightPos0 doesn't work in any other layer than default layer
                  float3 delta = (_WorldSpaceLightPos0.xyz - mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz);
                  float dist = length(delta);
                  o.light = (dot(delta / dist, v.normal) + (1.0 + _Offset))  / (2.0 + _Offset);
                  o.light /= pow(dist+1.0,_DistExp);
                  TRANSFER_SHADOW(o);
                  return o;
              }
              float4 _Color;
              
              uniform float4 _LightColor0;
              fixed4 frag(v2f i) : SV_Target
              {
                  float light = i.light;
                  if (SHADOW_ATTENUATION(i) < 0.9) {
                      light *= 0.4;
                  }
                  return _Color * light * _LightColor0;
              }
              ENDCG
          }
      }
  }