Pixel-Perfect Rendering on Locked Resolution

I am currently working on a retro-styled game with a specific, small render resolution.

I am limiting the render resolution by rendering at 160x144, and upscaling it with this image effect script:

 using UnityEngine;
 using System.Collections;
 
 public class LimitRenderRes : UnityStandardAssets.ImageEffects.ImageEffectBase
 {
     public int horizontalResolution = 320;
     public int verticalResolution = 240;
 
     void OnRenderImage(RenderTexture source, RenderTexture destination)
     {
         RenderTexture scaled = RenderTexture.GetTemporary(horizontalResolution, verticalResolution);
         scaled.filterMode = FilterMode.Point;
         scaled.anisoLevel = 0;
         Graphics.Blit(source, scaled,material);
         Graphics.Blit(scaled, destination);
         RenderTexture.ReleaseTemporary(scaled);
     }
 }

However, my game doesn't have appear to be pixel perfect.

Without image effect:

With Image effect:

Both the texture and the white cube have interpolation going on, when they should display as solid colors. Any idea on how I might fix this?

For reference, I am using the Sprites/Default shader in my effect.