How do I make sure my sprites are fitted properly to my textures?

I'm working on an orthographic top-down 2D sprite game, and I finally have my world working the way I want... except for one small thing: for some reason it looks like the sprites and textures aren't aligning properly for my world terrain tiles.

I noticed that the world terrain tiles weren't lining up properly, so I scaled them down to 0.975 * their normal size and added a 50 pixel border to the texture so I could more clearly see what's going on. See the attached image.... you can clearly see that the tiles are lining up properly in the world (the thin white line is the space between them due to being scaled down).

But that border, tho... what gives? Why are the textures not aligned properly? Image, texture and sprite are all created at perfectly square 1024x1024 (I've doubled checked this dozens of times). So why are things not lining up properly?

You may notice in the code that I'm loading my images in as a byte[]. This is because I'm reading them straight from a .zip file to a memory stream (.zip -> memory -> texture2d -> sprite). Could doing it this way cause anomalies, or am I just missing something obvious here?

Obviously I could simply adjust the offset of the texture relative to the sprite... if I knew how to do that, but I can't seem to figure it out.

  e = zip["s" + sectorID.ToString("D3") + "_" + (tileID+1).ToString("D3") + ".jpg"];
 
             if (e == null)
             {
                 Debug.Log("s" + sectorID.ToString("D3") + "_" + (tileID+1).ToString("D3") + ".jpg does not exist!");
                 return;//Don't be tryin' to draw no images that don't exist, fool!
             }
 
             MemoryStream data = new MemoryStream();
             e.Extract(data);
             byte[] imageBuffer = data.ToArray();
 
            
             Texture2D tex = new Texture2D(tileSize, tileSize);
             tex.LoadImage(imageBuffer);
 
             int borderWidth = 50;
             var borderColor = new Color(1, 0, 0, 1);
 
             //Nasty slow code for adding 50px red border.  For testing purposes only!
             for (var x =0; x < tex.width; x++) {
                 for (var y = 0; y < tex.height; y++) {
                     if (x < borderWidth || x > tex.width - 1 - borderWidth) tex.SetPixel(x, y, borderColor);
                     else if (y < borderWidth || y > tex.height - 1 - borderWidth) tex.SetPixel(x, y, borderColor);
                 }
             }
 
             tex.Apply();            
 
             //Create the sprite and apply to renderer----top,left,width,height--top left pivot, scale = 100 pixels per world unit  
             Sprite sprite = Sprite.Create(tex, new Rect(0, 0, tileSize * 0.975f, tileSize * 0.975f), new Vector2(0f, 1f), 100f);           
         
 
             //Store it in the buffer for later use.
             sBuffer.loadedSprites.Add(tileID, sprite);            
 
             renderer.sprite = (Sprite)sBuffer.loadedSprites[tileID];