Smooth Camera Angle Scroll

So I'm working on a camera script that will work similar to Diablo 3's camera that will zoom in on the player and change the camera angle as the scroll wheel moves forward - and do this smoothly. Just working with the angle for now I seem to be running into a problem. When I start up, I seem to be getting the desired result, the scrolling is smooth, but upon each scroll, it seems to get less smooth - like something in the script is compounding it. It's probably something obvious, I'd just like another set of eyes if possible. Thanks!

 using UnityEngine;
 using System.Collections;
 
 public class CameraScroll : MonoBehaviour {
     
     
     public float zoomSpeed = 5;
     public float minAngle = 0;
     public float maxAngle = 180;
     private float newAngle;
     public float zoomUpdateSpeed = 10;
     private float angle;
 
     // Use this for initialization
     void Start () {
     
         newAngle = transform.localEulerAngles.x;
     }
     
     // Update is called once per frame
     void Update () {
     
         newAngle -= Input.GetAxis ("Mouse ScrollWheel") * zoomSpeed;
         newAngle = Mathf.Clamp (newAngle, minAngle, maxAngle);
         
         transform.localEulerAngles = new Vector3 (Mathf.LerpAngle(transform.localEulerAngles.x, newAngle, Time.time * zoomUpdateSpeed),0,0);
         
         }
 }