Fireball not moving after time change

Okay so I'm working on a wizard game and I was doing great today up until I noticed a bug when I went to try to record a bit of progress. I'm spawning a fireball and it works great when I'm running around and just shooting fireballs. I can use my teleport ability with no problems and still continue to fire the projectiles. The problem I'm running into is, once I use my slow motion/slow time ability and I come back to full speed and go to shoot. The fireball just dies off and falls limp, it doesn't look like it's adding any kind of force after I come back to normal speed. I'm not great with the who time stuff so I'm not sure what is making it all happen this way. Was hoping someone else might have an idea or has ran into a similar problem.

I will include the scripts for where the spell should spawn, the fireball and the slow motion so that you can try to replicate the problem or see if you can figure out what's wrong with it just from the code. I've also uploaded a build of the game to DropBox, I will post the link here and you can test it out and see what I am talking about.

Playable DropBox Download:

https://www.dropbox.com/s/4h74cl3g1hs5dk6/Build.zip?dl=0

Keys: WASD T = Teleport G = Slow Time F = Gravity Off v = Gravity On

CODE - Fireball using UnityEngine; using System.Collections;

 public class Fireball : MonoBehaviour {
 
     [SerializeField]
     private float appliedThrust = 150.0f;
     public GameObject explosionPrefab;
     private Rigidbody rb;
     private Vector3 direction;
 
     // Use this for initialization
     void Start () {
         rb = GetComponent<Rigidbody>(); //Grab our rigidbody
         direction = new Vector3(0, 2, 15); //What direction we will send our fireball, 2 up, 15 forward
         rb.AddForce(transform.TransformDirection(direction) * appliedThrust); //Apply our Thrust to our Rigidbody in our direction
     }
     
     // Update is called once per frame
     void Update () {
     
     }
 
     private void OnCollisionEnter(Collision collision)
     {
         //Prevents our fireballs from hitting each other and exploding if we adjust speed back up
         if (collision.gameObject.tag != "Fireball")
         {
             Instantiate(explosionPrefab, transform.position, transform.rotation); //Instantiate an explosive prefab
             Destroy(gameObject); //Destroy this fireball
         }
     }
 }

CODE SpellSpawner

 using UnityEngine;
 using System.Collections;
 
 public class SpellSpawner : MonoBehaviour {
 
     public GameObject fireballPrefab; //We need to be able to tell our script what item to spawn in the game
     private bool canShoot; //We set a bool to let our engine know if we are able to shoot or not
 
     // Use this for initialization
     void Start () {
         //When starting the game, our character should be able to shoot
         canShoot = true;
     }
     
     // Update is called once per frame
     void Update () {
     
     }
 
     public void Fireball()
     {
         //If the player is able to shoot
         if (canShoot == true)
         {
             //Start our Coroutine, we use a Coroutine so we can prevent the player from shooting rapidly
             StartCoroutine(FireballDelay());            
         }
         else
         {
             //If we can't shoot, we return to the top to try the procedure over again
             return;
         }
     }
 
     //We need a way to prevent the player from constantly spamming fireballs
     private IEnumerator FireballDelay()
     {       
         canShoot = false; //Tell our engine that the player can no longer shoot
         yield return new WaitForSeconds(1.5f); //Wait for 1.5 seconds in order to line up our animation and fireball spawn
         //This also keeps the player from firing for 1.5 seconds
         Instantiate(fireballPrefab, transform.position, transform.rotation); //Spawn a fireball
         canShoot = true; //Tell our engine that the player is able to shoot again
     }
 }

CODE - SlowMotion

using UnityEngine; using System.Collections;

public class SlowMotion : MonoBehaviour {

 [SerializeField]
 private float slowdownFactor = 0.05f;
 [SerializeField]
 private float slowdownLength = 2.0f;

 private void Start()
 {
 }
 private void Update()
 {
     Time.timeScale += (1.0f / slowdownLength) * Time.unscaledDeltaTime;
     Time.timeScale = Mathf.Clamp(Time.timeScale, 0.0f, 1.0f);
 }

 public void EnableSlowMotion()
 {
     Time.timeScale = slowdownFactor;
     Time.fixedDeltaTime = Time.timeScale * 0.02f;
 }

}

I know the canShoot isn't really used, I changed the code and just left it in so I could use it later when I go back to change things, it's not really hurting anything so I figured why bother taking it out.