2D Platformer Character Acts Weird When Collide Side of a Platform

I'm trying to put a platformer game together. I wrote a character controller script, but character acts weird when collide with side of a platform. Here's a video: https://www.youtube.com/watch?v=QJlCmYONnZ0

My character movement script:

 if(Input.GetKeyDown(KeyCode.W) && isGrounded)
             {
                 //rigidBody.AddForce(new Vector2(0, jumpSpeed * 100));
                 rigidBody.velocity = new Vector2(rigidBody.velocity.x, jumpSpeed * 2);
                 isGrounded = false;
             }
     
             if((Input.GetKeyDown(KeyCode.S) || Input.GetKeyDown(KeyCode.DownArrow)) && !isGrounded)
             {
                 //rigidBody.AddForce(new Vector2(0, -1 * jumpSpeed * 100));
                 rigidBody.velocity = new Vector2(rigidBody.velocity.x, -jumpSpeed * 2);
             }
     
             if(Input.GetKey(KeyCode.A) || Input.GetKey(KeyCode.D))
             {
     
                 if (Input.GetKey (KeyCode.A)) {
                         rigidBody.velocity = new Vector2(-speed, rigidBody.velocity.y);
                 }
     
                 if (Input.GetKey (KeyCode.D)) {
                         rigidBody.velocity = new Vector2(speed, rigidBody.velocity.y);
                 }
     
                 if(isGrounded)
                 {
                     changeState(STATE_WALK);
                 }
             } else
             {
                 if(isGrounded)
                 {
                     changeState(STATE_IDLE);
                 }
             }

Player object contains a box collider and rigidbody2D also. What can cause that, and how I get rid of that?

Thank you in advance.