Button passes click to scroller

Hello, I have a scroll view and inside it is an object that blocks the scrolling. Inside that object, I have a button that if I press and drag it I can scroll. Is there a way to block the button from enabling the scrolling?