What needs to be done to remove In Stock Reminder app completely?

You just need to remove our script from Script Manager section of your Bigcommerce store according to the initial instruction: What needs to be done after In Stock Reminder app installation.