Tell us about yourself!
About problem of burning low side mosfet. Did You tried to add pulldown resisors at SD pin and IN pin of mosfet driver? You can add also the shottky diode in place of low side mosfet. You can also add blocking dc diode at output of converter. This will lower the efficiency but converter will work. Also, You can use another mosfet driver with two separate inputs for low side and high side mosfet. Then You will have a opportunity to use a comparator for detecting reverse current and closing low side fet before short circuit will happen. If You will have more questions feel free. I am also working at my own design but I want to use current mode dc-dc controller like uc3843 (or newer) and arduino for monitoring and mppt :)
With IR2104 converter will not work also in light load environment. This is half bridge driver, low side mosfet stay open whole time when high side mosfet is closed - this will also short the battery circuit without blocking diode at output. I am recommending changing topology to non-synchronous buck converter / adding blocking diode at output / changing mosfet driver with two separate inputs HIN/LIN + comparator for example at VS node (checking negative voltage when circuit is closing through low side fet in continous conducution mode).