I have a suggestion that might help on your battery wiring.
I don't have a down rigger, but I do have a trolling motor that I use all the time. I'm also bobbing around a longways out so I'm a little paranoid about my starting battery.
I have a Blue Sea Automatic Charging Relay (ARC)
https://www.marineoutfitters.ca/index.c ... 2085076016 , there's a bigger one if required.
This ACR is easy to wire and protects your starting battery voltage.
How it works is the starting and house (trolling motor) batteries are connected in parallel so that the running gas motor constantly charges both batteries. The house battery runs the big voltage items. The ACR prevents the house battery from drawing on the starting battery. This means you'll only drain the house battery and still have sufficient starting voltage in your starting battery.
http://assets.bluesea.com/files/resourc ... 014350.pdf When I'm not running my gas motor and just using my trolling motor, my trolling motor won't drain my starting battery. When I use my gas motor both batteries charge as usual.
The ACR also allows your to charge both batteries by charging either one. In my case, I have a single bank charger that is connected to my house battery and it also charges my starting battery.
I've used this setup for about 12 yrs on two different boats and have never drained a staring battery. I also carry one of those Lithium jumps starters as well.
Like I said, I'm paranoid.