growing up in a low stress home will probably help. stress (cortisol) will methylate your DNA and inhibit testosterone which are never good for a guy regardless of his age.
Just stop. Most women have no idea about penis size and will pretty much believe whatever you tell them as long as you don't say you have a foot long pecker.
Depression pretty much comes down to feeling like a worthless human being and thinking that you'll never feel better. None of the things you mentioned necessarily prevent someone from feeling that...