Halle Berry's ex-husband Olivier Martinez worries about their son spending time with Halle because she has "impulse control" and "anger management" issues ... at least according to Olivier's lawyer. Patrick Baghdaserians joined us on "TMZ Live"…