Zac Affleck won't be telling his side of the story on the upcoming "Secret Lives of Mormon Wives" spinoff ... 'cause he didn't film a single scene with Jen Affleck before she pulled the plug on their marriage ... TMZ has learned. Production sources…