Olsen and husband 'clashed over having kids'