Keep Your Transformer Working Like a Well-Oiled Machine