Hugo Barra, was the face of the smartphone maker's global expansion and the vice president in charge of global operations for the last three and a half years.
Xiaomi was briefly the world's most valuable startup and had hopes to be China's Apple. But it saw slowing smartphone sales and fell out of the top five in China for smartphone vendors in 2016, after reaching No. 2 in 2015.
Barra said that living in such a singular environment have taken a huge toll on his life and started affecting his health.
"Seeing how much I've left behind these past few years, it is clear to me that the time has come to return."
Barra was based in Beijing, though often traveled to other strategically important markets including India and his home country of Brazil.
In a letter released to employees earlier this month, Xiaomi Chief Executive Officer Lei Jun said "the worst is over", referring to the company's recent struggles to keep up with an ever increasing number of local competitors.
However Barra saw many roadblocks, including a halt of production in Brazil due to the country's economic instability, and a temporary ban in India in 2014 over patent infringements.
Barra said he would officially leave the role after Lunar New Year, which runs until Feb. 2, and take time off before embarking on a new role. He did not specify what that was.
A company spokeswoman confirmed to Reuters that Barra has a new role lined up in Silicon Valley, and that Senior Vice President Xiang Wang will lead the company's global efforts going forward.