Independent Alexander Van der Bellen beat the Freedom Party's Norbert Hofer by just 31,000 votes among the 4.64 million cast. The victor accepted there was a "rift" but said: "We are two sides of the same coin. Together we make up Austria."
Mr Hofer had run on a Eurosceptic, anti-immigration platform. If he had won, he would have become the first far-right head of state of a European Union nation.
Austrian Interior Minister Wolfgang Sobotka announced the results Monday in Vienna, revealing that Van der Bellen had won 50.3% of the ballot (2,254,484 votes) to Hofer's 49.7% (2,223,458 votes).
Van der Bellen, a 72-year-old economist, ran as an independent, although the Green Party, of which he was a former leader, financially backed his campaign.
Hofer had held a narrow lead in the neck-and-neck race on Sunday evening, with 51.9% of the vote to Van der Bellen's 48.1%, according to Austria's Interior Ministry.
In the end, the race was decided by more than 700,000 mail-in votes, accounting for 14% of eligible voters.
Van der Bellen, who fared better with urban voters, had been expected to secure more of the mail-in votes than his 45-year-old rival, a trained aeronautical engineer, who drew strong support in rural areas. Hofer had won 35% in the first round of voting last month to Van der Bellen's 21%.
In a Facebook post conceding to Van der Bellen on Monday, Hofer said he was thankful for the opportunity given, urged voters not to despair and said he would have been happy to take care of the country.
Jeremy Cliffe, political correspondent for The Economist, said that the campaign was notable for the "polarization" of the political landscape.
Candidates backed by the two centrist parties who had dominated Austrian politics since World War II - the left-leaning Social Democratic Party and conservative People's Party - had been eliminated in the first round of voting, resulting in a situation where the center was "really nowhere to be seen."
This was due to public disillusionment with the centrist parties -- seen as too "cozy" and "complacent" - combined with the migrant crisis, which had been a key issue on the campaign trail, over which Van der Bellen and Hofer had clashed.
"You're left with this clash between quite a conventionally left-wing politician in Mr. Van der Bellen and a far-right politician, Mr. Hofer."
While Hofer's Freedom Party campaigned against migration, Van der Bellen -- whose parents spent time in a refugee camp before settling in Austria -- has championed liberal migration policies.