The candidates are incumbent Basuki Tjahaja Purnama, a Christian of Chinese descent, and Anies Rasyid Baswedan, a Muslim.
Critics of Mr Purnama have accused him of blasphemy, heightening racial and religious tensions.
Officials are expected to announce early results soon.
Hardline Islamist groups say Mr Purnama insulted a Koranic verse during a campaign speech and have rallied large crowds against him in recent months. He is now on trial for blasphemy, which he denies.
Correspondents say this has made the election a choice between secularism and a growing hardline Islamist movement in Indonesia.
How did the vote go?
That though extra security officers have been deployed to polling stations, the mood in Jakarta has not been overly tense, and there is still a festive atmosphere.
Police said the election proceeded "smoothly and safely with no significant disruption", although there were a few "incidents". Reuters reported that 15 people were detained after disturbances at polling stations.
Convoys of special military police have been put on standby as well
A coalition of hardline Islamic groups supporting Mr Basedan previously said that it would send at least 100 activists to each polling station to monitor voting. But correspondents said they had a very limited presence.
Election officials are now counting the votes. A preliminary result based on approved private polls will be announced by the end of the day, with the official result due in early May.
Mr Purnama, also popularly known as "Ahok", voted with his family in north Jakarta early on Wednesday morning.
He told reporters: "Jakartans must use their voice as the future of Jakarta is in their hands. Don't be afraid, the police are here providing security."
Mr Baswedan (centre) is a former education minister
Police had warned against voter intimidation.
Rizieq Shihab, leader of the Islamic Defenders Front which has been leading protests against Mr Purnama, also cast his vote.
Asked by the BBC if his group was damaging Indonesia's pluralist democracy, he said: "Democracy doesn't stop someone from voting for a person from the same religion as you.... Christian vote for Christian, Muslims vote for Muslim."
Riziek Shihab (left) is the leader of the Islamic Defenders Front
The election has also seen anti-Chinese sentiment, sparking unease in a country that has seen violence against its Chinese minority previously.
A number of Chinese Indonesians who turned up to vote told the BBC that they were not intimidated.
"Politics is cruel. There will be threats, physically or verbally. But I still feel comfortable even though I'm of Chinese descent.... So far they only scream out hatred but haven't really acted on it," one voter, Rudi Irmawan, told.
Chinese voters were seen at several polling stations
What is the controversy about?
Mr Purnama is the first Christian and minority ethnic Chinese leader of Jakarta in over 50 years.
He stepped into the role from the deputy post without election in 2014, when his predecessor, Mr Joko Widodo, became president.
Mr Purnama was accused last year of insulting a Koranic verse during a campaign speech, which he has denied, saying his comments were aimed at politicians "incorrectly" using the Koran against him.
Hardline Islamists have cited that verse from the Koran to support an argument that Muslims should not vote for a non-Muslim leader.
The hardline Islamist Tamasya Al Maidah group sent activists to polling stations, but their presence was limited
Correspondents say his rivals have also heavily capitalised on these allegations to harness the Muslim vote, with Mr Baswedan meeting the Islamic Defenders Front twice publically.
If convicted, he faces a maximum five-year jail sentence, although he could still govern while appeals are heard. His trial is due to resume on Thursday.
How has it affected the outcome?
The controversy has clearly hurt Mr Purnama's chances. Once considered the clear frontrunner, he won the first round of the election in February with only 43% of the vote, while Mr Baswedan had 40%.
The run-off is now taking place as Mr Purnama failed to win that first round by a sufficiently large margin.
The Jakarta Post has described the campaign as "the dirtiest, most polarising and most divisive the nation has ever seen".
Indonesia is the world's most-populous Muslim country. About 85% of its population are Muslim, but the country officially respects six religions.