The Republican People's Party (CHP) has questioned the legitimacy of the close result, citing irregularities in the electoral process.
President Recep Tayyip Erdogan's push for an executive presidency succeeded with just over 51% of the vote.
The win was met with both celebrations and protests across Turkey.
The CHP is refusing to accept the Yes victory and is demanding a recount of 60% of the votes, criticising a decision to pass unstamped ballot papers as valid unless proven otherwise.
Three of Turkey's biggest cities - Istanbul, Ankara and Izmir - all voted No to the constitutional changes.
Opposition supporters took to the streets of Istanbul to bang pots and pans - a traditional form of protest - in a series of noisy demonstrations.
Meanwhile, flag-waving supporters of Mr Erdogan celebrated as their president praised them for their "historic decision" that could keep him in office until 2029.
With 99.97% of ballots counted, the Yes campaign had won 51.41% of the votes cast, while No had taken 48.59%. Turnout was said to be as high as 85%.
Separately, three people were shot dead near a polling station in the south-eastern province of Diyarbakir, reportedly during a dispute over how they were voting.
Responding to Sunday's result, the European Commission issued a statement urging Mr Erdogan to respect the closeness of the vote and to "seek the broadest possible national consensus" when considering the far-reaching implications of the constitutional amendments.
Death penalty next?
"Today... Turkey has taken a historic decision," Mr Erdogan told reporters at his official Istanbul residence, the Huber Palace. "With the people, we have realised the most important reform in our history."
He called on everyone to respect the outcome of the vote.
Supporters of Mr Erdogan celebrated the referendum result in Istanbul
The president also said the country could hold a referendum on bringing back the death penalty - a move that would end Turkey's EU negotiations.
Deputy Prime Minister Veysi Kaynak admitted the Yes vote had been lower than expected.
What's in the new constitution?
The draft states that the next presidential and parliamentary elections will be held on 3 November 2019.
The president will have a five-year tenure, for a maximum of two terms.
- The president will be able to directly appoint top public officials, including ministers
- He will also be able to assign one or several vice-presidents
- The job of prime minister, currently held by Binali Yildirim, will be scrapped
- The president will have power to intervene in the judiciary, which Mr Erdogan has accused of being influenced by Fethullah Gulen, the Pennsylvania-based preacher he blames for the failed coup in July
- The president will decide whether or not impose a state of emergency
Mr Erdogan says the changes are needed to address Turkey's security challenges nine months after an attempted coup, and to avoid the fragile coalition governments of the past.
The new system, he argues, will resemble those in France and the US and will bring calm in a time of turmoil marked by a Kurdish insurgency, Islamist militancy and conflict in neighbouring Syria, which has led to a huge refugee influx.
Critics of the changes fear the move will make the president's position too powerful, arguing that it amounts to one-man rule, without the checks and balances of other presidential systems such as those in France and the US.
They say his ability to retain ties to a political party - Mr Erdogan could resume leadership of the AKP he co-founded - will end any chance of impartiality.
Some protesters banged pots and pans in the Turkish capital after the results were announced
CHP deputy leader Erdal Aksunger said he believed there had been irregularities in the count: "Many illegal acts are being carried out in favour of the 'Yes' campaign right now.
"There is the state on one side and people on the other. No will win in the end. Everybody will see that."
The pro-Kurdish Peoples' Democratic Party (HDP) also challenged the vote.
Many Turks already fear growing authoritarianism in their country, where tens of thousands of people have been arrested, and at least 100,000 sacked or suspended from their jobs, since a coup attempt last July.
The campaign unfolded under a state of emergency imposed in the wake of the failed coup.
Mr Erdogan assumed the presidency, meant to be a largely ceremonial position, in 2014 after more than a decade as prime minister.
Under his rule, the middle class has ballooned and infrastructure has been modernised, while religious Turks have been empowered.
Relations with the EU, meanwhile, have deteriorated. Mr Erdogan sparred bitterly with European governments who banned rallies by his ministers in their countries during the referendum campaign. He called the bans "Nazi acts".