It blamed Awa Dabo, a Gambian national, for the "deteriorating performance" of her office, driven by what it called her "strained relations" with staff.
Neither she nor the UNDP has commented.
But local media have linked the shock move with Ms Dabo's alleged criticism of controversial elections on the Tanzanian island of Zanzibar last year.
In a statement, the foreign ministry did not comment on the reports, but urged the UNDP to remind its staff that their first priority was to work closely with the Tanzanian government to help it meet its development targets.
Speaking on condition of anonymity, a UNDP official said that Ms Dabo had been deported on the same day the government issued the order, under a security escort.
Zanzibar's controversial election re-run in March 2016 prompted a US government aid agency to withdraw nearly half a billion dollars of funding from Tanzania, describing the vote as "neither inclusive nor representative".
The opposition in the semi-autonomous archipelago boycotted the presidential poll, which was won by a landslide by the candidate from Tanzania's ruling Chama Cha Mapinduzi (CCM) party.
Opposition MP Paskal Haonga condemned the move to expel Ms Dabo, saying it soiled the reputation of Tanzania in international diplomatic circles.