The Kenya Red Cross tweeted that response teams were at the scene, in the Kware Pipeline Embakasi area.
The Star newspaper said dozens of people had been evacuated moments before the collapse.
Witnesses told the paper that the building had been condemned after cracks appeared in its walls.
The co-ordinator of the rescue efforts, Pius Masai, said that more than 100 people had been accounted for, but added that some people may still be trapped.
"Rescue efforts are ongoing," he said in a statement, and appealed for people with access to "cutters, drillers and any other extrication equipment" to help with the search.
Rescuers appealed for help from the public in the search for the missing
The National Disaster Management Unit said that most families acted when ordered to leave the building prior to its collapse, with 121 people making it to safety.
Local media reports that some people re-entered the building apparently to collect their belongings when it caved in, possibly trapping them. Police said they do not know how many people are trapped.
Authorities said 121 people had made it to safety before the collapse
Building collapses are a problem in Kenya with many people in Nairobi living in low-income areas or slums. Housing is in high demand, and developers often bypass regulations.
In April 2017, 49 people died after a building collapsed in heavy rain in Nairobi.
At the height of Kenya's rainy season in April 2016, a six-storey building collapsed killing 52 people in Nairobi's poor Huruma district.
Rescue workers are searching the rubble for survivors