A bearded man, wearing black trousers and a grey sweatshirt, was seen being held against the fence with his arms behind his back by a police officer outside the Palace of Westminster.
The incident is not thought to be terror related. The Carriage Gates entrance to Parliament is closed.
Scotland Yard said the man - aged in his 30s - was arrested on suspicion of possessing a knife.
He was held with his arms against his back just yards from the forecourt of the building, where PC Keith Palmer was stabbed to death by terrorist Khalid Masood in March.
BBC deputy political editor John Pienaar said the latest incident ended very swiftly with the man being taken away in a police van.
While the gates are closed, the road and pavement opposite remained open and the incident had no affect on traffic in Parliament Square, he said.
Security around Westminster was tightened following the attack by Masood, who was shot dead after fatally stabbing PC Palmer as he guarded Carriage Gate.
A review of security at the perimeter of the parliamentary estate was ordered by Commons Speaker John Bercow following the incident.
The use of armed officers and physical barriers have become more visible, while a permanent barrier has been erected to protect pedestrians on Westminster Bridge, where Masood murdered four people and injured more than 50 others by mounting the pavement in a car.