Prime Minister Shinzo Abe was away at the time, attending a regional conference in Indonesia.
The drone was discovered close to the building's heliport by a worker at around 10:20 a.m., who raised the alarm and called the police, according to local media reports.
Images from TV news helicopters initially showed the drone covered with a piece of cardboard while police gathered. It was later covered by a larger blue sheet as about 50 investigators crowded around the area.
Later, local media reported that trace amounts of radiation were discovered on the drone.
Japanese police described the drone as about 50 centimeters in size with four propellers.
They said they have no idea how it got there and no one reported seeing it landing.
In January, a small drone was found crashed onto the lawn of the White House in Washington, D.C.
That turned out to be the result of a drunken late night flight by an unidentified worker for a government intelligence agency. No charges were filed in that case.
Martyn Williams covers mobile telecoms, Silicon Valley and general technology breaking news for The IDG News Service. Follow Martyn on Twitter at @martyn_williams. Martyn's e-mail address is martyn_williams@idg.com