The work, which consists of a panel known as the Tavola Doria, depicts a scene from the famous Battle of Anghiari of 1440 in which Florence and its allies defeated troops from Milan during the Wars in Lombardy.
It was believed to be part of a mysterious painting known as the 'Lost Leonardo' which Da Vinci is believed to have painted on a wall of Palazzo Vecchio, the medieval town hall in the heart of Florence, in 1505.
The Tavola Doria went on display this week at the Quirinale Palace, the official residence of President Giorgio Napolitano and shows four soldiers on horseback locked in furious combat.
"A universally recognised masterpiece has returned to Italy's possession," Mr Napolitano said in a statement.
The painting measures 45x34in and was last seen in public in Milan.
But Italian art police say after the panel was stolen from its owners in Naples it passed through a Swiss art dealer, before being sent to Germany for restoration in the 1960s.
The artwork then travelled to a New York art gallery before ending up in Japan where it was eventually bought by the Tokyo Fuji Art Museum in 1992.
The Japanese museum agreed to send it back to Italy after lengthy negotiations and an arrangement which will give it joint access to the work for 26 years.
In a joint statement released by Italy and Japan, Roberto Cecchi, the Italian culture ministry's under-secretary, said he was "thrilled" to see the work return to Italy.
"We are immensely grateful to the Tokyo Fuji Art Museum for their most generous donation and look forward to our co-operation with the museum,' he said.
After its stint at the palace ends in January, the painting is expected be housed at the Uffizi Gallery in Florence.
Art historians have been looking for the remains of the Battle of Anghiari, which they believe lies behind a wall painted by Giorgio Vasari when he was commissioned to do another fresco in Florence's medieval municipal headquarters.
In March this year a team led by Professor Maurizio Seracini said it had found evidence that the painting still exists behind a cavity, underneath a section of Vasari's fresco in the grand chamber, the Hall of the Five Hundred.