In 1581, Sassa Narimasa, a retainer of Oda Nobunaga became lord of the castle. After the Honnoji Incident of 1585, there was a falling out between Sassa Narimasa and Hideyoshi, and Hideyoshi attacked the castle with 100,000 soldiers, destroying it. Maeda Toshinaga later rebuilt the castle and temporarily used it as his retirement home until much of it burned down in 1609. In 1661 Maeda Toshitsugu rebuilt the castle again and made it his home. His descendents ruled over Toyama from here until the Meiji Restoration, when it was dismantled in 1870. Post-WWII, it was rebuilt as a modern concrete structure in 1954. Nowadays, the castle remains were renovated as the Toyama Castle Park, a place to relax and it is utilized as 富山市郷土博物館ーToyama Local History Museum, and 富山市佐藤記念美術館ーSato Memorial Museum, noted for its collection of utensils for Tea ceremony and antiques.