The Voronezh Palace is the only building of palace type in the city. It was built in 1777-1779 in the Baroque style as the residence of the Voronezh governor, General-Poruchik I. A. Potapov. Although there are legends that the palace was intended for Empress Catherine II or the last Crimean Khan, Shahin-Giray, these versions are not confirmed.

Despite its relatively small size, the palace impresses with its architectural grandeur and precision of proportions, evoking associations with the St. Petersburg palaces of B. F. Rastrelli. It was believed that the palace could have been built by Rastrelli himself; however, it is more likely that it was designed by the first provincial architect N. N. Ievsky, a student of D. V. Ukhtomskiy, known for his Baroque style. The first floor of the building is decorated with large rustication and is topped with a cornice, serving as the base for the two upper floors, which are united by Ionic columns. These columns support the cornice and adjoin the wall in the central part of the facade, crowned with a pediment. The high windows of the second floor stand out, above which are reliefs on the theme of weapons from different eras. The side walls are adorned with triangular pediments and stucco garlands.

After the owner's death, the palace changed hands several times. In the early 20th century, it was transferred to the city for the creation of a museum. In 1912 and 1914, restoration work was carried out under the guidance of V. I. Gain. However, during World War I, the building was temporarily used by medical and military institutions, and after the revolution, by the provincial archive bureau and health department with warehouses. Only in the mid-1920s was it transferred to the local history museum. From 1928 to 1932, the palace was overshadowed by the building of the Southeast Railway Administration, but during the post-war restoration, three high arches were made to open a view of the palace. Since 1959, it has housed the regional art museum named after I. N. Kramskoy. In 1984, an exhibition hall was added to the rear of the building, blocking the view of the rear facade with side risalits.