From the second quarter of the 18th century, in the area of Bolshaya and Malaya Polyanka, where trading and craft settlements once existed, noble estates began to be built. The house at modern Malaya Polyanka, 9, has been known since the 1750s as part of the estate of Prince Fyodor Andreyevich Baryatinsky, a judge and state councilor. He lived there with his wife and 11 children. After his death in 1760, the estate was divided between his wife and sons. The site included stone chambers, wooden buildings, and a garden.

Later, the estate was owned by representatives of well-known noble families: court councilor A. M. Voeykov, major Y. P. Rtishchev, and second-major N. D. Bibikov. In 1819, when the estate belonged to the Polish noblewoman Elizaveta Nikolaevna Naranovich, a two-story mansion with a stone first floor was built, which became the basis of the modern property.

By 1835, the owner became the lady-in-waiting Elizaveta Nikolaevna Ofrosimova, presumably changing her surname after marriage. The house, which stood out from the estate, faced the red line of Malaya Polyanka and was an Empire-style mansion with an Ionic portico and a stepped attic.

From the mid-19th century, the house was owned by merchants. Initially, its owner was the wife of a Moscow merchant, Maria Petrovna Antonova, and by 1875, the estate passed to merchant A. E. Viktorov. At this time, wooden buildings were replaced with stone ones. The house was inherited by Viktorov's daughter Zinoviya, who sold it in 1900 to Ivan Akimovich Vorontsov.

Vorontsov significantly remodeled the mansion according to the design of architect I. P. Mashkov. The building received a modern facade in an eclectic style with decorative elements. Extensions were added, and a two-story service wing was built. Along the southern boundary of the estate, another wing was erected, housing various utility rooms and a garden.

After nationalization, the estate became communal housing, where about 50 people lived in 1925. Since 1976, it has housed a psychoneurological dispensary. Despite this, the layout and some interiors of the building have been preserved. Restoration is planned in the near future, including the restoration of monumental painting.