The first documented owner of the building in the early 19th century was merchant Pyotr Giryakov, who was listed as the owner of the plot in 1804. Later, in 1828, the house was rented by Mikhail Semyonovich Shchepkin, a famous artist of the Maly Theatre. However, he soon moved to the Sretensky area. By 1829, the building passed into the ownership of "merchant's wife A.S. Ryabushinskaya," and this surname, later changed to "Ryabushinsky," remained the owner of the house for a long time.

The facade of the building is adorned with a high portico with four pilasters set on the arcade of the ground floor, where the vaults have been preserved. The exterior of the house is quite modest for its time: there is no stucco, bas-reliefs, or paintings; it is a simple merchant mansion located away from major streets. However, the fame of the new owners quickly spread throughout the Empire.

The Ryabushinsky family traces its origins to a peasant from the Kaluga province, from the village of Rebushevo, Mikhail Yakovlev, who arrived in Moscow in 1802 and traded in the Canvas Row of the Gostiny Dvor. Bankrupted due to the Patriotic War of 1812, he was forced to transition to the merchant class but did not lose heart. In 1820, Mikhail Yakovlevich adopted Old Belief, which many wealthy merchants of that time adhered to. Support from his co-religionists helped him return to the merchant class and resume trading. At that time, he took the surname "Rebushevskiy," which later changed to "Ryabushinskiy." Over time, Mikhail Yakovlevich founded several textile manufactories in Moscow and the Kaluga province and left his heirs more than 2 million rubles. This capital became the foundation of the dynasty's prosperity: his son Pavel Mikhailovich increased the family fortune tenfold, leaving his sons 20 million rubles.

The Ryabushinskys rented out the estate in Golutvino but retained ownership until the revolution. In the 1840s, the tenants were the Ryabushinskys' neighbors, the Tretyakov dynasty, whose house, where brothers Pavel and Mikhail were born, has been preserved in the neighboring alley near the Church of St. Nicholas the Wonderworker in Golutvino. By the end of the 19th century, the Ryabushinsky estate was converted into a charitable institution and dining hall, managed by the Imperial Society for the Promotion of Humanity. For this purpose, the building was expanded with an annex from the courtyard and service areas. Behind the estate rise the buildings of the Golutvino manufactory, founded by Mikhail Yakovlevich Ryabushinsky and later sold to merchants Grigory and Alexey Istomin.

After the revolution, the estate was nationalized, and for some time it housed apartments, then offices. Today, the house stands closed and unused. Meanwhile, it could become an ideal venue for a local museum, where there would be room for Shchepkin, the Ryabushinskys, the Tretyakovs, and the Istomins.