The current building is based on stone chambers built in the last third of the 17th century, when this property belonged to brothers Mikhail and Ivan Guryev, wealthy merchants with guest status. This status exempted them from paying duties and city taxes, as well as granting them the right to travel abroad freely for trade. The Guryev family originated from Yaroslavl, where they engaged in trade. In the 1640s, they built a fort on the Yaik River (now Ural), which became an important Russian outpost in the northern Caspian region, later the city of Guryev, now Atyrau. At that time, they also began developing fishery operations on the Yaik River.

In 1671, the Guryev brothers became the owners of a estate in Potapovsky Lane, where they likely erected the stone chambers. These single-story buildings on a high basement were located deep within the property and faced the alley with their end facade. In the 1688 "Description of Yards," the house was described as "the courtyard of the Guryev guests, with chambers in their courtyard, half covered with turf, and the other half with boards, and a high tented attic, covered with boards."

In 1728, Alexey Guryev sold the estate to Rodion Koshelev, the stable master of the Palace Horse Department. The new owner added a stone annex and updated the facades in accordance with Baroque fashion: the windows were adorned with Baroque moldings, semi-columns were installed between them, and the walls were decorated with rustication.

The Kosheleves owned the estate until the 1780s, when they sold it to merchant Mikhail Kolosov, who then transferred it to English merchant Doughty. Kolosov expanded the house, and Doughty built a new wing along the alley in the late 1790s, which was later combined with the main house. The estate was not damaged in the 1812 fire.

In 1821, Kaluga merchant Pyotr Zolotarev acquired the estate and made it a profitable property: the house and wings were rented out. Over the years, private educational institutions were located here, where well-known professors from Moscow University taught. Among the students were notable figures such as doctor Sergey Botkin and historian Vladimir Gerye.

From 1870, the estate briefly passed to merchant and patron Vasily Kokorev, who used it as a profitable property. Under him, part of the house was raised, an arch for access to the courtyard was arranged, and the facades acquired a modern appearance.

In 1881, the estate was purchased by the Abrikosov merchants, owners of a famous confectionery factory in Moscow. The large family of Alexey Ivanovich Abrikosov occupied two large apartments, while the others were rented out. The interiors of the Abrikosov apartment were exquisitely decorated: stucco, marble window sills, and patterned parquet.

The estate remained in the Abrikosov family until the 1917 revolution, after which it was nationalized and turned into communal apartments. In the 1930s, the house was raised by two floors using short-lived materials.

The interiors of the Abrikosov apartment were preserved almost in their original form until the fire in 2009, when the Soviet annex was damaged, and firefighters flooded all floors with water. The premises required urgent restoration. As of 2018, emergency works had been carried out on the site.