In the 1700s, the property belonged to the clerk of the Povar Sloboda, Ivan Ryabinkin. Around 1818, it was purchased by Prince S. Gagarin, who began construction. For a long time, authorship was attributed to O. Bove, but in 1975, during a scientific restoration led by architect O. Cherevata, it was established that the author is D. Zhilyardi.

The main house, unusually large for its time, incorporated an older structure, which is why the building is located deep within the main courtyard. Its atypicality is also evident in the facade design, where instead of an Empire-style colonnade, three arched niches are used at the second-floor level. The house is adorned with ornamental inserts-reliefs depicting eagles and laurel wreaths, as well as a richly decorated frieze under the central pediment. The keystones of the first floor are decorated with masks in the form of lion faces, modeled after G. Zamarayev.

Inside, the building is also unusual. The central feature is a grand staircase under a skylight, leading to the second floor, which houses a large ballroom, an anteroom, and a suite of open parlors, a grand study, and other rooms. The rooms are decorated with columns made of artificial marble with exquisite capitals, rich stucco work, and sculptural compositions, including works by I. Vitali and S. Campioni. In the grand study, a white marble fireplace with a tall mirror has been preserved, adorned with bas-reliefs of winged female figures on a globe and with a laurel wreath in hand, symbolizing Victory.

Simultaneously with the main house, a one-story stone building for the riding hall was constructed deep in the courtyard.

According to drawings from the early 19th century, the estate had greenhouses, utility rooms, and gazebos located in the garden behind the house.

After the construction was completed, Gagarin sold the estate to horse breeder and patron Okhotnikov, who in 1843 transferred it to the Moscow State Horse Breeding Administration. Horse exhibitions and sales were often held here, and the main house housed apartments for staff. In one of them, in 1875, settled the manager of the Imperial Horse Breeding Farms in Tula and Moscow, Major General L. Hartung, with his wife and eldest daughter of A.S. Pushkin. In 1877, a tragedy occurred: accused of embezzlement, the general shot himself in the building of the district court. Maria Hartung never remarried but lived a long life and died in 1919 in Moscow, never receiving the pension assigned to her by Lunacharsky.

After the revolution, military courses, the horse base of the Soviet government, and the International Lenin School were located here. In the 1930s, it housed the Budyonny Cavalry School, where Vasily Stalin and the sons of Anastas Mikoyan studied.

In 1937, the house was transferred to the Gorky Institute of World Literature of the USSR Academy of Sciences, which is still located here today. In 1957, a monument to Gorky, created by V. Mukhina and A. Zavarzin, was installed in front of the central entrance.