The wing of the estate, owned by the president of the justice collegium and the chief member of the Moscow Mint Expedition, Alexei Alexandrovich Yakovlev, grandfather of the revolutionary and writer Alexander Ivanovich Herzen, was built in the 1770s. Initially, it was a one-story building. In 1812, the wing was completely destroyed in a fire, but was soon restored after the war with the French.

In the late 18th century, the estate was acquired by Elizaveta Mikhailovna Yermolova, sister of the well-known prince Sergei Mikhailovich Golitsyn, whose house was nearby. Around 1790, she married the former favorite of Catherine II, Alexander Petrovich Yermolov, who brought a significant dowry. After her husband left for abroad around 1800, Yermolova remained in Moscow, where she lived with her son Mikhail. After her death on August 16, 1833, Mikhail also moved to France. Elizaveta Mikhailovna was buried in the Donskoy Monastery.

In 1831, the state treasury acquired the building for the First Male City Gymnasium, which was already located nearby at Volkhonka, 16. The new premises housed classrooms, an office, and the director's apartment. On May 7, 1836, when Matvey Alexeyevich Okulov was the director of the gymnasium, he was visited by Alexander Sergeyevich Pushkin, a close friend of Pavel Voinovich Nashchokin, the brother of Okulov's wife. Pushkin mentions this visit in a letter to the poet P.A. Vyazemsky.

Since 1852, part of the premises was transferred to the administration of the Moscow educational district, for which the wing was expanded with a second floor. The institution remained here until the 1917 revolution.

In 1872, women's courses were opened in the building by Professor Vladimir Ivanovich Gerye, the first higher educational institution for women in the city, where future physicists, philologists, and doctors were trained.

In 1918, the presidium of the Russian Association of Research Institutes of Social Sciences (RANION) and six scientific institutions, including the Institute of Language and Literature under the leadership of Vladimir Maximovich Friche, were located here. The association was dissolved in 1929, and all institutes came under the management of the Communist Academy, which existed until 1936. In the same year, the building was transferred to the Academy of Sciences of the USSR. Currently, it houses the V.V. Vinogradov Institute of the Russian Language.