The main house of the estate of Alexei Alexandrovich Yakovlev, who was the president of the justice board and the chief member of the Moscow Mint Expedition, as well as the grandfather of the revolutionary and writer Alexander Ivanovich Herzen, was built in the 1770s. Initially, it was a single-story building. Later, the house belonged to Prince Nikolai Fedorovich Lodyzhensky, a lieutenant general and participant in the capture of Paris, as well as to the noble family of the Stolypins.

In the late 18th century, the property was acquired by Elizaveta Mikhailovna Yermolova, sister of the famous Moscow nobleman, actual state councilor of the 1st class Prince Sergey Mikhailovich Golitsyn, whose house was nearby. Around 1790, she married the former favorite of Catherine II, Alexander Petrovich Yermolov, who brought a significant dowry. After her husband permanently moved abroad around 1800, Yermolova remained living in Moscow. As a result of the fire in 1812, the house was completely burned down but was restored immediately after the war with the French. It was raised by one more floor, the portico with columns was replaced with pilasters, and the pediment was removed. Elizaveta Yermolova died in Moscow on August 16, 1833, and was buried in the Don Monastery.

In 1831, the state treasury acquired the building for the needs of the First Men's City Gymnasium, which was already located nearby at Volkhnka, 16. It seems that the entire intellectual elite of Moscow emerged from these walls. Outstanding Russian historians Sergey Solovyov and Mikhail Pogodin, playwright Alexander Ostrovsky, writer and anarchism theorist Peter Kropotkin, professors N.V. Bugayev and N.A. Umov, and doctor V.P. Serbski studied here.

The gymnasium classes and office were placed in the new premises. In 1852, part of the premises was transferred to the administration of the educational district of the city of Moscow. In 1872, women's courses of Professor Vladimir Ivanovich Gerye opened in the building, the first higher educational institution of its kind in the city, where future physicists, philologists, and doctors were trained.

In 1918, the buildings at Volkhnka, 18 housed the presidium of the Russian Association of Research Institutes of Social Sciences (RANION) and six scientific institutions under its auspices, including the Institute of Language and Literature, which was headed by Vladimir Maksimovich Friche. The association was dissolved in 1929, and all institutes came under the management of the Communist Academy, which existed until 1936. In the same year, the building was transferred to the balance of the USSR Academy of Sciences. It now houses the V.V. Vinogradov Institute of the Russian Language.

By the end of 2018, the first stage of work on the preservation of the object was completed, and emergency measures were taken. Specialists reinforced the structures of the house, and now the premises are fully prepared for the main stage of restoration.