In the late 18th century, there was a estate of staff captain P.N. Dmitrov at this site, which later passed to Prince A.N. Vyazemsky. In the 1840s, the property was divided: one part went to the Danilov sugar factory and restaurateur I.Y. Testov, while the other was purchased by merchant Yakov Yakovlevich Ermakov. A former serf of the Sheremetev counts, he established a weaving production in his native village of Meshcherino in the Kolomna district, which soon became the largest in the area. In 1838, Ermakov bought his freedom along with his family, moved to Moscow, became a merchant, and opened a calico printing and dyeing factory behind the Three Hills Gate. His son, Frol Yakovlevich, expanded his father's business, making the firm 'Yakov Ermakov and Sons' one of the most successful in Moscow.

However, in the early 1880s, the factory behind the Three Hills Gate was closed. The reasons for this are unknown, but it is likely that the deaths of close relatives affected Frol Yakovlevich: his wife Maria, sons Andrei and Pavel, brother, and uncle. In the last ten years of his life, Ermakov dedicated himself to charity. On the site of the factory, a charity house for 500 people with a church was established; the building had a G-shape with a bell tower and altar apse. The Church of Philaret the Merciful and the charity house were consecrated in 1889, and later a lower church in honor of the Dormition of Righteous Anna was added. The church was built in memory of Metropolitan of Moscow Philaret (Drozdov). By the order of Frol Yakovlevich, an orchard was planted nearby, named Ermakovskaya, which is now commemorated by Ermakovskaya Grove street.

The charity house had a second branch in Sokolniki, designed for 500 people. Ermakov also donated to other charitable institutions in the Moscow province, fed the poor with free meals, gave money to those in need, and allocated funds for the construction of churches and monasteries.

Around 1922, the house church of Philaret the Merciful was closed, and the charity house was liquidated. Part of the Ermakov charity house was demolished, but the Church of Philaret was preserved. Later, a new infectious disease building of the GKB No. 9 named after G.N. Speransky was built on the site of the demolished part and around the remaining structure.

At the suggestion of Patriarch of Moscow and All Russia Alexy II, restoration work began at the Church of Philaret. According to the project by M.V. Isaev (CTRК 'Preobrazhenskoe'), the building regained its historical appearance: the two-light hall of the church was restored, divided by arches with the altar, the bell tower with an entrance from the second floor, and a grand staircase uniting three floors. The restorers preserved the Monier vaults, paying special attention to the decoration of the metal beams.

In 2012, the site became a laureate of the Moscow Government's competition for the best project in the field of preservation and popularization of cultural heritage sites 'Moscow Restoration.'