The wooden church at this site is first mentioned in 1625, although there are reasons to believe it existed earlier. At that time, there was the State Kazennaya Sloboda, which was responsible for storing the royal treasury and other property. In 1676, merchant Daniil Pivovarov built a stone church, the main altar of which was consecrated in honor of the Kazan Icon of the Mother of God. Two chapels, Nikolsky and Yakovlevsky, were arranged in the refectory, and the church received its name from the latter. The church took on its final appearance in the second quarter of the 19th century after significant reconstruction.

Initially, the stone church of the 17th century had a characteristic appearance for that time: a five-domed church with kokoshniks and facades decorated with patterned bricks. The tent-like bell tower on the west side was replaced in the mid-18th century with a three-tiered baroque one. In 1831 and 1833, instead of the five domes, the church received a wide dome on a drum with large windows, three altar apses were replaced with one wide one, the brick decor from the facades was removed, and the building acquired a restrained Empire style while retaining the foundation of the 17th century. The reconstruction work was commissioned by merchant Ivan Matveyevich Yartsev, who was then the mayor of Moscow. In 1841 and 1844, landlady Tatyana Ivanovna Tumakova updated the refectory and chapels in the Empire style.

In the early 20th century, a parish charity for the poor and a small almshouse operated at the Yakovlev Church. In 1932, services ceased, and the appearance of the church changed significantly: pipes, extensions, and stairs appeared, and a garage and mechanical workshop were placed inside. Over time, the dome collapsed, the roof became overgrown with trees, serious cracks appeared in the walls, and the church practically turned into ruins. Only in 1995 did services resume, and later the church was restored. At that time, the alley originally called Yakovlevsky, renamed in 1961 to Elizarova Street, became Yakovoapostolsky in the 1990s.