The Sign Church that exists today was built in the 1890s, but it is far from the first building on this site. As early as 1736, at the request of Igumen Ion and by the decree of Archbishop Joakim, a chapel was added to the Vlasyevskaya Tower of the Afanasyev Monastery. This tower served as the fortress gate of the Earthen City, and on its eastern facade was depicted the image of the Mother of God Sign, while on the western facade was the image of the Savior, which, according to tradition, was a copy of the image on the Savior Tower of the Moscow Kremlin. The image of the Mother of God symbolized heavenly protection for the city. Initially, a wooden staircase led to it, and later a wooden chapel belonging to the Vlasyev Church was added. In 1736, due to a lack of funds at the Vlasyev Monastery, Archbishop Joakim ordered that services be conducted by the monks of the Afanasyev Monastery, and donations were directed to its benefit.

In 1779, the wooden chapel was replaced with a stone one, but in 1785, Governor A.P. Melgunov decided to dismantle the Sign Tower to use the bricks for the construction of the House of Charity. However, thanks to the intervention of the city mayor Andrei Barsov, who offered the governor 15,000 bricks, the tower was preserved. Shortly after this, the governor's wife fell ill and dreamed of the image of the Mother of God, which foretold her recovery on the condition that the tower be preserved.

In 1850, the stone chapel was expanded and decorated with paintings. In 1861, with the funds of A.I. Olovyanishnikov, a stone church was built in honor of the icon of the Mother of God Sign. In the 1890s, thanks to the efforts of Bishop Ioannikiy and architect A.A. Nikiforov, the church was reconstructed and expanded. On May 18, 1897, its solemn consecration took place. The image of the Savior on the western facade of the tower was also venerated, with a service held in front of it annually on August 1.

Shortly before the October Revolution, an altar in honor of Anna Kashinskaya appeared in the church, mentioned in inventories from the 1920s. This was likely initiated by Archimandrite Cornelius.

In the 1930s, when local authorities planned to demolish the church, Bishop Sophronius appealed to Glavnauka to preserve the icon of the Mother of God Sign as an archaeological value. However, after inspection, the fresco was deemed secondary, and soon it was destroyed. The Sign Church was closed on March 8, 1932, and its property was transferred to the Church of the Friday on Tugovaya Mountain.

In the 1990s, the church's dome was restored, and the building housed the Youth Film Studio. In 2001, Archbishop Mikhail proposed to create an Orthodox cultural center in Yaroslavl and return the church to the church, but the issue was delayed due to the lack of space for the film studio. Only thirteen years later, in 2015, the church was returned to the Yaroslavl Diocese, and on January 6, the first service was held after an 80-year break.