The Kazan Icon of the Mother of God church on the monastery grounds was built from 1874 to 1885 under the guidance of Abbess Arsenia. The church project was developed by Professor I.I. Gornostayev, an academician of the St. Petersburg Academy of Arts. Mother Arsenia was the daughter of the famous General Mikhail Vasilyevich Sebryakov, the leader of the Don Cossacks, who also contributed donations for the construction of the church. After the completion and consecration of the church, Arsenia, along with three nuns, began to dig caves beneath it. These labyrinths, according to the abbess's design, symbolized the history of the last days of Christ, including Holy Week, the betrayal of Judas, and the crucifixion of Jesus. The caves are divided into two paths: one symbolizes the path of Christ, the other, that of the Mother of God, and both lead to Golgotha. In 1905, Arsenia passed away at the age of 72 and was buried in the crypt of the lower church of the monastery.

With the advent of Soviet power, the monastery was closed, the church was looted, and the coffin with Abbess Arsenia's body was opened. The body was found to be incorrupt, but the next morning the coffin disappeared and has not been found since. Currently, the Ust-Medveditsky Spaso-Preobrazhensky Monastery houses the upper church of the Kazan Icon of the Mother of God and the lower church of St. Arseny the Great. The underground church, Mother Arsenia's cave, where the Holy Stone is located, is also being restored.