One kilometer from the monastery, across the Oskol River, in the chalk mountains, there are caves that house a church dedicated to Ignatius the Godbearer. It is unknown who exactly began the creation of these caves, but since 1897, monks have been arranging and expanding them. The work was difficult and was once halted by an order from the authorities. In 1910, the caves were cleaned and inspected by an architect, who confirmed their safety and the stability of the ground. The caves stretched for more than 300 fathoms (about 640 m) and were located at a height of approximately 50 fathoms (106 m) above the water level of the river. The width of the caves was about 89 cm, and the height was about 3 m. Inside the caves, a temple was created, approximately 8 m long, 6.5 m wide, and 5.5 m high.

The underground temple was consecrated on May 4, 1914, in honor of Ignatius the Godbearer, who suffered martyrdom for his faith in the first century AD. A chapel was built at the entrance to the caves, and in 1916, the construction of a church in the name of the Transfiguration of the Lord began at the top of the mountain, which remained unfinished. A skete was established at the site where the caves were dug, where up to 10 people lived in the summer and somewhat fewer in the winter. The caves, along with the temple and chapels, became a decoration of the monastery. T.M. Oleinikov wrote that this place resembles Mount Tabor, hence it was named the 'Tabor Godbearer-Ignatius Cave Skete.'

In 2005, a board of trustees was created to restore the monastery's shrines. From 2005 to 2007, a wooden church of St. Ignatius the Godbearer was built at the top of Lysaya Mountain above the caves. During this time, the underground passages and the cave temple were cleaned, a viewing platform and adjacent areas were arranged, and a park zone was created.