Many Russian monasteries had hermitages that served as places of solitude and silence for hermit monks. Thus, the monks of the Valuy Monastery initially built a wooden house in the forest, but later established a hermitage in chalk caves located a kilometer from the monastery across the Oskol River. Research by speleologists conducted in the late 20th century showed that the caves in the Belgorod region are of natural origin. It is difficult to determine the exact date of the founding of the cave hermitage, but by indirect evidence, it can be attributed to the mid-17th century.

In the early 20th century, due to the poor condition of the underground galleries, a comprehensive restoration was carried out in the caves. Dangerous areas of the walls and ceilings were reinforced, and cracks and gaps were sealed with plaster. The arrangement of the hermitage began in 1890, mainly funded by donations from parishioners. The work was carried out by both the monastery brotherhood and trusted local residents. The caves were cleared, expanded, and reinforced, and in 1910, an architect declared them safe. A temple was created in the caves in honor of St. Ignatius the Godbearer. The caves are located at an altitude of 106 meters above the water level of the river.

It is estimated that the total length of the underground passages is about one kilometer, of which 640 meters have been cleared to date. The width of the caves reaches one meter, and the height is about three. The underground temple was consecrated on May 4, 1914, in honor of Ignatius the Godbearer, a saint who suffered martyrdom for the Christian faith in the 1st century AD. On the mountain above the caves, construction began in 1916 on a church in the name of the Transfiguration of the Lord, but it remained unfinished. In 1917, the church was burned down.

Restoration work began in 2005, and in 2007, the cultural and historical complex "Cave Monastery in the Name of Ignatius the Godbearer" was opened, transferred to the Russian Orthodox Church. Currently, there is a temple of Ignatius the Godbearer at this site.