The gorge, located 15 km from Sudak towards Feodosia, is known as "Kiziltash," which translates from Tatar as "red stone." This name is associated with the reddish hue of the local rocks, especially the medium-sized ones. It is believed that in the 7th century, this was the summer residence of St. Stephen of Surazh, the archbishop of Sudak. St. Stephen of Surazh, a historical figure, is known as a spreader of Christianity in Southeastern Taurida. The Greek synaxarion of the 12th-13th centuries mentions that December 15 marks his death, as he converted the inhabitants of Sudak to Christianity. Due to the secrecy of the Kiziltash area, now Krasnokamenka, archaeological excavations have not been conducted, so exact dates are unknown. Remains of medieval settlements dating from the 8th to the 15th centuries have been found around the gorge, confirming the possibility of the archbishop's residence existing here.

On the slope of one of the rocks, there is a wedge-shaped cave where, according to tradition, St. Stephen prayed. Deep inside the cave, there is a small pond where, in the 1820s, a Tatar shepherd found a floating wooden icon of the Dormition of the Mother of God. Rumors of the find attracted many pilgrims, and the water from the cave was considered healing, especially for treating eye diseases, deafness, and nervous disorders. Modern studies have confirmed the healing properties of the water, which has a sodium bicarbonate composition and contains hydrogen sulfide.

Recently, the entrance to the cave was closed off with a stone wall and an iron door. Security was installed at the entrance to the monastery, along with signs stating "Entry Forbidden for Tourists," which led to the spring drying up. In 2013, the monastery well was closed, and water can now be obtained at the church or purchased in Feodosia. Currently, tourists can only access the monastery from the side of Sandyk-Kaya, while entry from the Krasnokamenka side is prohibited.

In 1855, a men's monastery was founded in the gorge under the rock in honor of St. Stephen of Surazh. The monastery thrived until the 1917 revolution. The cave with the spring turned into a hermitage with cells for 50 novices, a church, and guesthouses. In 1894, there were 65 people living in the monastery. The forest turned into a garden and vegetable patch, a mill operated, and livestock was raised. The monastery attracted many pilgrims, who called it an earthly paradise or the Swiss of Crimea.

Although Kiziltash Monastery is mountainous, it is not a cave monastery. All its buildings were located on rocky terraces. The cave with the spring is of natural origin, while another cave, the Robber's Cave, served as a hideout for the robber Alim in the 19th century, according to legends.

Igumen Parfeny, a prominent abbot of the monastery, left a significant mark in history due to his ascetic life and martyr's death. He was killed on September 4, 1866, after clashes with local residents. In 2000, Parfeny was canonized by the Russian Orthodox Church.

In 1923, the monastery was closed, and a children's colony was established in its place, later becoming a sanatorium. During the Great Patriotic War, it served as a base for partisans. In 1950, construction began on a medium machine engineering enterprise, and the monastery buildings were destroyed. The territory was strictly guarded, giving rise to many legends.

In 1995, Archbishop Lazar of Simferopol and Crimea requested the transfer of church buildings. In 1997, the monastery was revived, and in 1998, Priest Nikolai Demyanyuk took monastic vows under the name Nikon. Currently, the monastery is slowly being restored, and its population consists of three people. The central church is consecrated in the name of All Saints of the Caves.

On July 28, 2007, the village of Krasnokamenka celebrated its first birthday, becoming an ordinary settlement. It is now accessible from Feodosia by public bus.