The Krestitel Church is located on the southwestern outskirts of the city, on the grounds of the Krestitel Cemetery, founded in 1772, among dense shrubs and trees. From 1774 to 1777, a stone church dedicated to John the Baptist was built.

In 1828, a refectory with a bell tower and two chapels dedicated to the icon "Joy of All Who Sorrow" and the Kazan Icon of the Mother of God were added to the church. In 1870, the church was expanded with stone annexes on both sides. In the 1820s, a two-story stone almshouse was built on the cemetery grounds. On September 12, 1863, it was visited by Tsarevich Nikolai Alexandrovich, the son of Emperor Alexander II, who passed away in 1866. In 1864, the cemetery was enclosed by a brick wall. Since 1867, the almshouse housed a shelter for widows and orphans of the clergy.

In the 1930s, the church was closed. It operated from 1942 to 1944 but was closed again afterward. It reopened in the 1960s. The church building is marked with a memorial plaque as a monument of architecture. The necropolis is the resting place of Metropolitan Palladiy of Oryol and Bryansk (1896, 1976), Archbishop Varfolomey of Oryol and Bryansk (1928, 1988), as well as local historians Y. I. Gorozhansky (1858, 1913) and P. I. Sizov (1931, 1989), and writer and literary scholar L. N. Afonin (1918, 1975).

In the southern part of the cemetery, there is a mass grave of Soviet soldiers who died in the battles for the liberation of Oryol in August 1943. In 1983, to mark the 40th anniversary of the city's liberation, a memorial was erected at this site. The names of the fallen soldiers and commanders are inscribed on metal plates. Here, on May 8, 1987, a restored individual monument to the commander of the 61st Army squadron, Major T. G. Verkhoviy, who died on May 3, 1943, was unveiled. He was originally buried in the Tankist Square, and in 1952, his remains were transferred to the Krestitel Cemetery.