The history of the village of Kastornoye dates back to 1590, when its name was first mentioned in written sources. At the end of the 16th century, there was a wooden fortress surrounded by a moat, rampart, and palisade, where the first inhabitants were service Cossacks. It is believed that they arrived from Ukrainian cities, as in 1575 Ivan the Terrible dismissed Cossacks from Liven and Elets who dared to submit a petition to him.

According to the 1859 census, Kastornoye was a state village on the Olym River with a church, a fair, and a market, where 1,105 people lived. Until 1923, the village was part of the Zemlyansky district of the Voronezh province. Under Catherine II, a stone church was built in Kastornoye, consecrated on August 15, 1763, in honor of the Assumption of the Blessed Virgin Mary. Archimandrite Dmitry (Sambikin) noted in documents from the 1880s: "The church in the village of Kastornoye, Zemlyansky district, with chapels of St. Barbara the Great Martyr and St. Nicholas the Wonderworker, is a stone building with a bell tower, built in 1779. There are 30 desyatins of arable land and 3 desyatins of hayfields. In the enclosure, there are two stone storages. There are 1,660 parishioners. Two states. Villages: Moscow Kolodez, Otradnoye, Nikolayevka, Pokrovskoye, Znamenskoye, Dmitriyevskoye, Klyuchi, Andreyevka, Arkhangelskoye, Znameno-Arkhangelskoye, Novo-Uspenskoye."

In 1872-1873, the priests Pavel Dionisievich Troitsky and Nikolai Skryabin served at the Assumption Church. At the turn of the 19th-20th centuries, the church staff included three priests (Ioann Nikolayevich Bogomolov, Dmitriy Ivanovich Nechaev, and Tikhon Peskov), a deacon (Alexander Nikolayevich Minervin), and two psalmists (Pyotr Savvich Smirnov and Ivan Kurbatov). The church had 33½ desyatins of land, and the parish counted 536 households with 4,125 residents.

P. Nikolsky in his works states that the church was built in 1799. Priest Dmitriy Nechaev is also mentioned in 1912: "On August 26, 1912, in the Kastornoye parish church, with a large gathering of people, the dean of the 3rd district, priest Dmitriy Nechaev, served a requiem liturgy and a thanksgiving prayer in honor of the 100th anniversary of the Patriotic War of 1812."

In 1937, the church was converted into a cultural house. Until the bells were removed, Father Dmitriy Nechaev conducted the services, who kept a chronicle of Kastornoye throughout his life. However, the chronicle did not survive, as it perished in the fire of the revolution. Currently, the Assumption Church is a monument of history and culture of the peoples of Russia and an object of cultural heritage.