From 1759 to 1771, a new stone church with two altars was built in Kolpna, replacing the old wooden one. It was located in a new place. One pre-revolutionary publication mentions that there were two churches in Kolpna. This likely refers to a short period in the mid-19th century when a new Peter and Paul church was being built to replace the dilapidated 18th-century church, which was remembered by the older generation. Its appearance has been preserved in photographs. Currently, part of this church's site is occupied by the Communications House and the adjacent square. The old brick church may have been nearby, on the site of the current asphalted area in front of the Communications House, where a chapel once stood.

In the book "Kolpna a Hundred Years Ago," Nikolai Chernov notes that the new church in the center of Kolpna, likely built in the 1870s, was indeed beautiful.

In the 1930s, the church in Kolpna was destroyed, and services were not held for several decades. In the early 1990s, the Orthodox community appealed to Bishop Paisiy of Oryol and Livny to resume services. Temporarily, services were held in the preserved church's baptismal guardhouse.

In 1993, with the blessing of Bishop Paisiy of Oryol and Bryansk, the foundation for a new church in honor of Saints Peter and Paul was laid outside the village, opposite the cemetery. In 1995, Bishop Paisiy consecrated the site of the future church, where a capsule and the first brick were laid under the altar. The construction was funded by donations from benefactors and local residents. For over ten years, the walls of the church were erected. Four crosses for the domes were purchased with the funds from the head of the Kolpna traffic police, I.V. Novgorodov. With the help of the former head of the district administration, M.N. Bukhtiyarov, a "Bell Tower" with six bells was acquired. Significant assistance was provided by V.A. Gromov, head of the Kolpna district, and the leaders of the sugar factory: I.V. Potapenko, M.V. Pavlenko, and Y.A. Sergeev. Continuous support was given by the chairwoman of the district consumer cooperative, I.A. Vasina, and farm leaders: G.D. Polyanchikov, V.M. Sergeev, V.M. Shelekhov, and V.I. Belyaeva.

An active donor was Grigory Gurevich, president of JSC NK "Nobel Oil," whose funds covered more than half of the interior finishing works, as well as the purchase of chandeliers, church utensils, and candlesticks. His main gift was a unique iconostasis.

The consecration of the new church took place on November 2, 2006, on the feast day of Saint Artemius the Great Martyr. A large icon of Saints Peter and Paul, gifted by Major General Nikolai Shebanov, hangs in a special place in the church.