Historically, the Balzer colony was Lutheran-Reformed. Initially, it was part of the evangelical parish of Messer, but in 1856 it became an independent parish. The first wooden church was built in 1777, and then it was rebuilt in 1821. It is known that the church was consecrated in 1839. The building, often seen in old photographs, was erected in 1849 and 1851. It was a spacious stone church with columns and a bell tower, designed for 1,350 seats, located in the center of the village. A parish school and pastorate operated alongside it. During the Soviet era, the church was closed, and now a cinema called 'Victory' stands in its place.

The Orthodox church in Krasnoarmeysk appeared after the collapse of the USSR. On May 2, 1993, a church in honor of the holy apostles Peter and Paul was consecrated in an old brick house on Kirov Street. A Sunday school for children was opened at the church. On July 12, 2004, Bishop Longin conducted the Rite of Foundation for a new church. While a spacious building with a bell tower is being constructed, services continue in the old premises.