Twenty years after the founding of Orenburg, the city had about ten churches, thanks to the efforts of the first governor I. I. Nepluev. One of the first was the church in honor of the apostles Peter and Paul, built at Nepluev's initiative for the garrison, when a battalion of the Nizhny Novgorod regiment was transferred here from Ufa with the blessing of Bishop Luka of Kazan and Sviyazhsk.

Construction of the church began in 1757, and it was consecrated in 1760. The stone temple with elegant vaults was considered one of the best in the city after the Transfiguration and Introduction cathedrals. However, as a result of a large fire in 1786, the building was severely damaged and remained in ruins for a long time. It was planned to be dismantled, but this was prevented by the military governor Grigory Semyonovich Volkonsky.

In 1804, Volkonsky appealed to Emperor Alexander I for the restoration of the military church, as regiments were constantly stationed in the city. The emperor granted the request, and funds were allocated from the treasury, with which the governor not only restored the church but also added a stone Cross Exaltation chapel to it. Church utensils from the burned wooden Cross Exaltation church, stored in the Introduction cathedral, were transferred here. In 1809, the temple was consecrated again. The following year, by the governor's order, all military personnel and officials of the city were assigned to the church, and in 1831, the Nepluev Cadet Corps, which did not have its own church until 1852, was also added. The newly consecrated temple featured a large glass entrance corridor. The church was surrounded by a fence, inside which was the grave of Orenburg's military governor Pavel Petrovich Sukhtelen, who died in 1833.

In 1853, the church updated the iconostasis, added new paintings, and acquired new utensils. The Peter and Paul church was a single-staff church, but after its transfer from the military department to the diocesan, it became a two-staff church.

In 1922, the church building was handed over to the House of the Red Army for a club. In the early 1940s, the temple was completely demolished. It was located at the corner of Soviet and Krasnoznamennaya streets. Now, on this site, there is a park named after Polina Osipenko, a pilot and Hero of the Soviet Union, where a monument to A.S. Pushkin and V.I. Dal is installed in 1998 in memory of the meeting of two great Russian writers in Orenburg.