The Christmas Church, also known as the Church of the Descent of the Holy Spirit, was located at the intersection of Christmas and Trinity Streets, now known as Gorky and Kirov. The view from there opened to the east, towards the Penza River.

The stone temple, dedicated to the Nativity of Christ, was built in the 1770s. It included a small chapel located at the intersection of Trinity and Christmas Streets, now known as Kirov and Gorky.

The name of Christmas Street originated from the church. It started at the Penza River and stretched to Popovka Street, now called Klyuchevskogo. This street also housed the city’s Alexandrovskaya almshouse.

In 1931, after the closure of the Church of the Holy Spirit, the wooden chapel at the corner of Trinity and Christmas Streets was converted into a kerosene shop, having first lost its dome. In the early 1960s, it was demolished due to dilapidation.