The Cathedral Square was bordered on the east by a street known as the Line of Government Offices, now called Belinsky Street. Here were located government institutions, including the First and Second Government Office Buildings, constructed in the 1780s and 1790s according to a unified design. Between them were the treasury buildings and the main guardhouse, which was transferred to the Penza Scientific Archive Commission in 1901.

Vissarion Grigorievich Belinsky was born in Sveaborg, but his true homeland is considered to be the town of Chembars (now Belinsky) in the Penza province. In August 1825, he arrived in the provincial city, successfully passed the entrance exams, and was enrolled in the first grade of the gymnasium located at the corner of Nikolskaya Street (now K. Marx) and Troitskaya Street (now Kirov).

At the end of 1828, Belinsky returned to Chembars, and at the beginning of the following year, he was expelled from the gymnasium for missing classes. Thus, he spent three years in Penza, which became a time of active intellectual work, intensive reading, and attending the theater.