The "Gardener's House," which likely originally served as a soap house, was built in the early 19th century on the grounds of the Keeper's Courtyard. It was used as a bathhouse for the numerous staff and soldiers who maintained the Tsaritsyn park and greenhouses. The architect's name is unknown, but it is believed to have been someone from the circle of V.I. Bazhenov.

In the 1840s, the building was repurposed for the needs of the Volost Administration. In the 1930s, it was reconstructed and expanded. From the 1930s to the 1970s, it housed the People's Court, and later various institutions, including a Medical-Labor Sanatorium in 1975. In the second half of the 1970s, the building was dismantled.

In 2006-2007, the Capital Archaeological Bureau conducted excavations, during which the original foundations of the building were discovered. They represented a rectangular structure with a semi-rotunda on the eastern facade.

According to the General Plan of the village of Tsaritsyn, a new fruit garden was laid out next to the "Gardener's House" in the early 19th century. Currently, groups of decorative shrubs such as spirea, mock orange, lilac, and hydrangea are planted here, enhancing the area.