The Kremlin hill, formed by nature, is surrounded on three sides by the Trubezh and Lybed rivers. On the fourth side, it is protected by an artificial dry moat, dug by the city's residents in the 13th century. This moat could be filled with water, connecting the rivers and creating a water ring around the hill.

The earth extracted during the digging of the moat was used to construct the Kremlin rampart. Currently, its length is 290 meters, and its height reaches 18 meters from the outer bottom and 8 meters from the side of Verkhneilinskaya Street at the Kremlin. Until the 18th century, defensive walls and towers stood on the rampart. Two other ramparts were located within the fortified outpost adjacent to the Kremlin but were demolished during the city's reconstruction in the 18th-19th centuries.

During the Great Patriotic War, one of the city's air defense points was located on the rampart, and at its foot was an artillery school that operated almost until May 1945.

Kremlin Rampart Street is named in honor of this defensive structure.