In the 19th century, beyond the Pokrovskaya outpost, there was an ancient village of Kalitniki, named after the Grand Duke of Moscow Ivan Kalita. Beyond the village stretched the Karacharovsky and Sukino swamps. According to another version, the village's name is connected to the neighboring Kalitnikov stream.

In 1771, during a plague epidemic, Kalitnikov Cemetery was established, becoming the most "rural" of all plague burials in the city.

The cemetery originated from the Simonov Monastery, where a plague quarantine was organized in 1771. Deceased monks were taken beyond the Pokrovskaya outpost for burial outside the city. Over time, a citywide cemetery emerged here.

The cemetery primarily buried local peasants, the poor, and workers. Graves were located around the wooden church of the Bogolyubskaya Icon of the Mother of God. Marble and stone monuments appeared here in the second half of the 19th century.

After the fire of the wooden church in 1834 and 1838, a stone temple was built at a new location according to the design of architect Nikolai Kozlovsky. It was dedicated to the icon of the Mother of God of All Who Sorrow and had chapels of Nicholas the Wonderworker and Alexander Nevsky in the Empire style. In 1881, architect V. Verigin reconstructed the temple, and the cemetery fence was created by architect N. Markov.

During the Soviet era, the Church of All Who Sorrow became an important church for Renovationists, where Metropolitan Alexander Vvedensky often served, buried behind the apse. Around the church are the graves of other clergymen and the Blessed Elder Schema Nun Mother Olga, revered by the faithful.

In the Soviet period, the cemetery began to bury artists, musicians, scientists, and architects. The most significant burial is that of the People's Artist of the USSR Yevdokiya Turchaninova, who played over 70 roles in the plays of Alexander Ostrovsky. Also resting here is avant-garde artist Robert Falk, one of the founders of the "Jack of Diamonds" association. In the central part of the cemetery is the monument-chapel of architect Afanasy Grigoriev, who designed the Lopukhina House on Prechistenka, now the Leo Tolstoy Museum, and the Khrushchyov estate, which houses the Pushkin Museum.

Also buried here is architect Fyodor Shestakov, who created the Provision warehouses at Krymskaya Square and the bell tower of the Church of Nicholas the Wonderworker in Tolmachi. The grave of the author of the monument to Alexander Pushkin on Tverskoy Boulevard, Alexander Opekushin, is also located here.

Among other notable figures buried at Kalitnikov Cemetery are murdered journalist and former intelligence officer Vadim Biryukov, legendary Soviet spy Yelizaveta Zarubina, and literary scholar Felix Kuznetsov.