Since the early 1680s, the property near the Yauza Gates, next to the walls of the White City (in the 1760s, boulevards were created where the demolished walls once stood), belonged to the chamberlain Alexei Petrovich Golovin. Wooden chambers were located here. His son, boyar Fyodor Alexeyevich, a well-known admiral and close associate of Peter I, one of the first in Russia to receive a count title, built a brick church in his urban estate between 1685 and 1689, consecrated in honor of the Kazan Icon of the Mother of God, with facades executed in baroque style.

Over time, in 1748, the old chambers of the Golovins burned down. Soon after, the property was acquired by Prince S. I. Shcherbatov. In the late 1750s, his widow Natalia Stepanovna built stone two-story chambers in the style of Elizabethan baroque and connected the residential building with the church by a passage.

In 1785, the estate was sold to Privy Councillor Andrei Dmitrievich Karpov and his wife Natalia Alexeyevna. After the demolition of the walls of the White City, the estate, which had previously faced west, turned towards the boulevards. At the end of the 18th century, the house was rebuilt in classical style.

The mansion was not damaged in the fire of 1812. In 1822, it was acquired by retired Major General Nikolai Zakharovich Khitrovo, after whom the neighboring Khitrovskaya Square was named. Nikolai Zakharovich, a combat officer, participated in the wars of 1805-1811 against France and Turkey, took part in the siege of Brailov, and retired after being wounded. He was a member of M. M. Speransky's circle, for which he was exiled to Vyatka and then to his estate near Tarusa. Thanks to the intercession of his father-in-law, Field Marshal M. I. Kutuzov-Smolensky, who was married to his daughter Anna, he was pardoned in 1813 and moved to Moscow.

Under him, the old Shcherbatov house was rebuilt, and its facades acquired decorations in the Empire style. A sculpted coat of arms of the Khitrovo family appeared in the tympanum of the pediment, and on both sides of the six-column portico, sculpted male profiles in antique armor. According to one version, this is a sculptural portrait of the estate's owner.

The house church was also rebuilt and consecrated in honor of the Tikhvin Icon of the Mother of God, especially revered in the Khitrovo family.

After N. Z. Khitrovo's death in 1826, the estate was sold by his heirs. In 1843, it passed to Guard Colonel Vladimir Ivanovich Orlov, and in 1851, to his widow Ekaterina Dmitrievna. The couple had no children, and according to V. I. Orlov's will, after his widow's death in 1889, the estate was transferred to the Moscow Charitable Committee for the Poor of the Imperial Philanthropic Society.

In the old estate, the "Orlov Hospital of the Moscow Committee for the Poor, for incoming indigent patients" was organized, under the patronage of Prince Alexander Petrovich of Oldenburg. The previously closed house church was reopened and consecrated in honor of the Smolensk Icon of the Mother of God. The hospital operated daily from 10 AM to 2 PM and had "a special department with 5 beds for surgical patients and a pharmacy with free dispensing of medicines." The charitable institution was intended for the residents of Khitrovka, where they were treated, underwent minor surgeries, and were fed in the dining room.

After the revolution, both the hospital and the church were closed. The Church of the Smolensk Icon of the Mother of God was demolished in the 1930s during the construction of the Residential Building of the Military Engineering Academy named after V. V. Kuybyshev.

In 1922, the main house of the Golovins, Shcherbatovs, Khitrovo, and Orlovs was occupied by paramedic courses, reorganized in 1923 into a three-year paramedic school named after the Department of Health of the All-Medical Labor. In 1928, the school was transformed into the Clara Zetkin Medical Polytechnic (since 1954, Moscow Medical School No. 2 named after Clara Zetkin). Currently, it houses the Clara Zetkin Medical College.

In the late 20th century, the building was restored. On the western facade, where the entrance to the medical college is now located, restorers restored decorative finishes in the baroque style of the mid-18th century. On the opposite facade, elements of decorative finishes from the time of N. Z. Khitrovo have been preserved.

The building of the former Orlov Hospital is under state protection as a cultural heritage site of federal significance.