The history of the building begins in the late 17th century, when its owner was Yakov Baibakov, who served in the Pomernaya Chamber, where bulk goods were measured and accounted for sale. After him, the plot often changed owners, and the buildings on it burned down multiple times.

In the late 18th century, a new house was built here, which became part of a building constructed in 1901 for the schools of the "Auxiliary Society of Merchant Clerks." These schools offered lectures on commercial disciplines, including accounting, which, although not widely known, were popular.

Among the staff of the "Historical Library," there was a legend that the building belonged to "Aunt Griboedov," and that Griboedov himself read his comedy "Woe from Wit" here. Although there is no documentary evidence for this, it is known that the house belonged to merchant A.A. Kumanyin, who was married to the sister of F.M. Dostoevsky's mother. Dostoevsky often visited here.

Since 1938, the building on Starosadsky Lane has housed the Historical Library, founded on the collections of renowned literary scholars and book collectors. The foundation of its collections was the library of historian and archaeologist A.D. Chertkov. The library also holds collections from Prince A.I. Baryatinsky, archaeologist P.M. Stroeva, orientalist A.I. Gulyanov, ethnographer A.F. Gilferding, collector A.P. Bakhrushin, historian I.E. Zabelin, and others. In 1938, the funds of the library of the Institute of the Red Professorship were added here.

In the mid-1980s, a new glass and concrete wing was added to the building, housing the periodicals department. In 1990, the local history commission "Old Moscow" resumed its work here under the leadership of writer and local historian V.B. Muravyov.

During the restoration in 2015-2016, the load-bearing structures were reinforced, white stone elements from the 18th century were restored, facades were cleaned, and brickwork was renovated. The grand staircase was returned to its original appearance, and metal griffins hidden under old panels were discovered. The historical layout and finishing of the premises were restored. In 2017, the library received awards from the "Moscow Restoration" competition for the best restoration project and high quality of work.