The territory of the estate is located on the site of an ancient settlement from the 14th century. The first mentions of the Kudryavtsev family in Panskoye can be found in the records of the wife of Major General Dmitry Ivanovich Kudryavtsev, dated 1829. During the war with the Poles, a wounded Russian officer found refuge with them and proposed to Sofia Alexandrovna. At that time, he was 40 years old, and she was only 12.5. Initially, the girl did not want to get married, but over time she changed her mind, and their wedding took place in 1796.

According to legend, in 1812, Panskoye was the headquarters of the Cossack ataman General M.I. Platov. By this time, the Kudryavtsevs had already built a manor house here. A few decades later, the founders of the estate passed away, and the estate passed to their son. In the mid-19th century, the estate began to decline and soon became impoverished.

After the October Revolution, the estate was used for sewing courses, after the Great Patriotic War, as a pioneer camp, and later as an orphanage.

In the second half of the 20th century, the building was used as a children's home. Now the abandoned house is gradually falling into ruin.

The main single-story building, clad in planks and featuring wooden mezzanines, has a wooden six-column portico and a balcony with stairs attached. A wing has been preserved 50 meters from the house.