The plot of land where the church now stands was purchased by the Russian Spiritual Mission in Jerusalem in 1909. However, the local Turkish authorities did not grant permission for construction. Archimandrite Leonid (Sentsov), who headed the mission, suggested first building a dining hall for pilgrims arriving in Haifa by sea. The construction of the church was overseen by engineer Boris Gurevich.

In 1912, the mission began to seek permission for the redesign of the dining hall. An altar was added to the building, a gray marble iconostasis was installed, and a bell tower was erected. At the beginning of 1913, permission was granted, aided by the celebration of the anniversary of the Romanov House. The Russian site in Haifa was proposed to be named the 'Romanov Courtyard.' The resolution of the issue took longer than expected and required additional formalities. On November 14, 1913, the church in honor of the Prophet Elijah was solemnly consecrated by Patriarch Damian of Jerusalem.

The report of the Russian Spiritual Mission states that 'the church of Elijah on Mount Carmel near the city of Haifa is very small. It is built in the shape of a Greek cross and in the Greek architectural style, but it is quite sufficient for the few pilgrims and local Orthodox Arabs. The iconostasis is small, single-tiered, but good. The vestry and utensils are modest. Most items were taken from the Trinity Cathedral in Jerusalem, some were donated.'

World War I disrupted the peaceful life of the church. The influx of pilgrims from Russia virtually ceased, and the church remained inactive due to the lack of a priest. In the 1920s-1930s, an Arab Orthodox priest, Archpriest Nikolai, served in the church, and after his death, he was buried next to the church on the Russian site. After that, services were held only sporadically when pilgrims arrived with priests.

After the establishment of the State of Israel, the properties of the Russian Spiritual Mission, including the Church of the Prophet Elijah in Haifa, came under the administration of the Moscow Patriarchate (previously they were under the jurisdiction of the Russian Orthodox Church Outside Russia).

In the spring of 1991, Miroslav Vityv was ordained a priest at the Church of the Holy Sepulchre, who became the rector of the Church of the Prophet Elijah. After the visit of Patriarch Alexy II of Moscow and All Russia in the same year, the pilgrimage life revived, churches began to be restored, and new priests appeared.