The Rambam (Hilchot Berachot, 8:2) rules that the blessing over olive oil is “Shehakol”, unless one drinks it to cure a throat ache, in which the blessing would be “HaEtz”. Nevertheless, the Shulhan Aruch (Orah Haim 202:4) says that, since olive oil has a detrimental effect to one’s health, one would not recite any blessing before drinking it. If one were to mix the olive oil with something else, however, the Shulhan Aruch says that one would recite “HaEtz”.
Rabbi Baruch Toledano (Kitzur Shulhan Aruch, §199:16) explains that nowadays it is common for people to consume olive oil and that not only is it not harmful, but on the contrary, many doctors recommend it. As such, he writes that the appropriate blessing would be “HaEtz”. The Ben Ish Hai (Shana Alef, Parashat Pinhas, § 9) rules similarly. On the other hand, Rabbi Ovadia Yosef (Yabia Omer, vol. VI, § 48:6) challenges this approach based on the ruling of the Shulhan Aruch.
Nevertheless, there are grounds for supporting the opinion of Rabbi Toledano and the Ben Ish Hai since the olive oil on the market today is much more refined and less acidic than what used to be available previously. Even though Rabbi Ben Zion Abba Shaul (Or LeZion, ch. 14, § 1) rules like Rav Ovadia Yosef and suggests that today’s olive oil may still have unknown harmful effects, there is sufficient support to recite “HaEtz”.
Summary: There is a Halachic foundation for reciting “Bore Peri HaEtz” when consuming olive oil.