Olive oil quality

I am very fond of butter, and cook with it a little too much.
I am trying to switch over to olive oil (or another oil )and wonder just how much difference there is in the quality of the products..
The grocery store shelves have a baffling array of different ones from tiny very expensive ones to large vats. Is there much difference in the tastes?