Matters of dispute in Jewish law are to be resolved in the highest court, the Beis Din Hagadol located near the Beis Hamikdash (Devarim 17:8,9, see Rashi). Everyone is commanded to follow the beis din's ruling, and not to stray right or left from it(17:10,11). In his introduction to the Mishna Torah, the Rambam says that this binding force of rabbinic rulings also applies to the decisions and enactments found in the Gemara, which was completed centuries after the Beis Din Hagadol ceased to function[1]. The Rambam explains that all, or most, chachmei Yisrael gathered and agreed to the Talmud's contents. As such, the decisions contained therein have the force of a beis din of all of Am Yisrael (Kovetz Shiurim, Kuntres Divrei Sofrim 2:4), and are binding on all of Am Yisrael, under pain of "lo sasur - you shall not stray."
In later generations the chachamim dispersed to many different locales, where each beis din ruled for the local residents who were bound by its decisions (but not bound by those of a different beis din). Halacha became decentralized, and the commandment to abide by the ruling of a single contemporary beis din no longer applied. The Minchas Chinuch (495:3) suggests that the mitzva of lo sasur applied only when the Beis Din Hagadol was near the Beis Hamikdash, as the pasuk implies.
The Sefer Hachinuch, on the other hand, cites the Torah's reference to "the judge in those days" (Devarim 17:9, 19:17) and the gemara's comment, "Yiftach (the least scholarly of the judges in Sefer Shoftim) in his generation is like Shmuel (the greatest scholar) in his generation" (Rosh Hashana 25b). Based on these sources, the Sefer Hachinuch (495) adds that just as we must obey Yiftach we must obey the greatest chachamim in every generation. Furthermore, he who does not obey the advice of the gedolei Torah of his generation violates this commandment and his punishment is great, because this is the strong pillar (amud) that the Torah depends upon.
The proof from Yiftach is questionable. Despite his relative ignorance, he presumably served on the Beis Din Hagadol, an institution whose rulings are binding on its generation. How, then, does this prove an obligation exists wowadays when there is no Beis Din Hagadol?
Harav Hershel Schachter suggests [Nefesh Harav p. 61,62] that Beis Din Hagadol's authority stems from the gedolim of the generation who serve on it. They are considered the rav muvhak, the primary rebbe, of all of Am Yisrael in their generation, and their rulings must be followed (see Tosfos Berachos 31b). However, if an individual has a rav muvhak, a personal rav, or a rav of his community, he is required to follow that rav's rulings, even his minority views. The townspeople of R' Eliezer, who chopped trees and made fires on Shabbos to make a mila knife, were rewarded. In R' Yosi Haglili's place, the people ate chicken cooked in milk (Shabbos 130a).
The Chazon Ish (Yoreh Deah 150:5) limits the right of a rav or beis din to argue against a greater rav or more numerous beis din. Only one who is worthy (higi'a l'hora'ah, see Sotah 22a), such as R' Eliezer and R' Yosi Haglili in their generation, may do so. Only then may their townspeople or students, broadly defined, follow the minority view which is disputed by greater scholars. One who is unqualified to rule, as well as the masses who do not have a qualified rav muvhak, must abide by the opinion of the greater of the disputants.
In his introduction to the Igros Moshe Rav Moshe Feinstein defines one who is qualified to rule (higi'a l'hora'ah) as someone who studies the sources in the Talmud and poskim to the best of his ability, with seriousness and fear of Hashem. If there is no contradiction to his ruling it is the "word of Hashem" (Eruvin 13b), even if the accepted psak halacha rejects his view. The rulings of R' Eliezer and R' Yosi Haglili are examples, and their followers properly obeyed them and were, therefore, rewarded[2].
The requirement to follow the greatest halachic authority in the absence of a rav muvhak is challenging for many individuals who are unable to determine which rav or beis din is greater. Presumably the common practice of scrupulous observant Torah Jews should be followed. The definition of a qualified posek is equally problematic. If in doubt, common practice of scrupulous observant Torah Jews should be followed. If more than one practice is common and traditional, one may rely on his rabbi to choose between them.
Innovative and non-traditional practices require a ruling of an eminently and undoubtedly qualified posek. One who honestly believes that his direct or indirect posek is so qualified may follow his rulings. Otherwise, the more common and traditional practices should be followed.
The Rambam (Hilchos Shemita V'Yovel 10:5,6) famously rules in accordance with tradition and common practice, against his own ruling (10:3,4). He explains that tradition and common practice are great pillars (amudim) in halachic ruling and it is appropriate to depend on them. In the merit of our best effort to follow the direction of the Chinuch and the Rambam, we will hopefully witness the restoration of the Beis Din Hagadol and the Beis Hamikdash.
[1] The term "beis din" is omitted in the Rambam's chain of mesora after R' Yochanan ben Zakai who lived during the churban
[2] If contradicted by the Gemara, the ruling has no validity. If it is contradicted by earlier authorities of which the later rav was unaware it is likewise invalid. If the later rav consciously argues with all recognized earlier authorities, it is matter of dispute whether the ruling is valid or not. See Rosh to Sanhedrin 4:6.