When you think about it, gov't is a very counterintuitive institution. Indeed, it only works because most people don't give it much thought. Since most of us were born into the system, we're conditioned to take the status quo for granted.
Like fiat money, gov't is ultimately an honor system. It works so long as enough people don't question the rules.
Gov't only works because the many obey the few. Gov't works because most people agree to do what a few people tell them to do.
But that raises an obvious question: why? What gives one adult the right to order other adults to do something? Why should I do what you tell me to do just because you tell me to do it? Even if you have the brute power to coerce compliance, what gives you the moral authority to obligate compliance?
There are different historical and theoretical justifications for legitimating the state:
I. Theological legitimation
i) Divine law
In the Mosaic theocracy, no one was above the law of God. Everybody, including the ruler, was answerable to the law. What legitimated the ruler was not so much how he became ruler, but whether he enforced the law of God. Moreover, the legitimacy of the ruler depended, in part, on the ruler's personal obedience. Everyone from the top down was accountable to the same standard.
At most, this amounted to a Constitutional monarchy or oligarchy (i.e. tribal elders).
But, ultimately the effectiveness of this system depended on divine enforcements. Credible threats of divine judgment in case of official disobedience or popular disobedience.
ii) Divine rulers
In paganism, you sometimes had the pretense of divine rulers. Rulers who were demigods, incarnations of a god, or descendants of a god. This made the ruler a superior being. Greater than mere mortals.
Although that made sense in principle, there were obvious practical difficulties. Those closest to the ruler, such as nannies and courtiers, could see firsthand that the ruler was all too human.
However, they played along with the ruse because they benefited from the system. It was not in their interest to blow the whistle on the ruler.
By the same token, this system required the ruler to be insulated from the masses Too much public exposure would give the lie to his divine pedigree.
Another vulnerability is that divine affectations raise expectations. If you claim to be a deity, you should be able to rise to the occasion. That's what was so devastating about the ten plagues of Egypt. What kind of god can't ward of natural disasters?
Likewise, a god who is defeated on the battlefield can't be much of a god. In that event, you were backing the wrong horse.
iii) Divine right of rulers
On this view, although the ruler is human, he is God's appointed vice regent. He is divinely authorized to speak and act on God's behalf. To oppose him is to oppose God. This was an underpinning for absolute monarchy.
In addition, it was often allied with a state church or state relation. The church legitimated the state while the state legitimated the church.
But it only works so long as the rulers don't render the theological presupposition implausible.
When the papacy becomes too corrupt or manifestly errant, the honor system breaks down. That's a problem with pope Francis. Does he speak and act like someone who enjoys special divine guidance? Isn't his behavior is indistinguishable from someone who lacks divine guidance?
Either the pope is divinely guided or not. How can you tell which proposition is true? How would Pope Francis conduct himself any differently if he wasn't divinely guided?
Likewise, when the French church, French monarchy, and French aristocracy became pervasively decadent, or when the Czar and the Russian Orthodox Church became pervasively decadent, the honor system broke down.
II. Anthropological legitimation
i) Warrior rulers
You can have a system in which the best warrior becomes chieftain. He defeats a challenger is mortal combat. Of course, that gives him a tenuous grip on power.
Similarly, a king who leads his armies into battle. He gains the respect of those who serve under him.
But, of course, that's very precarious. It makes him an easy and very inviting target. Instead of having to defeat an entire army, you just pick off the king, and his army will scatter or surrender.
ii) Elected rulers
Here the principle is: I will obey you because I granted you authority to act on my behalf, and that's revocable authority. You have authority over me because I ceded authority to you. And that's conditional. If you don't act in my interests, I will replace you with a different representative.
However, this begins to break down when you have judicial supremacy. What is there to legitimate the prerogative of judges who subvert the consent of the governed? Absent legitimation, why obey them?
Of course, the executive branch has the power to compel compliance to judicial rulings. But that just pushes the question back a step: Why should soldiers and policemen obey executive officials whose orders are illegitimate? If a judicial ruling is illegitimate, doesn't that delegitimate executive enforcement of said ruling?
There's the Hobbisean view that anarchy is worse than tyranny. Certainly anarchy is a worse-case scenario. But tyranny can be a worse-case scenario as well, viz. Stalinism, Maoism, &c.
If it comes down to force, why should the many defer to the few? The many can overpower the few.