Thoreau questions whether he’s a libertarian. Read the whole thing, I’m not going to even attempt to break that up.
There’s a lot of people out there, people that get way more attention than they deserve, who claim libertarianism yet have no real philosophical grounding to it (and no, selfishness does not count; you’re thinking Objectivism, not libertarianism). That the term started out as in effect a euphemism for anarchy has to be kept in mind when discussing it: anyone who can’t imagine some form of time-line the end of which would mean people could live in the absence of concentrated authority (not perfectly, I’m not talking utopia, but in a way that would be clearly worth maintaining compared to what came before) is not a libertarian. This doesn’t mean you have to be all Smash The State tomorrow, or even within the next few years, just that somewhere in your mind you can think about no central control & say “yeah, we could do that”.
I get the sense that Thoreau holds such a view. You pretty much have to in order to attach such a name to your own words. In contrast, most of the people catching hell over “libertarianism” do not. Merely wanting your central source of order to be something not considered a State — say, religious authority, big business, or some sort of throwback to old-world aristocracy — is not libertarian.

