This is Burke's theory of government in his own words:
"That the State ought to confine itself to what regards the State, or the creatures of the State, namely, the exterior establishment of its religion; its magistracy; its revenue; its military force by sea and land; the corporations that owe their existence to its fiat; in a word, to every thing that is truly and properly public, to the public peace, to the public safety, to the public order, to the public prosperity."
I think he might well have added public health to this list if the notion had existed back then, rather than the harsh brutallism of modern Republican theory. He was about continuity of government within a traditional framework, rather than minimising it. He was in favour of incrementalism rather than reduction. He preferred pragmatism to ideology. He believed in government function rather than obstruction.
Alongside the state, and not instead of it, he believed in the free market. For me, in modern terms, he was a cautious liberal rather than a tub-thumping conservative.