In fact private enterprise supplies many public goods, although few commentators think they should provide all public goods. Lighthouses are often cited as an example of essential services that only the state can provide, but the Nobel laureate Ronald Coase showed that many lighthouses were indeed built and operated by private enterprise.
They had their origin in the hilltop fires that were lit near ports to guide incoming ships. These eventually evolved through wooden or stone towers into their modern form with steady improvement in their illumination. They were financed by contributions from nearby ports, which incorporated the costs into landing fees charged on boats entering the harbour. The state's role was to allow operators to levy such charges, in order to counter free riders who might seek to benefit from the lighthouse without contributing to its upkeep. When the state took over their maintenance and operation, it was not because they were failing, but to standardize the charges which were then subject to wide local variations.
Britain's Royal National Lifeboat Institution (RNLI) is an example of an independent public service supported by voluntary contributions rather than out of taxation. For a few years in the 19th century the RNLI did take government money, but found its private contributions dropped off by more than it received in tax support, so it reverted to voluntary finance, which it maintains to this day. Because contributions are voluntary, there are undoubtedly freeloaders who benefit without contributing, but there are enough public-spirited people to sustain it through their support.
The usual way of providing public goods privately is by a charge levied on users, as the early lighthouses did. Modern technology makes it easier to identify users and to charge those who wish to benefit from the service. The BBC was originally financed by a licence fee to provide and broadcast its programmes, but later media providers have used first advertising, as with ITV, and then subscription services, as with Sky. Many would say that it is fairer and more appropriate for public services such as these to be paid for by those who benefit from them, rather than use taxpayer funds.