I think you're assuming the boats should be relevant to cruisers. That's like saying F1 should be relevant to city commuters so they should have accommodation for a passenger.
Many people who own boats have them just for racing, or fun sailing, so hard core racing boats are relevant to them. Those who don't race or sail but like watching races don't care about relevance. They just want to see good racing. I expect the vast majority of people following F1 never expect to drive such a car and don't consider them relevant to their daily drive at all, but they love watching them race.
Carbon fibre isn't really that expensive in the overall cost of a boat, however where cost matters, other materials get the job done well enough to make the saving worth while.
It's notable that many if not most race series do drive cars that have "accomodation" for passengers. LeMans prototypes, for example, are technically two seaters. NASCAR, touring cars, and all the other GT cars are based off of passenger cars. Dedicated single seaters are found mostly in open wheel racing.
As far as carbon fiber being "not that expensive": that's a relative term. Let's give it a fixed point of reference so that we know that we're discussing the same thing. How much ought a cruiser cost? I would posit that it ought to be affordable for a significant percentage of the American population, as anything less than that makes it unaffordable for most of the world (The countries on a similar economic footing as the US comprising a very small percentage of the world population as a whole).
As far as "most people" being able to drive a formula 1 car: it's true that they can't drive one of those. But I would argue that's to F1's detriment. The fact that I can build or buy a circle track car (new) for $30k and be competitive in the lower classes I would argue is one of the things that keeps NASCAR as popular as it is (despite being, in my opinion, much less interesting racing). In short, I think sailing has a problem with pricing itself out of a market. Even the few people who have $1 million to drop on a hobby are going to think long and hard about dropping that on a boat (or more likely, they buy a cheaper boat, a cheaper plane, a cheaper Ferrari, etc.). My grandfather was in this category. As a partner in the biggest law firm in a major city, he retired in his forties and took to various hobbies, like sailing. But even he never spent anywhere near the amount of coin you'd need for a carbon racing yacht, and he would have been solidly in the top 2-3% of income earners. That's a really small market (albeit a potentially profitable one).
But coming back to getting people to watch a race: I think at some level, the competition needs to be relatable for the people watching. I don't know anyone who doesn't sail who watches sailing, and sailing is competing with other kinds of competitions for viewership. So making racing that's disconnected from the viewer's experiences is, in my opinion, a good way to kill the sport.