I think it really comes down to how much the car is worth relative to your income/available cash.
I mostly agree with this, but I also think that the 'fun' of a car is really hard to define. There are a lot of factors that appeal to a lot of different kinds of people.
I work on exotic cars for a living, and have driven many different ones, from Ferrari 308,599,275, etc to DeTomaso, Lamborghini, ISO, Bizzarrini, Ford GT, etc.
Some cars are really magic to drive, others are terrible, and it is often hard to tell which one it will be until you actually get in and drive it. Between the look and sound of the car, throttle response, steering feel, suspension dynamics, transmission feel, interior quality and comfort, there are a huge number of factors. Then there is also what you are using a car for.
I used to have a junk miata that was my beater car. I had a huge amount of fun in that car, because I drove it at 110% all the time and didn't care if got scratched.
I have a customer that is a little more affluent than me, and has a similar relationship with a Ferrari 308.
My Elise is fun because of the speed and handling, the looks, and because it represents a kind of purity that is missing from most modern cars.
On the other hand, my FC race car is way faster than the Lotus, and more fun when actually racing, but there is an additional stress level there from lower reliability and more pressure to win that cuts into the fun factor somewhat.
A couple of my favorite customer cars that I have driven are a Ferrari Dino, which just drives really nice and had that history and beauty, and a Lamborghini 400GT, because it is a classic, fast V12 car, but still has 4 usable seats, and reserved looks and luxury compared to many sports cars. A more modern example is a Ferrari 599GTB, which is similar to the 400GT, just newer and faster, but also more complex and less rare.
I was super excited to drive a customer's Panoz Esperante once, and found it didn't have the magic for me, the carbon shell is kind of 'boomy' and flexy, and even though the car drives, looks, and handles great, it just didn't give me the feel that I was looking for.
Anyway, I'm not sure if I am explaining this well, other than to say that 'fun' isn't so simple, and based on my experience, doesn't have a super direct relationship with how fast or expensive a car is, but more how you use it and what it means to you.