My personal oval experience (2 Night Before the 500s at Indy) is that car/setup is more important than driver on an oval. On a road course (100+ starts in US, Canada, and England), the driver can make up somewhat for missing the car setup but on the oval if you miss the setup keeping it off the wall is an accomplishment. If you get the car setup right, the oval track gets almost easy (if you can avoid other people's wrecks) -- on a road course, you can challenge yourself even with a perfect car.
No simple answer from me, admittedly someone biased toward formula cars on road courses, but my two oval races were generally enjoyable experiences. Guess that for me racing is racing. Choosing between oval and road course ends up being a very personal choice -- just be sure to try both in the type of car you plan to drive.
Also, there is a lot more to racing than just driving the car. Is a race a one, two, or three day process? Do you paddock in mud or on pavement, how much crew and support equipment is required, etc., etc.?
2008 Solar Yellow Exige S240 (Torque 300)
1991 Elan M100
2015 Cayman GTS
2016 Ford Flex--Tow Vehicle