My two cents would be, do you like driving a race car on the weekend or a performance car? Elise is pretty much a race car on the road, it's cramps, hard to get in/out, uncouch, uncomfortable, loud, ride really low, I can barely see into the window of the adjacent sedan. Evora is going to perform quite similar, but will be on the other end of the spectrum with everything else. It's much more comfortable, real audio, real AC, actual supportive seats, decent ride height, and your passenger would likely enjoy the ride.
With all that said, I would pick the Elise over the Evora just because I enjoy the rawness of the drive in the Elise. It's pretty much you and the car. There's almost nothing else in between.