I think it would matter more if you had a hard top. With the top open, I can imagine that at higher speeds the car would be less aerodynamic and potentially slower than if the car had its top on. I dont know if that would be the same with a soft top because it is not rigid.
Top on just makes more sense to me. Less drag, don't need to wear elbow restraints, less of the annoying buffeting, possibly less of the marbles flying up and hitting you on the head. People talk about small increments in tire pressure to get the last fractions of a second better per lap, and the added top weight of the top is negligible while gaining the above advantages, IMHO. OTOH, if you're looking for fun fast cruising around the track rather than testing speed abilities, it would make sense to do top off if the organizers will let you.
Autocross, definitely top off. Aerodynamics less of an influence, if any, and it's just plain more fun.