I don't feel it is the least bit unsafe. I live in the Central West End about 5 blocks from the park. I've never felt unsafe anytime day or night whether in my car or on foot. My wife and I are out walking around the area all the time.
Now, if you're talking about hanging out near North Kingshighway and Natural Bridge, that might be a different story.