Yep, I removed mine a few years ago and didn't even notice a difference. I've read that some XJs didn't even come with a rear sway bar.

Did the bar pin eliminator too, works well on the rear. ...