While you can drive with a leak for quite some time, there are a few things you need to be aware of. For one, the leaking oil WILL contaminate the lining of your e-brake shoes and if it's bad enough, your brake pads as well. Not a good thing. For two, if you allow your fluid level to drop too low, your axle bearing will become starved of lubricant and can fry. If that happens on the highway, your seized bearing will super heat to the point where they'll cause your shaft to melt off and send your wheel off into a direction all its own as you loose your brakes the control of your Jeep. This is something I have personally seen and my friend is lucky to be alive today after rolling his Jeep 4 times on the interstate.
Not trying to scare anyone here as I too have driven quite a while with axle seal leaks in the past - just posting up facts and sharing what I've seen.