It’s possible, but involves kind of “hacking” the core excerpt function which is a little messy and can be buggy.
I typically suggest that people use the More tag in situations that call for it.
I haven’t looked at preserving HTML in excerpts in a long time, so I’ll see if there’s any updates recently that are less messy 🙂