I don't have an idea how that would look in detail, but
Creation of an OSGi artifact e.g. "smack-osgi" where we merge the most critical bundles
Could you elaborate that? It sounds like we would duplicate code here.
Introduction of a service interaction layer where we can define dependencies between services (objects) explicitly.
That sounds like the "best" solution. How would such a service interaction layer look like?
Mixture of both and resorting to the fragment approach, if interaction with Smack should still be done on the package and not the service layer.
We already had a fragments based approach in Smack and switched away from it. Resorting again to fragments seems like a step backwards.
Would be great if we could discuss this in a real-time chat, e.g. if you could join #smack (freenode) or open_chat@conference.igniterealtime.org (MUC).
I'm completely depending on your judgement here, since I'm not as nearly as far into OSGi as you. But there are already further consumers for Smack's OSGi support (e.g. Bug 438006 – Update to Smack 4), so it's definitely a often requested feature for which I'm willing to put some effort into a clean, nice and non-hacky solution, while not breaking Smack's modularity or Android compatibility.