For inference research specifically, what Bob mentions, the plugin architecture gradient based and otherwise is available in R and Python (and in C++, but minority of inference research is done directly in C++). My group is using this “plugin” interface. There are open examples how to use it, we have helped people on discourse, but it doesn’t seem to be popular. Using the current log density plus gradient interface is easy. and my group is in progress making more examples and case studies for this.
For implementing an inference algorithm in C++, I think the C++ code is modular enough for adding other algorithms, although there is additional work to enable those in interfaces, ie there is no plug-ang-play-feature that you could put you algorithm module somewhere and interfaces would figure out how to show help and check arguments, etc.
What seems to be forgotten sometimes is that, these two above “plugin” approaches assume that algorithm needs just the log density or the log density plus gradients. However, there have been people asking support for algorithms which would require, for example, mix of log densities and log probabilities (sampling for mix of continuous and discrete parameters), Markov blankets (alternating or nested inference for different parameter groups), or streaming data (some of sequential Monte Carlo methods). All these would be huge changes requiring a lot of work. Even if someone would offer to do most of the work, the changes would have effects all over Stan. Making Stan to allow plug-in algorithms for these would be a very big project.
See also Modularity for making Stan more Pythonic and Rthonic - #30 by bgoodri and Algorithm Inclusion Policy