Interaction of user-defined and integral fallbacks with inference

Here is an open PR that implements the behavior for #1. I also have a couple of other branches with different strategies implemented.