Scheduling LORA

[nom]

In Comfy, it's possible to schedule loras and their weights. This is a really helpful feature, is it already possible or easy to implement in Diffusers?

https://blog.comfy.org/p/masking-and-scheduling-lora-and-model-weights

[rileyafox]

You could try something like this:

Load your LoRA adapters into a copy of the U-Net (or keep two copies: one with LoRA, one without).

In your sampling loop, rather than calling pipeline(...) in one shot, you manually step through timesteps and swap/scale in the adapter weights before each denoise call

I think it wouldn't be very performant however.