> so after the inhibition of the thing i got excited about in my little > httptransfomer repository > > there's been energy around, rather than completing that thing, instead > trying to make it work for diffusion, making images > > of course since that thing wasn't implemented along with in general > the work having been for small data, diffusion ends up finding a space > to use the work for what it hasn't been designed for
paths forward might include - working on optimizing large data rather than large models - figuring out to use a diffusion model with small data maybe similar to how with transformers i limited the context window to focus on the weight streaming they're reasonable! they just take some [conscious informed pursuin'
