A few days ago, I had a simple experience with sdxl, and I must say the effect was very stunning. Although the original model of sd is just average, the idea of xl is nothing short of opening a new generation for AI drawing.
Optimization Refine#
Unlike sdxl and other ordinary models, this model has released two versions. One is the base model, which is commonly used for generating images. However, there is also a refiner model that does not participate in image generation. Its job is to intervene in the final stage of iteration to optimize and repair images. In other words, sdxl actually requires running two models for one image. After the base model completes the basic image generation, it hands it over to the refiner for optimization. This approach indeed produces much better results than direct image generation, solving many areas prone to errors in drawing.
In fact, in combination with lora and cn, it is basically possible to produce a very good image.
Drawbacks#
The drawbacks are also quite apparent. This time, sdxl's model is very large, with the two models totaling over 14GB (perhaps due to the variety of styles). Additionally, sdxl's requirements for computer configuration have been raised again. The webui version requires 8GB of VRAM to run smoothly. For low-end machines, it is recommended to use comfyui for a better experience.
Image Generation Method#
For image generation, if you are using a version before 1.6.0, you can first generate the image with the base model and then run the refiner model through the image-to-image tool, which should achieve similar results. If your sdwebui version is 1.6.0 or later, the webui comes with the refiner plugin built-in. You just need to select the model and adjust the switching timing to generate images directly.
Multi-Style#
Another feature of the sdxl model is multi-style, meaning it can draw in various styles (although the quality may vary, being decent for realistic portrayals but average for anime characters). It is recommended to install a plugin called sdxl_styles to easily switch between styles, avoiding the need to remember a bunch of tags.
Conclusion#
Although the original model of sdxl is still not impressive, it requires fine-tuning and secondary development. However, this new approach to image generation is nothing short of revolutionary for AI drawing, promising more impressive results in the future!
This article is synchronized updated by Mix Space to xLog.
The original link is https://blog.xiaohan-kaka.me/posts/ai/sdxl