Free entry-level local AI image generation tool on the Home Data Center series Mac (Part 2): Draw Things installation and usage tutorial

Preface

除了”上篇”中介绍的Diffusion Bee之外,还有另一个同样是免费的、适合在M系列mac上运行的本地AI图像生成工具:Draw Things。

Draw Things Introduction and Installation

Software Introduction

Draw ThingsIt is a local image generation tool optimized for macOS and Apple M series chips, and also supports multiple models such as Stable Diffusion, ControlNet and LoRA. It has rich functions, including text generation image, image repair, parameter adjustment, etc., suitable for users who need to customize the generation details. At the same time, it provides a simple interface and built-in model management functions, taking into account ease of use and flexibility. It is a multi-functional image generation tool that integrates advanced functions and high performance.


Many options involved in this article (such as LoRA, ControlNet, strength, seed, number of steps, text guidance, sampler, etc.) are basic concepts in diffusion models (such as Stable Diffusion). The following is a brief explanation:

  1. LoRA(Low-Rank Adaptation): A lightweight model fine-tuning method for optimization of a specific style or task.
  2. ControlNet: An extension tool of the diffusion model that can control the generation results by input images or other conditions (such as edge maps).
  3. strength(Strength): Used for image-to-image generation, controls the influence of the input image on the final generated result.
  4. seed(Seed): Controls randomness. Fixed seeds can reproduce the same results.
  5. Steps(Steps): The number of iterations when the model generates an image, which affects the image quality and generation time.
  6. Text guidance(Text Guidance): Influence the generation results through text prompts.
  7. Sampler(Sampler): Determines the sampling strategy of the diffusion process, such as Euler, DDIM, etc., which affects the generated quality and style.

For detailed explanation and setting effects, please refer to the previous article:Free entry-level local AI image generation tool on the Home Data Center series Mac (Part 1): Diffusion Bee setting parameter detailed explanation and practical demonstration.


Draw Things vs Diffusion Bee

The following is a simple comparison between Draw Things and Diffusion Bee in terms of functionality, usability, performance, and usage scenarios:

  • Feature Comparison
characteristic Draw Things Diffusion Bee
Support Model Support multiple SD models Support multiple SD models
Multi-model switching Supports basic models and extended models (ControlNet, LoRA), no need to reload when switching basic model versions Supports loading up to 3 LoRA model combinations. Changing the base model version requires reloading
Tool scalability Support multiple plugins No plugin support
Generate Mode Supports txt2img, img2img and other generation modes Supports txt2img, img2img and other generation modes
  • Usability comparison
characteristic Draw Things Diffusion Bee
Interface Design Intuitive and simple Simple but less functional
Difficulty Flexible configuration, requires little familiarity Simple configuration, ready to use
Difficulty of operation Lower Low
Community Support active limited
  • Performance Comparison
characteristic Draw Things Diffusion Bee
Hardware Requirements Support M series chips Support M series chips
Spawn Speed fast medium
Memory usage Moderate Low
stability high high
  • Comparison of usage scenarios
characteristic Draw Things Diffusion Bee
Suitable for beginners yes yes
Suitable for advanced users yes no
Recommended scenarios High demand generation tasks Basic generation tasks

对比总结: Diffusion Bee更强调一键生成体验,但灵活性较低,适合完全没有技术基础的新手用户;Draw Things的默认配置更适合有一定技术基础的新手和进阶用户,保留了一定的灵活性。因此,作为”入门级本地AI图像生成工具”的下篇,Draw Things可以视为Diffusion Bee的进阶版。

Install

Draw Things' official website address is as follows:https://drawthings.ai/,要安装APP直接在Mac的App Store里搜索”Draw Thing”进行安装即可(也可以直接从以下链接进行安装:App Store Link)。同时,Draw Things相比Diffusion Bee有另一个优势,就是除了Mac版之外,还有Iphone版和Ipad版(这也是下文中会讲到的Draw Things为什么会有”服务器协助”这个功能的一个原因):

image.png

Draw Things Practice

initialization

和Diffusion Bee类似,开始正式使用Draw Things之前的第一步,都是需要下载一个基础模型,不同的是,Diffusion Bee有一个默认推荐的基础模型:”Default_SDB_0.1″(对应Stable Diffusion的核心版本是”Stable Diffusion v1.5″),而Draw Things需要自行选择:

image.png

Start downloading the model (there are 4 ckpt format files with a total size of nearly 5G):
image.png

set up

Draw Things的设置部分是生成图像的核心,如果只是生成类需求,设置中”基本”模式下的选项就能覆盖:

image.png

image.png

然后看看使用”基本”模式”下的默认参数+默认提示词的效果的生成效果:
image.png

Below the canvas area on the lower right is the toolbar, and the descriptions of each button are as follows:
image.png

Right-click on the icon to insert a picture for more options:
image.png

注1:上面的这些图片里左侧红框中出现的选项,都是在””基本”模式时出现的选项,如果选择”高级”,选项会变成一些高级功能的开关(比如图像生成图像、放大器等);如果选择”全部”,就是同时包含了”基本”和”高级”中的所有选项(类似于Diffusion Bee中的”高级选项”)。不过一来,这些选项基本都在上篇中详细解释过,二来,在Draw Things中这些选项都有中文解释,有一定基础的朋友其实看看也能明白是什么意思,所以我在本文中就不浪费口水来解释了(Diffusion Bee中的选项说明都是英文~)。

注2:在Diffusion Bee中,各个功能是分开,比如”文字生成图像”、”图像生成图像”、”修复”、”放大器”等,而在Draw Things中,这些功能都是以选项的方式出现,比如图像生成图像是在强度选项中:

image.png

放大器(提升照片分辨率)功能是在”高级”模式中的放大器选项:
image.png

From these differences, we can see that there are still differences in the usage concepts of the two softwares, Diffusion Bee is more suitable for beginners with zero basic knowledge (each function is separated and clear at a glance), while Draw Things is suitable for friends who have a certain understanding of diffusion model. For me, I obviously prefer Draw Things.

Note 3: Diffusion Bee and Draw Things can both support Chinese prompts in theory, but the model you download needs to be trained with Chinese content, so whether it can be supported depends on the basic model you download. The ones I downloaded do not support Chinese prompts and need to be translated into English.

Note 4: This article mainly introduces the interface and functions of Draw Things and its major differences compared to Diffusion Bee. As for the impact of different parameter settings on the effect of generating pictures, this article will not cover it. If you need, you can try it yourself, or refer to the examples of similar operations of Diffusion Bee in the previous article.

script

In Draw Things, the scripting part is mainly used to extend and customize the generation process (Diffusion Bee does not have this feature). It provides users with another way to adjust the generation details, add additional effects or implement advanced generation tasks besides adjusting the settings options.

Take the preset script as an example:

image.png

其实和”设置”-“基本”-“强度”-“图像到图像”选项效果是一样的:
image.png

In this case, the effect is the same no matter which method is used. But more often, a script contains the effects of multiple options combined, such as the functions provided by community scripts:
image.png

以第一个”Flux Auto Workflow”为例:
image.png

As can be seen from the comments at the beginning of the script above, the script integrates multiple functions such as text generation image, batch processing, random prompt word generation, and extended canvas, optimizes the parameter usage of Flux and other popular models, and provides flexible performance and mode selection. The degree of automation is high. Specifically, it mainly includes the following points:

1. Support multiple generation tasks

Text-to-Image: Basic generation task.

Batch Image Refine: A group of images can be batch optimized and refined.

Outpainting: Supports the extension of existing images to generate larger canvas content.

Batch Prompts: Multiple prompt words can be processed at the same time to generate multiple corresponding pictures.

Random Prompt: Built-in random prompt word engine, which can automatically generate high-quality prompt words and also supports customization according to user preferences.

2. Support multiple models and automatic optimization

Designed for Flux models: Two models (Flux Dev 8-bit and Dev to Schnell LoRA) need to be used at the same time, and the most suitable model will be automatically selected based on the accuracy priority.

Compatible with other popular models: Supports automatic parameter setting and is compatible with common models including DreamShaper Turbo, Kolors, SDXL, SD3, Schnell, etc.

3. Performance optimization and multi-mode selection

• Provides three different performance modes, including optimization parameters, to suit different needs (e.g. performance priority, effect priority, etc.).

• Especially suitable for beginners, it simplifies parameter adjustment during the generation process.

4. Flexible workflow and mode switching

• Supports multiple workflows and generation modes to achieve different image generation effects.

• Allow users to switch freely between different modes to meet specific creative needs.

From the above descriptions, we can see that scripts can provide higher flexibility and realize automated operations such as dynamic adjustment, batch processing, and multi-stage generation. They are suitable for complex or efficient needs. It is difficult to achieve these functions by manually adjusting options alone.


For novice users, scripts can be used to quickly achieve the effect of combining and setting multiple function options: if the task requires repeated adjustment of multiple parameters or multi-step operations (such as gradually adjusting the noise intensity), manual operation will become very cumbersome; for advanced users, if Draw Things introduces new algorithms or model functions, but the default interface does not support them, scripts can provide interfaces to call these functions so that users can try new features.

所以,不管是对于初级用户还是高级用户,脚本功能都可以大大提升工作效率,该功能也是Draw Things相比Diffusion Bee来说是”进阶版”的主要原因之一。


PEFT

PEFT (Parameter-Efficient Fine-Tuning) Module is a functional module in Draw Things that is specifically used to manage and apply lightweight fine-tuning models. Its main function is to help users achieve advanced generation effects such as style customization and domain migration by efficiently loading and dynamically adjusting fine-tuning technologies such as LoRA (Low-Rank Adaptation) (there are many options in the red box in the figure below, so I won’t take screenshots one by one):

image.png

与”设置”-“基础”中的 LoRA 配置相比,PEFT 提供了更深入的管理能力,包括自动优化模型兼容性、多 LoRA 模型叠加、动态参数调整,以及对其他 PEFT 技术(如 P-Tuning 或 Adapter)扩展的支持。它还能与脚本和批量生成功能深度集成,为复杂创作流程提供更大的灵活性和效率,是用户进行个性化图像生成和高级操作的核心工具。


exist Draw Things 中,”设置”选项中的 “LoRA 配置” 和 “PEFT 模块”, 虽然都涉及 LoRA 的加载和使用,但它们的用途和功能范围有所不同,主要体现在”管理深度”和”灵活性”。以下是两者的区别和联系:

1. LoRA configuration in settings

Scope: The LoRA configuration in the settings is mainly used to quickly load and adjust one or more LoRA models for direct application in the generation process.

How to use: In the "Basic" option, you can manually select multiple LoRA models and specify their weights (degree of influence); suitable for simple scenarios, such as superimposing multiple LoRA models to generate images of a specific style.

flexibility: Parameter adjustment is relatively basic, focusing mainly on strength (weight) and model loading order, suitable for users who do not need complex operations.

2. Functions of PEFT module

PEFT moduleProvides more advanced and granular LoRA management capabilities, with greater focus and capabilities than the LoRA configuration in Settings:

Advanced management features:Automatically optimize loading: The PEFT module will perform adaptability optimization for different LoRA models to ensure compatibility and performance between models;Dynamic model adjustment: Different parameter layers of LoRA can be dynamically switched, superimposed, or even individually controlled during the generation process.

Support more PEFT technologies:PEFT is not limited to LoRA, but may also support other lightweight fine-tuning technologies (such as P-Tuning, Adapter, etc.), expanding the toolset available to users.

Workflow Integration: The PEFT module may be more tightly integrated with scripts or other modules (such as batch generation) and can be flexibly called in complex workflows.

3. Core differences

Function LoRA configuration in settings PEFT module
Usage scenarios Quickly load and apply LoRA Advanced management, dynamic adjustment, and support for multiple lightweight models
Supporting technologies Mainly for LoRA Expand support for other technologies of PEFT
Complexity Basic functions, easy to use More powerful features for advanced users
Dynamic Control Basic parameter adjustment Support dynamic switching and optimization during the generation process
Integrate with workflow Independent operation, less related to other functions Can be deeply integrated with scripts, batch generation and other functions

If you just need to quickly load one or several LoRA models and adjust their influence weights, the LoRA configuration in "Settings" is sufficient. But if you need more sophisticated management of LoRA (such as dynamic switching, parameter optimization) or want to explore a wider range of lightweight fine-tuning techniques,PEFT moduleIt provides higher flexibility and functionality. It is more suitable for users who need complex generation effects and process control.

注:关于LoRA的实操我在”上篇”中也用专门的篇幅介绍过,也包括LoRA模型如何下载,感兴趣的朋友可以参看”上篇”中的说明,本文中就不多讲了。

Server Assistance

Function Introduction

Draw Things 的”服务器协助”功能允许用户通过启用”gRPC API”服务,将图像生成任务分配到同一网络中的其他设备(如具有更强算力的计算机)运行,从而加速处理过程,同时降低主设备的资源占用和发热问题。这种方式支持本地与远程计算协同,保持数据隐私的同时显著提升性能,适用于高分辨率图像生成和多任务协作等场景:

image.png


What is gRPC: gRPC is a high-performance Remote Procedure Call (RPC) framework that is widely used for cross-language and cross-platform service communications. It uses Protocol Buffers (protobuf) as the interface definition language and data serialization protocol, and is fast and efficient.

The gRPC API service in "Draw Things" can be understood as distributing computing tasks to other devices in the local area network through gRPC, thereby utilizing their computing resources (such as more powerful GPUs) to accelerate image generation tasks.


Features and advantages

Features of Server Assist

  1. Combining local and remote computing

• When running an application on a host device (such as an iPhone or iPad), you can delegate part or all of the computing tasks to a remote computer in the same network (as mentioned earlier in the article, Draw Things supports deployment on MAC, iPhone, and iPad at the same time).

• Avoid the main device from affecting performance or generating heat due to insufficient computing power.

  1. gRPC API Service

• Leverage gRPC (remote procedure call framework) to communicate efficiently with remote computing devices.

• The remote device needs to run a server program that specifically supports the gRPC API, which can handle the image generation task and return the results.

  1. Cross-device collaboration

• Supports collaborative work of multiple devices within a local area network, for example, the main device is used for control interface and some lightweight computing, and the remote devices focus on heavy-load tasks (such as model inference).

  1. Open source and flexible

• Draw Things uses open standards, allowing it to seamlessly integrate with community-maintained server-side programs.

Advantages of Server Assistance

  1. Performance Improvements

• Use high-performance remote GPUs to speed up inference for complex tasks.

• Reduce resource consumption of the master device.

  1. flexibility

• The task execution location can be switched at any time to adapt to different usage scenarios.

  1. Local Privacy

• Compared to a fully cloud solution, server-assisted functions remain local and no sensitive data is uploaded to the public cloud.


Possible use cases:

High-resolution image generation:Mobile phones have limited computing power, so desktop-level GPUs are used to accelerate the generation of large-size images.

Multi-tasking collaboration: Run multiple tasks simultaneously and distribute some tasks to remote devices.

Energy saving and heat management: Reduce the load on the main device to avoid overheating of the device.

Note:

  1. Model compatibility: Make sure that the master and remote devices use compatible Stable Diffusion model versions.
  2. Network stability: The server assistance function is sensitive to network latency and is recommended to be used in a high-bandwidth, low-latency LAN environment.
  3. Server configuration requirements: The remote device needs to have sufficient computing power (such as NVIDIA GPU and corresponding driver support).

How to enable Draw Things' server assistance

Here are the brief steps for setup and use:

1. Enable gRPC service on the remote device

• Make sure a server that supports the Stable Diffusion model (such as diffusers or other PyTorch-based implementations) is installed on the remote device.

• Install required dependencies:

pip install grpcio protobuf diffusers

• Start the gRPC API server. For example, run a sample server program to handle image generation requests.

2. Configure the Draw Things client

• In the Draw Things settings on the master device, find Server Assistance options.

• Enter the IP address and port of the remote device (make sure both devices are on the same network).

• Test the connection to ensure that the client can access the remote server normally.

3. Start generating tasks

• Choose to use server-assisted mode in Draw Things.

• The image generation task will be assigned to the remote device for processing, and the processing results will be returned to the main device.

4. Network and Firewall Settings

• Make sure that the remote device's firewall allows communication on the specified ports.

• If you cannot connect, check that the network settings support direct communication between devices.

Note: The above are just brief steps. In reality, it is not that simple. I will write an article to explain it in detail later.

附加知识:支持类似”服务器协助”功能的AI应用

This type of AI application is mainly concentrated in areas that require a large amount of computing resources, such as deep learning reasoning, training tasks, and generative AI applications. This type of application usually allows users to use the computing power of remote devices for distributed computing through a local area network or the Internet.

Here are some apps and tools with similar functions. Similar to Draw Things’ “Server Assist” feature, these apps and tools can offload high-load computing tasks to remote devices to provide users with more efficient computing power usage while retaining a certain degree of local control and data privacy:

1. Stable Diffusion related tools

Automatic1111 WebUI

• Features: Supports connecting to a remote server through a web interface and running Stable Diffusion inference using the server’s GPU.

• Implementation: Users can set up the API service of the WebUI, or call remote device resources through the SD API interface.

• Usage scenarios: Perform model inference and control on lightweight devices (such as laptops), but transfer the actual calculations to remote servers.

ComfyUI

• Features: Modular image generation interface, support for distributed deployment, control of remote inference nodes via WebSocket or API.

• Implementation: Multiple devices can be connected and different tasks (such as processing, synthesis, etc.) can be assigned to remote nodes.

2. Text generation applications

Oobabooga Text Generation WebUI

• Features: Used to run large language models (such as LLaMA, GPT-J, etc.), and supports setting up remote servers as inference devices.

• Implementation: Start local or remote inference services through server.py, and the user's front-end interface can connect to these services.

• Usage scenario: Users access the model on a regular device, but the computation is performed by a remote GPU or TPU.

3. Video generation and editing

Runway ML

• Features: Supports video generation and editing tasks based on Stable Diffusion, and supports connecting to custom remote inference servers in some scenarios.

• Implementation: Users can deploy the inference part to the cloud or a server in the local area network through the API provided by Runway.

• Usage scenario: Video editing and preview are performed on a local lightweight device, and complex calculations (such as stylized rendering) are completed by the remote GPU.

4. AI-assisted painting and 3D modeling

Blender (AI Acceleration Plugin)

• Features: Support for integration of Stable Diffusion into Blender for texture generation via plugins (such as Dream Textures), allowing connection to remote servers for high-performance tasks.

• Implementation: Users can set up remote computing nodes to work with local Blender software.

• Usage scenarios: Perform modeling tasks on the local device and outsource complex texture generation or high-resolution rendering to a remote server.

5. Distributed Deep Learning Training

Ray Serve

• Features: Ray is a distributed computing framework that supports deploying AI reasoning tasks to multiple machines.

• Implementation: With Ray Serve, you can easily implement distributed reasoning or training tasks and automatically distribute the load.

• Usage scenarios: AI models need to be run on multiple devices simultaneously, such as real-time recommendation systems or massively parallel reasoning.

TensorFlow Serving and PyTorch Serve

• Features: Supports deployment of AI models as API services, allowing client calls in the LAN or the Internet.

• Implementation: Configurable multi-instance services, providing load balancing and distributed reasoning capabilities.

• Usage scenarios: Remotely call AI models to perform prediction tasks, suitable for developers and enterprise-level applications.

6. Other Generative AI Applications

DeepFaceLab

• Features: A tool for face replacement that supports distributing compute-intensive tasks (such as model training or video processing) to multiple devices.

• Implementation: Speed up the face swapping process by setting up remote inference nodes.

• Usage scenario: Process video materials on a regular computer and offload heavy-load calculations to a remote server.

DreamBooth Training Tools

• Features: Supports distributing DreamBooth model training and generation tasks to remote devices.

• Implementation: Call high-performance devices through the local area network or cloud services to perform custom model training.

7. Cloud and local collaboration tools

NVIDIA Omniverse

• Features: A real-time collaborative 3D design platform that supports AI and rendering tasks accelerated by NVIDIA GPUs and supports remote computing collaboration.

• Implementation: Users can operate model designs on lightweight devices and utilize remote GPUs for real-time computing and rendering.

Hugging Face Inference API

• Features: Provides cloud-based AI inference services, allowing users to call models through APIs and combine them with local processing.

• Implementation: Data pre-processing and management are performed locally, and complex tasks are sent to the cloud for execution.

Afterword

本文的重点并不在于详细讲解如何使用 Draw Things 生成 AI 图像,因为像这类 AI 图像生成工具,无论是对初学者友好的 Diffusion Bee,还是功能更全面的 Draw Things,”能用”是一回事,但要想真正的”用好”,需要用户具备一定的知识储备并进行大量实践:从理解扩散模型的基本原理,到熟悉各类参数(如采样步骤、CFG Scale 等),再到选择合适的基础模型、LoRA 等辅助模型,以及探索提示词和参数组合的最佳搭配,这些都需要时间积累和试错过程。这些内容不仅超出了本文的篇幅范围,也需要远比一篇文章更系统的学习和总结(我自己也只是刚刚入门的初学者,很多知识和经验都需要慢慢学习和积累)。

我写这篇文章的真正目的,是希望从”工具特性和使用思路”的角度出发,阐明为什么说 Draw Things 相较于 Diffusion Bee 是一款”进阶版”工具,并通过介绍它的功能特点(如脚本、PEFT、服务器协助等),帮助有一定基础的朋友可以快速上手,从而更快更高效地发挥其能力,省去不少学习摸索的时间。

For those who are interested in in-depth research on generative AI image creation, this article hopes to be your first step in getting started with Draw Things and provide a higher starting point for subsequent deeper exploration.

The content of the blog is original. Please indicate the source when reprinting! For more blog articles, you can go toSitemapUnderstand. The RSS address of the blog is:https://blog.tangwudi.com/feed, welcome to subscribe; if necessary, you can joinTelegram GroupDiscuss the problem together.
No Comments

Send Comment Edit Comment


				
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
(╯‵□′)╯︵┴─┴
 ̄﹃ ̄
(/ω\)
∠(ᐛ 」∠)_
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ`)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ( ̄∇ ̄o)
ヾ(´・ ・`。)ノ"
( ง ᵒ̌ᵒ̌)ง⁼³₌₃
(ó﹏ò。)
Σ(っ°Д °;)っ
( ,,´・ω・)ノ"(´っω・`。)
╮(╯▽╰)╭
o(*////▽////*)q
>﹏<
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
Emoticons
Emoji
Little Dinosaur
flower!
Previous
Next
       
error:
en_US
Spring Festival
hapiness