TouchDesigner implementation for real-time Stable Diffusion interactive generation with StreamDiffusion.

Benchmarks with stabilityai/sd-turbo, 512x512 and 1 batch size.

Notice: This repository is in an early testing phase and may undergo significant changes. Use it at your own risk.

Includes preinstalled configurations, ensuring everything is readily available for immediate use.

1. Download and extract archive
2. Run webui.bat. It will provide url to web interface (ex. http://127.0.0.1:7860)
3. Open install & update tab and run Update dependencies.

You can follow YouTube tutorial

Required TouchDesigner 2023 & Python 3.11

1. Install Python 3.11
2. Install Git
3. Install CUDA Toolkit 11.8 (required PC restart)
4. Download TouchDiffusion.
5. Open webui.bat with text editor and set path to Python 3.11 in set PYTHON_PATH=. (ex. set PYTHON_PATH="C:\Program Files\Python311\python.exe")
6. Run webui.bat. After installation it will provide url to web interface (ex. http://127.0.0.1:7860)
7. Open install & update tab and run Update dependencies. (could take ~10 minutes, depending on your internet connection)
8. If you get pop up window with error related to .dll, run Fix pop up
9. Restart webui.bat

Models in .safetensors format must be in models\checkpoints folder. (as for sd_turbo, it will be auto-downloaded).

Internet connection required, while making engines.

1. Run webui.bat
2. Select model type.
3. Select model.
4. Set width, height and amount of sampling steps (Batch size)
5. Select acceleration lora if available.
6. Run Make engine and wait for acceleration to finish. (could take ~10 minutes, depending on your hardware)

1. Add TouchDiffusion.tox to project
2. On Settings page change path to TouchDiffusion folder (same as where webui.bat).
3. Save and restart TouchDesigner project.
4. On Settings page select Engine and click Load Engine.
5. Connect animated TOP to input. Component cook only if input updates.

• Fix Re-init. Sometimes required to restart TouchDesigner for initializing site-packages.
• Code clean-up and rework.
• Custom resolution (for now fixed 512x512)
• CFG not affecting image
• Add Lora
• Add Hyper Lora support
• Add ControlNet support
• Add SDXL support

Based on the following projects:

• StreamDiffusion - Pipeline-Level Solution for Real-Time Interactive Generation
• TopArray - Interaction between Python/PyTorch tensor operations and TouchDesigner TOPs.