AI艺术相关的工具连接

Looking to get started with AI art? A good place to start is one of the  popular apps  like  DreamStudio ,  midjourney ,  Wombo , or  NightCafe . You can get a quick sense of how you can use words and phrases to guide image generation. Read up on  prompt engineering  to improve your results. Then you may want to move on to using Google Colab notebooks linked below like Deforum.  If you have a good nVidia GPU of your own then you can also use  NMKD Stable Diffusion GUI  or  Visions of Chaos  to run the most popular notebooks locally. If you want to train your own Ai models check out the  Ai art model training  page, for animations check Stable Diffusion animations .

Text to Image

There are a TON of shared Google Colab notebooks floating around for doing text to image with pre-trained GAN and diffusion models. I’ve been compiling the ones I come across and try out and find interesting. Please hit me up on twitter (@pharmapsychotic) if you know a cool notebook that I am missing! Stable Diffusion is most popular right now. 

• Stable Diffusion WebUI by automatic1111 – run SD local with lots of features and extensions
• StableStudio – local webui using Stability API for inference
• Deforum Stable Diffusion 0.7 – group effort for ultimate SD notebook (discord) (youtube tutorial) (guide)
• Disco Diffusion v5.6 by Somnai, gandamu, zippy721 (guide) (new guide) (youtube tutorial)
• Huemin Jax Diffusion 2.7 by nshepperd, huemin_art (guide) (stitching guide)
• pytti-tools v0.10 by DigThatData and sportsracer
• VQGAN+CLIP by remi_durant

• [2023/09/15] llamas.ipynb by @osanseviero QR ControlNet + SD1.5 for optical illusions ( tweet )
• [2023/04/28] DeepFloyd IF (huggingface) (github)
• [2023/04/05] Kandinsky 2.1 Batching+Dynamic prompting Colab by @jrobocat
• [2023/04/03] Kandinsky 2.1 (huggingface) (site)
• [2023/03/23] Image-to-text-to-image Colab by @jrobocat  – batch CLIP Interrogator + SD generations
• [2023/03/20] ModelScope text-to-video Colab by @camenduru (youtube) (github)
• [2023/03/18] ModelScope text-to-video huggingface space
• [2023/03/14] Unidiffuser – unified diffusion framework (github)
• [2023/02/20] Stable Diffusion Auto Stitching by @oleg_ai_art (guide)
• [2023/02/15] ControlNet – control Stable Diffusion with extra conditioning (youtube) (huggingface) (github) (models)
• [2023/02/14] Pix2Pix video with coherence by ​@johnowhitaker – stylize video inputs!
• [2023/01/30] Tune-a-Video – create short text2video sequences (github) (paper)
• [2023/01/21] KLMC2 Animation – @DigThatData’s fork with lots of additions
• [2023/01/20] InstructPix2Pix – use text instructions to modify images (huggingface)
• [2023/01/19] Image Mixer by @Buntworthy – mix up to 5 images together with SD
• [2023/01/14] Latent Blending by @j_stelzer – smooth transition between SD latents (github)
• [2023/01/10] Custom Diffusion – fast SD finetune with multiple concepts (github)
• [2022/12/22] Karlo – unCLIP architecture like DALLE-2 (huggingface) (github)
• [2022/12/08] Stable Diffusion KLMC2 Animation by @RiversHaveWings
• [2022/11/30] BAOAB-limit sampler – new SD sampler that can also make anims hella fast (paper)
• [2022/11/25] Stable Diffusion 2.0 Web UI – by @anzorq (run SD 2.0 in colab using Diffusers)
• [2022/11/24] Stable Diffusion 2.0 w Diffusers – by @amrrs (youtube)
• [2022/11/08] Midjourney v4 Style – (dreambooth SD finetune on midjourney v4 outputs)
• [2022/11/03] All-in-one Private Diffusions Colab – fork and upgrades to WD notebook (website)
• [2022/10/25] Fast Dreambooth by TheLastBen (easy fast finetune of stable diffusion in colab)
• [2022/10/08] Stable Worlds by @NaxAlpha (create panoramas with SD!)
• [2022/09/29] MathRockDiffusion by ethansmith2000 (mods and improvements on Disco) ( guide )( cuts )
• [2022/09/29] robo_diffusion_v1 by @nousr (a DreamBooth fine tune of stable diffusion)
• [2022/09/27]  Video Killed The Radio Star Diffusion by @DigThatData (transform music videos from YouTube)        
• [2022/09/25] fast-stable-diffusion – automatic111 ui, hlky ui, github (+25% speed and low VRAM)
• [2022/09/18] Doohickey Diffusion by aicrumb (stable diffusion with CLIP guidance, perlin init, lots more)
• [2022/09/18] optimized colab by neonsecret (stable diffusion with nice gradio gui in colab)
• [2022/09/13] Stable Diffusion Batch by visoutre (includes tiled upscaling!) (tutorial)
• [2022/09/11] Easy Diffusion by WASasquatch and NOP (stable diffusion with lots of still image features)
• [2022/09/07] NMKD Stable Diffusion GUI (nice easy Windows GUI for stable by Noomkrad)
• [2022/08/30] Simple Stable Diffusion by @ai_curio (supports prompt weighting)
• [2022/08/29] Stable Diffusion WebUi by @altryne (fancy Gradio UI for stable diffusion)
• [2022/08/28] Prompt Parrot v2.0 by @KyrickYoung (train gpt2 on prompt list then generate with stable-diff)
• [2022/08/23] Stable Diffusion Interpolation by @ygantigravity (animate from own prompt to another!)
• [2022/08/23] Deforum Stable Diffusion (discord link) 🔥
• [2022/08/23] FunkyHorses Stable Diffusion by  Coskaiy/Corran (has neat import from spreadsheet)
• [2022/08/23] NOP’s Stable Diffusion Colab v0.19 by NOP#1337
• [2022/08/23] Stable Diffusion Lite by @future__art (prompt queueing and seed mining)
• [2022/08/23] Interactive notebook for Stable Diffusion  
• [2022/08/22] Stable Diffusion HuggingFace space by stabilityai
• [2022/08/22] Stable Diffusion notebook by @pharmapsychotic 🔥 (easy to use and batch to gdrive) (tutorial)
• [2022/08/22] Official Stable Diffusion notebook – requires hugging face account
• [2022/08/22] DiscoStream v1.1 by @WASasquatch
• [2022/08/20] Disco Diffusion v5.6 with Inpainting by @cut_pow
• [2022/08/18] DiscoArt [w/ Batch Prompts + GPT3 generator] by Skquark
• [2022/08/16] WAS’s Disco Diffusion v5.6-9 Portrait Generator Playground by WASasquatch
• [2022/08/08] Paint Pour Diffusion by @EclecticBeams (diffusion trained on paint pour art)
• [2022/07/31] Huemin Jax Diffusion 2.7 August 2022 by @huemin_art
• [2022/07/30] CLIP Prior + VQGAN by @RiversHaveWings and @jd_pressman (a new VQGAN notebook 😮)
• [2022/07/23] Textile Diffusion by @KaliYuga (diffusion trained on textiles)
• [2022/07/21] Floral Diffusion by @jags111 (fine tunes for floral)
• [2022/07/18] Liminal Diffusion v1 by @BrainArtLabs (diffusion trained on liminal photographs)
• [2022/07/18] DifNESfusion 1.35 by @LufiQ (fork or PixelArtDiffusion with NES dataset)
• [2022/07/18] Medieval Diffusion by @KaliYuga (diffusion trained on medieval art)
• [2022/07/17] FeiArt_Handpainted CG Diffusion by @FeiArt_AiArt
• [2022/07/17] Fantasy Diffusion by @LaVista (diffusion trained on fantasy art)
• [2022/07/15] Ukiyo-e Portrait Diffusion by @avantcontra
• [2022/07/15] Lithography Diffusion by @KaliYuga (diffusion trained on lithographic landscapes and portraits)
• [2022/07/06] Disco v5.2 Dynamic Prompting (dynamic prompt variations –  tutorial video )
• [2022/07/06] Watercolor Diffusion by @KaliYuga (diffusion trained on watercolor paintings)
• [2022/07/05] EnzymeZoo edits to Huemin Jax Diffusion by @EnzymeZoo (brought over masking from Majesty)
• see older notebooks in the archive

Upscaling / Super-resolution

Check out the Upscaling Guide

• Gigapixel AI by Topaz Labs (costs $99) <- voted #1
• Real-ESRGAN – ( github ) <- voted #2
• Real-ESRGAN Sber – a nice fine tuned ESRGAN model
• chaiNNer – node base tool that can batch process ESRGAN upscale and more
• Cupscale – Windows GUI for ESRGAN
• Latent-SR – Nightmare Ai latent diffusion super resolution (slow but nice!)
• PASD image super resolution – (github) pixel aware Stable Diffusion
• Neural Love – credit based system for diffusion upscaling 
• Stable Diffusion Upscaler – latest and greatest 🔥
• SuperRes Diffusion – Batch upscaling and super resolution with latent-diffusion
• SwinIR – Hugging Face space
• Upscale Model Database – big set of pretrained models for upscaling different types of content
• Waifu2x (github) – designed for anime / manga
• WaifuXL – newer and beats Waifu2x in quality
• LetsEnhance.io – credit based web service for image super resolution

StyleGAN

• [2022/08/23] Painting with StyleGAN by @jmoso13 (tutorial) – use VAE to navigate and animate!
• [2022/04/25] StyleGAN-Humans + CLIP modified by Diego Porres to use StyleGAN3
• StyleGAN2-ADA – train your own StyleGAN2 model from an image set you create
• StyleCLIP – Text-drive manipulation of StyleGAN imagery
• Structured Dreaming – Styledreams With helpers
• Structured Dreaming (CLIP+StyleGAN) by @ArYoMo (tweet)
• StyleGAN 2 pretrained models – can use these with Structured Dreaming
• StyleGAN 2 awesome pretrained models – BIG collection of models
• StyleGAN 3 training – train a StyleGAN and do interpolation video by @dvsch (currently busted)
• StyleGAN 3 music video generation – (tweet)
• StyleGAN 3 + CLIP by Annas
• StyleGAN3 + CLIP by @nshepperd1 and @RiversHaveWings
• StyleGANXL + CLIP by Eugenio Herrera and Rodrigo Mello
• Lucid Sonic Dreams – animate path through StyleGAN latent space with music (github)

Text

• GPT4All Chat – run local windows/linux/mac app like ChatGPT
• oobabooga text-generation-webui – it’s like auto1111 sd web ui but for text models
• StableLM space – huggingface space for language model from Stability AI

• Goose.ai Playground – can use their playground to generate text with GPT-Neo
• GPT Neo Colab notebook – use GPT-neo 1.3B and 2.7B from Google colab
• GPT Neo HuggingFace – run GPT-neo 2.7B on HuggingFace
• Neuralism Generative Art Prompt Generator – generate prompts to use for text to image
• OpenAI GPT3 Playground – generate text with GPT-3 (requires free account)
• Textsynth Playground – text completion using large language models

• Dalle-2 Prompt Generator – nice site that let’s you generate interesting text prompts
• Prompt Parrot by @KyrickYoung – train GPT2 on a list of your prompts
• MadLib Prompt Generator – makes interesting prompts for you, by @remi_durant
• Noodle Soup Prompts v2.1 by WASasquatch
• Neuralism Prompt Generator – generative art prompt generator

Video

Image to video

• [2023/11/22] Stable Video Colab by @mkshing

Text to video

• camenduru text-to-video Colabs – great collection of Zeroscope, potat1, modelscope notebooks
• AnimateDiff (colab) (github) – short video clubs with your own LoRA
• ModelScope (colab) (huggingface) – super fun but prominant shutterstock watermarks
• Text2Video-zero (colab) (github) (huggingface) (webui ext) – zero shot video from Stable Diffusion

Interpolation

• Video Enhance AI by Topaz Labs – commercial upscaling and frame interpolation <- excellent
• AnimationKit AI – video upscaling and interpolation tool <- great
• FILM colab – by @KyrickYoung has pause, loops, reverse <- my fave FILM
• 3D Ken Burns Effect from single image – animated video from 2D image
• 3D Photo Inpainting – cool 3D effects for 2D images
• Animating Pictures with Eulerian Motion Fields – code not out yet, looks like it’ll be awesome
• DAIN colab – depth aware interpolation
• EbSynth – stylize video by giving it ai or hand painted key frames from video
• ESRGAN 4 Video – increase resolution of video with ESRGAN
• FILM: Frame Interpolation for Large Motion – (replicate link) smooth interpolation/morphing
• Flowframes – free Windows tool with patreon option, uses RIFE and other models
• PyTTI-Tools: FILM – @DigThatData ‘s version of FILM for video frames
• RIFE – smooth interpolation of video to increase frame rate
• Sequence Frame Interpolation – batch version of FILM
• Super Slomo – another way to increase frame rate of video
• Video Art and Styling Tools – by @Coskaiy (style transfer, interpolation, superres, and more)

Animation

• [2022/11/03] FrameSync.xyz – Automate Deforum Keyframe animations with waveforms
• [2022/10/26] Tulpa Prompter by @dreamingtulpa – helper to build animation prompts (tweet)
• [2022/08/15] AnimationPreview by @pharmapsychotic – quickly preview Deforum camera animations
• [2022/08/04] DALL-E 2.5D Depth Warped Zoom by @deKxi
• [2022/03/31] PyDub Audio to Disco Diffusion Keyframe Generator v0.1 by austinhquinn
• [2022/02/26] Wiggle animation key frame generator by @zippy731 
• [2022/02/23] audio-reactive-video – by @vsewall2motion, skip video frames based on volume
• Keyframe string generator for AI animation notebooks
• Audio to keyframe string generator for AI animation notebooks

Prompt Engineering

To get good results with CLIP guided diffusion and VQGAN+CLIP you need to find the right words and phrases that will direct the neural network to the content and style you are looking for.

Image to Text

• Antarctic-Captions by @dzryk
• BLIP image captioning HuggingFace space
• CLIP Interrogator by @pharmapsychotic – image to prompt! (huggingface) (lambda) (replicate) 🔥
• CLIP prefix captioning inference notebook (github)
• LLaVa: Large Language and Vision Assistant – ask vision model to describe image
• personality-clip by @dzryk
• PEZ: Prompts made EZ – prompt from image or long to short prompt  (huggingface) (colab)

Prompt Guides

• [2023/08/04] Stable Diffusion XL reference library – great guides for SDXL!!
• [2022/11/29] Stable Diffusion V2 CFG Scale Comparison – nice ref of samplers and cfg scale
• [2022/09/16] krea.ai search stable diffusion prompts and browse by modifiers
• [2022/09/07] libraire.ai search 10 million stable diffusion prompts and images
• [2022/08/28] Prompt Parrot v2.0 by @KyrickYoung (train gpt2 on prompt list then generate with stable-diff)
• [2022/08/24] Lexica stable diffusion prompt search engine
• [2022/08/13] Promptomania by @wszp – cool prompt building tool!
• [2022/08/08] Stable Diffusion Artist Studies by @proximasan @EErratica @KyrickYoung @surrailabs
• [2022/08/08] Stable Diffusion Modifier Studies by @proximasan +
• [2022/07/30] Disco Diffusion Portrait Study by @enviraldesign
• [2022/07/13] Dall-e 2 prompt book by @GuyP
• [2022/03/25] Disco Diffusion Modifiers Study by @KyrickYoung and @sureailabs
• [2022/03/21] DiscoDiffusion Model Comparison Study – by @KaliYuga
• [2022/03/05] Midjourney Artist Dump – spreadsheet or artists and example render
• [2022/02/26] Disco Diffusion 70+ Artist Studies
• A Guide to Writing Prompts for Text to Image – Google Doc guide and advice
• CLIP Retrieval Tool – see what kinds of images match strings for CLIP (wait a long time for it to load)
• CLIP Prompt Engineering for Generative Art – nice long guide by Matthew McAteer
• CLIP + VQGAN keyword comparison by @kingdomakrillic
• Artist Studies by @remi_durant – big collection of results using different artist names
• Art Movements and Styles as perceived by VQGAN + Clip (Imagenet 16k, ViT-B/32)
• Art Movements and Styles as perceived by VQGAN + Clip (Imagenet 16k, ViT-B/16)
• Art Movements and Styles as perceived by VQGAN + Clip (Imagenet 16k, RN50x16)
• Art Movements and Styles as perceived by VQGAN + CLIP (Imagenet 16k, RN50x4)

Music

You can generate music with AI using OpenAI’s Jukebox. You can prompt Jukebox with an artist and music genre or with a short audio clip in WAV format. It generates new music for you in phases of increasing quality (level_2, level_1, level_0) and takes about 8 hours on Colab.

• aiva – ai composition of soundtracks and music
• amper – royalty free ai music creation
• AudioLDM – text to audio latent diffusion model (huggingface) (replicate) 🆕 🔥
• boomy – let’s you create and publish music with Ai but they hold the copyright
• D3Net-MSS – colab for splitting music into separate clips for drums, vocals, etc 
• Dance Diffusion – audio diffusion! (guide) 
• Dance Diffusion Finetuning – fine tune on your own audio dataset
• Easy One Click Jukebox – this is my favorite currently
• Jukebox Community Build – download this ipynb and put in Colab Notebooks folder on Google Drive to use
• lalal.ai – commercial music to stems service
• Moises.ai – ai audio separation
• mubert – nft friendly music remixed by Ai
• Official OpenAI Jukebox – the official notebook from OpenAI
• riffusion – stable diffusion fine tuned on audio spectrograms! (web) 
• Spleeter colab – split music into stems
• Zags Jukebox v3.7 – (youtube tutorial)

Other

• sdtools.org – cool wiki covering tools and methods related to Stable Diffusion
• JAX CLIP Guided Diffusion 2.7 Guide – Google doc from huemin
• Zippy’s Disco Diffusion Cheatsheet – Google Doc guide to Disco and all the parameters
• EZ Charts – Google Doc Visual Reference Guides for CLIP-Guided Diffusion (see what all the parameters do!)
• Hitchhiker’s Guide To The Latent Space  – a guide that’s been put together with lots of colab notebooks too
• Resources for GAN Artists – another big Google Doc with notebooks and resources for AI art
• Way of the TTI Artist – pytti guide 
• Guide to install Disco Diffusion 5 on Windows with WSL – haven’t tried this yet challenge is pytorch3d
• Great explanation of VQGAN+CLIP – https://ljvmiranda921.github.io/notebook/2021/08/08/clip-vqgan/
• Nice overview of lots of different optimization algorithms SGD, Adam, RMSProp etc and their differences (also covered in this lecture)
• Stanford’s Convolutional Neural Networks class on YouTube – https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv 

• ClipMatrix – text controlled 3D mesh deformation and stylization
• CLIP-Mesh – text to 3D mesh with texture and normal map (still pretty simple and mixed results)
• DreamFields – latest text to 3D (github)
• ImageSorter by @pharmapsychotic – sort images by similarity (nice for StyleGAN/FiLM animated loops)
• PIFuHD Colab – Human photo to 3D mesh of the human
• Point-E – OpenAI’s text to 3d point clouds (github)
• text2mesh – Kaggle notebook for text to 3D mesh
• Watermark images – little notebook to add text watermark to images
• Zero-Shot Text-Guided Object Generation with Dream Fields – text to 3D render

AI Art Discord Servers

There are quite a few Discord servers dedicated now to AI artists or discussing text to image techniques.

• Ai NFT Discord – AI NFT Consortium. Has especially useful StyleGAN training resources
• Disco Diffusion Discord – chat and tech support for the Disco notebook
• EleutherAI Discord – researchers and good art room with more technical discussions
• Jukebox Community Discord – server for using OpenAI Jukebox for music generation
• LAION Discord – group working on replicating a full DALLE-E
• NeuralismAI Discord –  AI art competitions and knowledge exchange
• Prompt Sharing Discord – community for sharing text to image prompts
• VQGAN+CLIP Discord – home of Instagram #vqganclipcommunitycolab
• Zoetrope Central Spoke Discord – support and discussion of the Looking Glass notebook 

Learn to Code Generative Ai

• The Illustrated Stable Diffusion – really nice overview of Stable Diffusion and the pieces that make it up
• AIAIART – really nice ongoing youtube series and discussion in its Discord
• Deep Learning for Art, Aesthetics, and Creativity – MIT course available on youtube
• Dive into Deep Learning (online, free, interactive)
• Deep Learning Foundations to Stable Diffusion – 4 videos from the fast.ai class
• Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play by David Foster [2019]
  Really enjoyed this and it’s a great book! It’s from 2019 so doesn’t cover the very latest like VQGAN, CLIP, guided diffusion though.
• HuggingFace Diffusion Models Class – nice coverage of the diffusers library and Stable Diffusion 
• The Artist in the Machine: The world of AI-powered creativity by Arthur I. Miller [2020]
  Not very technical but engaging and inspiring view of many Ai art projects so far.
• ml4a.net – online textbook, classes, and learning resources

Cool Apps

No Code AI Art tools

• Artbreeder – StyleGAN model with “genes” (directions in latent space) for editing
• Artbreeder Collage – CLIP guided diffusion on top of simple collages
• Astria.ai – nice and easy Dreambooth training – upload images, and get finetuned SD model
• BlueWillow – text to image Discord like MidJourney (appears to use Stable Diffusion finetunes)
• CogView – text to image, Chinese model like DALL-E ( interview )
• conjure.art – new text to image site currently in beta
• craiyon – formerly known as dall-e mini, free and makes quick grids of 9 outputs
• Dall-e 2 – OpenAI’s text to image
• DeepDreamGenerator – deep style, thin style, deep dream
• DreamStudio – easy to use text to image from creators of Stable Diffusion 🔥🔥🔥
• Genmo – short animations (looks like KLMC2)  🎥
• Kaiber – create short animations (looks like Deforum) 🎥
• midjourney – text to image via discord bot  🔥🔥🔥
• murf.ai – text to speech with Ai voices
• neural.love image-upscale – credit based image upscaling service
• NightCafe – style transfer, VQGAN, diffusion image generation
• Ostagram – style transfer
• Playform – style transfer, train stylegans, images morphs
• pollinations.ai – run lots of popular notebooks
• ProsePainter – interactive tool to “paint with words”
• runwayml – video editing powered by AI 🎥
• snowpixel – text to image and variations
• StarryAI – text to image with easy selection of styles
• synth.run – text to image app for iOS, Android, and web
• tokkingheads – animate portraits with Ai
• uberduck.ai – text to speech with lots of different voices
• Visions of Chaos – run the popular AI notebooks locally on Windows (see the Machine Learning setup steps)
• Wombo –  Super fast and free
• wzrd.ai – give it a music file and produce animation from big set of pretrained GANs

Create Game Assets

• Layer – 2D assets and variations for games
• Luma AI Imagine 3D – alpha test of text to 3D models
• MirageML – 3D assets and prototyping
• Scenario.gg – AI-generated game assets
• withpoly – AI-generated textures and materials

Online Galleries to Showcase Art

• OnCyber art galleries – https://oncyber.io – Cool 3D art gallery to showcase your art with links to NFT market
• Spatial – https://spatial.io

本文转载地址：https://pharmapsychotic.com/tools.html