#001 - Hello World

Stable Diffusion in VR, How models shrink, Making of Interiorai.com , GPT-3 in Google Sheets & more...

Oct 11, 2022

Welcome! CRW RIDE (🔊curve-ride) is your weekly navigator to the world of LLMs & CreativeAI. There’s so much happening out there; I curate the chaos weekly.

Whether you’re an Artist, Engineer, Founder, Investor, Content Creator or an ML Enthusiast- there’s something for everyone! The latest Research Papers, Tutorials, Open-source projects, Prompt tips, Industry Stats, Insights, Art Inspiration, Memes …

so let’s dive in 👇

Alright, the 1st issue is finally here!

But first a quick apology to my first 10 subscribers who came from a tweet a few months back🙏. Better late than never!

🕵️‍♂️ Sneak Peek

Why OpenAI Whisper, Dreambooth are a big deal
Pieter Levels explains how he made Interiorai.com in 5 days
Stable Diffusion in Google Sheets? Yes!

🦘Why OpenAI Whisper is a major leap:

If you're wondering why is Whisper so special, this is the perfect example. I had used Snipd a few months back but it had very inaccurate transcripts. Just see the difference before/after Whisper!

Snipd | Unlock the Knowledge in Podcasts @snipd_app

Here's a side-by-side comparison between our previous transcription tech and @OpenAI's Whisper for a snip from the @hubermanlab podcast:

🔬 Honey, We Shrunk the Models!

I was curious how rapidly these models keep getting smaller and smaller. This thread explains that in detail. Useful if you’re looking for ways to optimize ML models.

swyx 🌴 @swyx

Open Source is accelerating AI: Aug 26: @natanielruizg announces Dreambooth Sep 7: @XavierXiao releases StableDiffusion port, but requires 48GB VRAM Sept 25: @CorridorDigital video 24GB Sept 26: @shivamshrirao reduces it to 18GB Sep 27: 12.5GB Oct 1: 11GB Today: 10GB OSS > APIs

Dreambooth + Stable Diffusion = 🌋:

Stable Diffusion enables generating fantasies. Dreambooth enables you to insert yourself into those fantasies.

Last decade was about Bitmoji, Animoji. Those still have a place just like folks still enjoy retro games. But Dreambooth and its successors will enable more customization.

Infinite Vibes @Infinite__Vibes

Here's the last vid from my first experiments with using #dreambooth to get my likeness into #stablediffusion. Not the most exciting video but the stability of the face when fine-tuned like this is noteworthy. (sound on, music is an oldie I just rediscovered) #aiart

Imagine a 3D version of Dreambooth to generate 360° avatars for some VR application ! Speaking of VR, check this out:

ScottieFox @ScottieFoxTTV

WILD! Working to achieve my goals! Lot's of test effort today. Stable Diffusion in VR + touchdesigner = realtime immersive latent space. This proof of concept is the FUTURE! #aiart #vr #stablediffusion #touchdesigner #deforum

👷‍♂️ The Making of Interior.ai :

Pieter Levels made Interiorai.com in 5 days. Here you can upload photos of your room and get interior design mockups in different styles. Kinda like Snapchat face filters for your room.

If you’re trying to productize Stable Diffusion or similar models, you’ll find this thread helpful. Also as of yesterday he’s crossed $1000 MRR!

💬 Improving Language Models’ ability to answer complex questions :

Self-ask is a method of generating subquestions, answer them before answering the main question. Pretty cool prompting technique to increase accuracy on complex questions. Basically, give the model more context.
Integrating Google with Self-ask for fact retrieval leads to better performance (demo)

Ofir Press @OfirPress

The structure of our prompt allows us to easily parse subquestions and let Google answer them instead of GPT-3. So GPT-3 is only in charge of asking the subquestions and returning the final answer, Google does all the fact retrieval. GPT-3 output in green, Google in cyan.

🧪 Diffusion Models for videos, 2D → 3D & protein folding

Text2Video: Google released Imagen Video & Phenaki
Imagen Video Features: different artistic styles, 3D understanding, and text rendering and animation.
Phenaki helps generate long coherent videos for a sequence of text prompts (examples).
2D → 3D: Also by Google. Generate 3D renders from a single image (paper).
Folding Diffusion: a diffusion model by Microsoft Research for protein structure inspired by physical protein folding. This could lead to new biological discoveries & treatments targeting yet incurable diseases (tweet).

🤿 CreativeAI Workflows

🛠 POISD (Pointcloud Occlusion Inpainting with Stable Diffusion):

Dmitrii’s AI Animation has received >210k views. He wrote a custom stable 3D animation algorithm (POISD) on top of Stable Diffusion to make it. (link)

Might get open-sourced this week. I’ll let you folks know next week! So..subscribe 😜!

⏳ Saving time by using DALLE-2 to generate characters:

Sergei Galkin @sergeyglkn

AI-generated character + a couple of hours of modelling. AI saved me several days of work, with looking for references, sketches and coordination with a team.

🔧 Toys N’ Tools

Character.ai enters open beta

It lets you create and talk to AI characters. I am very curious about how they’ll turn this into a business.

Their team includes five of the authors of the Google LaMBDA paper (h/t Simon). This 👇 trend is gonna continue.

Simon Willison @simonw

Shipping AI-driven products at companies like Google and Facebook is really hard, because there are so many incentives NOT to ship these potentially risky systems I imagine we'll see a lot more researchers from big cos split off their own AI startups - or get poached by them

Murf.ai 2.0 launched on Product Hunt (link)

Create natural sounding voiceovers in a variety of accents. It gives a ton of control for tweaking voice expressions. No recording hassles!

I am thinking about using this to make weekly CRW RIDE roundup videos (for twitter, IG etc to grow this newsletter). I’ve looked at other options, Murf seems to be the best choice right now (on the basis of pricing, feature set etc.)