Text-to-Video Prompts: Runway Tutorial

The most recognized & famous Text-to-Video AI model

👋 Hey, I’m Daniil, and welcome to a ✨ subscriber-only edition ✨ of Creators’ AI. Each week, we post AI Tools, Tutorials, News, and practical knowledge aimed at improving your life with AI. To get full access to our content, consider upgrading.

---

Generative AI models have only been getting more and more interesting recently. Things got even more fun when you combine LLM and Generative AI.

Today, we go through the famous Claude AI + text-to-video generator Runway ML Gen-2 and see how they can be used together to produce an abstract video from a text description, using minimal human input - barely in a few minutes!

What is Runway?

Runway is a generative AI for creating videos.

Runway ML has many incredible features, but its most impressive feature is its ability to generate short video clips from just a text prompt. While the technology is still early, it gives us a fascinating glimpse into the future of video.

Free version is limited to producing 4 second videos and 320 character prompts. But thats enough to make fun videos even commercial: You're 100% free to use any content you create using Runway commercially or non-commercially - RunwayML

RunwayML released its last version, GEN2, in March 2023, and now it’s available for everyone. Let’s try it out.

Before starting with Prompts

Creating videos in Runway is just like the creation of images with Midjounrey or Stable Diffusion, but there are a few unique things to keep in mind:

• 🎥 Think like a screenwriter, not a Director of Photography, when writing prompts for Runway Gen-2 to focus on describing what you want to see in the shot rather than specific camera models and lenses.

• 📷 Using different types of shots like a medium shot, close-up shot, and wide angle shot can enhance the visual appeal of the footage.

• 🌅 Nature and landmarks can be captured well in footage, even if they don't exactly resemble the real thing, by using aerial drone shots and adding cinematic elements like the time of day (golden hour, sunrise, sunset, twilight).

---

Sharing is caring! Refer someone who recently started a learning journey in AI. Make them more productive and earn rewards!

A quick tutorial with Claude

Creating the Prompt → www.claude.ai/

To develop a compelling text prompt, you can use Claude — an AI assistant that directly challenges Open AI’s Chat GPT.

1. The first step is to ask Claude to craft a prompt. E.g., a spiritual, abstract animation reminiscent of the surreal work of artist Hilma af Klint and similar to the artistic evocation within the Holy Mountain.

2. Then, iterate on the wording and length of the Prompt until creating a 320-character description (Runway limitation) that acted as the perfect script for the vision:

3. Paste Prompt to the Runway ML platform (sign-up required, no card needed). In the text-to-video generator, we configured the output settings to produce a 4-second (sadly limited) 1080p video. Within seconds, we received a video that translated what Runway understood into a visual, moving piece.

   The result:

   

   The entire process, from Idea > Prompt > video, didn’t even take 5 minutes.

   The completed AI-generated video encapsulated the essence of the Prompt, with a psychedelic feel deeply rooted in physical Nature and a spiritual one, utilizing colors that expertly evoked these feelings. Thx, Zacharia, for the excellent tutorial and spiritual vibe!

Do you necessarily need an LLM to make prompts for Runway? - Absolutely not. After trying 20 different AI-generated prompts, I ended up writing myself and found it more productive. Try yourself, and let us know what works for you in the comments!

Different modes in Runway