Generative-AI Newsreader

Presenting to you, Keezum AI Newsreader!

Welcome to the AI Newsreader repository! This project is a demonstration of how generative AI tools can be used together to create a sophisticated and dynamic newsreader avatar.

⚠️ Disclaimer: Just kidding feel free to use it as per your comfort.

visit this link to watch the demo video to see Macy in action, where she briefly talks about two commonly-prescribed medications.

The outcome is pretty impressive and will only get better over time, given the speed at which generative AI is improving. To create this demo, it took me 0 dollars and only 25 minutes.

Here are the tools I used:

1. Midjourney to generate an image of a female pharmacist
2. OpenAI's ChatGPT to generate script for newsheadlines
3. ElevenLabs's Prime Video AI to generate audio from the ChatGPT script
4. D-ID's Creative Reality Studio to generate realistic animated avatar video synced with audio (Free trial)

---

Step-by-Step Guide

(1) Midjourney - Image Generation

• We need a face to represent our avatar, and we can use image generation tools like Midjourney to do just that
• Midjourney is a free AI service by OpenAI that creates images from textual descriptions
• Setup:
  1. Midjourney works entirely on Discord, so make sure you sign up for a Discord account (which is free).
  2. Visit this Midjourney site link, which automatically takes you to a Discord invite.
  3. Accept the Discord invite to Midjourney. Choose to Continue to Discord.
  4. Click on the Midjourney button (with the ship icon) and select any of the Newcomer rooms e.g., newbies-24
  5. In the chat line, type /imagine followed by your description prompt. For example, the prompt I used was "High-quality upper body professional photo of a female media news reporter in a red coat with a newsroom background ". Press Enter after typing it in, and give Midjourney some time to generate the images.
  6. Once done, you will see an output of four images. Below the image set, you will see a set of buttons U1-U4, and V1-V4.
  7. The four images are numbered going clockwise from the upper left. To get a new variation on one you like, select "V1" (or V2, V3, or V4), and to get a high resolution copy, select "U1" (or U2, U3, or U4).
  8. Once you got your selected high-resolution variant by selecting one of the U buttons, click on the image and select 'Open in Browser'. You can then save the high resolution image on your local machine

(2) Playground ChatGPT - Text Generation

• Since everyone using chatgpt so it got a bit slow these days and not everyone interested in paying money so we can use ChatGPTPlayground to perform our task very quickly.We need a news headlines script that can provide news. To do that, we can use ChatGPTPlayground.
• ChatGPT is a chatbot developed by OpenAI and launched in November 2022. It is built on top of OpenAI's GPT-3 family of large language models.
• Setup:
  1. Visit this link to access ChatGPT (you will need to login accordingly)
  2. In the prompt section, enter the relevant description for the counselling script. For example, the prompt I used is as follows: "Create a script that first introduces oneself as a newsreader called Keezum, and then talks about the headlines of news with points in start and give some news info with the limit of two to three lines max".
  3. From the output on the ChatGPT screen, copy and save the generated text in a text file on your local machine.

(3) ElevenLabs - Text-to-Speech Generation

• Next, we want to convert the ChatGPT script text into a natural sounding audio clip. We can do so with free tools like Prime Voice AI (by ElevenLabs)
• Prime Voice AI is a realistic and versatile AI speech software that brings the most compelling, rich and lifelike voices to creators and publishers seeking the ultimate tools for storytelling.
• Setup:
  1. Visit the ElevenLabs page and create an account for free.
  2. On the Speech Synthesis page, select a specific voice in the settings, paste the text script into the text section, and click Generate. The voice I chose was premade/Domi as I found it to be the most lively and natural. The settings can also be adjusted accordingly for things like stability and clarity.
  3. I shortened the script slightly by removing the section on the drug Amlodipine because I did not want the demo to be too long.
  4. There is a credit limit for the free account, so make sure you use them wisely for the audio you want to generate.
  5. Download and save the .mp3 (titled 'synthesized_audio.mp3') file on your local machine.

(4) D-ID - Photorealistic Talking Avatar (and Audio Sync) Generation

• Lastly, it is time to piece the pharmacist image and counselling audio together into a photorealistic video. To do so, we can use tools like D-ID.
• D-ID’s creative AI technology takes images of faces and turns them into high-quality, photorealistic videos. At the click of a button, it can combine images with audio or text to give them expression and speech.
• Setup:
  1. Visit the D-ID website and create a Free Trial account
  2. Select the Create Video button to start creating a new video
  3. Add your Midjourney pharmacist image as a presenter image
  4. Upload our ChatGPT scripted audio in the Upload your own voice section on the right.
  5. Click the Generate Video button at the top right and wait for your masterpiece to be ready for download!

---

Dependencies

This project requires the following dependencies:

Python 3.6 or higher
Midjourney
OpenAI's GPT-3 API
ElevenLabs' Prime Video AI
D-ID's Creative Reality Studio

---

Future Work

We plan to continue to refine and improve this project by integrating more generative AI tools and expanding the functionality of the newsreader avatar. We also welcome any contributions or suggestions from the community. Acknowledgments

We would like to thank the developers and researchers at Midjourney, OpenAI, ElevenLabs, and D-ID for their excellent generative AI tools, which made this project possible.

---

License

This project is licensed under the MIT License - see the LICENSE file for details.