ReLive

Team Details

Samarth Rawat

Pramod Goyal

Gourav Mohanty

Midatana Sai Praneetha

Description of Project

We are dedicated to the mission of preserving the memories of our loved ones through the power of artificial intelligence. By bringing back our loved ones as AI, we can continue interacting with them and keeping their memories alive. ReLive can provide comfort and closure to those who have lost loved ones, and it has the potential to revolutionize the way we grieve.

The application takes in user input through photos, audio files, previous chats, and personality traits and then creates a digital model of the deceased. The images are then used to create a realistic 3D model, while the audio files are converted to a voice clone that can speak like the deceased. The personality traits and chat history are used to generate personalized text conversations.

User input elements are explained below:

1. Audio Sample: Conversations that Echo the Past; To ensure a truly immersive experience, we have focused on converting the provided audio files into voice clone models. By integrating sophisticated speech synthesis techniques, we aim to replicate the deceased individual's unique vocal patterns, intonations, and nuances. This enables the AI to engage in conversations with the user, speaking with a voice that closely resembles that of their loved one.

2. Conversation information: Nurturing Meaningful Dialogue; Including personality traits and chat history, adds depth and authenticity to the AI-generated conversations. Our project aims to understand the individual's unique traits by incorporating natural language processing algorithms and sentiment analysis. This understanding enables the AI system to generate personalized responses and dialogue that align with the loved one's distinctive character, fostering a more meaningful and intimate interaction.

   1. Chat history - The user will export chat history as a .txt file from any messaging platform like WhatsApp.

   2. Personality traits: Users are prompted to choose personality traits from a pre-defined exhaustive list. Users are given the flexibility to input and customize any quirks or personality traits according to their preferences.

3. Photos: Realistic Videos from Photos; At the core of ReLive lies the ability to transform static photos into dynamic, lifelike videos. Leveraging cutting-edge computer vision techniques and 3D modeling, we meticulously create a digital model of the deceased from the provided photographs. This digital representation captures the true essence and appearance of the individual, immersing users in a visually authentic experience.

Our technical workflow can be understood easily via the flowchart below.

ReLive is still in development, but it has the potential to revolutionize the way we grieve. By bringing back our loved ones as AI, we can continue interacting with them and keeping their memories alive.

The ReLive team is deeply committed to harnessing the power of technology to impact people's lives positively.

Tech Stack

• React -> Front-end Framework

• Vercel -> Front-end Deployment

• Appwrite Cloud

  • Database

  • Cloud Functions -> Python

• APIs

  • Voice Clone

  • Text-to-Speech

  • Chat Generation

  • Video Generation

Challenges We Faced

1. We initially developed our frontend in Angular. However, we faced challenges with authentication due to inadequate documentation. To overcome this hurdle, we decided to migrate our codebase to React. This shift enabled us to leverage the resources and support available in the React ecosystem.

2. We encountered a significant challenge when connecting the frontend to the database. We spent approximately 5 hours attempting to write data into the storage bucket but couldn't identify the error. Eventually, we discovered through the Discord channel that the cloud server was down for a while, causing the connectivity issue. If this information had been promptly displayed on the website, it would have saved us valuable time and effort.

3. We encountered a challenge when trying to deploy our functions. Initially, we assumed that uploading the function definition would suffice, but we soon realized we needed to manually write our code into the central part. This discovery was made through trial and error, which consumed valuable time and effort. Having comprehensive documentation would have been immensely helpful in guiding us through the deployment process.

4. We encountered challenges related to model generation using multiple APIs. Our process involved several I/O conversions to ensure compatibility and produce the desired results. These conversions added complexity and required careful handling to maintain data integrity. Additionally, we had to prioritize minimal execution time to ensure optimal performance. Overall, these hurdles pushed us to be resourceful in integrating multiple APIs and optimizing our code for efficient execution.

5. We faced a formidable challenge during the integration process with messaging platforms. The task of importing chat history from platforms such as WhatsApp as a .txt file proved to be quite intricate. Our developers had to meticulously design and implement a robust import mechanism capable of parsing and extracting crucial data from the exported file. Furthermore, they had to address the complexities arising from various formatting styles encountered in these files and adeptly map the extracted data to the appropriate data structures within our application.

Public Code Repo

Repo Link

Demo Link

If the video is not playing, please use the following google drive link to view it - Video Link

#appwrite #AppwriteHackathon #BuiltWithAppwrite