Introduction

Sound Byte is an app that provides summaries and audio output of articles from their URLs. With this app, you can receive news not only visually but also through auditory input, allowing you to listen to the news instead of reading it.

Links

GitHub Repository

Application

How to Play

Sign in to the app.
- Username: soundbyteamplify@gmail.com
- Password: amplify@12345
Send a URL.
Press the play button.

Demo Video

https://youtu.be/GFXXYi8BxDc

Why we do

Problem

Busy modern people do not have much time. However, we are forced to have a large amount of input to survive. We don't have time to read the news as it is of interest to us.

Solution

Reading a text article in its entirety is time-consuming and is only possible in an environment where you can see the device. However, this makes it impossible to input news on the move or when the device is not at hand. So we summarized the text articles and output them in audio.

How we built🚀

<AWS>

Cognito: Authentication

Amplify can easily implement authentication using Cognito. This time, we used the Amplify CLI to build authentication.
You can build authentication with a single command from the Amplify CLI.

amplify add auth

AppSync: GraphQL API

AppSync is used with the GraphQL API.
Using the GraphQL category of the Amplify CLI, we have created a database-based GraphQL API that can also be built with a single command from the Amplify CLI.

amplify add api

Data Modeling

We created a data model with schema.graphQL created by amplify add api.
By using @model, Amazon DynamoDB database tables are automatically created.
We also use @auth and @hasMany to build the data model.

// amplify/backend/api/soundbyte/schema.graphql

type User
  @model
  @auth(rules: [{ allow: private, provider: userPools }, { allow: private, provider: iam }]) {
  id: ID!
  email: String!
  articles: [Article] @hasMany
}

type Article @model @auth(rules: [{ allow: private, provider: userPools }]) {
  id: ID!
  title: String!
  content: String!
  sourceUrl: String!
  speechUrl: String!
  userId: ID! @index(name: "byUserId", queryField: "articlesByUserId")
  user: User! @hasOne(fields: ["userId"])
}

type GenerateSummaryTextResponse {
  title: String!
  summaryText: String!
}

type Query {
  generateSummaryTextQuery(url: String!): GenerateSummaryTextResponse!
    @auth(rules: [{ allow: private, provider: userPools }])
    @function(name: "generateSummaryText")
  textToGenerateSpeech: String
    @auth(rules: [{ allow: private, provider: userPools }])
    @predictions(actions: [convertTextToSpeech])
}

DynamoDB: Database

Automatically creates Amazon DynamoDB database tables based on schema.graphql

Lambda: Function

Lambda is run via AppSync to retrieve and summarize articles and output audio.
This Lambda was built using a container image because the use of newspaper, a library for Python scraping, could only be achieved by building with a container image.

## This function is generated by Dockerfile in the same directory, so you need to build the image and push it to ECR.

import json
import os
import openai
from newspaper import Article

def handler(event, context):
  arguments = event.get("arguments")
  url = arguments.get("url")

  openai.api_key = os.environ["GPT_API_KEY"]

  if not url:
    raise Exception("The attribute url is required.")

  if not openai.api_key:
    raise Exception("There is a missing environment variable.")

  try:
    article = Article(url)
    article.download()
    article.parse()

    title = article.title

    text = article.text

    chat_completion = openai.ChatCompletion.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": f"The following is the text of a news article. Please summarize this sentence in plain English.{text}"}])

    return {
      "summaryText": chat_completion["choices"][0]["message"]["content"],
      "title": title
    }
  except Exception as err:
    print(err)
    raise Exception("internal server error")

// amplify/backend/api/soundbyte/schema.graphql
type Query {
  generateSummaryTextQuery(url: String!): GenerateSummaryTextResponse!
    @auth(rules: [{ allow: private, provider: userPools }])
    @function(name: "generateSummaryText")
/// ...
}

Storage: To use amazon polly

I am adding storage to use amazon polly. You can build authentication with a single command from the Amplify CLI.

amplify add storage

Amazon Polly: Text to Speech

Amazon Polly is used to convert summarized content into speech.
We chose Amazon Polly because of its natural voice, support for dozens of languages, and ease of use in Amplify projects.

<Design>

We organized the model, designed the database, and designed the page in Figma.

<Frontend>

MUI

Various UI components could be used for free and easily.

Next js

Next js could be incorporated into Amplify.

<Backend | Summary>

newspaper: scraping

Using Python’s newspaper library, we were able to get the full text of the article.

ChatGPT: summary

Using ChatGPT’s API, the obtained text can be summarized with high accuracy.

<Hosting>

Vercel

Since the latest Next js version cannot be hosted by Amplify Hosting, we adopted Vercel for hosting this time.
Vercel was also able to host projects using Amplify and Next js with high speed.

Tech Stack

AWS

Cognito: Authentication
AppSync: Graphql API
Lambda: Function
Polly: Text to Speech
DynamoDB: Database
Storage: S3

Design

Figma: design
Figjam: modering

Frontend

MUI
NextJS

Backend | Summary

ChatGPT: summary
newspaper: scraping

Hosting

Vercel

Conclusion

This time we were able to achieve our goal of creating a minimalist app at high speed using Amplify. Amazon Polly and Chat GPT are easy to connect to AppSync, and Amplify provides a very comfortable development experience by making it easy to incorporate AWS services. It is a great service to try out products. Thanks to Hashnode and AWS Amplify for this opportunity. Thank you.

Sound Byte - Article Summarizer App with Text-to-Speech!

Table of contents