Empower Your Email Routine with LLM Agents: 10X Efficiency Unlocked

6 min readSep 24, 2024

Nowadays, almost every service revolves around communication, whether it’s plumbers, receptionists at clinics, or clerks in banks. Every day, countless people read emails, review attached scanned documents, and PDFs; and sort, organize, and forward them elsewhere. Imagine how much time we could save if we automated this process! With the advancements in Generative AI and Large Language Models (LLMs), it’s no longer just a dream — it’s possible now.

Problem ✨ Solution

You receive an email with a long thread of messages and attached files — many of them with random names. What do you really want? To quickly understand what the email is about and know what actions to take.

That’s where EmailSnap comes in. Simply forward the email to review@emailsnap.app, and EmailSnap will apply AI to analyze the email and its attachments, then send you back a new, actionable, and well-formatted response.

The processed email contains:

Title: [Priority] Subject
From: Sender of the email
To: Recipient
Highlights: Key action items in bullet points
Summary: A concise review of the email content and attached files
Attachments: Renamed to reflect their actual content

Thread of email with attached files <VS> Well formatted email with insights and action items

You need to be manually whitelisted to send emails to review@emailsnap.app

Magic, right?🪄 Now, let’s take a look at how it works

EmailSnap.app 📩

The implementation consists mainly of two parts: the underlying infrastructure and the LLM flow. Let’s start with the more interesting part.

AI Review ✨

For the LLM implementation, we use the LangChain stack: LangChain, and LangSmith. Despite not using LangGraph itself, I apply the same graph-like idea for executing EmailSnap’s LLM processing, as it requires multiple LLM calls with different prompts while using various tools, like saving intermediate states to a database and S3.

Out execution graph contains the following nodes:

Format Email: Reads the EML file and converts it into well-formatted text, handling threads and forwarded messages while removing redundant HTML tags and other irrelevant information.
Read Attachments: If the attached file is an image or PDF, we convert it to Base64 for the LLM to read the content. For PDFs, each page is converted into a new image, potentially creating a long loop.
Review Attachment: This node creates a summary with highlights and action items from each document, suggests the recipient, and proposes a better file name.
Email Summary: The final node combines the formatted email and the summaries from all attachments to generate a complete email with insights, action items, highlights, and an overall summary.

After the final content is generated, we will send out a new email back to the sender including reorganized attachments.

Infrastructure 🛠️

To process emails, we needed a scalable tech stack, and after a quick search, I found AWS Simple Email Service (SES) to be the perfect solution. The flow works as follows: SES receives an email, stores it in S3, and sends a notification via SNS (Simple Notification Service).

My service subscribes to an SQS (Simple Queue Service) queue, which listens for SNS messages. From the queue, I retrieve a message ID and use it to load the email from S3. The emails are stored in EML format, which isn’t the easiest to process, but fortunately, there are plenty of libraries to handle that. This setup applies to a specific email address that you register with SNS.

Below is the most interesting part of the Terraform code to configure it:

# Setup SNS 

resource "aws_ses_domain_identity" "ses_domain" {
  domain = var.domain
}

resource "aws_sns_topic" "email_notifications" {
  name = "email-notifications"
}

resource "aws_ses_receipt_rule_set" "main" {
  rule_set_name = "default-rule-set"
}

resource "aws_ses_receipt_rule" "email_receipt_rule" {
  rule_set_name = aws_ses_receipt_rule_set.main.rule_set_name
  name          = "store_emails_to_s3_and_notify"

  recipients = ["review@${var.domain}"]

  enabled      = true
  scan_enabled = true


  s3_action {
    bucket_name       = aws_s3_bucket.email_bucket.bucket
    position          = 1
    object_key_prefix = "emails/"
  }

  sns_action {
    topic_arn = aws_sns_topic.email_notifications.arn
    position  = 2
  }
}

# Create SQS 

resource "aws_sqs_queue" "email_queue" {
  name                       = "email-processing-queue"
  visibility_timeout_seconds = 30
  message_retention_seconds  = 86400
}

resource "aws_sns_topic_subscription" "email_sqs_subscription" {
  topic_arn = aws_sns_topic.email_notifications.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.email_queue.arn
}

After that, we subscribe to the queue on the service side to process emails:


while True:
  # Subscribe to the SQS 
  response = sqs_client.receive_message(
      QueueUrl=SNS_QUEUE_URL,
      MessageAttributeNames=['All'],
      MaxNumberOfMessages=5,
      WaitTimeSeconds=20
  )
  messages = response.get('Messages', [])
  for message in messages:
    email_id = get_email_id_from_sns_message(message)
    eml = fetch_email_from_s3(email_id)

# Find email ID from the SQS message
def get_email_id_from_sns_message(message):
    body_string = message.get('Body')
    body_json = json.loads(body_string)
    message_string = body_json.get('Message')
    message_json = json.loads(message_string)
    mail = message_json.get('mail')
    email_id = mail.get('messageId')
    return email_id

# Load and parse Email from EML
def fetch_email_from_s3(email_id: str):
    response = s3_client.get_object(Bucket=EMAIL_BUCKET_NAME, Key=f"emails/{email_id}")    
    body = response['Body'].read()
    return email.message_from_bytes(body,  policy=policy.default)

Besides that, I set up a MySQL RDS and an EC2 instance to run a Dockerized service. I chose this setup for convenience, ease of debugging, and to have more control over the running machine. However, in the long term, it makes more sense to migrate to Fargate for better scalability and management.

SES Setup

Setting up a Simple Email Service (SES) requires a few manual steps after provisioning the infrastructure. Once your infrastructure is in place, follow these steps by accessing the SES dashboard:

Navigate to Configuration -> Identity -> DKIM and press Generate.

Copy the CNAME records to your domain’s DNS provider.

2. Add an MX record to your DNS provider with the following details:

Name: Domain
Mail server: 10 inbound-smtp.us-east-1.amazonaws.com (link changes depending on your region)

3. Open Email receiving and activate the rule: default-rule-set.

This setup works in the SES sandbox. To gain production access, you will need to apply for production access from the SES homepage by selecting “Get Production Access.” AWS may require you to justify your use case and potentially ask further questions.

Note: You can receive emails without the verification process.

Sample

Finally, let’s take a look at a real example. I took an MRI report in PDF format from an online source, sent it to one email address, and then forwarded it to review@emailsnap.app. EmailSnap generated a new subject, highlights, and summary. It also extracted the recipient from the PDF — Dr. Ross Banner — and finally suggested a new file name: Regina Doe MRI Report, based on the patient’s name and procedure.