Episode 6: Handling incoming Email in your application

Welcome to Episode 6 of this series. In this episode, we will learn how your Google App Engine Java (GAEJ) Application can receive incoming email.

In an earlier episode of this series, we have covered how to use the Email Service of GAEJ to send out emails from your application. At that point in time, we were using version 1.2.5 of the AppEngine SDK. That version did not provide support for handling incoming email. Since then a newer version of the AppEngine SDK for Java 1.2.6 has been released. And one of the nice features of this new release is support for incoming email in your application. What it means is that anyone can send an email to your GAEJ hosted application, it can receive an email and then you can process and perform some business logic with that as required.

Prerequisites

Before we move on, it is important that you have a working setup of the environment and are comfortable with developing a new project using the Google Eclipse plugin. If not, please go through earlier episodes that contained detailed setup information along with a few development episodes like using Email Service, using the URL Fetch service, etc.

The most important prerequisite is to make sure that you have upgraded your Eclipse environment to the latest AppEngine SDK i.e. 1.2.6. Please go through the following : Episode 5: Upgrading to Google App Engine 1.2.6

A quick check: To make sure that you  have version 1.2.6 of the SDK installed, do the following:

1. In your Eclipse IDE, go to Window –> Preferences.
2. Navigate to Google –> App Engine
3. You should see version 1.2.6 of the SDK installed. And make sure that it is the default one by selecting it. By selecting it, it will be added to the Build Path of your Google Web Application projects. And we need the latest SDK since it will have support for the Incoming Email feature.

Receiving Email Feature

App Engine now supports incoming email in your applications. Read the official documentation here. Your applications can now receive email and you can parse out the email and determine any business logic that needs to be processed. This opens up a whole new range of applications where you can fulfill requests and send information from your application by allowing users to simply send an email from their favourite email client. Think of it as a Instant Message itself that your application can receive and react to it. We had seen in an earlier episode how through XMPP Support, we can write our own Agent that can receive XMPP messages directly and respond to them. Now with version 1.2.6 of the SDK, the same functionality has got extended to email too. And the best part about it is the consistency with which Google has implemented it.

The steps to follow to receive an email is identical to the ones required to receive XMPP messages:

1. Configure your application to receive incoming email by configuring the Mail Service

2. Write and configure a servlet to receive email

3. Once the application is deployed, anyone can send an email to SomeID@YourApplicationId.appspotmail.com. SomeID is any id like test, admin, support,etc. It is just an id. And YourApplicationId is the application id of your hosted Google App Engine application.

Let us look at each of the above points in detail now. But before we begin, create a New Google Web Application Project (If you wish you can continue to use an existing project to add the incoming Email support, which is what I have done personally, but the choice is yours) . Follow these steps to create a new project:

1. Either click on File –> New –> Other or press Ctrl-N to create a new project. Select Google and then Web Application project. Alternately you could also click on the New Web Application Project Toolbar icon as part of the Google Eclipse plugin.
2. In the New Web Application Project dialog, deselect the Use Google Web Toolkit and give a name to your project. I have named mine GAEJExperiments. I suggest you go with the same name so that things are consistent with the rest of the article, but I leave that to you.
3. Click on Finish

This will generate the project and also create a sample Hello World Servlet for you. But we will be writing our own Servlet.

Configuring the incoming Email Service

This is straightforward and all you need to do is add the following element to the appengine-web.xml file. The appengine-web.xml file as you know is specific to the Google Java Web Application project and is used for configuring certain services. You need to configure the Incoming Email Service so that your application is enabled to receive it, being one of them. It is found in the warWEB-INF folder of your Web Application Project. The XML fragment to add at the end but before the </appengine-web-app> element

<inbound-services>
<service>mail</service>
</inbound-services>

Configure and code a Java Servlet that will receive the incoming Message

All Email messages to your application are delivered via POST to following URL path in your application: /_ah/mail/ as per the Google AppEngine documentation. So you will need to configure the servlet like the following snippet in the web.xml file, present in the warWEB-INF folder of your Web Application Project.

We need to add the <servlet/> and <servlet-mapping/> entry to the web.xml file. This file is present in the WEB-INF folder of the project. The necessary fragment to be added to your web.xml file are shown below. Please note that you can use your own namespace and servlet class. Just modify it accordingly if you do so.

 

<servlet>
<servlet-name>emailhandler</servlet-name>
<servlet-class>com.gaejexperiments.email.GAEJReceiveEmailServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>emailhandler</servlet-name>
<url-pattern>/_ah/mail/*</url-pattern>
</servlet-mapping>

<security-constraint>
<web-resource-collection>
<url-pattern>/_ah/mail/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>admin</role-name>
</auth-constraint>
</security-constraint>

 

In the above snippet, you will find the fixed URL path /_ah/mail/* configured as the <url-pattern/>. And then I have a Java Servlet class com.gaejexperiments.email.GAEJReceiveEmailServlet as the <servlet-class>. The security constraint has been added so that in case anyone invokes your url directly, then only Google Account authenticated users will be able to do that.

Now, all we have to do is write our Servlet. As mentioned, the incoming Email messages will be POSTed to our Servlet, so we need a simple doPost(…) implemented in our Servlet. The code is shown below:

 

package com.gaejexperiments.email;

import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.mail.MessagingException;
import javax.mail.Multipart;
import javax.mail.Part;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;
import javax.servlet.http.*;

@SuppressWarnings("serial")
public class GAEJReceiveEmailServlet extends HttpServlet {
public static final Logger _log = Logger.getLogger(GAEJReceiveEmailServlet.class.getName());

@Override
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {

try {

Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage message = new MimeMessage(session, req.getInputStream());

//Extract out the important fields from the Mime Message
String subject = message.getSubject();

_log.info("Got an email. Subject = " + subject);

String contentType = message.getContentType();
_log.info("Email Content Type : " + contentType);

printParts(message);
//Parse out the Multiparts
//Perform business logic based on the email
}
catch (Exception ex) {
_log.log(Level.WARNING, "Failure in receiving email : " + ex.getMessage());
}
}

private static void printParts(Part p) throws IOException, MessagingException {
Object o = p.getContent();

if (o instanceof String) {
System.out.println("This is a String");
System.out.println((String)o);
}
else if (o instanceof Multipart) {
System.out.println("This is a Multipart");
Multipart mp = (Multipart)o;

int count = mp.getCount();
for (int i = 0; i < count; i++) {
printParts(mp.getBodyPart(i));
}
}
else if (o instanceof InputStream) {
System.out.println("This is just an input stream");
InputStream is = (InputStream)o;
int c;
while ((c = is.read()) != -1)
System.out.write(c);
}
}

}

 

Let us discuss the main parts of the code:

1. We have a doPost() method that gets invoked by the Google App Engine when an email is received.

2. In the doPost() method, we build out the email message (a instance of class MimeMessage) using the javax.mail.* classes as shown below:

Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage message = new MimeMessage(session, req.getInputStream());

3. We extract out key attributes from the Message like subject, content type, etc.

 

//Extract out the important fields from the Mime Message
String subject = message.getSubject();
_log.info("Got an email. Subject = " + subject);

String contentType = message.getContentType();
_log.info("Email Content Type : " + contentType);

 

4. We have a utility method printParts(), that helps simply print out the contents of the message. But you could explore the Java Mail API to parse out the multiparts as required and then incorporate your business logic.
5. To help debug our Servlet, we have put in some log statements along with System Out statements, which we shall look for to verify that the Application did receive email.

Finally, we have used the INFO level to log if the message was sent out successfully or not, so we will have the change the logging level by modified the logging.properties file present in the warWEB-INF folder. The necessary line after modification is shown below:

# Set the default logging level for all loggers to INFO
.level = INFO

Deploying our application

To deploy the application, you will need to first create your Application ID. The Application Identifier can be created by logging in at http://appengine.google.com with your Google Account. You will see a list of application identifiers already registered under your account (or none if you are just getting started). To create a new Application, click on the Create Application button and provide the Application Identifier as requested. Please note this name down since you will be using it for deployment.

For e.g. I have registered an application identifier named gaejexperiments.

To deploy the application, follow these steps (they should be familiar to you now):

  1. Click on the Deploy Icon in the Toolbar.
  2. In the Deploy dialog, provide your Email and Password. Do not click on Deploy button yet.
  3. Click on the App Engine Project settings link. This will lead you to a dialog, where you need to enter your Application ID [For e.g. my Application Identifier gaejexperiments]
  4. Click on OK. You will be lead back to the previous screen, where you can click on the Deploy button. This will start deploying your application to the GAEJ cloud. You should see several messages in the Console window as the application is being deployed.
  5. Finally, you should see the message “Deployment completed successfully”.

Testing our Application

To test our application, send an email from any mail client to an address within your application. As explained below, the email address to be used is shown below:

SomeID@YourApplicationId.appspotmail.com

For e.g. my Application Id is gaejexperiments, so I can send email to any of the following:

  • test@gaejexperiments.appspotmail.com
  • user1@gaejexperiments.appspotmail.com
  • and so on…

Once the email has been sent successfully from a Google Mail Account or Yahoo Mail or Outlook/Thunderbird, etc – you can use the App Engine console to verify if your application received the email or not. To do that, perform the following steps:

  1. Go to http://appengine.google.com and log in with your account.
  2. You will see a list of applications registered. Click on the application that you just deployed. In my case, it is gaejexperiments.
  3. When you click on a particular application, you will be taken to the Dashboard for that application, which contains a wealth of information around the requests, quotas, logs, versions, etc. This will be a subject of a future episode but for now, it is sufficient to say that you can come over here to monitor the health of your application and also to analyze what is going on.
  4. Click on the Logs link as shown in the screenshot below:console1
  5. This will display the application log. And all your application log statements that you code using the Logger class can be visible here.
  6. By default, the severity level is set at ERROR and we can change that to DEBUG, the lowest level and you should be able your log statements that had the log level of INFO. This was the log level at which we had logged statements like Received an Email, etc in our Java Servlet, so that is what we need to check for.
  7. If you see the statements, it means that the message has been received. Shown below is a screen shot of the log for a message that I sent.console2

Once this mechanism is in place, it means that your application has been correctly setup and deployed for receiving email messages. You can now build in business logic on what you need to do when an email is received.

Hope you had a good time reading this episode.

Read more Episodes on App Engine Services

 

About these ads

7 thoughts on “Episode 6: Handling incoming Email in your application

  1. Loved the tutorial. It was a very nice, clear, every single step of the way – with code – tutorial. Great work! Thanks!

    Google’s got similar examples posted, but I like the flow and explanations you included. I initially followed google’s examples – with their code – but I couldn’t receive email. Tried it with your code – still no luck. I’ve searched and searched for anyone reporting similar problems and can’t find any. When I look in the logs, it simple notes (only when viewing “All requests”):

    0.1.0.20 – – [28/Jul/2010:09:36:46 -0700] “POST /_ah/mail/test@[appid].appspotmail.com HTTP/1.1″ 200 0 – – “[appid].appspot.com” ms=25 cpu_ms=38 api_cpu_ms=0 cpm_usd=0.001281

    And that’s it – besides, of course, the initial log indicating it had to load my code for the first time. Nothing else! I was originally using the GAE 1.3.1 SDK, then I decided to try and downgrade to 1.2.6 because that’s what you used in this post – no luck. I then bumped it up to the current version, 1.3.5 – still no luck. I’m totally frustrated. I know you’re not responsible for maintaining or trouble shooting GAE problems, but any help would be appreciated. Any ideas?

    1. Hi Kelv,

      Thanks for your comments.

      I would definitely want to help you out to ensure that the code is working. If you are ok, we can work together on this. If it is possible, send me your Eclipse project (remove out parts that I should not worry about). And the additional thing that I would need is to understand what is your AppID and the exact email id that you are sending the email to.

      I can look at your sources to see if there is anything missing. And we can also move to the latest version of the AppEngine SDK … that would be good too.

      Cheers
      Romin

      P.S: My email id R O M I N . K . I R A N I @ G M A I L . C O M

  2. Phew, as you pointed out in an email to me, I had simply forgotten a small but important step – setting the default log level property in logging.properties file to look like:

    1 # Set the default logging level for all loggers to INFO
    2 .level = INFO

    Oops. Thanks for your help!

  3. Tack friend, you help me to understand about receving mail i Google apps.

    I’m have done the code in Python.

    Realy thanks.
    //Percy

  4. This tutorial is so much help ful , but i need a help ?
    Thing is that can i see my inbox for the mails i’ve received.

  5. Hi Romin

    I am developing an application using ‘Receiving Email on App Engine’ feature and I need to perform some database operations after I receive an email which can be either to Datastore or Cloud SQL. Everything works fine when I do it locally but after deploying it does not perform the database operations. I have tried Datastore but entities are not created in datastore after deploying. Even with the CLoud SQL same issue. Can you suggest me some reasons as why this might be happen or if there is any timeout Limit or something or any sort of limitations. Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s