Episode 2 : GAEJ + XMPP and rolling your own Agent

I hope all of you have got started with development of GAEJ applications using the nifty Google Eclipse plugin. If not, I strongly recommend that you read it here. In this episode I will be discussing one significant enhancement that Google has brought about to their latest GAEJ release and that is XMPP support.

XMPP Protocol has for long been the backbone for a lot of systems till now and most likely in the future. It stands for Extensible Messaging and Presence Protocol. Whew! That was quite a complicated expansion if you ask me. But fear not. It emerged from the Jabber protocol and is in use today in a lot of Instant Messaging Systems. Google Talk is a prime example of an application that utilizes XMPP as its backbone. Several other IM clients utilize XMPP too. Several XMPP APIs in a variety of languages are available too for you to hack your XMPP client any time.

Agent in Action

What are we going to build today, you ask? We are going to a build a XMPP Agent that we can add in IM clients like Google and talk to it. We are going to cover here all that is needed for you as a Java developer to use GAEJ and get a basic XMPP Agent application working.

To see it in action, I suggest you do the following:

  1. Start Google Talk on your machine.
  2. Add gaejxmpptutorial@appspot.com as a friend. If it asks you to add itself as a friend, please do so.
  3. Send it any message you like. For e.g. Hello.
  4. It will reply back with whatever you typed.

Here is a running sample screenshot of what we will be achieving today:

post2-5

We will cover how to write the same XMPP Agent (gaejxmpptutorial) using GAEJ. Once you get the mechanics in place, the world is your oyster when it comes to writing Agents that do what you want it to do. All you need to then write is your Agent logic and GAEJ platform does the rest for you.

First things first

Here are a few things that I shall assume:

  1. You have an appropriate Eclipse setup along with the GAEJ for Eclipse plugin configured correctly
  2. You have a Google Account and know what it means to register an application under your account

If you are unclear, I suggest to read up my first episode.

Our “Hello World” XMPP Agent

At a high level, we will be creating a XMPP Agent that will do the following:

  • The Agent will be hosted in the GAEJ system. The application name will be any application id that you select for your account (More on that while deploying).
  • This Agent can be added as your friend/buddy in Google Talk or any XMPP IM client
  • You can type out a message to the Agent and it will simply send back the message that you typed.

Alright, let’s go.

Create a New Project

The first thing to do is to create a New Google Web Application Project. Follow these steps:

  1. Either click on File –> New –> Other or press Ctrl-N to create a new project. Select Google and then Web Application project. Alternately you could also click on the New Web Application Project Toolbar icon as part of the Google Eclipse plugin.
  2. In the New Web Application Project dialog, deselect the Use Google Web Toolkit and give a name to your project. I have named mine GAEJXMPPTutorial. I suggest you go with the same name so that things are consistent with the rest of the article, but I leave that to you.
  3. Click on Finish

This will generate the Google Web Application Project, which consists of a sample Servlet that prints out “Hello, World”.

Understand the GAEJ XMPP API

Before we write our Agent, we need to understand first the Google XMPP API. I will cover it in brief over here and the rest of the details you can understand from the link provided, once you need to dig deeper. Make no mistake, the API is detailed and you will need to refer to the finer points as the scope of your XMPP interaction increases but for our starter version, we do not need to know too much.

Think for a moment that you are the Agent that people are going to add to your Google Talk IM client. What do you need to do at a basic level. It all boils down to this:

  1. Receiving a message
  2. Interpreting it and composing an appropriate response message (This is where your Application logic will come in).
  3. Sending a message.

Let us look at each of the 3 areas now and what the API looks like. Please note that all XMPP API classes are present in com.google.appengine.api.xmpp.* packages.

Receiving a message

Since XMPP support is now part of the GAEJ infrastructure, all GAEJ applications are capable of receiving an incoming XMPP Message. So once your application is hosted in the GAEJ cloud, the application can receive messages from other XMPP clients like Google Talk. To enable this receiving of XMPP messages from a code level, you need to do two things:

- Enable your application to receive XMPP messages by configuring the XMPP Service in your application. This is straightforward and all you need to do is add the following element to the appengine-web.xml file. The appengine-web.xml file as you know is specific to the Google Java Web Application project and is used for configuring certain services, XMPP being one of them. It is found in the warWEB-INF folder of your Web Application Project. The XML fragment to add at the end but before the </appengine-web-app> element.

<inbound-services>
<service>xmpp_message</service>
</inbound-services>

We will cover this in detail again later, but this is sufficient for the moment.

- Configure and code a Java Servlet that will receive the incoming Message. All XMPP messages to your application are delivered via POST to following URL path in your application: /_ah/xmpp/message/chat/. So you will need to configure the servlet like the following snippet in the web.xml file, present in the warWEB-INF folder of your Web Application Project.

 

<servlet>
<servlet-name>xmppreceiver</servlet-name>
<servlet-class>com.gaejexperiments.xmpptutorial.XMPPAgentServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>xmppreceiver</servlet-name>
<url-pattern>/_ah/xmpp/message/chat/</url-pattern>
</servlet-mapping>

 

In the above snippet, you will find the fixed URL path /_ah/xmpp/message/chat/ configured as the <url-pattern/>. And then I have a Java Servlet class com.gaejexperiments.xmpptutorial.XMPPAgentServlet as the <servlet-class>.

Now, all we have to do is write our Servlet. As mentioned, the incoming XMPP Message will be POSTed to our Servlet, so we need a simple doPost(…) implemented in our Servlet. The skeleton is shown below:

 

package com.gaejexperiments.xmpptutorial;
import com.google.appengine.api.xmpp.*;
//Other imports
public class XMPPAgentServlet extends HttpServlet {

public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException
{
//get the incoming message from the Request object i.e. req
// interpret it and compose a response
//send the response message back
}
}

 

Interpreting the incoming message and composing a response

To interpret the incoming message, we need to extract out the message that has been passed to us in the HTTP Request. It is done in the following manner:

XMPPService xmpp = XMPPServiceFactory.getXMPPService();
Message msg = xmpp.parseMessage(req);

The msg object will contain the XMPP Message. There are several elements in the XMPP message that are of interest to us and chief among them being who has sent the message and what is the text of the message that has been sent to us. This is extracted out by the getter methods available on the Message class as shown below:

JID fromJid = msg.getFromJid();
String body = msg.getBody();

The JID class represents the Jabber ID of the sender whereas the body will contain text of the message sent to us.

Once you have the message, you can interpret it and then create a XMPP message that you need to send back i.e. respond with. To compose an XMPP Message, here is the API:

Message replyMessage = new MessageBuilder()
.withRecipientJids("JABBER_ID_OF_RECIPIENT")
.withBody("TEXT_TO_SEND_TO_RECIPIENT")
.build();

The above method is straightforward and we will see it more in the next section.

Sending a message

Finally, we have our message and all we need to do is send it. To do that, here is the API. We first check if the recipient is available i.e. Online and then send the message using the sendMessage method.

XMPPService xmpp = XMPPServiceFactory.getXMPPService();
//Compose the Message Object i.e. replyMessage
if (xmpp.getPresence("JABBER_ID_OF_RECIPIENT").isAvailable()) {
SendResponse status = xmpp.sendMessage(replyMessage);
//Take appropriate action based on status SUCCESS or FAIL i.e. log an error, update database, etc
}

Write our Servlet and configure it

Create a new Java Servlet class named XMPPAgentServlet in the package com.gaejexperiments.xmpptutorial. You can use your own namespace and Java class name if you wish. The entire source code for the Agent is shown below:

 

package com.gaejexperiments.xmpptutorial;

import java.io.IOException;
import javax.servlet.http.HttpServlet;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.google.appengine.api.xmpp.*;
import java.util.logging.Level;
import java.util.logging.Logger;

@SuppressWarnings("serial")

//STEP 1
public class XMPPAgentServlet extends HttpServlet {
public static final Logger _log = Logger.getLogger(XMPPAgentServlet.class.getName());
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException
{
try {
String strStatus = "";
XMPPService xmpp = XMPPServiceFactory.getXMPPService();
//STEP 2
Message msg = xmpp.parseMessage(req);
JID fromJid = msg.getFromJid();
String body = msg.getBody();
_log.info("Received a message from " + fromJid + " and body = " + body);
//STEP 3
String msgBody = "You sent me : " + body;
Message replyMessage = new MessageBuilder()
.withRecipientJids(fromJid)
.withBody(msgBody)
.build();
//STEP 4
boolean messageSent = false;
if (xmpp.getPresence(fromJid).isAvailable()) {
SendResponse status = xmpp.sendMessage(replyMessage);
messageSent = (status.getStatusMap().get(fromJid) == SendResponse.Status.SUCCESS);
}
//STEP 5
if (messageSent) {
strStatus = "Message has been sent successfully";
}
else {
strStatus = "Message could not be sent";
}
_log.info(strStatus);
} catch (Exception e) {
_log.log(Level.SEVERE,e.getMessage());
}
}
}

 

The code should be familiar now but let us go through each of the steps above:

  1. We have written a Java Servlet that extends the HttpServlet class and implements the doPost(…) method since the XMPP message will be delivered via POST to the application.
  2. In this step, we extract out the XMPP Message object from the HTTP request. Then we invoke a couple of getter methods on the Message object to get the Jabber Id from where we got the message  and also the body of the message. We log this into the log file.
  3. We compose a string named msgBody that simply echoes back what was sent by the sender and then we compose a XMPP Message object named replyMessage to be sent to the Jabber Id that sent us this message i.e. fromJid along with the text that we want to send in the message i.e. msgBody
  4. We detect if the recipient that we are sending the response to i.e. fromJid is available i.e. online. If yes, we send the message using the sendMessage method. Then we determine if the message was sent successfully or not i.e. messageSent.
  5. Finally, depending on the message status, we log the appropriate message in the log file.

To complete our Servlet development, we will also need to add the <servlet/> and <servlet-mapping/> entry to the web.xml file. The necessary elements to be added to your web.xml file are shown below. Please note that you can use your own namespace and servlet class. Just modify it accordingly if you do so. But make sure that the fixed path URL i.e. /_ah/xmpp/message/chat/ is correctly mentioned in the <url-pattern/>, since that is the path over which the Google XMPP Infrastructure will deliver the message to you via a POST.

 

<servlet>
<servlet-name>xmppreceiver</servlet-name>
<servlet-class>com.gaejexperiments.xmpptutorial.XMPPAgentServlet</servlet-class>
</servlet><servlet-mapping>

<servlet-name>xmppreceiver</servlet-name>
<url-pattern>/_ah/xmpp/message/chat/</url-pattern>
</servlet-mapping>

 

Finally, we have used the INFO level to log if the message was sent out successfully or not, so we will have the change the logging level by modified the logging.properties file present in the warWEB-INF folder. The necessary line after modification is shown below:

# Set the default logging level for all loggers to INFO
.level = INFO

Configure the XMPP Service for our Application

To enable your application to receive XMPP messages, you need to configure the XMPP service in your application. This is done by adding the XMPP service to the appconfig-web.xml file that is found in the warWEB-INF folder of the Web Application project. Shown below is appconfig-web.xml and the relevant portion that you need to add to the file.

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
...
<inbound-services>
<service>xmpp_message</service>
</inbound-services>
...
</appengine-web-app>

Deploy the Application

To deploy the application, you will need to first create your Application ID. The Application Identifier can be created by logging in at http://appengine.google.com with your Google Account. You will see a list of application identifiers already registered under your account (or none if you are just getting started). To create a new Application, click on the Create Application button and provide the Application Identifier as requested. Please note this name down since you will be using it for deployment.

For e.g. I have registered an application identifier named gaejxmpptutorial.

To deploy the application, click on the Deploy Icon in the Toolbar. This will present the dialog as shown below:

post2-8

You will need to provide your Email and Password. Do not click on Deploy button yet. Click on the App Engine Project settings link. This will lead you to the screen as shown below, where you will need to enter your Application ID [For e.g. shown below is my Application Identifier gaejxmpptutorial]

post2-7

Click on OK. You will be lead back to the previous screen, where you can click on the Deploy button. This will start deploying your application to the GAEJ cloud. You should see several messages in the Console window as shown below as the application is being deployed:

post2-6

You should see the message “Deployment completed successfully” as shown above. This means that you application is ready to serve.

See it in Action

Now that your Agent application has been deployed, it is time to communicate with it. To do that you will need to add your Agent as a friend/contact in your IM application. To do that you will need to know the ID of the Agent. This is made simple for you by GAEJ. If your Application ID is for e.g. myagent and your application is XMPP enabled, then the ID for your application will be myagent@appspot.com. In other words the ID is APPLICATION_ID@appspot.com where APPLICATION_ID is the application identifier for your application that you registered.

In my case, it was gaejxmpptutorial, so all I did was start up Google Talk and clicked on Add Contact to add the Agent as shown below.

post2-2

Once you click on Next and finish the invitation, you will receive a message in your Google Talk as shown below, where the Agent is requested to add you as friend. It is important that you accept that , so that it can send messages to you.

post2-3

Once you do that, you will find that the Agent is available as shown below. The Agent is available because your Application is online and running in the cloud. Cool, isn’t it?

post2-4

Now you can start the conversation and see your Agent in action as shown below:

post2-5

Moving forward

You should now be in a position to think, code and deploy XMPP Agents of your choice using GAEJ. I would love to see you write a few and sharing it out here with all of us. You could write Agents that could do the following:

  • Play a word game
  • Provide weather, sports, traffic,etc  updates
  • Examination Results
  • Ticket Confirmation
  • Train, Air Tickets routes… almost anything that you can think of.

If you would like to get an idea of a sample XMPP Agent written using GAEJ, look no further than the Mumbai Blood Bank Locator Agent that I wrote a while back. Take a look and inspire all of us with your next cool Agent 007. Till the next episode …

Read more Episodes on App Engine Services

 

About rominirani

Google Developer Expert Cloud 2014. Harnessing the power of software by learning, teaching and developing simple solutions. I love learning about new technologies and teaching it to others.
This entry was posted in Cloud Computing, Google App Engine. Bookmark the permalink.

38 Responses to Episode 2 : GAEJ + XMPP and rolling your own Agent

  1. Pingback: Twitter Trackbacks for GAEJ + XMPP and rolling your own Agent « Google App Engine Java Experiments [gaejexperiments.wordpress.com] on Topsy.com

  2. Prashant Thakkar says:

    This is really an well written article to get started. With the help of this article I have already developed todays-special@appspot.com agent.

  3. beeman says:

    Cool tutorial, got my Bot running… Now only extend it… :)

  4. James says:

    Thanks for this wonderful article. Will try that out!

  5. rominirani says:

    Everyone — Thanks for the positive comments so far. I would like to see what agents you folks have written. I plan to cover each of them, highlight what it does and give credit to the author. Do send me an email or paste it here for all to use and try.

    Please stay tuned for more episodes of GAEJ, which I plan to convert into a free online book for all.

    Cheers… Romin.

  6. SVGrulez says:

    Hi Romin, I just finished my first agent, but would like to know if it is possible to communicate with the agent using the browser instead of an IM client like GTalk, perhaps using flash or java applets.

    Anyone had any luck?

  7. asianCoolz says:

    hi, may i know can the gae-xmpp use to connect to external jabber server. if yes how? from my understanding it can only use to connect to gtalk?

    • rominirani says:

      Hi,

      Do you mean if you can use any other IM client like Spark, Pidgin and use an account from Jabber.org? And then connect to your Agent hosted in Google App engine?

      If yes, that is possible but a small change needs to be made in the code. I will be detailing that in another post shortly. But in short, Google App engine XMPP Support can only detect the presence of another user on its own network of Google Server. It cannot detect the presense of another ID on another XMPP Server Network like Jabber. So all we have to do in the code is to comment out the check for checking the following:
      if (xmpp.getPresence(fromJid).isAvailable()) {
      ….
      }
      Just remove that check and it should work fine. More details in a Blog Post coming shortly.

      Thanks
      Romin

    • rominirani says:

      I posted a new blog entry that details how to communicate to your XMPP Bot running inside Google App Engine, from another Jabber Network. Please read it here : http://gaejexperiments.wordpress.com/2009/10/09/episode-2-update-communicating-to-another-xmpp-account-via-your-bot/

  8. Pingback: Episode 2 : Update : Communicating to another XMPP Account via your Bot « Google App Engine Java Experiments

  9. asianCoolz says:

    rominiran, great article. do you know whether it is possible to create “component” bot like subdomain.appspot.com rather than agent bot using GAE?

    reference:http://metajack.wordpress.com/2008/08/04/thoughts-on-scalable-xmpp-bots/

  10. rominirani says:

    I am not sure if I understand your question. Thanks for the reference link too but I guess that is some advanced stuff and not within my realm :-).

    Additionally, as far as GAEJ is concerned, a subdomain.appspot.com means that subdomain is one application id. So I guess conceptually you can create more applications but within the context of GAEJ, it is another web application!

  11. asianCoolz says:

    do you think it is possible to use GAE to create pubsub bot? can elaborate … ?

    • rominirani says:

      I do not know too much about the pubsub hubbub protocol… though I have read a little about it. A lot of details are here : http://code.google.com/p/pubsubhubbub/

      I think it might be perfectly possible to write a bot. This is because of the following points:
      1. The pubsub protocol works on the premise that you should get notified when the particular information that you are interested in is updated.
      2. Now when it is updated, one of the ways that you could get notified is by providing your URL that you should invoked when an update is available.
      3. We could technically provide our Bot endpoint in GAEJ that could be invoked. Which in turn would then notify other Jabber accounts?

      While I am not that aware of the finer details of the pubsubhubbub protocol, I think it should work in a generalized way that I have given in the above points and given that, it should be straightforward to get the XMPP endpoint invoked in our GAEJ Application.

  12. elwiz says:

    Hi, and thanks for some great articles.
    However, my bot won’t work (nasty one) and I’m thinking of one could debug this one?

    regards

    • rominirani says:

      Thanks for the positive comments on the articles so far. Debugging the Bot might be difficult since Google App Engine at runtime does all the XMPP protocol heavy lifting for you and simply invokes your endpoint at /_ah/xmpp/message/chat/.

      Do debug, I suggest the following:
      1. Use the java.util.logging.Logger class to insert some log statements in your code. I would go with the INFO level and make sure that your logging.properties file is set to that level.

      2. Use the AppEngine Console to view the log statements.

      3. Make sure that your XMPP bot is getting invoked by putting in some log statements in the doPost() handlers. Be aware that a POST is done by the GAEJ Engine. And not a GET.

      Hopefully this should get you somewhere.

  13. Pingback: Episode 7: Writing your First Google Wave Robot « Google App Engine Java Experiments

  14. Chinmay says:

    Awesome sir,
    it took only 5 mins to make the first bot :)

  15. Chris says:

    Hi!

    Using your code brings on Google App Engine/1.2.6.
    “XMPPAgentServlet doPost: Must set a body”

    Do you know what could have happened?

  16. Chris says:

    Hi!

    I’m using either Pidgin on Ubuntu 9.10 (might be v2.5.5) or using Google Talk via plain WebIf (no flash).
    But in neither way a message gets delivered back.
    Do I understand right that the message to the servlet is delivered correctly?
    – wait!
    I did the check in Google Talk again and now it works!

    I’ll check on the afternoon again if the errors persists on my home machine using pidgin, but now it’s working via Google Talk.

    Anyway, thank you for your reply!

    Did you notice that you could use like in the mail API ‘anything@app-id.appspotchat.com’ as adress for incoming messages? It’s like a catch-all configuration

    More is described at
    http://code.google.com/intl/de-DE/appengine/docs/java/xmpp/overview.html#XMPP_Addresses

    • rominirani says:

      Chris,

      Interesting! I think that is what the issue also says that it seems to be working fine with Google Talk. And it looks like certain XMPP IM clients are not packaging the message correctly at times. And I think it sort of goes well with what you are experiencing. But is definitely a cause of concern at times since the author of the Agent has no control over what XMPP client the end user might be using.

      Thanks for the other observation. I had read the docs but had skimmed over this one quickly. Thanks for pointing it out.

      Cheers
      Romin

  17. Amith GC says:

    is there a way to put an Avatar to the bot i create?

  18. asianCoolz says:

    hi, anyone tried connect to jabber.org and try communicate with appspot bot? the bot exist on my jabber roaster but i cannot talk to it. is it because google block it?

  19. Kartik says:

    hi I tried. But for somereason mine application is not working. It says no error in deploying. But I dont get any reply from kartikpanjabi.appspot.com

    • rominirani says:

      Hi Kartik,

      I cannot help based on this information since it is not clear to me what your code is and why it may be failing. Try putting in a logger for your agent. Track via logging if a request was received or not. If request was received, then look at your Agent logic for something going wrong. If request itself was not received, then look at your web.xml settings and servlet settings, something could be wrong there.

      Hope this helps.

      Thanks
      Romin

  20. hi rominirani
    first i have to say your work here is super cool :) it helped a lot i made a dictionary using google chat with the help of your tutorial and some of my past work
    “chatdictionary@appspot.com” is the friend you need to add
    again thanks a lot for this

    • rominirani says:

      Thanks for the feedback. I am glad to learn that the posts have helped you. Interesting Bot you have there — should be useful to lot of us.

      Cheers
      Romin

  21. Pingback: problem with XMPP service of Google App - Java Forums

  22. John Fairfield says:

    Romin: Many thanks for your example, it’s been a huge help. But I can’t get group chat to work. I don’t see a fix, given that the logs show that the doPost never gets called, as soon as I add anybody else to the chat. What should I be looking at?

  23. Thanks a lot for this tutorial :)

  24. Juhani says:

    Thank you for sharing this tutorial!

  25. dennis says:

    Very helpful post!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s