Google Cloud Endpoints Tutorial – Part 1

Welcome to a Tutorial Series on Google Cloud Endpoints.

A Brief History of APIs

Public APIs are all over the place and ProgrammableWeb, the premier directory for public APIs reports that we are well over 10,000 public APIs. This is just the tip of the iceberg. The number of APIs that are private in nature or in other words undocumented and officially unavailable to us is probably 10x or 20x times that (I could be way lower in my estimates!).

Public APIs are now considered to be the most effective mechanism to integrate two applications. Typically the APIs are hosted in a Server side application and exposed to the clients to consume. The clients could be other server side applications, native mobile applications or even browser applications (mobile or desktop).

When we speak about a Public API, few things are common to them:

  • They are available over a popular and well defined protocol, usually HTTP.
  • It uses JSON or XML as the data format.

The above are not absolutely binding but the majority of the Public APIs have that in common.

Additionally, there has been a lot of debate over SOAP vs REST. I cannot get into the details of the two protocols and neither do I want to debate the pros and cons of both the approaches. Suffice to say that the currently predominant style is REST. REST makes use of various HTTP verbs in an effective manner to perform basic operations CRUD Operations or in simpler terms :

  • List (or Get) All Records
  • Get One or more Records based on some search criteria or ID(s)
  • Create a Record
  • Update a Record
  • Delete a Record

One key point about Web Services is that it provides a great layer of separation between the client and the server in terms of the technology used. Client applications are completely agnostic to the programming language/environment that you are using on the Server side. HTTP is the common language that binds the two together and as a result, you could write your Server side API implementation in Java but still have the ability to invoke the same HTTP API via a Java client, iOS client or even a Web client.

Google Cloud Endpoints

The series of tutorials that I wish to cover, starting with this episode is focused around Google Cloud Endpoints, which is a solution from Google to help create a Public API for your App Engine application. Cloud Endpoints also provides facilities to generate client libraries for Android and iOS, thus easing your task to integrate your backend application functionality into your mobile applications on Android and iOS.

The diagram below shows the Cloud Endpoints solution (taken from the official documentation):

cloudendpoints

The point to note is that you need to have an App Engine application. Apart from that, how you store your data or retrieve your data is completely up to you. You could opt to work with the data store to meet your API functionality or simply talk to external services too to aggregate and present the response.

The tutorial series is more about the mechanics of Cloud Points, the toolset that Google provides rather than best practices around writing your API or API backend functionality.

This tutorial series will cover the following topics to give you a complete feel of the total solution:

  • Part 1 : Overview and writing our first public API using Google Cloud Endpoints (We will be writing the API manually)
  • Part 2 : Auto generate the Google Cloud Endpoints classes from your JDO/JPA class in App Engine.
  • Part 3 : Using your Cloud Endpoints API in an Android app
  • Part 4 : Using your Cloud Endpoints API in a Web Application
  • Part 5 : Securing your API
  • Part 6 : Calling a Secured API from a Web Application (JavaScript).
  • Part 7 : Calling a Secured API from an Android application

I might eventually end point combining one or more parts into a single blog post but we will leave that for another day. I might stumble along the way, but together we can and will cross over to the finish line.

What does this part cover?

In this part, we shall look at how we can write our API layer manually with the help of Cloud Endpoint APIs. By API Layer, we will write a REST interface that exposes our sample entity object (Famous Quotes) and provides API methods for Adding, Updating, Deleting and Searching/Retrieving Quotes.

Additionally, we will look at how to test your API locally and finally deploy it.

But why write the API manually? One could argue about why write the API manually since Cloud Endpoints does provide a mechanism to generate the API class from existing JDO/JPA annotated Entity classes. Well, it is important to understand this in detail too since it could help a lot in debugging when you face issues. And rest assured, in the next part, we will look at generating the API layer that we wrote here in an automated fashion and see the best in breed code that Google generates for the API class.

What do you need ?

  • Basic understanding of Java Web Development, which includes Servlets, JSP, WAR file structure, etc.
  • You have a working development environment for Google App Engine. This includes the Google Eclipse plugin.

My Development Environment

  • Eclipse Juno
  • Google Eclipse plugin with App Engine SDK 1.8.7
  • Mac machine (but Windows will do too!)

Attention-> Cloud Endpoints became GA (General Availability) in Release 1.8.7 of the App Engine SDK. The latest release as of App Engine at the time of writing is 1.8.8.

Download Full Source Code

I suggest that you begin with a full download of the project source code.

Go ahead & download the code from : 

https://github.com/rominirani/MyAPIProject

Note: This is an Eclipse project that you can import directly. For the sake of reducing the code size, I have removed the App Engine SDK Jars from the WEB-INF\lib folder. So depending on the version of your App Engine SDK in the Eclipse development environment on your machine, please link to the appropriate App Engine SDK that you have for the project to build successfully.

If you are successful with importing the project into your Eclipse setup, the project directory should look something like this:

cepproject1

Famous Quotes

Our goal is going to be to write a public API that will allow us to manage Famous Quotes. I deliberately chose a Quote to make it easy for all of us to follow. A Quote is a famous/philosophical statement said by an eminent person.

For example:

“Be The Change That You Wish To See In This World.” – M.K.Gandhi

We will keep our Quote object simple. It will contain the following 3 attributes:

  • id: A unique identifier for the quote in our system.
  • author : The person to whom this quote is attributed to.
  • message: The quote text.

Quote Entity

Let us keep aside the API business for the moment. We will first model the Quote as a Java class. The class is present in the com.mindstorm.famousquotes.entity package and it also contain additional methods for equals/hashcode, etc. The source code Quote.java is shown below:

Quote Service

This section is not required as such but I am including it here so that we define both our interface and the implementation details for the different methods to handle Quote. So assume that you were not writing a web service or public API and had to create a Service class or Utility class to manage different operations like add, modify, delete and list for Quotes, then the class could look something like this.

The class is present in com.mindstorm.famousquotes.service package and the file name is QuoteService.java. The code is shown below:

Let us discuss a few things here since they are important:

  • The QuoteService class contains methods for add, update, delete, get, search and so on.
  • The implementation of these methods has been kept simple and you should be able to understand what is going on.
  • Pay special attention to the signature for each of the methods because that has a role to play later on in the article because Google Cloud Endpoints does look for specific signatures to help map the different operations to HTTP methods like GET, PUT, POST and DELETE.
  • The add method typically contains the different attribute fields as the parameter and returns back the Quote object that was successfully added to the list.
  • The update method has the Quote object that needs to be updated, as the method parameter. This means that the id, author and message attribute needs to be populated correctly in the Quote object that is passed as the parameter. This method too returns the Quote object that was update.
  • The remove method takes an Id as the parameter for the Quote that needs to be removed and has no return type.
  • The get* methods are used to retrieve one or more Quotes that match as criteria as needed. Notice that the return type is a List of Quotes.
  • Also note that we are throwing back the normal java.lang.Exception object, in case of any issues.

Hope this makes things clear on how we would have written a standard Java class to help work with the Quote entity and manage the Quote collection.

Converting it to a Google Cloud Endpoint class

Now, let us think for a moment for what we would need to do in order to expose the above service as a REST API.

At a high level, we will need to do the following:

  • Identify how we will map the different methods to the equivalent HTTP Verbs. In order words, as per best practices in REST APIs, we will need to expose the get* methods over GET, the update method over PUT, the add method over POST and the delete method over DELETE verbs. Sounds good ?
  • Additionally, we also need to make sure that we are throwing back the correct HTTP Response Codes. You would have heard about HTTP 200 for OK, HTTP 404 for Not Found, etc. So for e.g. if we try to update a Quote for which the Quote ID is not found, we need to make sure that we are passing back the correct HTTP Error code say 404 NOT FOUND. You will find public APIs that don’t follow this strictly but it is good practice.
  • Once we have all this in place, we need to figure out how to test this API.

Given the above points, let us see first how to convert our existing QuoteService.java file to a Cloud Endpoints class that will help expose the REST API. You will find that it is all about using the correct annotations, Exception classes from the Cloud Endpoints libraries and some magic that Endpoints does behind the scene based on your method signatures.

Let us look at the converted class with the API Annotations. The Java class is present in com.mindstorm.famousquotes.service package and the file is QuoteServiceAPI.javaThe source code is shown below:

Let us go through the important points:

  • We have replaced the java.lang.Exception class with the NotFoundException class that is part of the Cloud Endpoints classes. The NotFoundException class will throw the correct 404 response code and details, as we shall see. Look at the entire set of Exception classes that Cloud Endpoints exposes : Exceptions and HTTP.
  • To identify this class as the API class, we have used the @Api annotation right on the top of the class as given below:@Api(name=”quoteapi”,
    version=”v1″,
    description=”An API to manage famous quotes”)

    We provide a name to our API, a version and also a description. If you do not provide these, default values will be taken but we will specify them here. Actually this is all that we need to do since Cloud Endpoints can then take all the public methods of the class, weave its magic based on the method signatures and expose the methods over HTTP verbs respectively. But we will be more specific as explained below.
  • Each method, we are annotating with the @APIMethod in which we are providing a name to the API method. Each method parameter that we want to the API Method to consider, we are used the @Named annotation where we specify the parameter name that will be actually used by the client while passing the value to it. We are keeping things simple for now, but if you are interested in the entire range of annotations, take a look at Annotating your API.
  • Notice that we did not specify which HTTP method (GET, POST, PUT, DELETE) is mapped to which Java method. It turns out that Cloud Endpoints does some clever behind the scenes work where it will use the correct HTTP methods. It seems to do that based on what key words a method starts with, the method signature, etc. From my findings, it is doing so on method signature but conceptually you should follow the general rules of thumb given below (taken from their documentation). You should ideally follow this best practice for beginning your method names with the key words listed below, it will makes things easier to understand for everyone. Screen Shot 2014-01-10 at 7.37.51 PMDo note that if we want we could still specify the HTTP Method to use via the @APIMethod annotation. The attribute httpMethod can be used to specify GET , POST, etc.

We are done with our API ! Simple … wasn’t it ? Yes, but how do we test out the API and see the raw HTTP Request / Responses that move back and forth.

Testing your API Locally

Now, comes the fun part and indicates the amount of work the Google Developers have put in to make it easy for you.

Follow these steps:

  • Make sure your Project is ready, compiles fine without errors. Run your Application.
  • Assuming that your Web Application starts on port 8888, Visit the following URL: http://localhost:8888/_ah/api/explorer

Make sure you are connected to the Internet. If all is well, you will see an API Explorer as shown below:

api1

How did something get exposed over the _ah/api endpoint. Well, visit your web.xml and you will find an entry created out there for you, which behind the scenes will invoke the API logic for you.

You will notice that the explorer has used our @API (name, version and description) as we provided.

Cool, now lets click on the quoteapi link, this will bring up a list of all our API methods. Notice that the method names were taken as the ones that we provided in the @APIMethod annotation.

api2

Now, lets add a Quote. We will click on quoteapi.add link. This will bring up a form where we can provide the id, author and message for the Quote as shown below:

api3

Clicking on the Execute button will actually make the call and what is interesting to note is that behind the scenes a HTTP POST call is made. Take a look at the HTTP Response too, where the return object type i.e. Quote is marshalled correctly into a JSON object.

api4

Go back to the list and click on the quoteapi.list method. This will invoke a GET as shown below and return the collection as a JSON response.

api5

Now, let us look at quoteapi.getQuote method where we need to provide the id field to retrieve the record. The API explorer brings up the form as shown below. We provide the id as 1 (for our only quote so far) and it fires a GET and retrieves the data.

api6

Pay attention to the entire GET format too to better understand the format. You will notice that the format is _ah/api/<APIName>/<APIVersion>/<Object>/<ID>

api7

Now, look at what happens if we provide an id that does not exist. 

api8

Since a Quote record with id = 2 does not exist, it throws a NotFoundException which behind the scenes gets converted into the correct HTTP Response 404.

If we invoke the quoteapi.update method, it will open up a form where we can enter construct the Quote object that we want to modify. We will provide the same id but different values for the message as shown below:

api13On executing it, we will see that it fired a HTTP PUT command now:

api14

If we retrieve the list of quotes, we will find the updated data:

api15

Finally, let us look at quoteapi.remove method shown below (In my sample data I added a second record too so I am deleting here with id = 2). Notice that it fired a DELETE HTTP command and since we did not return any type, it sent back a 204.

api12Similarly, if we try to delete an non-existing record i.e. id = 3, we will get the correct 404 responses as shown below:

api11

Next Steps

This brings us to the end of this episode. In this episode, we saw how simple it was to use the Google Cloud Endpoints library to expose a public API. You can go ahead and deploy the same to a live instance too, things should work seamlessly.

In the next part of this tutorial, we are going to take a look at how we could take an existing JDO/JPA annotated Entity object and ask Google Cloud Endpoints to generate the API for us, instead of us having to write it by hand as we saw in this tutorial. But our basics here will help us understand better the code that it generates.

Hope you liked the episode. Till the next one, Happy “API”ing … !

About these ads

38 thoughts on “Google Cloud Endpoints Tutorial – Part 1

  1. Romin, Thank you for practical approach to helping others learn AppEngine. Two things I really appreciate are: 1. Not getting into SOAP vs REST and other personal preference for programming discussions , 2. Willingness to make note of simple, but often frustrating first try mistakes (like remembering to add the sdk jars). I hope you continue to post your excellent style.
    -Will
    wsutton17@gmail.com

  2. Hi Sir. I’m a little confused as to how to actually start this project. Do I go to Eclipse, select New Project–>Google–>Web Application Project? I’m not quite sure how to get your directory structure in Eclipse. Can you please provide a step by step for using the wizards to at least get the starter files? Thanks again!

    1. Yes. You are right. You can create the new project with Eclipse = New Project -> Google -> Web Application Project. Please ensure that you deselect the Google Web Toolkit option.

  3. Come here to say thank you. You did exactly what i missed on RequestFactory framework! Google show only the automatic way.. and when I’ve got some bugs, it was a hell in earth! So i have to learn how it works manually and study it’s source code.

    Your totorial is an example to follow! You showed how to change from “actual code” to api. I’ve already have a complex project at AppEngine, i’ll do the same thing you did here… i really don’t need the automatic way… maybe in future projects…

    You deserve the GDE title!

    I’m going to part II….

    1. Thank you for your kind comments. It motivates me to keep writing more such articles. If you have any specific feedback on what I could have done better, do let me know. Cheers.

      1. Sure, i will let you know. First, i’ll build an API from my Project and if i have any issue that wasn’t covered here.. i’ll report… Cheers o/!

  4. thank you for the lessons
    but when I run the project I got this problem:
    Problem accessing /_ah/api/explorer. Reason:

    SERVICE_UNAVAILABLE

    1. A couple of things:
      1) Can you tell me the full url that you are trying to access i.e http://…../_ah/api/explorer.
      2) Did the Web application start up normally or was there some error. At times it is due to some web.xml issue due to which the application is not available.

      1. Thank you for your reply
        The problem has been solved
        It was probably because of blocking the service for some countries , I used a proxy software and it was all ok.

  5. Thanks for such a clear and helpful Post. I was looking for a tutorial to learn more about Android App and App Engine communication.
    I have started with Part 1 of your blog, but I am stuck because my app engine is not working.
    It shows Warning: mysqli_connect(): (HY000/2002): No connection could be made because the target machine actively refused it.
    Fatal error: Call to a member function query() on a non-object in

    Also, for Google app engine integration what language are you using? I would really appreciate you reply.
    Thanks,

    1. Hi Nidhi,

      1. Your problem seems to be around some connectivity issues to CloudSQL. I am unable to help you debug that but best would be to check out the connection parameters, etc. I also suggest that if you are in the process of just learning Cloud Endpoints through my tutorials, then you do not need CloudSQL. Just follow the tutorial, which uses the Cloud Datastore (NOSQL option)

      2. My tutorials use the Java language for both Cloud Endpoints and App Engine development/deployment. Additionally, the front-end integration with Cloud Endpoints has been demonstrated with Android (Java) and Web (JavaScript).

  6. Hi Romini
    Thanks for your reply. Which one of your tutorial uses Cloud Datastore(NOSQL option)?

    I guess the Google Cloud Endpoints Tutorial – Part 3 uses tutorial Part 2 for the API’s. I am looking for a complete tutorial(backend as well as front end.)

    Thanks,

  7. Hi Romini,

    Thanks for the very useful tutorial. It has been really helpful so far. When I tried to deploy it to App Engine, it went to the “Hello Cloud Endpoints API Tutorial” page, but if I try to go to http://localhost:8888/_ah/api/explorer, I get a http error: 503. In my eclipse project, there are errors in the war and web-inf directories, but I can’t find the errors in any files. It says it deploys successfully, but for some reason, the server is not working properly. Do you have any idea about what might be going on? Thanks! My error log in eclipse reads as follows:

    Info
    Fri May 16 16:53:29 CDT 2014
    Analytics ping response: 200: OK
    GIF89a

    Info
    Fri May 16 16:53:28 CDT 2014
    Analytics ping request parameters: v=1&t=event&el=id%3D1396146332483%26ev%3D3.8.101.v20130717-0806%26v%3Dcom.google.gdt.eclipse.suite.e43.feature_3.5.1.v201312301723-rel-r43%26gwtv%3D2.6.0%26gaev%3D1.9.1%26p%3Dorg.eclipse.epp.package.standard.product%26action%3Dgae_backend_deploy%26appIdHash%3D6787A973A699E7D582CBB10870B64C85352DDCBF%26isGoogleCloudSqlUsed%3Dfalse%26isCloudEndpointProject%3Dtrue%26isSignedIn%3Dtrue&cd1=org.eclipse.epp.package.standard.product&cd2=6787A973A699E7D582CBB10870B64C85352DDCBF&an=Google%20Plugin%20for%20Eclipse&tid=UA-43239122-1&ea=DEPLOY_GAE&cid=1396146332483&ec=com.google.gdt.eclipse.core

    Info
    Fri May 16 16:53:28 CDT 2014
    MyAPIProject successfully deployed to Google App Engine

    1. Without having a view into your errors in wat and web-inf directories, it is difficult for me to pin point where the problem could be. From the log that you have posted, there is nothing too visible in terms of what the problem could be.

  8. Hi Romini,
    Your tutorial helped me understand the concept of connecting GAE and android client. My friend created the back-end and I’m working on the front end. How to pass parameters form android client to python back-end. I’m not sure how to post — email and password ? I would appreciate your help.Thanks,

    Python
    #Validate user
    LOGIN_REQUEST_RESOURCE = endpoints.ResourceContainer(email=messages.StringField(1),password=messages.StringField(2))
    @endpoints.method(LOGIN_REQUEST_RESOURCE, BoolResponse, path=’login’, http_method=’POST’, name=’account.login’)
    def LoginUser(self, request):
    success = False
    qry = Account.query().filter(Account.email==request.email).fetch(1)
    if len(qry) > 0:
    success = qry[0].pwd == (request.password)
    return BoolResponse(value=success)

    Android Client
    public LoginAsyncTask(Context context) {
    this.context = context;
    }
    protected Account doInBackground(String…string){
    Account response = null;
    Toto.Builder builder = new Toto.Builder(AndroidHttp.newCompatibleTransport(), new GsonFactory(),null);
    Totoservice = builder.build();
    Account account = new Account();
    account.setEmail(string[0]);
    account.setPw(string[1]);
    ## response = service.account().login(account.put());
    return response;
    }

    1. You will need to make sure that these parameters are defined in your APIMethod on the Server. Additionally, just a suggestion, I would not be passing passwords like that.

  9. Every time I click on login button I get this error. Not sure what is happening.
    06-09 14:32:49.776 9881-9922/com.xx.xx E/LoginActivity﹕ Exception during API call
    java.io.IOException: Hostname ‘turtinapi.appspot.com’ was not verified
    at libcore.net.http.HttpConnection.verifySecureSocketHostname(HttpConnection.java:223)
    at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:446)
    at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:290)
    at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:240)
    at libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:81)
    at libcore.net.http.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:205)
    at libcore.net.http.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:281)
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:965)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
    at com.turtalabs.turtin.LoginActivity$LoginAsyncTask.doInBackground(LoginActivity.java:119)
    at com.turtalabs.turtin.LoginActivity$LoginAsyncTask.doInBackground(LoginActivity.java:102)
    at android.os.AsyncTask$2.call(AsyncTask.java:287)
    at java.util.concurrent.FutureTask.run(FutureTask.java:234)
    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
    at java.lang.Thread.run(Thread.java:838)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s