Archive for November, 2009

Comparison of JAX-RS Client Proxies

November 18, 2009

Though JAX-RS has done wonders for standardizing server-side implementation of Java REST services, there is definitely a need for some client-side standardization. Considering that for every REST implementation there are order-of-magnitude more clients, it is rather puzzling that more movement hasn’t occurred in this space.

Section 1.3 Non Goals of the JAX-RS 1.0 (Sep. 2008) spec states:

Client APIs The specification will not define client-side APIs. Other specifications are expected to provide such functionality.

Nevertheless, vendors have not been idle and have used their particular client-side frameworks as a value-add selling point. In this blog, I report some first impressions on CXF and RESTEasy‘s client APIs. Next report will be on the Jersey and Restlet versions.

Perhaps the reluctance to tackle a standard client API has something to do with the complexity associated with SOAP client proxies, but in the absence of REST client proxies, every single customer has to recreate the wheel and implement rather mundane low-level plumbing with httpclient. A chore best avoided.

Both CXF and RESTEasy support a nearly equivalent client proxy API that mirrors the server-side annotated resource implementation. They differ in two ways:

  • Bootstrap proxy creation
  • Exceptions thrown for errors

“Naturally”, each provider has a different way to create a proxy. Both are rather simple, and since they are a one-time action, their impact on the rest of the client code is minimal.

The “happy path” behavior for both implementations is the same – differences arise when exceptions are encountered. RESTEasy uses its own proprietary org.jboss.resteasy.client.ClientResponseFailure exception while CXF manages to use the standard JAX-RS exception javax.ws.rs.WebApplicationException. Therefore, round one goes to CXF since we can write all our client tests using standard JAX-RS packages. In addition, this enables us to leverage these same tests for testing the server implementation – an absolute win-win.

Note that the test examples below use the outstanding testng.

Proxy Interface

Here’s the client proxy VideoServiceProxy that is the same for both CXF and RESTeasy. Very nice guys!

public interface VideoServiceProxy {
@GET
@Path("genre")
@Produces("application/xml")
public GenreList getGenres(); 

@GET
@Path("genre/{id}/")
@Produces("application/xml")
public Genre getGenre(@PathParam("id") String id) ; 

@POST
@Path("genre")
@Consumes("application/xml")
public Response createGenre(Genre genre) ; 

@PUT
@Path("genre/{id}/")
@Consumes("application/xml")
public void updateGenre(@PathParam("id") String id, Genre genre) ; 

@DELETE
@Path("genre/{id}/")
public void deleteGenre(@PathParam("id") String id) ; 

}

Proxy Bootstrapping

The client side bootstrapping for the proxy is shown below.

CXF Bootstrapping

import org.apache.cxf.jaxrs.client.JAXRSClientFactory;
import org.apache.cxf.jaxrs.client.JAXRSClientFactoryBean;

public class GenreTest {
    static String url = "http://localhost/vapp/vservice/genre";
    static VideoServiceProxy rservice ;

    @BeforeSuite
    public void initSuite() {
        rservice = JAXRSClientFactory.create(url, VideoServiceProxy.class);
    }
}

RESTEasy Bootstrapping

import org.apache.commons.httpclient.HttpClient;
import org.jboss.resteasy.client.ProxyFactory;
import org.jboss.resteasy.plugins.providers.RegisterBuiltin;
import org.jboss.resteasy.spi.ResteasyProviderFactory;

public GenreTest {
    private static VideoServiceProxy rservice ;
    private static String url = "http://localhost/vapp/vservice/genre" ;   

    @BeforeClass
    static public void beforeClass() {
        RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
        rservice = ProxyFactory.create(VideoServiceProxy.class, url, new HttpClient());
    }
}

As you can see, the CXF version is slightly less verbose and simpler in that it has less imports.

Exception Handling

Happy Path – No Errors

For a happy path the test code is the same for both CXF and RESTEasy.

@Test
public void createGenre()  {
    Genre genre = new Genre();
    genre.setName("Animals");
    genre.setDescription("All animals");
    Response response = rservice.createGenre(obj);
    int status = response.getStatus();
    Assert.assertEquals(status,Response.Status.CREATED.getStatusCode());
    String createdId = ResponseUtils.getCreatedId(response); // get ID from standard Location header
    Assert.assertNotNull(createdId);
}

CXF Unhappy Path – Error

    @Test
    public void getGenreNonExistent() {
        try {
            Genre genre = rservice.getGenre(nonExistentId);
            Assert.fail();
        }
        catch (WebApplicationException e) {
            Response response = e.getResponse();
            int status = response.getStatus();
            Assert.assertEquals(status, Response.Status.INTERNAL_SERVER_ERROR.getStatusCode()); // TODO: fix 404 not being thrown?
            //Assert.assertEquals(status, Response.Status.NOT_FOUND.getStatusCode());
        }
    }

RESTEasy Unhappy Path – Error

    @Test
    public void getGenreNonExistent() {
        try {
            Genre obj = rservice.getGenre(nonExistentId);
        }
        catch (ClientResponseFailure e) {
            ClientResponse response = e.getResponse();
            Response.Status status = response.getResponseStatus();
            Assert.assertEquals(status, Response.Status.INTERNAL_SERVER_ERROR); // TODO: fix 404 not being thrown
            //Assert.assertEquals(status, Response.Status.NOT_FOUND);
        }
    }

Note, that there is a problem in correctly throwing a 404 with WebApplicationException on the server-side. Thought my current server implementation is CXF, I have also verified that this problem exists for RESTEasy and Jersey. The framework always returns a 500 even though I specifiy a 404. This is definitely not OK. Its a TBD for me to further investigate.

    throw new WebApplicationException(Response.Status.NOT_FOUND);

I definitely plan to check out Jersey and Restlet in more detail, so stay tuned!

JAX-RS: Java™ API for RESTful
Web Services
Version 1.0
September 8, 2008
Advertisements

Foray into Flex AIR

November 3, 2009

Adobe AIR has really excited me since I’ve been a long time fan of desktop applications. There is a time and place for web apps as well as for desktops. This post describes some of my reflections on a partial port of a Swing application to Flex.

Globesight is an economic modeling package geared toward scenario playing with a complex visualization metaphor as well as some heavy-duty binary floating point storage. It was my  Swing/UI opus magnum that I crafted a few years back – lots of UI navigation, charting, plugins, XML and binary I/O crunching and transfers. Too bad I implemented this before I came across Spring!

Although I won’t have time to port the whole application from Java/Swing, I want to see how some of the key Java technologies are handled by Flex/AIR.

  • Binary file IO for floating point numbers – I have concerns regarding Flex’s performance
  • Charting – I used the versatile JFreeChart and I expect Flex will excel here
  • Multiple windows, dialog boxes, popups
  • General screen navigation
  • How well does the Java code and classes translate into ActionScript?

Screenshots

Globesight Swing

swing-gs-views

Globesight Flex

flex-gs-views

Initial Reaction

Designing the Main Workbench screen and populating it from the XML project definition files was easy enough. I can’t imagine doing that so quickly in Java! So far so good. Reading in the XML is a charm.

I’ve also been able to leverage the elaborate MVC Java class hierarchy representing the domain model and visual components into ActionScript. The package hierarchy and class names almost match one for one. This leads me to think, might a Java to ActionScript translator help here? I’ve already found some info on the topic. Too bad that I simply can’t reuse the Java classes directly from Flex – too bad Flex doesn’t use the  JVM. I guess I got spoiled using Groovy which can be seamlessly (and confusingly) mixed with Java.

I’m already running into major hiccups regarding binary IO. I’ve got two dimensional arrays stored in binary floating point format (and some longs and ints in the header), and I see that ActionScript doesn’t have a long or FP data datatype! There’s Number and byte array and ultimately I assume I’ll be able to read the data, but how efficiently?

Field Manual for Lightweight WS/SOA governance

November 2, 2009

I’ve recently worked for several companies that were publishing web services without any clear-cut consistency or strategic vision. The following blog outlines some observations and suggestions for a coherent approach to developing and managing web services. The focus is on SOAP services but most points equally apply to REST as well.

The context is a medium-sized company that needs to develop one or more public-facing web services. Most of the following guidelines are applicable for internal-facing services, but public services have more of an emphasis on security and interoperability.

The focus is on solidly grounded pragmatics. I’ve been inspired by some of Thomas Erl’s writings but I find much of his advice overwhelming. The intent of this blog is to distill some essential SOA guidelines that I have seen work in practice. This blog can be seen as a field manual for an SOA lite framework.

Developing web services – especially public ones – is one place where “agile/iterative” development can come up short. Once an API is published it id basically fixed in stone. As the number of clients grow, the cost of change proportinally increases. Having a solid grasp of some of the industry best practices and key tenets of SOA/WS development can pay huge dividends in the overall life cycle of your service.

Some service features to be considered:

  • Common namespace
  • Common service endpoint
  • Service implementation
  • Schema validation
  • Security
  • Interoperability
  • Contract versioning and evolution
  • Error Handling
  • Client usage and packaging
  • Testing
  • Documentation

Common Namespace

The namespaces used in each service – both for the WSDL and external imported XSDs should be be based on a common prefix.

Note that the namespace is not an addressable URL – its not tied to any running server – a common confusion.  So we have the liberty to choose whatever we want. Some prefer URNs over URLs to avoid this confusion though the latter are more familiar to the masses. Some APIs such as Flick don’t even use a namespace! eBay interestingly enough uses both URLs and URNs for different APIs! How about some “SOA governance”! If this kind of stuff turns you on, go surf the programmableweb for more details. For example:

Sample URNs:

  • eBay Shopping API: urn:ebay:apis:eBLBaseComponents
  • eBay Trading API: urn:ebay:apis:eBLBaseComponents
  • eBay Merchant Data API: urn:ebay:apis:eBLBaseComponents
  • eBay PayPal: urn:ebay:api:PayPalAPI
  • Yahoo Search API: urn:yahoo:srch
  • Yahoo Map API: urn:yahoo:maps
  • Microsoft Health API schemas: urn:com.microsoft.wc.thing.types
  • Salesforce API: urn:partner.soap.sforce.com

Sample URLs:

eBay WSDL using URN style:

<wsdl:definitions
xmlns:wsdl=”http://schemas.xmlsoap.org/wsdl/&#8221;
xmlns:xs=”http://www.w3.org/2001/XMLSchema&#8221;
xmlns:wsdlsoap=”http://schemas.xmlsoap.org/wsdl/soap/&#8221;
xmlns:ns=”urn:ebay:apis:eBLBaseComponents
xmlns=”urn:ebay:apis:eBLBaseComponents”
targetNamespace=”urn:ebay:apis:eBLBaseComponents”>

Amazon S3 WSDL using URL style:

<wsdl:definitions
targetNamespace=”http://s3.amazonaws.com/doc/2006-03-01/
xmlns:tns=”http://s3.amazonaws.com/doc/2006-03-01/&#8221;
xmlns:wsdl=”http://schemas.xmlsoap.org/wsdl/&#8221;
xmlns:wsdlsoap=”http://schemas.xmlsoap.org/wsdl/soap/&#8221;
xmlns:xsd=”http://www.w3.org/2001/XMLSchema”&gt;

Common Service Endpoint

Having a common service endpoint is similar to the problem of a common namespace. The structure of the URL depends on the kinds of  services you support (SOAP or REST).

If you only have one type of web service (SOAP or REST), then you obviously do not need to have a top-level qualifier. If you support both services, then a separation such as the following  makes sense:

In either case, a bit of “governance” can give your services a professional feel. I’ve seen all too many cases where every developer invents their own URL without any coordination. Non-technical managers might not care but customers do notice. Look at it as a branding issue. Don’t you want all your services to sport the snappy same brand name? An example of “anarchy” is:

Schema Validation

You need to make an informed decision if you will support schema validation. This has two dimensions: outbound and inbound message validation.

The (JAX-WS) 2.1 spec only provides for optional validation – here’s what section 1.1 says:

JAX-WS 2.0 will allow full use of JAXB provided facilities including binding customization and optional
schema validation.

Mandating validation simplifies the service provider implementation as well as the client – it removes the inherent ambiguity of non-validation mode but sacrifices “flexibility”. Though validation by definition cannot be faster than non-validation, the guarantees of a validating mode are much stronger, and do provide well-known benefits of type-safety. There are many more ambiguities and undefined behaviors resulting from a non-validating mode.

For example, the CXF JAX-WS implementation doesn’t even accept its own advertised validation mode, so you are force to operate in non-validating mode.

Here’s the Spring-based configuration stanza that is supposed to turn on validation. Unfortunately it prevents any requests from suceeding by complaining about illegal XML when stub-generated client code is invoked! Go figure!

  <jaxws:endpoint id="videoServic" implementor="#videoServiceImpl" address="/soap/video" >
       <jaxws:properties>
          <entry key="schema-validation-enabled" value="true"/>
       </jaxws:properties>
  </jaxws:endpoint>

The corresponding server-side error message for a prefectly legitimate request is:

    Nov 7, 2009 12:55:47 AM org.apache.cxf.phase.PhaseInterceptorChain doIntercept
    WARNING: Interceptor has thrown exception, unwinding now
    org.apache.cxf.interceptor.Fault: Marshalling Error: cvc-elt.1: Cannot find the declaration of   element 'ns1:VideoAssets'.
    at org.apache.cxf.jaxb.JAXBEncoderDecoder.marshall(JAXBEncoderDecoder.java:160)
    at org.apache.cxf.jaxb.io.DataWriterImpl.write(DataWriterImpl.java:169)

The well-known SOA tenet “be strict in what you emit but be generous in what you consume” has a nice ring to it in the abstract, but with the “all or nothing” nature of XSD schemas, it becomes more difficult to implement. Other XML schemas such as RelaxNG have better abilities to handle these issues but unfortunately they have less mindshare and tools in the Java WS space.

It is interesting to note that Atom and AtomPub do not even provide a definitive schema though mention is made of it in RelaxNG. Although this is not relevant to a SOAP web services, it has a major impact on REST services. In section 1.3 the Atom Syndication Format spec states:

Some sections of this specification are illustrated with fragments of a non-normative RELAX NG Compact schema [RELAX-NG]. However, the text of this specification provides the definition of conformance. A complete schema appears in Appendix B.

Service implementation

The focus of this blog is on Java-based solutions especially JAX-WS . There are currently three open source JAX-WS implementations:

  • Metro – Sun’s RI intended to be production quality
  • CXF – Apache merger of XFire and Celtix
  • Axis2 – Another Apache implementation not related to Axis1

The first issue that you must tackle is whether you want to develop your service in a contract-first or code-first style. JAX-WS annotations have blurred this division, since you can now specific WSDL features from within your Java code using annotations. Not only do you have to have knowledge of WSDL, you also have to know how the annotations map to WSDL which can be challenging and depends on how faithful each vendor is.

There are advantages and disadvantages to both approaches.

Another issue is toolkit evaluation. I’ve noticed that many developers do not do an adequate job in “taking the toolkit around the block for a spin” before investing all their eggs in one basket. Ultimately it is the manager’s job to make sure that developers do not skip this step. If you know that you will be using WS security or MTOM, you should develop some non-trivial “spike” proof-of-concepts. Its risk-mitigation – better to be surprised earlier than later. I’ve seen some quite spectacular production blow-ups due to the lack of platform evaluation that literally ended up costing millions of dollars. Draw up a list of key features to evaluate. And don’t forget performance since SOAP as XML dialect is hardly efficient.

Interoperability

You should definitely make sure you comply with latest WS-I profile. Your customers’ platforms will drive this issue. Adequately testing different platforms in which you don’t have experience can be a challenge. If you are a Java shop, it might prove too costly (time consuming) to go out and implement a full suite of tests in C#. You should have at least one platform that has a comprehensive set of tests, and certainly have smoke tests for the platforms you plan to support. In one of my recent projects, just doing a small C# smoke test revealed problems with the name “System” as it seems to be a reserved word in the C# world.

Security

Once your services are being accessed outside the firewall, its a whole new ballgame. WS* security standards can be overwhelming, and I’ve found organizations often skimp here and hope for the best.

Dennis Sonoski has written some great articles on the practical implications of WS security:

Contract versioning and evolution

If you intend to change your service contracts, then you must at some point be aware of this issue. I’ve found that many organizations are in a hurry to roll out services and do not address this topic at all, and ultimately  pay the price downstream. A bit of foresight can alleviate some of the inherent pains of contract versioning.

Some versioning resources from the “big boys”:

Error Handling

There is no reason not to have a common reusable set of SOAP faults for all your services. This is a good example where you can get a “big bang for your buck” with XSD schema reuse between WSDLs.

Make sure you don’t “leak” provider-specific stack traces or error messages. Not only is it bad form and confusing for a user to see a cryptic Hibernate-related error message in a SOAP fault, its also a security risk since this gives the bad guys hints on how to crack your system. Remapping and “cleansing” error messages can be a surprisingly big chore, but if you tackle this problem early on, it will be much simpler to solve.

Instead of  a try-catch for every provider method, you can use Spring’s AOP exception advice mechanism to remap all lower-level exceptions to your SOAP fault. Below is an example that shows how all exceptions thrown within the service provider class VideoServiceImpl are mapped into a VideoServiceFault.

Spring Exception Advice Java

    package com.andre.video.ws.soap.aop;
    import org.springframework.aop.ThrowsAdvice;
    import java.lang.reflect.Method;
    import com.andre.video.ws.soap.VideoServiceFault;

    public class ExceptionAdvice implements ThrowsAdvice  {
        public void afterThrowing(Method method, Object[] args, Object target, Exception ex) throws Exception {
            throw new VideoServiceFault(ex.getMessage());
      }
    }

Spring XML Configuration

    <bean id="exceptionAdvice"  class="com.andre.video.ws.soap.aop.ExceptionAdvice"/>
    <aop:config>
        <aop:pointcut id="exceptionPointcut"
           expression="execution(* com.andre.video.ws.soap.server.VideoServiceImpl*.*(..))" />
      <aop:advisor id="exceptionAdvisor" advice-ref="exceptionAdvice" pointcut-ref="exceptionPointcut" />
    </aop:config>

Client Usage and Packaging

You must absolutely test you service with client stubs generated from the WSDL. Depending on your resources and customers, you should have clients in several languages. For example, if the service is implemented in Java, and you have Java and Flex clients, then it behooves you to have tests in both languages.

Some issues:

  • Stub generation
  • Client packages – stubs, javadoc, samples

You must decide if you want to provide your clients with pre-generated stubs. Perhaps its enough to provide them with just a WSDL. Maybe the customers expect something more especially if they have made a specific request for this.

Testing

This is closely related to the Client Usage and Packaging section.

Some issues:

  • Emedded web server tests vs. external web server
  • Leverage testng testing framework for its suite fixture capability that allows you to load only once an expensive package instead of for every class with JUnit’s class fixture
  • How to pump in bad XML that cannot ever be exercised by stubs-based clients?
  • Performance testing – JMeter, Grinder. testng has some simple multithreaded annotations that can be used for simple load tests

Documentation

Last but not least is documentation. A WSDL is a necessary but hardly sufficient form of documentation. The semantics of some services can be quite self-evident and need minimal explanation. Other services can have complex semantics that require extensive text and examples. Again, examples from major API publishers can be useful:

You can use XSD’s ability to have documentation elements. I’m not sure how JAX-WS supports this if at all – this is a TBD research item for me.

One of the most thrilling experiences I recently had was to write the entire user documentation with a business analyst for our REST API. Trust me, you’ll never look at your API in the same way after having to document it in detail every feature.

Other

Some other issues:

  • Throttling
  • Service management
  • Client service consoles

Using Flex’s WebService and auto-binding of SOAP data to DataGrids, you can very quickly develop useful service consoles that can exercise much the functionality of you service. The service console can be leveraged by developers, testers, professional service folks and even business analysts.

For an example, see a sample Flex web service console.