Navigating SharePoint Folders With Axis2

I’ve just written some test code to get a list of items in a SharePoint folder with Apache Axis2 and since this was “not so easy”, I’ll share my insights here.

First, you need Axis2. If you’re using Maven2, put this in your pom.xml:

    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-kernel</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-adb</artifactId>
        <version>1.4.1</version>
    </dependency>

Next stop: Setting up NTLM authorization.

import org.apache.axis2.transport.http.HttpTransportProperties;
import org.apache.commons.httpclient.auth.AuthPolicy;

        HttpTransportProperties.Authenticator auth = new
            HttpTransportProperties.Authenticator();
        auth.setUsername ("username");
        auth.setPassword "password");
        auth.setDomain ("ntdom");
        auth.setHost ("host.domain.com");

        List authPrefs = new ArrayList (1);
        authPrefs.add (AuthPolicy.NTLM);
        auth.setAuthSchemes (authPrefs);

This should be the username/password you’re using to login to the NT domain “ntdom” on the NT domain server “host.domain.com”. Often, this server is the same as the SharePoint server you want to connect to.

If the SharePoint server is somewhere outside your intranet, you may need to specify a proxy:

        HttpTransportProperties.ProxyProperties proxyProperties =
            new HttpTransportProperties.ProxyProperties();
        proxyProperties.setProxyName ("your.proxy.com");
        proxyProperties.setProxyPort (8888);

You can get these values from your Internet browser.

If there are several SharePoint “sites” on the server, set site to the relative URL of the site you want to connect to. Otherwise, leave site empty. If you have no idea what I’m talking about, browse the SharePoint server in Internet Explorer. In the location bar, you’ll see an URL like this: https://sp.company.com/projects/demo/Documents2/Forms/AllItems.aspx?RootFolder=%2fprojects%2fdemo%2fDocument2%2f&FolderCTID=&View=%7b18698D80%2dE081%2d4BBE%2d96EB%2d73BA839230B9%7d. Scary, huh? Let’s take it apart:

https:// = the protocol,
sp.company.com = The server name (with domain),
projects/demo = The “site” name
Documents2 = A “list” stored on the site “projects/demo”
/Forms/AllItems.aspx?RootFolder=... is stuff to make IE happy. Ignore it.

So in out example, we have to set site to:

        String site = "/projects/demo";

Mind the leading slash!

To verify that this is correct, replace “/Documents2/Forms/” and anything beyond with “/_vti_bin/Lists.asmx?WSDL”. That should return the WSDL definition for this site. Save the result as “sharepoint.wsdl” (File menu, “Save as…”). Install Axis2, open a command prompt in the directory where you saved the WSDL file and run this command (don’t forget to replace the Java package name):

%AXIS2_HOME%binWSDL2Java -uri sharepoint.wsdl -p java.package.name -d adb -s

This will create a “src” directory with the Java package and a single file “ListsStub.java”. Copy it into your Maven2 project.

Now, we can get a list of the lists on the site:

        ListsStub lists = new ListsStub
            ("https://sp.company.com"+site+"/_vti_bin/Lists.asmx");
        lists._getServiceClient ().getOptions ()
            .setProperty (HTTPConstants.AUTHENTICATE, auth);

If you need a proxy, specify it here:

        options.setProperty (HTTPConstants.HTTP_PROTOCOL_VERSION,
            HTTPConstants.HEADER_PROTOCOL_10);
        options.setProperty (HTTPConstants.PROXY, proxyProperties);

We need to reduce the HTTP protocol version to 1.0 because most proxies don’t allow to send multiple requests over a single connection. If you want to speed things up, you can try to comment out this line but be prepared to see it fail afterwards.

Okay. The plumbing is in place. Now we query the server for the lists it has:

        String liste = "Documents2";
        String document2ID;
        {
            ListsStub.GetListCollection req = new ListsStub.GetListCollection();
            ListsStub.GetListCollectionResponse res = lists.GetListCollection (req);
            displayResult (req, res);

            document2ID = getIDByTitle (res, liste);
        }

This downloads all lists defined on the server and searches for the one we need. If you’re in doubt what the name of the list might be: Check the bread crumbs in the blue part in the intern explorer. The first two items are the title of the site and the list you’re currently in.

displayResult() is the usual XML dump code:

    private void displayResult (GetListCollection req,
            GetListCollectionResponse res)
    {
        System.out.println ("Result OK: "
                +res.localGetListCollectionResultTracker);
        OMElement root = res.getGetListCollectionResult ()
                .getExtraElement ();
        dump (System.out, root, 0);
    }

    private void dump (PrintStream out, OMElement e, int indent)
    {
        indent(out, indent);
        out.print (e.getLocalName ());
        for (Iterator iter = e.getAllAttributes (); iter.hasNext (); )
        {
            OMAttribute attr = (OMAttribute)iter.next ();
            out.print (" ");
            out.print (attr.getLocalName ());
            out.print ("="");
            out.print (attr.getAttributeValue ());
            out.print (""");
        }
        out.println ();

        for (Iterator iter = e.getChildElements (); iter.hasNext (); )
        {
            OMElement child = (OMElement)iter.next ();
            dump (out, child, indent+1);
        }
    }

    private void indent (PrintStream out, int indent)
    {
        for (int i=0; i&lt;indent; i++)
            out.print ("    ");
    }

We also need getIDByTitle() to search for the ID of a SparePoint list:

    private String getIDByTitle (GetListCollectionResponse res, String title)
    {
        OMElement root = res.getGetListCollectionResult ().getExtraElement ();
        QName qnameTitle = new QName ("Title");
        QName qnameID = new QName ("ID");
        for (Iterator iter = root.getChildrenWithLocalName ("List"); iter.hasNext (); )
        {
            OMElement list = (OMElement)iter.next ();
            if (title.equals (list.getAttributeValue (qnameTitle)))
                return list.getAttributeValue (qnameID);
        }
        return null;
    }

With that, we can finally list the items in a folder:

        {
            String dir = "folder/subfolder";

            ListsStub.GetListItems req
                = new ListsStub.GetListItems ();
            req.setListName (document2ID);
            QueryOptions_type1 query
                = new QueryOptions_type1 ();
            OMFactory fac = OMAbstractFactory.getOMFactory();
            OMElement root = fac.createOMElement (
                new QName("", "QueryOptions"));
            query.setExtraElement (root);

            OMElement folder = fac.createOMElement (
                new QName("", "Folder"));
            root.addChild (folder);
            folder.setText (liste+"/"+dir); // &lt;--!!

            req.setQueryOptions (query);
            GetListItemsResponse res = lists.GetListItems (req);
            displayResult (req, res);
        }

The important bits here are: To list the items in a folder, you must include the name of the list in the “Folder” element! For reference, this is the XML which actually sent to the server:

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
    <soapenv:Body>
        <ns1:GetListItems xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/">
            <ns1:listName>{12AF2346-CCA1-486D-BE3C-82223DEC3F42}</ns1:listName>
            <ns1:queryOptions>
                <QueryOptions>
                    <Folder>Documents2/folder/subfolder</Folder>
                </QueryOptions>
            </ns1:queryOptions>
        </ns1:GetListItems>
    </soapenv:Body>
</soapenv:Envelope>

If the folder name is not correct, you’ll get a list of all files and folders that the SharePoint server can find anywhere. The folder names can be found in the bread crumbs. The first two items are the site and the list name, respectively, followed by the folder names.

The last missing piece is displayResult() for the items:

    private void displayResult (GetListItems req,
         GetListItemsResponse res)
    {
        System.out.println ("Result OK: "
                +res.localGetListItemsResultTracker);
        OMElement root = res.getGetListItemsResult ()
                .getExtraElement ();
        dump (System.out, root, 0);
    }

If you run this code and you see the exception “unable to find valid certification path to requested target”, this article will help.

If the SharePoint server returns an error, you’ll see “detail unsupported element in SOAPFault element”. I haven’t found a way to work around this bug in Axis2. Try to set the log level of “org.apache.axis2” to “DEBUG” and you’ll see what the SharePoint server sent back (not that it will help in most of the cases …)

Links: GetListItems on MSDN, How to configure Axis2 to support Basic, NTLM and Proxy authentication?, Java to SharePoint Integration – Part I (old, for Java 1.4)

Good luck!

32 Responses to Navigating SharePoint Folders With Axis2

  1. max says:

    Hi, Dark,

    Thanks for this sharepoint java integration post! This is the best example I got so far to use Java to access sharepoint web service!

    I got problem to use your simple code and wish to see or download your completed code example. My problem is your declarations on req and res:

    ListsStub.GetListCollection req = new ListsStub.GetListCollection();
    ListsStub.GetListCollectionResponse res = lists.GetListCollection (req);

    Later on you declare:

    ListsStub.GetListItems req = new ListsStub.GetListItems ();
    GetListItemsResponse res = lists.GetListItems (req);

    Even they seems to be in different scopes, but they are calling same method: displayResult (req, res);

    The type will not match there.

    Because I can’t see all your code, I don’t know how that works out.
    Would you please post a completed code?

    Many Thanks

    Max

  2. Michael says:

    > If the SharePoint server returns an error, you’ll see “detail
    > unsupported element in SOAPFault element”. I haven’t found a way to
    > work around this bug in Axis2.

    I’m not even sure if this is a Axis2 bug. Looking at the SOAP response there is a element where Axis2 expects a elements as required by the spec. I don’t know whether the spec allows for a element. If not the error would be with Sharepoint.

    • digulla says:

      In this context, “bug” is left binding as in “I don’t know enough about Axis2 to work around the bug in SharePoint” 🙂

      • michid says:

        Yes sure. Only if we knew it was a Axis2 bug we could report it as an issue on the Axis JIRA and probably have it fixed.

        Well, we could report an issue anyway 😉

  3. Amit Sachdev says:

    org.apache.axiom.soap.SOAPProcessingException: First Element must contain the local name, Envelope , but found html

    I try your method, but i get the above exception. Can you please help?

    • digulla says:

      That sounds like you’re using the wrong URL in your soap request. Instead of the SOAP reply, you get the HTML page (like the browser).

  4. Pawel says:

    Hi!
    this is a great post. I’m using WSS 3.0 and after making wsimport I don’t have ListsStub class. Is there a easy way to use e.g. ObjectFactory class or ListsSoap(ListsSoap12 is not created neither)? I’ve lost all my hair because of this.
    Can you help?
    cheers,
    Pawel

  5. Pawel says:

    Ok, I resolved it. ListsStub is generated only when using Axis 🙂

  6. Pawel says:

    Hi, have you managed to put some element on the SP list with axis?
    Or maybe even something simple: add a list to SP.
    Could you put some sample code?
    My code throws exception:
    “detail unsupported element in SOAPFault element” (if I use Lists12Soap)
    and “Microsoft.SharePoint.SoapServer.SoapServerException” exception was thrown when using ListsSoap.

    Thanks in advance

  7. Pawel says:

    btw. your example with GetListCollection works fine.

  8. Pawel says:

    I solved the problem with “detail unsupported element in SOAPFault element” and “Microsoft.SharePoint.SoapServer.SoapServerException”.

    Options options = stub._getServiceClient().getOptions();
    options.setSoapVersionURI(Constants.URI_SOAP11_ENV);

    cheers,
    P.

  9. Pawel says:

    Here comes my another post 🙂 :
    Have you managed to add list elements through WebService?

  10. Eric A. Gravel says:

    HI Aaron,

    I saw your post on stack-overflow and wanted to see if you might of found a solution on how to upload documents to sharepoint? I was looking through the list of services and the only one of potential interest is AddAttachment. But I’m not sure that’s the right one.

    Any luck implementing this?

  11. satya says:

    java2sharepoint.axisadb.ListsStub.GetListCollection belongs to which jar

  12. digulla says:

    401 means “permission denied”. And the SOAPFault should also contain some error message which you can try to understand.

    If everything else fails, post a question with as much useful information as possible on stackoverflow.com

  13. satya says:

    please chek my code.and log
    //code
    HttpTransportProperties.Authenticator authenticator = new HttpTransportProperties.Authenticator();
    List authSchemes = new ArrayList();
    authSchemes.add(HttpTransportProperties.Authenticator.NTLM);
    authenticator.setAuthSchemes(authSchemes);
    authenticator.setHost(host);
    authenticator.setPort(80);
    authenticator.setDomain(domain);
    authenticator.setUsername(username);
    authenticator.setPassword(password);
    authenticator.setPreemptiveAuthentication(true);

    ListsStub lists;
    try {
    lists = new ListsStub(“http://abc-xyz.com/_vti_bin/Lists.asmx”);
    lists._getServiceClient().getOptions().setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, authenticator);

    lists._getServiceClient().getOptions().setSoapVersionURI(Constants.URI_SOAP11_ENV);
    ListsStub.CheckOutFile req=new ListsStub.CheckOutFile();

    req.setCheckoutToLocal(“C:/siva.pdf”);
    req.setLastmodified(“2/2/2012 9:54 PM”);
    req.setPageUrl(“http://abc-xyz.com/zen/Invocie/Testing.pdf”);
    OMFactory factory = OMAbstractFactory.getOMFactory();
    QName ss=new QName();

    ListsStub.CheckOutFileResponse res=null;
    System.out.println(“1 “+req.toString());
    res = lists.checkOutFile(req);

    } catch (Exception e1) {
    System.out.println(“AXISF”+e1);
    e1.printStackTrace();
    }

    //log
    org.apache.axis2.AxisFault: Connection refused: connect
    at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
    at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:203)
    at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
    at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
    at samples.SHAREPOINT.CLIENT.ListsStub.checkOutFile(ListsStub.java:2295)
    at com.test.Test.main(Test.java:85)
    Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(Unknown Source)
    at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)

  14. satya says:

    now i,m getting org.apache.axis2.AxisFault: Exception of type ‘Microsoft.SharePoint.SoapServer.SoapServerException’ was thrown

    • digulla says:

      I know how you feel 🙂

      Go on trying, eventually, you’ll get it to work. You won’t know why and how but if it was great, simple or useful, it wouldn’t be from M$…

      • satya says:

        thanks for you co-operation,still i’m getting 401,same credentials from browser it’s working,but java/burp suite i,m getting Ntml authentication fail is there any IIS server configuration changes required?

      • satya says:

        now im getting SOAP EXCEPTION when i,m trying to get list
        Guid should contain 32 digits with 4 dashes
        (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)

      • satya says:

        Hi Digulla,now am able to all checkout web service ,i got response HTTP code 200,but i’m not able to get file and getCheckOutFileResult is false,what are causes,please help me.

        Response is
        HTTP/1.1 200 OK
        Cache-Control: private, max-age=0
        Content-Length: 391
        Content-Type: text/xml; charset=utf-8
        Server: Microsoft-IIS/7.0
        X-AspNet-Version: 2.0.50727
        Set-Cookie: WSS_KeepSessionAuthenticated=80; path=/
        X-Powered-By: ASP.NET
        MicrosoftSharePointTeamServices: 12.0.0.6421
        Date: Mon, 20 Feb 2012 13:09:54 GMT

        false

  15. michid says:

    Try configuring IIS for basic auth and see if it works then. There might be an issue with NTLM and Apache http-client not supporting NTLM2. That is, if you use NTLM2 (i.e. Sharepoint 2010), then its likely that this is not supported by the http-client library used by Axis. This was at least the case when I last checked.

  16. […] Navigating SharePoint Folders With Axis2 Like this:LikeBe the first to like this. […]

  17. Jan says:

    had the same issue with Sharepoint 2010 and NTLMv2. I got 401 HTTP return code. I found a hint that NTLMv2 is not supported by axis2 / HttpClient 3.1. There is a workaround with JCIFS:

    http://devsac.blogspot.de/2010/10/supoprt-for-ntlmv2-with-apache.html

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s