Showing posts with label XML. Show all posts
Showing posts with label XML. Show all posts

Monday, February 6, 2017

Pretty Print JSON and XML from Mac OS Command Line

JSON


Must have Python 2.6 or higher version installed.

From Clipboard


After copying unformatted JSON to clipboard (ctrl+c), run this:

pbpaste | python -m json.tool

From File


After saving unformatted JSON to a file called, for example, ugly.json, run this to write it to a file called pretty.json:

cat ugly.json | python -m json.tool > pretty.json

XML


From Clipboard


After copying unformatted XML to clipboard (ctrl+c), run this:

pbpaste | xmllint --format -

pbpaste | xmllint --format -

From File


After saving unformatted XML to a file called, for example, ugly.xml, run this to write it to a file called pretty.xml:

cat ugly.xml | xmllint --format -o pretty.xml -


Monday, February 1, 2016

Jackson UnrecognizedPropertyException

For complicated APIs, I use the XJC binding compiler included with JAXB to generate Java classes using the XSD schema of the XML results of a web service.  This is a huge time saver.

For JSON results, there isn't a standard binding compiler to generate Java classes but since JSON and XML are almost interchangeable, I generate the Java classes the same way and fix the differences after the classes have been compiled.

One common error I get is an "unrecognized field" because of the slight differences between XML and JSON representations of data.

For example, I got this error:
com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "geonames" (Class org.learn.ws.model.jaxb.api.geonames.org.postal.countryinfo.types.GeonamesType), not marked as ignorable
when unmarshalling this JSON:
{
  "geonames": [
    {
      "numPostalCodes": 7,
      "maxPostalCode": "AD700",
      "countryCode": "AD",
      "minPostalCode": "AD100",
      "countryName": "Andorra"
    },
    {
      "numPostalCodes": 20260,
      "maxPostalCode": "9431",
      "countryCode": "AR",
      "minPostalCode": "1601",
      "countryName": "Argentina"
    }
    ...lots more rows...
  ]
}
to this GeonamesType class:
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlType;
import java.util.ArrayList;
import java.util.List;

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "geonamesType", propOrder = {
    "country"
})
public class GeonamesType {

    protected List<countrytype> country;

    public List<countrytype> getCountry() {
        if (country == null) {
            country = new ArrayList<countrytype>();
        }
        return this.country;
    }
}
and this CounryType class:
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "countryType", propOrder = {
    "countryCode",
    "countryName",
    "numPostalCodes",
    "minPostalCode",
    "maxPostalCode"
})
public class CountryType {

    @XmlElement(required = true)
    protected String countryCode;
    @XmlElement(required = true)
    protected String countryName;
    @XmlElement(required = true)
    protected String numPostalCodes;
    @XmlElement(required = true)
    protected String minPostalCode;
    @XmlElement(required = true)
    protected String maxPostalCode;

    public String getCountryCode() {
        return countryCode;
    }

    public void setCountryCode(String value) {
        this.countryCode = value;
    }

    public String getCountryName() {
        return countryName;
    }

    public void setCountryName(String value) {
        this.countryName = value;
    }

    public String getNumPostalCodes() {
        return numPostalCodes;
    }

    public void setNumPostalCodes(String value) {
        this.numPostalCodes = value;
    }

    public String getMinPostalCode() {
        return minPostalCode;
    }

    public void setMinPostalCode(String value) {
        this.minPostalCode = value;
    }

    public String getMaxPostalCode() {
        return maxPostalCode;
    }

    public void setMaxPostalCode(String value) {
        this.maxPostalCode = value;
    }

}
The GeonamesType class is the root-level class generated and annotated automatically with the JAXB XJC shell script. For more on XJC, see http://www.thoughts-on-java.org/generate-your-jaxb-classes-in-second/.

The XJC shell script is used to generate Java classes from an XSD schema file. The XSD schema file can be generated automatically from sample XML with any number of online XML schema generators, for example this one: http://www.freeformatter.com/xsd-generator.html.

The error is telling me that the root-level node in the returned JSON, "geonames", doesn't have a correspondingly-named property in the root-level class, GeonamesType.  In the class, its name is "country".  The class was generated with xjc from the XML version of the result returned from the web service, which looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<geonames>
    <country>
        <countryCode>AD</countryCode>
        <countryName>Andorra</countryName>
        <numPostalCodes>7</numPostalCodes>
        <minPostalCode>AD100</minPostalCode>
        <maxPostalCode>AD700</maxPostalCode>
    </country>
    <country>
        <countryCode>AR</countryCode>
        <countryName>Argentina</countryName>
        <numPostalCodes>20260</numPostalCodes>
        <minPostalCode>1601</minPostalCode>
        <maxPostalCode>9431</maxPostalCode>
    </country>
    ...lots more rows...
</geonames>
In the XML version, geonames is a list of country objects, which is how it is represented in the Java classes. In the JSON version, geonames is an array (list) of objects, but the objects aren't named. I could fix this by renaming "country" in the GeonamesType class to "geonames" like this:
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "geonamesType", propOrder = {
    "geonames"
})
public class GeonamesType {

    protected List<countrytype> geonames;

    public List<countrytype> getCountry() {
        if (geonames == null) {
            geonames = new ArrayList<countrytype>();
        }
        return this.geonames;
    }
}
This solves the problem for JSON results, but it doesn't work anymore for the XML results.

Use Same Classes With XML And JSON


By modifying the original GeonamesType class (at the top) with a Jackson annotation that is specific to JSON, it is possible to use the same Java classes for both XML and JSON versions of the results.

Just add the @JsonElement annotation to the country property to tell Jackson that the name of the element in the JSON result will be "geonames", not "country", like this:
import org.codehaus.jackson.annotate.JsonProperty;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlType;
import java.util.ArrayList;
import java.util.List;

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "geonamesType", propOrder = {
    "country"
})
public class GeonamesType {

    @JsonProperty(value = "geonames")
    protected List<countrytype> country;

    public List<countrytype> getCountry() {
        if (country == null) {
            country = new ArrayList<countrytype>();
        }
        return this.country;
    }
}
Now the classes work for both the XML and JSON results as shown above.

Thursday, January 14, 2016

Web Services With JSON and XML

XML and JSON are the most popular formats for exchanging data over the Internet.  JSON has become more popular recently because it's less verbose than XML.  For example, this XML:
<geonames>
    <country>
        <countrycode>AD</countryCode>
        <countryname>Andorra</countryName>
    </country>
    <country>
        <countrycode>AR</countryCode>
        <countryname>Argentina</countryName>
    </country>
</geonames>
Looks like this in JSON:
{"geonames": [
    {
        "countryCode": "AD",
        "countryName": "Andorra"
    },
    {
        "countryCode": "AR",
        "countryName": "Argentina"
    }
]}
JSON uses fewer characters, takes up less space and is easier to read.

Because JSON and XML representations of data can be used almost interchangeably, many web services are offered in both versions for convenience.  Here are a couple showing postal codes in London's W1J postal area:

XML:
http://api.geonames.org/postalCodeSearch?postalcode=W1J&maxRows=2&username=demo

JSON:
http://api.geonames.org/postalCodeSearchJSON?postalcode=W1J&maxRows=2&username=demo

They have slightly different URL endpoints, postalCodeSearch and postalCodeSearchJSON, and they return the same data, just in different formats.

Using Java to Produce and Consume RESTful Web Services


Java is one of the oldest and most popular languages for web development, especially for applications deployed on web servers. It includes packages to simplify producing and consuming RESTful web services. In addition, there are many popular Java community-built APIs for web services. Some of the APIs are so popular that they have been incorporated into the Java JDK.

JAXB - Java Architecture for XML Binding


Java objects can be serialized into an XML string using JAXB. Likewise, an XML string that represents a Java object can deserialized into a Java object. Serializing is called "marshalling" and deserializing is called "unmarshalling". Java classes to be marshalled into XML are marked up using annotations from the javax.xml.bind.annotation.* namespace, for example, @XmlRootElement and @XmlElement.

JAXB includes a couple tools to simplify creating annotated Java classes or XML schemas:

xjc - creates annotated Java classes from an XML schema (XSD file)

schemagen - creates an XML schema from annotated Java classes

See here for examples and usage.

(show example Java class, XML and XML schema)

JAX-RS - Java API for RESTful Web Services


JAX-RS is the Java API for RESTful Web Services that simplifies creating and consuming web services using XML and JSON.  There are several popular implementations:
I'll talk about Jersey because it's the one I know best.  I use Jersey 1.19, the last version of Jersey before version 2. Version 2 has features I don't find useful.

Jersey uses a reference implementation (RI) of JAXB version 2.x or higher to serialize Java objects to XML and deserialize XML to Java objects.

Since JAXB 2.x or higher is included in the Java JDK starting with Java 6, no additional dependency is required if you're using Java 6, 7, 8 or higher.  Here are the versions of JAXB included with Java 6 and 7.

To serialize the JAXB beans (Java classes) to JSON and deserialize JSON to JAXB beans when using the MIME media type application/json, you do need to include an additional dependency. With Maven, it looks like this:
<dependency>
    <groupId>com.sun.jersey</groupId>
    <artifactId>jersey-json</artifactId>
    <version>1.19</version>
</dependency>
See 11.4.1.1. JAXB RI (reference implementation) for more.

Jersey XML Support

Jersey JSON Support





YouTube 5.1 Channel Audio Fail in Browsers

Ignore any blog posts or YouTube videos that claim to show you how to play 5.1 surround from YouTube videos in a browser. They don’t work. A...