Implementation of Product Importer in AEM

Reading Time: 6 minutes

Nowadays, every serious company has options on its website to present the products to potential customers. When we talk about big companies, with a lot of products and huge traffic, AEM is one of the best solutions. But, how are the products imported in AEM, where are they placed in AEM are?… you can learn here. We will cover the following aspects of the problem:

  • How to fetch the products from the server
  • How to convert server response (JSON String) into Java object
  • Where to place products in AEM repository
  • Product node structure
  • How to persist in products
  • How to start product importer and follow the process

How to fetch the products from the server

Typically large companies are keeping their products on a separate dedicated server. With the API that they will provide to you, a connection to the server will be established, and you could fetch the products. In most cases, an OSGI service is created that keeps the configuration data for connecting the remote server. Typically we got the response as a JSON String. Bellow is just one idea of how to get a response. The URL parameter and the access token is provided by the client, and usually, we keep it in OSGI service configuration.

private String getAPIResponse(String url) {
		String accessToken = getAccessToken();
		CloseableHttpClient httpclient = HttpClients.createDefault();
		String authorizationString = "Bearer " + accessToken;
		HttpGet request = new HttpGet(URI.create(url));
		request.addHeader("Authorization", authorizationString);
		try {
			HttpResponse response = httpclient.execute(request);
			return getResponseBodyAsString(response);
		} catch (IOException e) {
			LOGGER.error("Failed httpclient.execute method", e);
		return null;
private String getResponseBodyAsString(HttpResponse response) {
		try (Scanner sc = new Scanner(response.getEntity().getContent())) {
			StringBuilder sb = new StringBuilder();
			while (sc.hasNext()) {
			return sb.toString();
		} catch (IOException e) {
			LOGGER.error("Failed to retreive response body", e);
		return null;

How to convert server response into Java object (APIModel)

Once we got a response as JSON String, the biggest challenge is converting response from the server (String apiResponse) into java class (APIModel). For that purpose we use the class. Sometimes it is unpredictable how Gson will de-serialize apiResponse into java objects. As for advice, if something goes wrong in mapping, just put “Object” in the mapping of the value, and later when debugging can check how Gson actually maps that value.

public APIModel convertIntoAPIModel(String apiResponse) {
 try {
 Gson gson = new Gson();
 return gson.fromJson(apiResponse, APIModel.class);
 } catch (RuntimeException e) {
 LOGGER.error("Error while converting into APIModel",e);
 throw e;
@Model(adaptables = Resource.class)
public class APIModel {
	private List<ProductImportedModel> results;

	public List<ProductImportedModel> getResults() {
		return results;
public class ProductImportedModel {
     private String nameOfProduct;
     private Date lastModifiedAt;
     private Date createdAt;

Where to place products in AEM repository

First, let’s look at the very base of this commerce part. We will look at the repository level in CRX to see the location and structure of the products in AEM. For that purpose most appropriate is an out of the box solution for we-retail which is part of the AEM installation. As we can see, products are stored in /var/commerce/products/your-company-name.

Product node structure

Let’s check the structure of one product in we-retail (on the image above “eqbisucos”). The product consists of one “master” product which contains the general properties of the product. These properties can be anything including price, rating, origin… and the most important properties are these two which mark it as a product:

  • cq:commerceType String product
  • sling:resourceType String commerce/components/product

Under this master node, there are subnodes as “image” and variants of the products. Regarding variants, it is important to mention that the difference with the product is that property commerceType has the value ‘variant’.

In the image above , we can see different variants as size-s, size-m, size-l.

Now, when we know the structure of the out of the box commerce product, let’s see how we can use our APIModel and transform it into node structure under /var/commerce.

The structure of the product node is not strictly defined. It depends on the concrete situation and the data for the product that we need to store. However, there are some rules to take in consideration:

  • define the master product node
  • create variants as sub-nodes of the master node. It could happen both (master and variant) to have very similar properties with a small difference, but that is acceptable. At least one property must be different
  • It is not required but would be nice to have “image” sub-nodes with an image for the product. It could be just one “image node” for master, or furthermore, every variant can have its own “image” node
  • It is possible to have other sub-nodes with different information for product or variants. The number of these “other nodes” is not limited. They can keep any information
  • Node structure of master or variant is not required to be the same for every product. Some sub-nodes could be missing for some masters or variants

How to persist products

Once we have determinate the node structure of the product, it is time to create node structure and store values as node properties. As first, for that purpose, we need service user with write permissions in /var/commerce part. The best approach is to use sling API, with all methods to create resources. Here is one example to create product node with properties.

Map<String, Object> properties = new HashMap<>();
		properties.put("sling:resourceType", "commerce/components/product");
		properties.put("cq:commerceType", commerceType);
		properties.put("jcr:primaryType", "nt:unstructured");
		properties.put("price", 1000);
		properties.put("color", "green);
Resource newProduct = resolver.create(productsResource, "myFirstProduct", properties);

How to start product importer and follow the process

Product importer is triggered from AEM JMX console. Possible options are to trigger it manually or periodic as a cron job. It is a separate thread that does not return a result, so it is very hard to follow the process without server logs. But this is possible just for the developer. So, any solution? Yes, best practice would be during the implementation of the importer, when exception happens, that exception to be mailed to the responsible person, with time and the exact point where exception happens.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *