Implementation of Product Importer in AEM

Reading Time: 6 minutes

Nowadays, every serious company has options on its website to present the products to potential customers. When we talk about big companies, with a lot of products and huge traffic, AEM is one of the best solutions. But, how are the products imported in AEM, where are they placed in AEM are?… you can learn here. We will cover the following aspects of the problem:

  • How to fetch the products from the server
  • How to convert server response (JSON String) into Java object
  • Where to place products in AEM repository
  • Product node structure
  • How to persist in products
  • How to start product importer and follow the process

How to fetch the products from the server

Typically large companies are keeping their products on a separate dedicated server. With the API that they will provide to you, a connection to the server will be established, and you could fetch the products. In most cases, an OSGI service is created that keeps the configuration data for connecting the remote server. Typically we got the response as a JSON String. Bellow is just one idea of how to get a response. The URL parameter and the access token is provided by the client, and usually, we keep it in OSGI service configuration.

private String getAPIResponse(String url) {
		String accessToken = getAccessToken();
		CloseableHttpClient httpclient = HttpClients.createDefault();
		String authorizationString = "Bearer " + accessToken;
		HttpGet request = new HttpGet(URI.create(url));
		request.addHeader("Authorization", authorizationString);
		try {
			HttpResponse response = httpclient.execute(request);
			return getResponseBodyAsString(response);
		} catch (IOException e) {
			LOGGER.error("Failed httpclient.execute method", e);
		}
		return null;
	}
private String getResponseBodyAsString(HttpResponse response) {
		try (Scanner sc = new Scanner(response.getEntity().getContent())) {
			StringBuilder sb = new StringBuilder();
			while (sc.hasNext()) {
				sb.append(sc.nextLine());
			}
			return sb.toString();
		} catch (IOException e) {
			LOGGER.error("Failed to retreive response body", e);
		}
		return null;
	}

How to convert server response into Java object (APIModel)

Once we got a response as JSON String, the biggest challenge is converting response from the server (String apiResponse) into java class (APIModel). For that purpose we use the com.google.gson.Gson class. Sometimes it is unpredictable how Gson will de-serialize apiResponse into java objects. As for advice, if something goes wrong in mapping, just put “Object” in the mapping of the value, and later when debugging can check how Gson actually maps that value.

public APIModel convertIntoAPIModel(String apiResponse) {
 try {
 Gson gson = new Gson();
 return gson.fromJson(apiResponse, APIModel.class);
 } catch (RuntimeException e) {
 LOGGER.error("Error while converting into APIModel",e);
 throw e;
 }
}
@Model(adaptables = Resource.class)
public class APIModel {
	private List<ProductImportedModel> results;

	public List<ProductImportedModel> getResults() {
		return results;
	}
}
public class ProductImportedModel {
     private String nameOfProduct;
     private Date lastModifiedAt;
     private Date createdAt;
     ........
     ........
}

Where to place products in AEM repository

First, let’s look at the very base of this commerce part. We will look at the repository level in CRX to see the location and structure of the products in AEM. For that purpose most appropriate is an out of the box solution for we-retail which is part of the AEM installation. As we can see, products are stored in /var/commerce/products/your-company-name.

Product node structure

Let’s check the structure of one product in we-retail (on the image above “eqbisucos”). The product consists of one “master” product which contains the general properties of the product. These properties can be anything including price, rating, origin… and the most important properties are these two which mark it as a product:

  • cq:commerceType String product
  • sling:resourceType String commerce/components/product

Under this master node, there are subnodes as “image” and variants of the products. Regarding variants, it is important to mention that the difference with the product is that property commerceType has the value ‘variant’.

In the image above , we can see different variants as size-s, size-m, size-l.

Now, when we know the structure of the out of the box commerce product, let’s see how we can use our APIModel and transform it into node structure under /var/commerce.

The structure of the product node is not strictly defined. It depends on the concrete situation and the data for the product that we need to store. However, there are some rules to take in consideration:

  • define the master product node
  • create variants as sub-nodes of the master node. It could happen both (master and variant) to have very similar properties with a small difference, but that is acceptable. At least one property must be different
  • It is not required but would be nice to have “image” sub-nodes with an image for the product. It could be just one “image node” for master, or furthermore, every variant can have its own “image” node
  • It is possible to have other sub-nodes with different information for product or variants. The number of these “other nodes” is not limited. They can keep any information
  • Node structure of master or variant is not required to be the same for every product. Some sub-nodes could be missing for some masters or variants

How to persist products

Once we have determinate the node structure of the product, it is time to create node structure and store values as node properties. As first, for that purpose, we need service user with write permissions in /var/commerce part. The best approach is to use sling API, with all methods to create resources. Here is one example to create product node with properties.

Map<String, Object> properties = new HashMap<>();
		properties.put("sling:resourceType", "commerce/components/product");
		properties.put("cq:commerceType", commerceType);
		properties.put("jcr:primaryType", "nt:unstructured");
		properties.put("price", 1000);
		properties.put("color", "green);
Resource newProduct = resolver.create(productsResource, "myFirstProduct", properties);

How to start product importer and follow the process

Product importer is triggered from AEM JMX console. Possible options are to trigger it manually or periodic as a cron job. It is a separate thread that does not return a result, so it is very hard to follow the process without server logs. But this is possible just for the developer. So, any solution? Yes, best practice would be during the implementation of the importer, when exception happens, that exception to be mailed to the responsible person, with time and the exact point where exception happens.

7 Websites Every Developer Should Know

Reading Time: 4 minutes

Programming is 10% writing code and 90% looking up how to do that.

We all know the scenes in Hollywood movies where a nerdy hacker sits in his poorly lit room, types seemingly random keys on his keyboard and a few seconds later the pentagon is hacked! In reality, coding looks very different. Countless times I had to look up simple stuff like “How to format a Java LocalDate” or “Rename git commit after push”.
I hope this blog post will introduce a few sites you didn’t know yet, to reduce your research time in future.


Let’s start with the elephant in the room:

stackoverflow.com

Stack Overflow is a question and answer site for professional and enthusiast programmers. It serves 120 million people every month, making it one of the 50 most popular websites in the world.
I’m pretty sure there’s a large number of developers spending more time on Stack Overflow than in their IDE.
Next to Stack Overflow, there are 172 more Q&A communities in the Stackexchange network. For example:


regex101.com

Working with regular expressions is hard and even if you exactly know what you’re doing, you might want to test your regex to verify that it is correct. Regex101 not only provides a regex tester but also a big library of community-created expressions for almost every use-case.


explainshell.com

Explainshell is a fairly unknown page that does exactly what it says. You can paste any Unix CLI command and the page explains in detail what every part of the command does. It helped me quite a lot in the past to get better at shell scripting.


mockaroo.com

Need some mock data to test your code? Mockaroo provides a simple way to generate a lot of realistic-looking Test-Data. You can choose between more than 140 different types of fake data. From email and IP-addresses to movie titles and many more. It provides also an API to get the test data directly from your code.


placeholder.com

The image above is not an error ;). It’s a placeholder image generated by placeholder.com! This page lets you create placeholder images in any resolutions and colours you like. Just open https://via.placeholder.com/900×512.png?text=placeholder.com and adjust the dimensions and text in the URL accordingly.


httpbin.org

httpbin.org is a simple website that provides REST-Endpoints for testing. For example:

  • https://httpbin.org/delay/x -> Waits for x seconds before responding
  • https://httpbin.org/status/x -> Returns a response with HTTP status code x
  • https://httpbin.org/ip -> Returns the public IP of the caller
  • …and many more! There are lots of other useful endpoints!

xkcd.com

Last but not least xkcd. “A webcomic of romance, sarcasm, math, and language”. While not necessarily a programming resource, bookmarking this page is almost required for every decent developer.

Do we love coding?

Reading Time: 6 minutes

a non-scientific try to investigate the passion between man and coding

Since you have managed to get to this blog, the mystical behavior of the internet is not completely foreign to you.
Things in your life such as your mobile phone, your washing machine, your TV, your car, your game console and much more are animated by magic of the same secret interaction and offer you comfort, information, benefit and joy.

🤖  Coding?  In order to make it all work, it requires instances that control the entire process of your inputs and their outputs to the (mostly) expected result. We exclude all imaginable forms of existence of the implementation of these seemingly complex processes and refer to the ordinary person known to us. To be precise, I would like to limit it to an ambitious and highly motivated employee, a software engineer, in our company.

  Love?  I’ll make it short … what we do is what we obviously can or should be good at. To put it even deliberately: That’s our job!
Isn’t it true that we only really succeed if we identify with it or feel emotionally connected? Yes, maybe I am exaggerating with this daring statement. So as not to go too deep into moral and ethical realms, we find that here we speak of an obviously great passion and affection.

so …

For outsiders, it really seems to have the appearance that we are here after cookie-cutter approach incorporate cryptic things in our machines, which then as a matter of course, e.g. can display a webpage on your device.
It seems unimaginable that the effort of making even a big, beautiful picture appear on the screen is really an enormously complicated matter. Also the click on a button and the following action is no big deal. And you are right! And not.

It is comparable to a cozy sports evening in front of the TV. We see all these athletes who make a lot of money with it, for example to chase a ball accurate, in a breakneck speed, on the tennis court and amuse us about balls that land in the net. Everything looks so easy and simple. In reality, if you try it yourself, you will first realize what a tremendous achievement is behind it and that it is not as all easy as it seemed to be. It is very very very much a lot of training, diligence, overcoming, will and experience behind it. Just like it is to generate code.

But there is also another aspect to consider. The artistic. Of course!

To generate code is actually somehow text-book or guidelines and standards to be followed (better yet). Nor should the geometric triangle be reinvented.
But as with an orchestral conductor, the art is now to bring together individual components that are obviously needed to get the desired result. Here, the nuances, interpretations, deviations and also new ideas set the tone.

The creative, innovative and courageous part makes it all. Even for problems that have not yet been solved satisfactorily or for this one special case, there is no solution … suddenly the creativity is in focus.
Of course, it requires the understanding and mastery of the tools necessary for this. The analytical and logical thinking. The art of composition leads to a comparable Mr.Spock & Cpt. Kirk relationship!

Now, looking at this, there seems to be more compelling reasons to ask “Do we love to generate code?” to agree in the affirmative. Or does it rather seem to be a community of convenience and thus a love-hate shape? It’s not just about typing, commonly as actual “coding”. People begin to rationally separate means and purpose.

You may know that too?
On the weekend you eat a really nice, maybe opulent, meal at a well-laid table; preferably with family and friends. (dishwashing not considered)

On the one hand, there is cooking and on the other, eating as an event. Both have emotionality.
The cooking is possibly comparable to coding. You prepare something. Different ingredients are put together or separately in the right condition and at the very end … it comes to the table, where even the consumers are waiting eagerly for it. The actual eating process is then, as well as I would almost say, meditative state to describe.

Anyway, …. finally, it is that this whole event has been divided into different phases aka love affairs. The chef and guest comparable to software engineer and internet user or service provider and customer.

We like to 
be your chef!

We gain more from our actions than just intrinsic satisfaction. Everything, except to bring you into rapture, is alien to us! We want that … we need that! That is our mission and passion.

Coding as a term of necessity.
The mechanical move of the fingers over the keyboard. The process routines such as e.g. to saving and committing the code. Do we love that (okay, some may have an obsession to it) or is it much more than only this?

If we imagine that we do that without achieving a result, then … yes, what then? I think that would be very unsatisfactory! But we don’t want to get into this condition!
We are more than that! We see ourselves as enablers. As a big hitter. As those who make your life a little bit easier, better, more enjoyable and more beneficial..

… we love it!