3 shell scripts to help you manage AEM Instances

Reading Time: 5 minutes

The Problem

Every AEM developers career starts by double-clicking the AEM jar file. And that’s where you can do your first mistake. AEM might shout at to you “Hey NOOB you need a license!”. You forgot to copy the license file next to the jar… *sigh*. But don’t worry, this happens to everybody.

After copying the license file, AEM will start and you will get a nice little GUI.

Congratulations! You are ready to go! You know where AEM is running (localhost:4502) and you even have a little stop button in the bottom left corner. But this is probably the only time you see this GUI. All the cool kids start AEM with the start script in crx-quickstart/bin/. But how do you know if AEM is running? How can you find out which port is used and how do you stop the instance?

This blogpost will provide you with three little shell scripts to address these problems and help you to manage your AEM instances without the need for a GUI.

#1 Get information about running AEM instances

To figure out which AEM instances are currently running you might smash something like this into your terminal window:

This method has several disadvantages. The command is hard to remember and the output is very hard to read.

Here’s a little challenge for you: Can you figure out the port, debug port and runmodes in the gif above before it loops? Give it a try.

And anyway, what does this command even mean?

Bonus Tip – Great page to explain shell commands (bookmark it!): https://explainshell.com/explain?cmd=ps+aux+%7C+grep+cq

Back to the topic. Our first script (“aeminfo”) provides a simple overview of all running AEM instances. It turns the complicated output into a readable form:

Nice! All you have to do is adding this script to your .bash_profile. The script does not use any external dependencies and should run many unix/linux based systems (tested on OSX and CentOS).

# Displays all running AEM instances
function aeminfo(){

    if [ "$(ps aux | grep q | grep crx)" ]
        then

            count=0;
            echo ""
            ps auxeww | grep q | grep sling | while read -r line ; do

                ((count++));
                params=($line);

                username=(${params[0]});
                pid=(${params[1]});
                port="not found";
                debugPort="not found";
                xmx="not found";
                runmodes="not found";
                path="not found";

                portRegex="-p ([0-9]+)";
                debugPortRegex="address=([0-9]+)";
                xmxRegex="-Xmx([^[:space:]]+)";
                runmodesRegex="-Dsling.run.modes=([^[:space:]]+)";
                pathRegex="PWD=([^[:space:]]+)";

                [[ $line =~ $portRegex ]] && port="${BASH_REMATCH[1]}";
                [[ $line =~ $runmodesRegex ]] && runmodes="${BASH_REMATCH[1]}";
                [[ $line =~ $debugPortRegex ]] && debugPort="${BASH_REMATCH[1]}";
                [[ $line =~ $pathRegex ]] && path="${BASH_REMATCH[1]}";
                [[ $line =~ $xmxRegex ]] && xmx="${BASH_REMATCH[1]}";

                echo "----------------------";
                echo "AEM Instance" $count;
                echo "----------------------";
                echo "username:  "$username;
                echo "pid:       "$pid;
                echo "port:      "$port;
                echo "debugPort: "$debugPort;
                echo "memory:    "$xmx;
                echo "runmodes:  "$runmodes;
                echo "path:      "$path;
                echo "----------------------";
                echo "";

            done

        else
            echo "";
            echo "No running AEM instances found";
            echo "";
        fi
}

#2 Death to all AEM instances!

Our next script (“killaem”) is pretty small. It just shuts down a running AEM instance:

function killaem() {
    kill $(ps aux | grep 'rx-quickstart' | awk '{print $2}')
}

But experienced AEM developers might know that asking AEM politely to stop is sometimes not enough.

Especially when it’s Friday evening and you’re looking forward to a cold beer in your local pub. For this emergency situation you could use a slight variation of the script (“KILLAEM”) (notice the capitalisation!). This will kill your instance definitely. No matter what. The consequences might be that the instance will be in the same condition as you are after too many beers in said pub. Broken and unable to work. So be careful with this command (and with drinking for that matter!)

function KILLAEM() {
    kill -9 $(ps aux | grep 'rx-quickstart' | awk '{print $2}')
}

#3 Improved debugging

So you destroyed your instance and it won’t start properly? What to do next? Have a look at your log files!

If you use the regular “tail -f error.log” command you will quickly notice that AEM logs a lot of stuff. How could you ever find that one useful log entry in the haystack of unnecessary INFO logs? This “aemlog” script might help you! It displays different log levels with different colours:

function aemlog() {
    tail -fn 128 "$@" | awk '
    /SEVERE/ {print "\033[35m" $0 "\033[39m"}
    /ERROR/ {print "\033[31m" $0 "\033[39m"}
    /WARN/ {print "\033[33m" $0 "\033[39m"}
    /DEBUG/ {print "\033[30m" $0 "\033[39m"}
    !/SEVERE|ERROR|WARN|DEBUG/ {print $0 }
';}
You could even observe multiple log files at once. Just pass them as parameters to the command: aemlog error.log access.log audit.log

Conclusion

The three little scripts might not do very much but they are easy to set up, don’t need any third-party software and provide some useful functionalities.

Just copy all shell scripts from this page into your ~/.bash_profile and you are ready to go!

If you have any suggestions for improvements feel free to leave a comment!

Editable Templates in AEM 6.5

Reading Time: 8 minutes

Editable templates have been introduced in AEM 6.2 and since then with each next version they are constantly improving. They allow authors to create and edit templates. Template authors can create and configure templates without help of development team. To be able to create and edit templates the authors must be members of the template-authors group.

Here are some of the benefits of using editable templates:

  • editable templates provide flexibility to define content policies to persist in design properties. There is no need for design mode which requires extra permission to give author to set design properties along with a replication of design page after making any design changes
  • it maintains a dynamic connection between pages and template which gives power to template authors to change template structure along with locked content which will be reflected across all pages based on the editable template
  • this doesn’t require any extra training for authors to create a new page based on an editable template. Authors can similarly create a new page as they create with static templates
  • can be created and edited by your authors
  • after the new page is created, a dynamic connection is maintained between the page and the template. This means that changes to the template structure are reflected on any pages created with that template (changes to the initial content will bot be reflected)
  • uses content policies (edited from the template author) to persist the design properties (does not use Design mode within the page editor)
  • are stored under /conf

Here are the tasks that the template author can do with the help of the AEM’s template editor:

  • create a new template or copy an existing template
  • manage the life cycle of the template
  • add components to the template and position them on a responsive grid
  • pre-configure the components using policies
  • define which components can be edited on pages created with the template

Create editable templates from the configuration browser

Go to Tools -> General -> Configuration Browser

It will create the basic hierarchy of templates in /conf directory.

There are three parts of template editor:

  • templates: all dynamic (editable) templates created by authors
  • policies: there are two types of policies
    template level policy: used for adding policies to the template
    component level policy: used for adding policies to the component level
  • template-types: base templates for the creation of new templates in runtime

There are three parts of a template:

  • initial: the initial content of the page created – based on the template. This content could be edited or removed by the author
  • policies: here a particular template is linked to a policy by using cq:policy property
  • structure:
    – the structure allows you to define the structure of the template
    – the components defined in the template level can’t be removed from the resulting page
    – if you want that template authors can add and remove components, add parsys to the template
    – components can be locked and unlocked to allow you to define initial content

How to create base template-type

To start working on editable templates, we need to create a page component. Navigate to /apps/47northlabs-sample-project/components/page and click on create component.

Next would be to create a template-type which helps the authors to create it’s editable templates. (Please note that the configuration is available on GitLab).

How can authors create editable templates

Next step would be to create the editable template. That can be done from Tools -> General -> Templates -> 47northlabs-sample-project -> Choose empty template.

Add template title, description and open the template. There should be a responsive grid available.

Configure policies

With defining policies, authors will be able to configure different components on the page for regular authors to place on the page. Since there is no defined policy on the template, no component is assigned to this template. Click on the first (Policy) icon will take the author to a new screen where the author can define what components can put on this template.

Create new policy.

Once done, components will be available on the page to drag & drop on the page.

Enable template

Once template authors are done with creating templates, authors must enable templates to make them available in sites section where regular authors can select templates to create pages.

Create editable templates from code

Another possibility is to We can create a sample project based on Adobe Archetype using the following command:

mvn archetype:generate -DarchetypeGroupId=com.adobe.granite.archetypes -DarchetypeArtifactId=aem-project-archetype -DarchetypeVersion=19

The generated sample project already contains content-page editable template by default. We are going to create three additional editable templates i.e.

  • home-page
  • landing-page
  • language-page

We are going to add some components as default in the templates, and few pages in ui.content project. The general idea is to have some test content and to play around with some corner cases. Let’s start.

The demo content is the like on the image below.

We have four editable templates available. With property cq:allowedTemplates we control the available templates that can be used to create child pages under a given page. For example, for the content-page we want to make available only the landing-page, so the initial content will look like:

<jcr:content
    ...
    cq:allowedTemplates="[conf/aem-editable-templates-demo/settings/wcm/templates/landing-page]">
    ...
 </jcr:content>

Similar, the initial content of the landing-page will have similar configuration:

<jcr:content
    ...
    cq:allowedTemplates="[conf/aem-editable-templates-demo/settings/wcm/templates/content-page]">
    ...
</jcr:content>

What happens in case we want to add a fancy new template which should be available only for a particular page types? These two solutions come to my mind:

  • write a groovy script that will update existing cq:allowedTemplates property of all the pages created for a given template with the new template
  • update the structure of the given template, so all the existing pages created with that template will be updated

With the second approach, for example, if we want to add that fancy page template to the content page template, we have to add the following configuration to the structure element of the content-page template:

<jcr:content
    ...
    cq:allowedTemplates="[conf/aem-editable-templates-demo/settings/wcm/templates/fancy-page]">
    ...
</jcr:content>

Given this, it is important to point out the differences between initial content and structure elements of the template definition.

Initial Content

  • is merged with the structure during page creation
  • changes to the initial content will not be reflected in all pages created with the given template
  • the jcr:content node is copied to any new page
  • the root node holds a list of components that defines what will be available in the resulting page

Structure

  • is merged with the initial content during page creation
  • changes to the structure will be reflected in all pages created with the given template
  • the cq:responsive node is responsible for the responsive layout
  • the root node defines the available components in the resulting page

Conclusion

With editable template, you give template authors the flexibility to create and modify the template as they want. It acts as a central place to manage everything about the template (structure, initial content, layout) and components used in the template. If you are migrating to use editable template, make sure you accessed the requirements for not only the template but also the components.

The code from this blog post is available on GitLab.

Exploring the headless CMS functionality of AEM 6.5

Reading Time: 5 minutes

Adobe Experience Manager (AEM) is one of the leading enterprise content management system (CMS), formerly knows as Day CQ. The initial version was introduced in 2002, at a time when web projects were mostly implemented as static, server-side rendered websites. Content as well as styling information were mixed up within the same HTML document. Nowadays traditional websites are being more and more replaced and complemented by (mobile) applications which come up with their own presentation layer. Thus there is a need for a more flexible approach that separates content from styling, and that is able to deliver the data to multiple channels in a universal format. This requirement led to the emergence of so called headless content management systems, which usually supply the data in a RESTful manner (as JSON) to their consumers. This blog post summarizes the headless CMS extension provided by AEM.

Headless CMS in AEM

The headless CMS extension for AEM was introduced with version 6.3 and has been continuously improved since then, it mainly consists of the following components:

  • Content Services: Provides the functionality to expose user-defined content through a HTTP API in JSON format. This allows to deliver data to 3rd party clients other than AEM.
  • Content Fragments: Allows the user to insert/edit content as structured data entities. The schema of each content fragment is defined by a corresponding Content Fragment Model.

Note that AEM follows a hybrid approach, e.g. content fragments can either be delivered as JSON through the content services API, or embedded within a traditional HTML page. Visit Adobe’s headless CMS website for further information.

Example Project

There is a tutorial provided by Adobe where the concept of content services is explained in detail. It describes how to model the entries of a FAQ list by using content fragments, and how to expose this data through a API as JSON. The complete article can be found here.

The example is based on the existing We.Retail demo project that comes with the installation file of AEM. In summary, the following steps have to be performed:

  • First content fragment models should be enabled for the We.Retail project. From the AEM welcome page, go to ToolsConfiguration Browser, open the properties of the We.Retail configuration and ensure that the Content Fragment Models property has been selected.
  • Navigate to Tools → Assets → Content Fragment Models → We.Retail to create or edit content fragment models. Select the FAQ model and click on the edit button to open the Content Fragment Model Editor. Here you can edit the model structure by adding/removing elements using drag and drop.
  • The model can be used to create new content fragments which contain the actual data . To do this, navigate to Assets → Files → We.Retail → English → FAQs → Company. There is already content available here: Each entry of the FAQ list is modeled as a single fragment. The picture below shows the editor view for a FAQ content fragment.
  • To access the data through content services from outside, the FAQ items have to be embedded within a content page. Content fragments can be added to the page by drag and drop in the same way as any standard content component.
  • The content of the page can now be delivered as JSON via the “model” selector. The code section below shows an extract of the response of the FAQ page /content/we-retail/language-masters/en/faq.model.json
...
":items": {
        "contentfragment": {
          "title": "The Company Name",
          "description": "",
          "model": "we-retail/models/faq",
          "elements": {
            "question": {
              "title": "Question",
              "dataType": "string",
              "value": "What's with the name \"We.Retail\"?",
              ":type": "string"
            },
            "answer": {
              "title": "Answer",
              "dataType": "string",
              "paragraphs": [
                ""
              ],
              "value": "<p>We're not sure, but it sounds good!<br>\n</p>\n",
              ":type": "text/html"
            }
          },
          ":items": {},
          "elementsOrder": [
            "question",
            "answer"
          ],
          ":itemsOrder": [],
          ":type": "weretail/components/content/contentfragment"
        },
        "contentfragment_1811741936": {
          "title": "History",
...
  • Finally the data can now be consumed by any 3rd party application. The screenshot below shows an example made with vue.js, where the FAQ list is loaded from AEM content services by XHR request.

Conclusion

AEM content services provide a flexible way to deliver structured content to multiple channels, the data as well as its corresponding schema can be created and modified without any need for a deployment. However, the main focus of AEM is still on the authoring of backend-side rendered websites, but content services may be a good addition for environments where AEM is already in use.

Project story: Automate AEM deployments for a Swiss bank

Reading Time: 5 minutes

A large bank in St. Gallen, Switzerland had the need to improve the AEM deployment process for its various staging environments. It was one of my first projects for 47 North Labs and was settled to run for 6 months starting in October 2018. The following blog gives a short project overview.

Getting started

Starting to work for a new customer is always exciting to me because every company and team has a unique mindset and culture. Usually, it takes a few days or weeks to get to know the new teammates. But this time it was completely different, as I already worked with each of the three team members and their supervisor together in one before my previous company. It was nice to meet old colleagues and we had a very good start.

Deployment process before automation.
Source: www.dreamstime.com

Technology Stack

The technology stack was already defined and the servers ordered. But it took a while until the infrastructure was ready and for the time being, I worked on my local machine.

Jenkins was set as the central tool for build orchestration, deployments, and various DevOps tasks. All the pipeline source code is stored in GitLab and the main business application we’re dealing with is Adobe Experience Manager (AEM).

A relatively large amount of work was needed for the initial setup like enabling connectivity to the relevant systems, basic shared library, and getting to know the internal processes. Read more about Jenkins behind a corporate proxy as an example for this setup: https://www.north-47.com/knowledge-base/update-jenkins-plugins-behind-a-corporate-proxy/

Implemented Pipelines

The bank has two different AEM projects: one for the corporate website and another for their intranet. They require a slightly different deployment pipeline and both have three environments: development, staging, and production.

Besides the deployment pipelines, there are pipelines for copying content from the production to the development environment and restoring a complete production environment into the staging environment in order to have an exact copy and a good baseline for approvals.

Many auxiliary jobs like start/stop AEM + Dispatcher, checking the health of instances, fetch last backup time, and execute Groovy scripts are used in the deployment pipelines as well in an independent executable job.

An example of a Jenkins Pipeline
Source: https://jenkins.io
An example of a Gitlab Pipeline
Source: https://docs.gitlab.com/ee/ci/pipelines.html

Advantages of automation

The automation of the various processes brought faster deployments. But more important transparency and centralized logs about what exactly happened and higher quality as repetitive tasks are always executed.

One example is the backup check, which needed coordination and forced to long waiting times. Now an API is used and the automation pipeline has instant feedback about the last backup time and shows a note if a backup is missing. Before, such a step might have been skipped in order to save some time.

With each built pipeline, some more little and reusable helpers were introduced which made it then again easier and faster to create the next pipelines. Think of a construction kit.

Deployment process with automation.
Source: www.wikimedia.org

Project finished ➞ client is very happy

After several months of close collaboration, more and more pipelines have been implemented and are used to support the crucial deployment processes countless times.

I enjoyed building-up the AEM automation and believe it’s a very good aid for higher quality and further extensions.

After a warm welcome and six months of working together, it was time to say good-bye as the project had a fixed time span. The client’s team was very kind and gave me even some great presents to remember the exciting time in St. Gallen.

Present from client: Swiss beer, chocolate, bratwurst and biber

How to get a service reference or BundleContext with no OSGi context

Reading Time: 2 minutes

Issue

In Adobe Experience Manager (AEM) projects developers are working a lot in services, filters, servlets and handlers. All of these classes are OSGi components and services using the Felix SCR annotations or the newer OSGi DS annotations. But sometimes you need an OSGi service or the BundleContext also in non OSGi / DS controlled class

Solution

You can use the OSGi FrameworkUtil [1] to get the reference to the bundle context from any object. The code below shows how to get reference to the BundleContext and the service.

Most of the time, you have the SlingHttpServletRequest ready to pass:

import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.scripting.SlingBindings;
import org.apache.sling.api.scripting.SlingScriptHelper;
import org.osgi.framework.BundleContext;
import org.osgi.framework.FrameworkUtil;
import org.osgi.framework.ServiceReference;

import java.util.Objects;

public class ServiceUtils {

    /**
     * Gets the service from the current bundle context.
     * Return null if something goes wrong.
     *
     * @param <T>     the generic type
     * @param request the request
     * @param type    the type
     * @return        the service
     */
    @SuppressWarnings({"unchecked", "rawtypes"})
    public static <T> T getService(SlingHttpServletRequest request, Class<T> type) {
        SlingBindings bindings = (SlingBindings) request.getAttribute(SlingBindings.class.getName());
        if (bindings != null) {
            SlingScriptHelper sling = bindings.getSling();

            return Objects.isNull(sling) ? null : sling.getService(type);
        } else {
            BundleContext bundleContext = FrameworkUtil.getBundle(type).getBundleContext();
            ServiceReference settingsRef = bundleContext.getServiceReference(type.getName());

            return (T) bundleContext.getService(settingsRef);
        }
    }
}

Or you just use the class that was loaded over a bundle classloader.

package com.sanitas.aem.core.utils;

import org.osgi.framework.Bundle;
import org.osgi.framework.BundleContext;
import org.osgi.framework.FrameworkUtil;
import org.osgi.framework.ServiceReference;

public class ServiceUtils {
    
    /**
     * Gets the service from the current bundle context.
     * Return null if something goes wrong.
     *
     * @param <T>     the class that was loaded over a bundle classloader.
     * @param type    the service type.
     * @return        the service instance.
     */
    @SuppressWarnings({"unchecked", "rawtypes"})
    public static <T> T getService(Class clazz, Class<T> type) {
        Bundle currentBundle = FrameworkUtil.getBundle(clazz);
        if (currentBundle == null) {
            return null;
        }
        BundleContext bundleContext = currentBundle.getBundleContext();
        if (bundleContext == null) {
            return null;
        }
        ServiceReference<T> serviceReference = bundleContext.getServiceReference(type);
        if (serviceReference == null) {
            return null;
        }
        T service = bundleContext.getService(serviceReference);
        if (service == null) {
            return null;
        }
        return service;
    }

}

[1] https://osgi.org/javadoc/osgi.core/7.0.0/org/osgi/framework/FrameworkUtil.html