Qlik Sense: Creating Your Own Narratives


Documentation for Arria NLG’s Extension for Qlik Sense Desktop


This documentation is organized as follows:

Overview

Create a custom narrative

Download and install Arria’s extension and sample Qlik Sense report

Add dimensions and measures

Configure your narrative

Create, modify, and publish a project in Arria NLG Studio

Link your published NLG Studio project to your Qlik Sense report

Using mapping scripts

Optimizing the narrative to handle filters

Contextualizing the narrative

Going further

Other documentation

 

Overview

This guide explains how to get and use Arria NLG’s extension for Qlik Sense Desktop.

You’ll begin by importing Arria’s extension and a sample Qlik Sense report. Next, using Arria’s extension, you’ll add a narrative widget to the report. After that, you’ll provide some information about your data and the kind of things you would like to say about it in a narrative.

Then, using NLG Studio, you’ll create a very simple NLG project, which you’ll publish to the cloud. Then, you’ll link the published Studio project to the narrative widget to bring its NLG into the report.

By moving through the steps of creating a very simple NLG project, you’ll get a glimpse of the amazing things you can do when you extend Qlik Sense with Arria’s NLG technology.

 

Create a Custom Narrative

In this section, you’ll get a sample report, then import Arria’s extension into it and add some data attributes.

Download and install Arria’s extension and sample Qlik Sense report

1. Download the Arria Qlik Sense extension.
2. Extract the ZIP file.
3. Copy the extracted folder (ArriaQlikSense2.0) to the Extensions folder in your Qlik Sense installation folder (\Qlik\Sense\Extensions).
4. Download this sample Qlik Sense report and copy it to \Qlik\Sense\Apps.
5. In Qlik Sense, open the report (Quick Start Project).

6. In the toolbar, click Edit.
7. In the assets panel, click Custom objects.

Under Extensions, you should see the Arria NLG extension.

8. Click the Arria NLG extension, then click the Add to sheet icon.
The Arria widget is added to the sheet:

Now you’re ready to add some dimensions and measures.

 

Add dimensions and measures

For this exercise we’ll add the dimensions Product, Segment, and Year. We’ll also add the measure Sales, with aggregation type Sum(Sales), and the measure Profit, with aggregation type Sum(Profit).


1. Click Add dimension, then select Segment.
2. In the properties panel at the right, click Dimensions.

You should see Segment listed.
3. Click Add dimension, then select another dimension: Product.
4. Do the same to add the dimension Year.
IMPORTANT: Make sure you select the dimension Year at the bottom of the Dimensions list (scroll down). Do not select Year under the Date hierarchy.
5. Now add some measures. In the properties panel, click Measures.
6. Add the measure Sales.
a. In the Sales row, click the arrow.

b. Choose the aggregation type Sum(Sales).

7. Add the measure Profit, and again choose the Sum aggregation type: Sum(Profit).

 

Configure your narrative

Now you can do the set-up in the Arria extension to bring in your custom report.

1. In the Arria widget, select the Create Custom Narrative tab and you should see the following:

2. Because you’ve already selected the data you’re working with in Qlik Sense (the fields Product, Profit, Sales, Segment, Year), you’re ready to bring that specific set of data into NLG Studio. Note: Expand the widget to full screen if you want to get a better look at the data you selected in JSON format.

3. In the Arria NLG widget, make sure the Simple/Advanced toggle is set to Simple.

4. Make sure the Studio Project Type is set to Describe a JSON Object. (This is because the NLG Studio starter project you are going to download is a JSON-type Studio project. If the Studio project you associate with your dashboard is based on CSV data, you would choose Describe a Table here.)

5. In the View Dataset in JSON section, at the top right, click the download icon to download the data.

6. In the dialog, click Save to save the data file.

7. Check your machine’s Download folder for the file called data.json. In the next section, you’ll import this into the BI Starter project in NLG Studio.

 

Create, modify, and publish a project in Arria NLG Studio

You are going to create a very simple sample project in NLG Studio in order to get a basic understanding of how you can use NLG Studio to augment your analytics.

1. Download the BI Starter project for Qlik Sense.
2. Log in to NLG Studio.
3. In NLG Studio, in your dashboard, click the Import a Project icon to import the BI Starter Project JSON file.

4. In the NLG Studio dashboard, click the starter project to open it.

Note: In the future, this project may become your starting point for all your BI projects created in NLG Studio. So if you like, you can rename the project to something generic. Or, you might want to give it a name that matches the name of the corresponding .QVF file. To change the project’s name, just click in the name at the top left of the screen.

You’ll see the data that comes with the starter project. When you’re ready to modify this starter project, you’re going to replace this data with the data from your field selections in Qlik Sense.

5. In the left navigation ribbon, click Compose to access the Compose editor.

In the editor, you’ll see the contents of the Main script, which is empty aside from a reference to a ReadMe sub-script. This ReadMe describes the BI data format used in Qlik projects, and it provides information about the helper functions in the project. The helper functions appear as a list of sub-scripts under the Main script. You can click on the sub-scripts to see their contents (the user-defined helper functions).

6. Make sure you’re in the Main script, then at the top right of the screen, click Preview. Read the ReadMe information.

In the next section, you’ll make some changes to this starter project.

 

Modify the starter project

You are going to compose a piece of narrative describing the total sales and total profit for the selected period. To generate the text for that, you’ll add some code in the Compose editor. But first, you need to import the data set that is specific to the fields you have selected in Qlik Sense: Product, Profit, Sales, Segment, Year.

1. In the left navigation ribbon, click Data to go to the Data view.
2. At the top right of the screen, click the import icon to bring in the file called data.json from your Downloads folder.

3. Click Compose to go to the Compose view.
4. Click Main to begin editing the Main script.
5. Remove the text that is already there ([[ReadMe]]).
6. Copy/paste the following:

For the [[inflectNoun('year',len(getUniqueValues(data,'Year')))]] [[getUniqueValues(data,'Year')]], the total sales were [[formatCurrency(columnAggregate(data,'Sales','sum'),'USD','#.00','')]] and the total profit was [[formatCurrency(abs(columnAggregate(data,'Profit','sum')),'USD','#.00','')]].

7. In the top right of the screen, click Preview.
Your previewed output should look like this:

Now add another piece of narrative. You’ll add a sentence reporting on the sectors that contributed the most and least profit. As an exercise, you’ll create some in-script variables that you’ll use to replace some of your existing code.

8. In Main, replace the existing script with the following:

[[global.currency='USD';
years=getUniqueValues(data,'Year');
totalSales=columnAggregate(data,'Sales','sum');
totalProfitLoss=columnAggregate(data,'Profit','sum');
profitLossbySegments=groupBy(data,'Segment','Profit','sum');
topSegmentbyProfit=top(orderDesc(profitLossbySegments),1)[0];
if (currency !=null){For the [[inflectNoun('year',len(years))]] [[years]], the total sales were [[formatCurrency(totalSales,currency,'#.00','')]] and the total [[direction(totalProfitLoss ,'profit','loss' )]] was [[formatCurrency(abs(totalProfitLoss),currency,'#.00','')]]. The [[printKey(topSegmentbyProfit)]] segment reported the most [[direction(topSegmentbyProfit[1], 'profit','loss')]]: [[formatCurrency(abs(printValue(topSegmentbyProfit)),currency,'0.##','')]].}]]

9. Click Preview.
Your previewed output should look like this:

10. Now add another sentence to your narrative, this time reporting on the loss made in some segments. Compose this sentence in a new script. Below the list of sub-scripts, click Add Script.

11. At the bottom of the list of scripts, overwrite the NewScript name with the name DescribeContrast.

12. In this new sub-script, copy/paste the following:

#define DescribeContrast(category,currency)[[salesForSegment=columnAggregate(filterOn(data,'Segment','==',category[0]),'Sales','sum');
if (category[1]<0){Even though the [[category[0]]] segment posted [[formatCurrency(salesForSegment,currency,'0.##','')]] sales, it reported a loss of [[formatCurrency(abs(category[1]),currency,'0.##','')]].}]]

In order to call the new sub-script, you need a new variable that represents the segment with the lowest profit (or highest loss). You can create an in-script variable in Main to do this.

13. Go back to the Main script and overwrite what you have there with the following code (which contains the new variable bottomSegmentbyProfit and the call to the new sub-script DescribeContrast):

[[global.currency='USD';
years=getUniqueValues(data,'Year');
totalSales=columnAggregate(data,'Sales','sum');
totalProfitLoss=columnAggregate(data,'Profit','sum');
profitLossbySegments=groupBy(data,'Segment','Profit','sum');
topSegmentbyProfit=top(orderDesc(profitLossbySegments),1)[0];
bottomSegmentByProfit=top(orderAsc(profitLossbySegments),1)[0];
if (currency !=null){For the [[inflectNoun('year',len(years))]] [[years]], the total sales were [[formatCurrency(totalSales,currency,'#.00','')]] and the total [[direction(totalProfitLoss ,'profit','loss' )]] was [[formatCurrency(abs(totalProfitLoss),currency,'#.00','')]]. The [[printKey(topSegmentbyProfit)]] segment reported the most [[direction(topSegmentbyProfit[1], 'profit','loss')]]: [[formatCurrency(abs(printValue(topSegmentbyProfit)),currency,'0.##','')]]. [[DescribeContrast(bottomSegmentByProfit,currency)]]}]]

14. Click Preview.
You should see the following:

Next, you’ll publish the project.

 

Publish the NLG Studio sample project

1. In NLG Studio, in your BI Starter Project, click Publish in the left navigation ribbon.
2. In the Publish view, click the Publish button.
3. At the Congratulations! message, make note of the URL for your project. (From inside the URL box, you can click the copy icon to copy the URL to your clipboard. Paste it to a notepad.)
4. Under Run Project, click API.
5. Scroll down, then click the Generate API Key button.
6. Make note of the API key generated. (Again, you can copy it to your clipboard from inside the box with the key. Paste it to a notepad.)

 

Link your published NLG Studio project to your Qlik Sense report

1. In Qlik Sense Desktop, select the Create Custom Narrative tab in the Arria widget. Expand the widget to full screen.

2. In the first empty field in the widget, enter the NLG service URL (the URL you made note of after publishing your NLG Studio project).

3. In the second empty field in the widget, paste in your API key.

4.  Minimize the widget and click the Generate Text button at the bottom of the widget:

5. Make sure you click the Done (or Stop editing sheet) button. Then you can click around on different pieces of the charts to see how the narrative responds. To select multiple bars at the same time, use SHIFT+Click.

6. (Optional) You can copy the narrative for pasting elsewhere by clicking the copy icon in the top right of the widget.

 

 

Using Mapping Scripts

With Arria’s extension, you can add custom JavaScript to modify the input data set — right inside the narrative widget. You do this in the Edit Mapping Script section.

About the Methods Used:

getData(): This method returns the underlying JSON data (the data you see in the View Dataset in JSON section).
setData(appSpecificData): This method sets the appSpecificData variable (the transformed data object). After using this method, the transformed data object is available for download and posting to the NLG service URL.

The following example shows how you can add some metadata to the input JSON data object:
var data=getData();

var metadata={};

metadata["currency"]="GBP";

var dataset=data.dataset[0];

dataset["metadata"]=metadata;

var datasets=[];

datasets.push(dataset);

data["dataset"]=datasets;

setData(data);

To try out a custom mapping script:
1. Copy the above script and paste it in the Edit Mapping Script section of the narrative widget.

2. In the View Dataset in JSON section, at the top right, click the download icon to download the data. This time you’ll get the sample data again, but with your customization.
If you examine the downloaded data, you’ll see the additional information: ("metadata":{"currency":"GBP"})

3. Click Open with Studio to open your BI Starter project.

4. Import the modified sample data in to the Studio project. (This is the data you modified using the custom mapping JavaScript.)
a. In the left navigation ribbon, click Data to go to the Data view.
b. At the top right of the screen, click the Import new sample data icon.
c. Browse, or drag and drop, to import the modified data file from your Downloads folder.
d. Click Continue.
In the Data view, you should see the new "currency": "GBP" metadata.

5. Now assign the currency metadata to a variable called currency. You can do this easily as an in-script variable.
Replace the first line of the current Main script with the following:
[[global.currency=WholeJSON['dataset']['0']['metadata']['currency'];

6. From the Main script, click Preview.

7. Now re-publish the Studio project. The URL endpoint and the API key remain the same, so you don’t need to update anything more in the configuration window of the narrative widget in your Qlik Sense report.

8. Go back to Qlik Sense Desktop. Refresh the narrative (click Generate Text in the Arria widget).

You should see the currencies expressed in sterling (£) instead of dollars (US$).

 

Optimizing the Narrative to Handle Filters

At this point, in your Qlik Sense report, if you click on a single sector within a graph, the narrative still describes the sector with the most profit or loss. But this information is redundant because the selected data represents only a single sector.

To see an example of this redundancy, try this:
1. In the top graph, click to select all the bars in the Government segment like this:

2. Now check out how the narrative changes.

Do you see the second sentence? It’s reporting on the greatest profit by segment, which happens to be the Government segment. But because we’ve selected only the Government segment, we’re already getting the profit figure for that segment in the first sentence. We don’t need the second sentence. The next exercise will show you how to filter that out.

Note: By removing that second sentence, we are removing only one redundancy. The narrative could be further optimized. But we’ll leave that for another time.

So, focusing on one fix here, you can filter out that redundant sentence — the sentence describing the segment with the most profit or loss — when there is only one segment selected.

To filter out the sentence when it is irrelevant:
1. Locate the following sentence in your Main script:

The [[printKey(topSegmentbyProfit)]] segment reported the most [[direction(topSegmentbyProfit[1], 'profit','loss')]] of [[formatCurrency(printValue(topSegmentbyProfit),currency,'0.##')]].

2. Add the following if statement surrounding the sentence (include the curly brackets and replace <above script> with the sentence above):

[[if (len(profitLossbySegments)>1){ <above script> }]]

Note: If you preview the narrative at this point, you’ll notice that the sentence still appears in the narrative within NLG Studio. This is because Studio is working with the data it currently has, not the data that comes from the selection of only the Government segment in Qlik Sense. To see that this redundant sentence has been removed, you need to re-publish the Studio project and then go back to Qlik Sense.

4. Re-Publish the Studio project again.
Now, in Qlik Sense, you should see that the redundant sentence does not appear in the narrative when you select the Government segment.

 

Contextualizing the Narrative

To make the narrative more responsive to selection events within visuals, we’re going to add two prepositional phrases to the first sentence, so we can describe narrowed-down selections in segments and products. Currently, the first sentence states For the years <years>, the total sales were <sales total> and the total loss was <loss total>. To report on narrowed-down selections, we’re going to add these two new phrases after For the years <years>: in the segment <segment name> and for the product <product name>. An example sentence would look like this (new phrases in bold):

For the years 2013 and 2014, in the segment Enterprise, for the product Montana, the total sales were £2.61MM and the total loss was £31.10K.

1. First, create two new sub-scripts, one called SegmentPhrase and the other called ProductPhrase:

#define SegmentPhrase()[[
segment=getUniqueValues(data,'Segment');
if (len(segment)<=2) {in the [[inflectNoun('segment',len(getUniqueValues(data,'Segment')))]] [[getUniqueValues(data,'Segment')]],}
]]


#define ProductPhrase()[[
product=getUniqueValues(data,'Product');
if (len(product)<=2) {for the [[inflectNoun('product',len(getUniqueValues(data,'Product')))]] [[getUniqueValues(data,'Product')]],}
]]

2. In the Main script, find the year phrase ( [[if (currency!=null){ For the [[inflectNoun('year',len(years))]] [[years]], ...}), and add calls to those two sub-scripts after the comma. The calls to the sub-scripts are shown in bold here:

[[if (currency!=null){ For the [[inflectNoun('year',len(years))]] [[years]], [[SegmentPhrase()]] [[ProductPhrase()]] the total sales were ...}

3. Preview the output. You won’t see the new phrases because they only show up with narrowed-down selections, but as long as you don’t get any errors, the code will work.

4. Re-Publish the project, and try it out in Qlik Sense Desktop. Below are examples of the results you should now see.

With all products in the Government segment selected:

With only the product Paseo selected, in the Government segment only:

 

Going further

Familiarize yourself with the existing NLG Studio code and try out the Getting Started tutorials. Once you are confident, try to produce the following sentence to add to your starter project narrative. The added sentence describes the sectors having the highest and lowest profitability, for example:

The most profitable segment is Channel Partners (73%) and the lowest is Enterprise (-3%).

If you need help, here is the code for the new sub-script (Profitability):

#define Profitability() [[
sales=groupBy(data,'Segment','Sales','sum');
profit=groupBy(data,'Segment','Profit','sum');
profitabilityBySegments=sort(map(sales,x->(x[0], percentage(filter(profit,y->y[0]==x[0])[0][1],x[1],0,true))), (p,q)->p[1]<=q[1]?1:-1); highestProfitable=profitabilityBySegments[0]; lowestProfitable=profitabilityBySegments[len(profitabilityBySegments)-1]; if (len(profitabilityBySegments)>1){The most profitable segment is [[highestProfitable[0]]] ([[highestProfitable[1]]]%) and the least is [[lowestProfitable[0]]] ([[lowestProfitable[1]]]%).}]]

Here is the call to the new sub-script, which you can place at the end of the paragraph in your Main script:
[[Profitability()]]

Other documentation for Arria’s extension to Qlik Sense

For more information, see:

Configuring Your Narrative

Extending narratives

Using drilldown with narratives

Arria’s showcase project for Qlik Sense

Full list of documentation