TIBCO Spotfire: Creating Custom Narratives – Table-Type Studio Projects


Documentation for Arria NLG’s Add-in for TIBCO Spotfire Analyst


 

This document is about BI integration with Table-type Studio projects. If you are working with JSON data see Creating Custom Narratives – JSON-Type Studio Projects.

Creating a custom narrative

Add Arria’s custom visual to a sample Spotfire analysis
Configure a narrative to describe the visuals
Create a project in Arria NLG Studio
Modify the Studio project
Publish the Studio project
Link your published Studio project to your Spotfire analysis

Using mapping scripts

Other documentation for Arria’s add-in for TIBCO Spotfire Analyst

 

Creating a Custom Narrative

This guide explains how to customize a starter project using Arria NLG’s add-in for TIBCO Spotfire and Arria NLG Studio. You’ll begin by downloading a sample Spotfire analysis. Next, you’ll import Arria’s custom visualization into the report. After that, using Arria’s custom visualization, you’ll add a narrative wizard to the report, in which you’ll provide some information about your data and the kind of things you would like to say about it.

Then, using NLG Studio, you’ll customize a very simple NLG JSON-type project, which you’ll publish to the cloud. By moving through the steps of customizing an NLG project, you’ll get a glimpse of the amazing things you can do when you extend Spotfire with Arria’s NLG technology.

Finally, you’ll link the published Studio project to the narrative wizard to bring its NLG into the report.

Note: To learn about creating NLG Studio projects from scratch and writing your own scripts, see the Arria NLG Studio documentation and tutorials.

Prerequisites

 

 

Add Arria’s custom visual to a sample Spotfire analysis

1. Download the Customizing Narratives sample Spotfire analysis (.DXP file).

2. In Spotfire, open the sample analysis.

3. In the Authoring bar, click Visualizations > Add visualization….

4. Select ArriaNLG at the bottom of the Visualization types… list.

5. Move and resize the Arria add-in to suit your screen.

 

 

Configure a narrative to describe the visuals

1. In the ArriaNLG add-in, click on the sliders icon to begin selecting data for your narrative.

2. From Dimensions, select SegmentProduct and Year.

3. From Measures, select Sales (not Sale Price or Gross Sales), Profit.

Now that you’ve selected the data you wish to describe, you’re ready to bring that subset of data into NLG Studio.

4. Click the Create Custom Narrative tab.

5. Make sure the Simple/Advanced toggle is set to Simple.

6. Make sure the Studio Project Type is set to Describe a Table. (This is because the NLG Studio starter project you are going to download is a table-type Studio project. If the Studio project you associate with your dashboard is based on JSON data, you would choose Describe a JSON Object here.)

7. Click on the Maximize Visualization icon.

8. Click on View Dataset in CSV to view your data selection in CSV format.

9. Click on View CSV Payload to view the data object you are sending as part of the API call to the Studio project you will publish later.

10. Consider whether you need the Add auto increment ID field to be selected or not.

Important: Describe the Table projects in NLG Studio require you to have a unique field as the first column. To accommodate this requirement, the Arria add-in by default adds an auto-increment field to the data as the first field. If your data already has an ID column and you want to use that instead of the default auto-increment field, de-select Add auto increment ID. If you do use your own ID column, make sure that in the Fields pane, it is the first column you select so the unique column is in the first position of the table data brought into NLG Studio.

11. In the View Dataset in CSV section, at the top right, click the download icon to download the data.

12. Check your machine’s Downloads folder for the file called data.csv. In the next section, you’ll import this into the BI Starter project in NLG Studio.

 

 

Create a project in Arria NLG Studio

You are going to create a very simple sample project in NLG Studio in order to get a basic understanding of how you can use NLG Studio to augment your analytics. (For instructions that show how to create a more elaborate narrative, see the NLG Studio documentation.)

1. Download the Customizing Narratives sample NLG Studio project. This BI starter project contains some helper functions that you’ll see as sub-scripts under the Main script in the Compose view. These helper functions are user-defined functions created specifically for working with BI data.

2. Check your machine’s Download folder for the CustomizingNarratives_TableData.zip file and extract the JSON file from the ZIP.

3. Log into NLG Studio.

4. In NLG Studio, in the Project Portal, click the Import a Project icon to import the extracted JSON file.

5. Rename the project if required.

Note: This project may become the starting point for all BI projects you create in NLG Studio in future. You may wish to rename the project to something generic, or to give it a name that matches the name of the corresponding .DXP analysis file. To change the name of the project, just click in the name field and start typing.

6. Click the sample project to open it.

You’ll see the data that comes with the starter project. When you’re ready to modify this starter project, you’re going to replace this data with the data from your field selections in Spotfire. These helper functions are user-defined functions created specifically for working with BI data.

7. In the left navigation ribbon, click Compose to go to the Compose view.

In the editor, you’ll see the contents of the Main script, which is empty aside from a reference to a ReadMe sub-script. This ReadMe describes the BI data format used in Spotfire projects, and provides information about the helper functions in the project that appear as a list of sub-scripts under the Main script.

8. Make sure you’re in the Main script, then at the top right of the screen, click Preview. You’ll see a preview of the information contained in the ReadMe in the Output Text section.

In the next section, you’ll make some changes to this starter project.

 

Modify the Studio project

You are going to compose a piece of narrative describing the total sales and total profit for the selected period. To generate the text for that, you’ll add some code in the Compose editor. But first, you need to import the data set that is specific to the fields you have selected in MicroStrategy: Product, Segment, Year, Profit, Sales.

1. In the left navigation ribbon, click Data to go to the Data view.

2. At the top right of the screen, click the import icon to bring in the file called data.json from your Downloads folder.

3. Click Compose to go to the Compose view.

4. Click Main to begin editing the Main script.

Note: When working through these tutorials, it is best to activate Plain Paste Mode before editing scripts in NLG Studio. This ensures that any HTML formatting (such as text color) is removed from copied text before you paste it. Click on the the clipboard icon in the editor ribbon to toggle between Formatted Paste Mode and Plain Paste Mode.

Studio remembers the last paste mode you selected.

5. Remove the text that is already there ([[ReadMe]]).

6. Copy/paste the following:

For the [[inflectNoun('year',len(unique(Year)))]] [[unique(Year)]], the total sales were [[formatCurrency(totalVal(Sales),'USD','#.00')]] and the total [[direction(totalVal(Profit) ,'profit','loss' )]] was [[formatCurrency(abs(totalVal(Profit)),'USD','#.00')]].

7. In the top right of the screen, click Preview. Your previewed output should look like this:

Now add another piece of narrative. You’ll add a sentence reporting on the sectors that contributed the most and least profit. As an exercise, you’ll create some in-script variables that you’ll use to replace some of your existing code.

8. In the left navigation ribbon, click Variables then +NEW VARIABLE to add a variable called “currency” of type STRING and source USD, as shown here:

9. Click Done to save the new variable.

10. In Main, replace the existing script with the following:

For the [[inflectNoun('year',len(unique(Year)))]] [[unique(Year)]], the total sales were [[formatCurrency(totalVal(Sales),currency,'#.00')]] and the total [[direction(totalVal(Profit) ,'profit','loss' )]] was [[formatCurrency(abs(totalVal(Profit)),currency,'#.00')]]. The best-selling product was [[getKey(topNInstancesbyDimension(Product,Sales,1)[0])]] with a total of [[formatCurrency(getValue(topNInstancesbyDimension(Product,Sales,1)[0]),currency,'#.00')]]. The [[getKey(topNInstancesbyDimension(Segment,Profit,1)[0])]] segment reported the most [[direction(getValue(topNInstancesbyDimension(Segment,Profit,1)[0]), 'profit','loss')]] ([[formatCurrency(getValue(topNInstancesbyDimension(Segment,Profit,1)[0]),currency,'#.00')]]).

11. Click Preview. Your previewed output should look like this:

 

Publish the NLG Studio sample project

Once you’ve built your Arria NLG Studio project and tested it with your sample data, you can publish it as an application. Before it is published, you can only run the application via the Preview mode in the NLG Studio interface. After it is published, it becomes a real application accessible in the cloud. You can continue to refine your application within NLG Studio, but the published version is fixed until you republish a new version.

To call your published project from outside Studio you will need to create an API endpoint (URL) and an API key (authentication key), as shown in the steps below.

See the NLG Studio documentation if you would like to learn more about deploying NLG Studio applications and using the NLG Studio applications API. However, it is not necessary to have an in-depth understanding in order to use the Arria add-in for Spotfire.

Publish the NLG Studio project and generate an API key

1. In your NLG Studio project, click Publish in the left navigation ribbon to go to the Publish view.

2. In the Publish > API tab, click the Publish button.

3. Make a note of the project URL. (From inside the URL box, you can click the copy icon to copy the URL to your clipboard. Paste it to a notepad.)

4. In the API Key section, click the Generate API Key button.

5. Make note of the API key generated. (Again, you can copy it to your clipboard from inside the box with the key. Paste it to a notepad.)

NOTE: The NLG Service URL and API key provided here are only examples; they will not work in practice. You must use your own Studio project URL and API key.

 

 

Link your published NLG Studio project to your Spotfire Analysis

Now you’re ready to configure the Arria add-in to use your newly published Studio project.

1. Back in Spotfire, in the first empty field in the Arria add-in, enter the published Studio project URL you made a note of earlier.

2. In the second empty field, paste in your API key.

3. Minimize the Arria extension window.

4.  Click the Generate Text button. You should now see your customized narrative.

5. Click around on different pieces of the charts and use the filters to see how the narrative responds. To select multiple bars at the same time, use CTRL+Click.

6. You can copy the narrative for pasting elsewhere (e.g. into Microsoft PowerPoint or Word) by clicking the copy icon in the top right of the widget.

 

 

 

Using Mapping Scripts

With Arria’s add-in, you can add custom JavaScript to modify the input data set — right inside the narrative widget. You do this in the Edit Mapping Script section.

About the Methods Used:

  • getData():
    This method returns the underlying table data (the data you see in the View Dataset in CSV section).
  • addMetaData(appSpecificData,key,value);:
    This method appends a key-value pair into the metadata object of the dataset (appSpecificData).
  • setData(appSpecificData):
    This method sets the appSpecificData variable (the transformed data object). After using this method, the transformed data object is available for download and posting to the NLG service URL.

The following example shows how you can assign some variable values (var data=getData() ;) to the input table data object that can be used in the narrative script:

var data=getData();
data.vars['currency']='GBP';
setData(data);

Note: In the above script, we add the "currency" : "GBP" key-value pair as metadata. 
However, with addMetaData, you can add any key-value pairs that you want to pass to the Studio project, for example a threshold as shown here: 

var data=getData();
addMetaData(data,”currency”,”GBP”);
addMetaData(data,”threshold”,5.3);
setData(data);

To try out a custom mapping script:

1. Click on the cog icon to take you back to the narrative add-in settings.

2. Copy the above example and paste it in the Edit Mapping Script section.

3. Click Generate Text in the Arria add-in.

You should see the currencies expressed in sterling (£) instead of dollars (US$):