Transformation tutorial

This section provides a "Hello World!"-type example of a transformation made easy by the use of Visual Mapper. We have three files to work with:

From the input, we are going to concatenate the street_address (for example: "24 Guild Street") with the city ("London") , into a full_address in the output ( "24 Guild Street, London") .

Here is an example of how to perform a transformation with Visual Mapper:

  1. Using your favorite text editor, create 3 json files, as indicated in the sections below
  2. Create a new Data map.
    • In Policy Studio: to create a new map , right-click the Data Maps node in the tree (under Resources) and select Add new Data Map.
    • Other means of creating new Data maps may be available at the time you are reading this.
  3. Enter the characteristics of the new data map:
    • Name: address
    • Source Schema details:
      • Type: JSON
      • Locate on disk: yes
      • Schema: click on [...] button to search for the address_schema_in.json file you created in step 1
    • Target Schema details:
      • Type: JSON
      • Locate on disk: yes
      • Schema: click on [...] button to search for the address_schema_out.json file
  4. The data map appears as a 3-pane window (see the Design tab screenshot below)
    • The left pane reflects the data structure of the file address_schema_in.json
    • The right pane reflects the data structure of address_schema_out.json
    • The center pane is empty for the moment (the screenshot below is a few steps ahead of us)

    The Design tab
  5. Drag to link streetAddress on the left to fullAddress to the right
    A rounded square appears in the center pane, materializing the new link
  6. Select Run tab among the Design - Source - Run tabs at the bottom of the data map
    The Run window appears (again, the screenshot is ahead of us, showing the result we intend to achieve: at this stage, the Input and Output fields are empty).

    The Run tab
  7. Click Browse... button to load the data file, named address.json in this example
  8. Click the Run button
    In the Output field, you should see the following text, showing that the data from the streetAddress in input has been copied verbatim to fullAddress on the output:
    { "address": { "fullAddress":"24 Guild Street" } }
  9. In the Design view, drag a link from city on the left to the rounded square in the center. The square now has two links coming from the input and one link going to the output.
    • Note that you can see the rounded square is selected, because the lines are bold and they have square points at the end.
  10. Right click on the square and select Set functionStringConcat
  11. Re-run the transformation as indicated above. The fullAddress in the output is now:
    "24 Guild StreetLondon
    ". 
    We are nearly there! All we need now is to add in a comma between Street and London.
  12. Return to the Design tab, right click on the square in the center, select Edit...
    The Edit Function Parameter window appears.

    It has two arguments in its center, named arg1 and arg2:
    • arg1: /address/streetAddress
    • arg2: /address/city
  13. Click the [+] button to add a new argument. An arg3 tab appears. Type in ", " (comma - space, without the quotes), then drag the new arg3 tab towards the center, to switch arg2 and arg3.
  14. Re-run the transformation as indicated above. The output fits the specifications!
  15. In the Design view, hover over the rounded square symbolizing the function. You will see:
    • 3 dots symbolizing the 3 arguments
    • a + sign, for adding a new argument

      Hovering over the concatenation function

Adding a condition

Now we are going to add some conditional processing between the input and the output.

Suppose the input data contains test records, recognizable because the city is equal to "Test_city". We don't want those records in the output. To do that, we will create an If condition, as follows:

  1. Drag a link between address on the left and address on the right . 
    A square appears in the center.
  2. Right-click on the square, then select Set functionIf
    The square becomes a lozenge, containing an icon symbolizing the If condition
  3. Drag a link between city (on the left) and the lozenge
  4. Right-click on the lozenge, then select Edit...
    In the Condition tab, you will see the following:
    /address/city != ''
  5. Edit the text to fit the required condition:
    /address/city != 'Test_city'
  6. Edit the data file to add a record with
    "city":"Test_city"
  7. To reload the new Source file, click the Run tab, then Browse... button again
  8. Re-run the transformation. 
    The content of the Output section now reflects the condition.
  9. If you need to debug your transformation, click on the > Settings entry and check the Log debugging information box.

Data files for the tutorial

address_schema_in.json

Copy the code below, then paste it into your favorite text editor.

Tip   http://www.jsoneditoronline.org/ is a handy resource for editing json files. This site provides an online editor to detect errors and indent source code, side-by-side with an editable expand-collapse tree view when the source has a correct syntax.
{"$schema":"http://json-schema.org/draft-04/schema#",
	"id":"http://jsonschema.net",
	"type":"object",
	"properties":{
		"addressItems":{
			"id":"http://jsonschema.net/address",
			"type":"array",
			"items":{
		"type":"object",
		"properties":{
			"address":{
			"type":"object",
			"properties":{
				"streetAddress":{
				"type":"string"
				},
				"city":{
				"type":"string"
				}
			},
			"required":[
				"streetAddress",
				"city"
				]
			}
		},
		"required":[
			"address"
			]
		}
	}
}
}

address_schema_out.json

{
"$schema":"http://json-schema.org/draft-04/schema#",
"id":"http://jsonschema.net",
"type":"object",
"properties":{
	"fullAddressItems":{
		"id":"http://jsonschema.net/address",
		"type":"array",
		"items":{
			"type":"object",
			"properties":{
				"address":{
				"type":"object",
				"properties":{
					"fullAddress":{
					"type":"string"
					}
				}
			}
		},
	"required":["address"]
	}
}
}
} 

address.json file

{"addressItems": [
	{"address": {
		"streetAddress": "24 Guild Street",
		"city": "London"}
	},
	{"address": {
		"streetAddress": "999 Lucky Strike", 
		"city": "Test_city"}
	},
	{"address": {
		"streetAddress": "221B Baker Street",
		"city": "London"}
	},
	{"address": {
		"streetAddress": "20 Forthlin Rd",
		"city": "Liverpool"}
	},
	{"address": {
		"streetAddress": "25 Brook Street",
		"city": "London"}
	}
	]}  

Related Links