Extracting Twitter Data and Using it in a Mapping - Informatica Powercenter Express

In the Previous blog, we saw how to create a Twitter App, configure Open Authentication and create a Twitter Connection. The next step is to create the Data Object

Create New Data Object

A Twitter data object is a physical data object that represents data based on Twitter resource such as Followers, Profile and Place.

1. Create a new Data Object by clicking File -> New -> Data Object and choose Twitter Data Object.

2. Give a name to the Data Object, location and choose connection created in the above steps. When creating a new data object Selected Resources section is empty. You need to choose resources by clicking Add button.

3. There will be various Twitter Resources shown like Search, Profile lookup, Followers and Friends. For now, just select Search as we are going to search for the tweets based on keywords. Click OK and complete the Data Object creation step by clicking Finish.

Create New Data Object Operation

Create new Data Object Operation on Data Object just created by right clicking on Twitter_Data_Object -> New -> Data Object Operation.

Name the Data Object Operation and select the resource that we added during the creation of data object i.e., Search. Click on Finish button.

Informatica creates two operations Search and Output. In the properties of Output read operation, go to Advanced tab. Use Query Parameter to specify search criteria for tweets to search and endsAfter property can be used to limit number of records fetched by the connection.

Query parameter: QUERY=Informatica

Twitter Data Object Operation Advanced Properties

Query Parameter is the Twitter search criteria. Each resource requires one of the mandatory parameters.
Followers - screen_name,user_id
Friends - screen_name,user_id
Profile Lookup - screen_name,user_id
Search - QUERY

For example, you specify the screen name in the following query parameter to extract the Twitter profile of the Twitter screen name xyzscreen_name=xyz

Token file  is the file name and format of a file that contains a list of access tokens and access secrets. The CSV file must be on the machine where Informatica Services is installed. The Data Integration Service uses the access token and access secret at run time to establish a connection. The Token File is applicale for Followers and Friends Twitter Resource.

Ends After  controls the number of rows requested. Specify a time or a positive numeral:
We can specify the amount of time that the Data Integration Service runs the mapping. Enter the duration in the format, hh:mm. Default is blank, which indicates that the Data Integration Service runs the mapping until you stop it or until there is data.
The Data Integration Service will request the minimum number of rows which is 100. The maximum number of rows requested is based on the rows returned by the Twitter application. Enter the number of rows as a positive numeral.

Create  Mapping

In the Developer, using above Data Object as source and any relational database or flat file as target, mapping can be created.

  

*In case of any questions, feel free to leave comments on this page and I would get back as soon as I can.