4. Getting Started with NetMiner Script
4. Getting Started with NetMiner Script
-2-
Copyright and Trademarks
The information in this publication is provided for information only and is subject to change without
notice. Cyram Inc. assume no responsibility or liability for any loss or damage that may arise from the
use of any information in this publication. The software described in this book is furnished under
license and may only be used or copied in accordance with the terms of that license.
NetMiner is a registered trademark of Cyram Inc. CYRAM and CYRAM logo are registered
trademarks of Cyram Inc.
Cyram Inc.
#904, U-Space 2B, 670, Daewangpangyo-ro, Bundang-gu, Seongnam-si, Gyeonggi-do, 13494,
South Korea
Tel: +82-31-739-8352
Fax: +82-31-739-8354
In order to reference NetMiner 4, please employ the following bold-faced reference, which is also
found at NetMiner Menu: “Help >> About NetMiner 4”.
Technical Support
-3-
The NetMiner Help file will guide you to use NetMiner 4. Simply press F1 function key or use
NetMiner menu bar for triggering Help feature.
Cyram is committed to provide a reliable, high-quality, yet easy-to-use product. If you have any
problem of installing or using NetMiner, please contact us by mail, phone and fax or preferably by
email and World Wide Web.
Cyram Inc.
#904, U-Space 2B, 670, Daewangpangyo-ro, Bundang-gu, Seongnam-si, Gyeonggi-do, 13494,
South Korea
Tel: +82-31-739-8352
Fax: +82-31-739-8354
Email: [email protected] Web: http://www.netminer.com/
-4-
Contents
1. INTRODUCTION ....................................................................................................... 2
-5-
1
1. Introduction
1.1 What is NetMiner Script?
Based on the Python syntax, NetMiner Script provides an environment for performing
analysis and visualization with respect to various aspects of NetMiner. Execution in
NetMiner Script is exactly the same as execution through the GUI not only in terms of
results but in terms of process as well. Therefore by looking at a script the user can easily
understand what specific task is to be performed in the overall process. NetMiner Script
supports the Python language constructs and modules by default and provides various
functions for network analysis and visualization. With NetMiner Script you can automate
analysis/visualization processes that have to be performed repeatedly as well as perform
condition-based analysis. In addition, external modules written purely in Python can be used
for application in a broader field. The Script Workbench is GUI program for more efficient
use of NetMiner scripts. The script generator of the Script Workbench allows the use of user
interfaces of control panels of an existing NetMiner session so that complex scripts of
analysis and visualization can be more easily written. The Script Workbench provides a
guide box for each NetMiner function as well as a list of functions.
1) Most features of NetMiner can be performed using a script rather than having to click
with mouse so that complicated series of commands can be stored in scripts and executed
repeatedly without additional work.
2) Analysis flows, such as loops and conditional branching can be controlled by the user in a
flexible way.
3) Various existing math and statistics modules written in Python can be imported without
modification for use, and ordinary data structures provided by Python can be defined.
2
4) Simple algorithms can be written by the user in order to be used in combination with
NetMiner features. In addition, user-written files can be added to the NetMiner menu to be
used like internal processes which can be shared with other NetMiner users as well.
1) Python-based scripting
Most analysis software packages with scripting feature use their own scripting language.
This requires the user to learn the language, which limits the user base and consequently it’s
not easy to find related material either.
NetMiner Script uses Python, a programming language distributed free of charge and
widely used all over the world. Non-computer majors often get cold feet when they hear the
word programming language, but Python is especially so easy that even complete beginners
who have little knowledge of programming can learn and use it without difficulties,
provided they put in some effort. Also, Python has a wide user base, which means there are
lots of easily-obtainable materials available on the Web for learning Python. To use analysis
algorithms involving math, statistics or data mining for example, you go to a place on the
Web where Python codes written by other people are shared and then, you can download
and use the pertinent source codes. This is especially helpful for Python beginners.
Therefore, learning NetMiner Script is not simply learning another script language – it is
also a gateway to help you to enter the vast world of Python.
2) WYSIWYG scripting
NetMiner is a GUI program and NetMiner Script is a GUI-based script, which means all
tasks executed through the GUI can be controlled by the use of scripts. Although NetMiner
is not the only analysis software that uses both GUI and script, it does differentiate itself
from others in the following respects: Generally, in the case of analysis software that uses
both of the interfaces, although the results of execution in GUI mode and scripting mode are
the same, their processes are not exactly the same. In most of the analysis programs, scripts
3
control data and processes in a way unrelated to the GUI. In contrast, the way of writing
NetMiner scripts, is the same. Therefore, while writing a NetMiner Script, the user can see
in their mind’s eye which task is corresponding to the execution in GUI. To make it more
convenient in generating scripts, NetMiner enables users to utilize the script generator in the
Script Workbench. By using it, scripts can be automatically generated according to items
selected through the GUI, which helps to avoid the hassle of manual coding in each process.
Because of this, NetMiner Script can be called a WYSIWYG (What You See Is What You
Get) way of scripting.
1.2 Installation
Refer to the installation guide included in the CD for the installation. The installation
guide can also be downloaded on the Web site, at the following address:
http://www.netminer.com>>Download>>Documents>>Installation_Guide.pdf
To use Script Workbench, NetMiner 4.0 or above is required. Therefore you have to use
NetMiner 4.0 in order to use Script Workbench. If you purchased NetMiner 3, you can get a
special discount when purchasing NetMiner 4. Please visit the Web site for details.
If you registered NetMiner 4 with a license key that includes the scripting package in the
initial installation process of NetMiner 4, you can immediately begin scripting. If you didn’t
register it with a license key that includes the scripting package, however, you can click
4
‘Help >> Register…’ in the main menu of NetMiner 4 and register it with a new license key
that allows for scripting. Once ‘Tools >> Script Workbench’ is activated, you are ready to
go to begin using NetMiner Script.
5
2. Getting Started
2.1 Starting and quitting Script Workbench
2.1.1 Starting
Step 1: Double click on the NetMiner 4 icon to run NetMiner 4, or execute the file
‘NetMiner 4.exe’ located in the NetMiner 4 installation folder.
Step 2: Click on ‘Tools >> Script Workbench’ in the main menu bar of NetMiner 4.
2.1.2 Quitting
Click ‘File >> Close’ in the Script Workbench window to quit Script Workbench.
6
2.2 Opening and running example scripts
Python files (extension *.py) are the primary file type of Script Workbench. To explain
how to open a script and run it, we’ll use the script sample file ‘GS_1.py’ and the NetMiner
sample data ‘ 01.Org_Net_Tiny1.nmf’. If you installed NetMiner 4 in the default directory,
they are located in the folders ‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started’
and ‘C:\Users\(Username)\NetMiner\SampleData\Cyram Dataset\01.Org_Net_Tiny1’
respectively.
2.2.1 Opening
Step 2: Click File >> Open in the menu or press the short-cut key (Ctrl+O) to open the file
‘GS_1.py’.
The content of the script file will be shown in the Script Workbench as follows.
7
2.2.2 Running
Step 1: Press Ctrl+R or click Script >> Run to run the currently open script.
The above results are the same as the ‘work interact’ 1-mode network which is drawn from
the Kamada & Kawai layout algorithm by clicking File >> Open, opening the file
‘01.Org_Net_Tiny1.nmf’, and executing Visualize >> Spring >> 2D session.
8
2.3 Manually writing and running scripts
NetMiner scripts are based on Python. Accordingly variables, data types and functions
used in Python can be used by default. And also the output can be checked using the ‘print’
command in the log window.
Variables can be created in the same way as a typical programming language. The end of
a line is denoted by a semicolon (;), which can be omitted.
Step 3: Click Script >> Run or press Ctrl+R to run the script.
NetMiner Script not only supports the Python language but provides its own functions. In
addition, you can use the auto-completion feature by pressing the shortcut key Ctrl+Space.
9
Step 2: Click Script >> Content Assist or press the shortcut key Ctrl+Space to bring up the
function list as shown below. _MAT, _NM and _SYS are unique function classes used in
NetMiner Script.
In NetMiner Script, there are three function classes: _NM (NetMiner-related functions);
_MAT (matrix data-related functions); _SYS (misc. functions). By adding ‘.’ to a function
class various classes and functions associated with that class can be used.
Step 1: In the editing window enter ‘_MAT.’ and use auto-completion (Ctrl-Space) to see a
list of sub classes that belong to the ‘_MAT’ class.
10
Step 2: Select a desired function to enter it automatically as shown below. Input variables to
the function are shown as well.
You can easily open NetMiner files (*.nmf) by using NetMiner Script. In order to explain
the way to open a project, we’ll use the sample script file ‘GS_2.py’ and the NetMiner
sample data file ‘01.Org_Net_Tiny1.nmf’. If you installed NetMiner 4 in the default
directory, they are located in the folders
‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started’ and
‘C:\Users\(Username)\NetMiner\SampleData\Cyram Dataset\01.Org_Net_Tiny1’
respectively.
Step 2: Enter ‘_NM.’ In the editing window, bring up a list of classes by pressing
Ctrl+Space, select Project, and then select ‘openProject(filepath)’ which is a function that
belongs to it.
Step 3: Enter the file name, the argument to openProject(). Prefix the file path with ‘r’ and
enter the file path enclosed by small quotations (‘ ‘).
_NM.Project.openProject(r'%UserProfile%\NetMiner\SampleData\Cyram
Dataset\01.Org_Net_Tiny1\01.Org_Net_Tiny1.nmf')
11
GS_2.py
Step 4: Click Script >> Run or press the shortcut key Ctrl+R to run the script. Confirm that
the file ‘01.Org_Net_Tiny1.nmf ‘ has been opened.
Using NetMiner Script, text files (*.txt, *.csv) and Excel files (*.xls) can be imported.
There are three network data types that can be imported. Here is described how you can
import edge list Excel files to the 1-mode network.
The file ‘work interact_weighted.xls’ is used to describe the way to import data. If you
installed NetMiner 4 with the default settings, you can find the Excel file in the folder,
‘C:\Users\(Username)\NetMiner\SampleData\Cyram Dataset\01.Org_Net_Tiny1\02.1-mode
Network Data\00.Edge List’.
Step 2: Write the script as follows in the editing window. You can either enter the class
and function names after ‘.’ or select one in a list that is to be automatically displayed
auto-completion by pressing Ctrl+Space.
importer.set1modeEdgeListMode('work_interact'); #-----(3)
importer.runImport(); #-----(4)
GS_3.py
(1): This creates an object for importing an Excel file. The argument is the file path.
(2): This sets whether headers are included in the rows and columns of the data that are to be
imported. As neither exists in this case, enter (False, False).
12
(3): This sets the type of data to be imported. The argument is the name to be entered in the
workfile.
(4): Execute the written import class and the data become imported.
Step 3: Click Script >> Run or press the shortcut key Ctrl+R to run the script. Check if
network data is imported as shown below.
The script generator automatically generates scripts for NetMiner session execution. In
this section we’ll write a script for analyzing ‘Degree Centrality’ of the network by using the
script generator. In order to describe the way to use it, the file ‘01.Org_Net_Tiny1.nmf’ is
used. If you installed NetMiner 4 with the default settings, it can be found in the folder
‘C:\Users\(Username)\NetMiner\SampleData\Cyram Dataset\01.Org_Net_Tiny1‘.
Step 3: Click Script >> Script Generator in the Script Workbench menu.
Step 5: Select a desired option and click [OK]. A script for ‘Degree Centrality’ analysis will
be automatically generated.
14
GS_4.py
Step 6: Click Script >> Run or press the shortcut key Ctrl+R to run the script. The ‘Degree
Centrality’ item will be added to the session and you can see the results of analysis.
After writing an analysis script for an arbitrary input data, it can be reused for other input
data at the same type. For the simple explanation on script reuse, the files
‘01.Org_Net_Tiny1.nmf’ and ‘GS_4.py’ are used. If you installed NetMiner 4 with the
15
default settings, they can be found in the folders
‘C:\Users\(Username)\NetMiner\SampleData\CyramDataset\01.Org_Net_Tiny1’ and
‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started’ respectively.
Step 3: Open the file ‘GS_4.py’ by clicking File >> Open. The content of the script will be
shown as follows. It is the same script as the one that script generator automatically
produces through a [Analyze>Centrality>Degree] analysis for the ‘work interact’ network.
GS_4.py
Step 4: Run the script by clicking Script >> Run. The following results will be shown.
16
Step 5: In the script, change ‘work interact’,
the argument to session1.setInputAdjacencyName(“work interact”), to ‘personal friend’.
###-- Auto generated code by Script generator : Analyze>Centrality>Degree : Begin --
session1 = _NM.Session.createSession("Analyze>Centrality>Degree");
session1.setInputAdjacencyName("personal friend");
session1.run();
report1 = session1.getMainReport("[R] Main");
report2 = session1.getTableReport("[T] Degree Centrality Vector");
report3 = session1.getMapReport("[M] Spring");
report4 = session1.getMapReport("[M] Concentric");
###-- Auto generated code by Script generator : Analyze>Centrality>Degree : End --
Step 6: Run the modified script to see the results of ‘Degree Centrality’ for the ‘personal
friend’ network.
Step 7: Change the input data and run the script again.
When performing the same analysis on several sets of input data, repetitive analysis can
be easily done by using the ‘loop’ construct. In addition, the decision for execution can be
made according to arbitrary conditions.
17
For the simple explanation on repetitive execution using ‘loop’, the files
‘01.Org_Net_Tiny1.nmf’ and ‘GS_5.py’ are used. If you installed NetMiner 4 with the
default settings, they can be found in the folders
‘C:\Users\(Username)\NetMiner\SampleData\Cyram Dataset\01.Org_Net_Tiny1’ and
‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started’ respectively.
Step 3: Open the file ‘GS_5.py’ by clicking File >> Open. The content of the script displays
as follows.
networkNameList = ['work interact', 'personal friend', 'personal help', 'personal knows', 'work
help'];
18
Analysis is repeated using the ‘for’ construct, and conditional branching is done using the
‘if’ construct. In the above script, networkNameList is a list of 1-mode network data names
that exist as data sets in ‘01.Org_Net_Tiny1’. ‘networkName’, an item of the list, returns as
an argument, to the analysis scripts of ‘Degree Centrality’ and ‘Node Betweenness
Centrality’ which were automatically generated by using the script generator. When above
process is proceeding, the ‘if’ construct branches ‘personal friend’ to ‘Node Betweenness
Centrality’ analysis while others to ‘Degree Centrality’ analysis.
As a result of running the script, an analysis session on five networks (‘work interact’,
personal friend’, ‘personal help’, ‘personal knows’, ‘work help’) is created. For the second
network, namely ‘personal friend’, ‘Node Betweenness Centrality’ was performed, while for
the rest ‘Degree Centrality’ was performed in order.
19
2.4.3 Using external libraries
NetMiner Script provides an environment for using external Python libraries. In this
regards, NetMiner Script can call an external library designed to collect twitter data. In this
section we’ll take a look at a script to import Twitter data to NetMiner by utilizing the
external library.
※ Unfortunately, this sample script is not working due to change of twitter API policy.
We plan to provide other sample script for using external libraries.
For the brief explanation on how to use an external library, the Twitter collection API and
the file ‘GS_6.py‘ are used. If you installed NetMiner 4 with the default settings, the
‘GS_6.py’ file can be found in the folder
‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started\GS_6’.
s_time = time.time()
custom = _SYS.Dialog.createCustomDialog('Crawling Box','Option')
custom.addStringVariable('Seed User Name :','evegood99')
custom.addComboBoxVariable('Select Relation :',['Followers'])
custom.addIntegerVariable('Depth :',1)
custom_result = custom.open()
seedName = custom_result[0]
relation = custom_result[1]
netDepth = custom_result[2]
api = api_.Twitter()
20
def searchFollowers(seed):
followersList = []
try:
result = api.statuses.followers(screen_name=seed)
for follower in result:
followersList.append(follower['screen_name'])
except:
pass
return followersList
def makeLinkListCSV(linkList):
try:
os.mkdir(r'c:\tmp_link')
except:
pass
fpout = open(r'c:\tmp_link\tmp_tweet_linkList.csv','w')
csv_writer = csv.writer(fpout, lineterminator='\n', quotechar='"',
quoting=csv.QUOTE_ALL)
for edge in linkList:
csv_writer.writerow(edge)
fpout.close()
def main():
seedQueue = [seedName]
nodeList = [seedName]
linkList = []
nodeDepth = {}
nodeDepth[seedName] = 0
endSeed = {}
while (len(seedQueue)!=0):
seed = seedQueue.pop(0)
_SYS.Dialog.showStatusDialog(str('['+seed+']'+' followers
searching...'))
followersList = searchFollowers(seed)
endSeed[seed] = True
for fol in followersList:
linkList.append([fol,seed])
if nodeDepth.has_key(fol):
pass
else:
21
nodeDepth[fol] = nodeDepth[seed] + 1
nodeList = list(set(nodeList + followersList))
for node in nodeList:
if nodeDepth[node] <= netDepth and endSeed.has_key(node) ==
False:
seedQueue.append(node)
seedQueue = list(set(seedQueue))
lastEdgeList = []
for edge in linkList:
if nodeDepth[edge[0]] <= netDepth and nodeDepth[edge[1]] <= netDepth:
lastEdgeList.append(edge)
makeLinkListCSV(lastEdgeList)
main()
GS_6.py
The BFS algorithm is used in designing ‘main’ function. Starting with a seed node BFS
algorithm enables to find the followers using the function ‘searchFollowers’, and then
followers of followers in sequence.
Step 3: Click on Script >> Run to bring up a dialog window where we can input the
twitter screen name as the starting point of data collection and the depth level of collection
can be entered.
22
Enter the screen name for ‘Seed User Name’ and enter the depth of collection for ‘Depth’.
Press OK to start crawling. When the crawling is completed, collected data is displayed.
Visualizing the above data, the following Twitter relationship map is drawn.
In NetMiner Script, you can write functions and algorithms and use them together with
various modules provided by NetMiner. In this section, ‘Node Betweenness Centrality’ is
measured at first, based on which the average and standard deviation of ‘Betweenness
Centrality’ are computed using a user-defined function.
For the brief explanation on how to use user-defined functions, the files
‘01.Org_Net_Tiny1.nmf’ and ‘GS_6.py’ are used. If you installed NetMiner 4 with the
default settings, they can be found in the folders
23
‘C:\Users\(Username)\NetMiner\SampleData\CyramDataset\01.Org_Net_Tiny1’ and
‘C:\Users\(Username)\NetMiner\SampleScript\Getting Started’ respectively.
Step 3: Open the file ‘GS_6.py’ by clicking File >> Open to bring up the following script.
def average(list):
tot = 0.0;
num = 0;
for value in list:
tot = tot + value;
num = num + 1;
return avg;
def stddev(list):
sum = 0.0;
num = 0;
avg = average(list);
for value in list:
sum = sum + (value - avg)**2;
num = num + 1;
std = (sum**0.5)/num;
return std;
attName = report2.getAttributeNames();
report2.addToNodeAttribute(attName[0], 'betw');
betwList = _NM.Dataset.getMainAttributeValues('betw')
24
print 'avg : ', average(betwList);
print 'std : ', stddev(betwList);
GS_6.py
In the above script a function for finding the average and the standard deviation is defined
by the user. After measuring ‘Node Betweenness Centrality’, the average and standard
deviation of the ‘centrality’ is found by using the user-defined function.
The functions ‘average’ and ‘stddev’ find the average and the standard deviation
respectively. Arguments to each function are made in a form of a list after adding as a main
node attribute by measuring the network’s ‘Node Betweenness Centrality’.
Step 4: Run the script by clicking Script >> Run. In the log window, the average and
standard deviation of ‘centrality’ values will be shown.
25