Mastering Cloud Computing:

More Example Programs

Answers to problem in Chapter 6: Cloud Programming and Software

Environments

Problem 6.13: Using a MapReduce programming model supported in Aneka, develop a program for
image filtering of hundreds of photos you have taken using digital camera. Conduct and report
results of scalability experiments by varying the number of computing nodes/workers and images of
different resolutions or file size on an Aneka-based enterprise Cloud. Aneka (prof. Buyya)

Answer: The following program illustrates a sample application for image filtering using Aneka’s
MapReduce Programming Model. Note that the actual image filtering is dependent on the problem
domain and you may use any algorithm you see fit.

class Program
{
/// <summary>
/// Reference to the configuration object.
/// </summary>
static Configuration configuration = null;

/// <summary>
/// Location of the configuration file.
/// </summary>
static string configurationFileLocation = "conf.xml";

/// <summary>
/// Processes the arguments given to the application and according
/// to the parameters read runs the application or shows the help.
/// </summary>
/// <param name="args">program arguments</param>
static void Main(string[] args)
{
try
{
//Process the arguments
Program.ProcessArgs(args);

Program.SetupWorkspace();

//configure MapReduceApplication
MapReduceApplication<ImageFilterMapper, ImageFilterReducer>
application = new MapReduceApplication<ImageFilterMapper,
ImageFilterReducer>("ImageFilter", configuration);
 //invoke and wait for result
application.InvokeAndWait(new EventHandler<Aneka.Entity.
ApplicationEventArgs>
(OnApplicationFinished));
}
catch (Exception ex)
{
Console.WriteLine(" Message: {0}", ex.Message);
Console.WriteLine("Application terminated unexpectedly.");
}
}

/// <summary>
/// Hooks the ApplicationFinished events and Process the results
/// if the application has been successful.
/// </summary>
/// <param name="sender">event source</param>
/// <param name="e">event information</param>
static void OnApplicationFinished(object sender,
Aneka.Entity.ApplicationEventArgs e)
{
if (e.Exception != null)
{
Console.WriteLine(e.Exception.Message);
}

Console.WriteLine("Press enter to finish!");

Console.ReadLine();
}

/// <summary>
/// Processes the arguments given to the application and according
/// to the parameters read runs the application or shows the help.
/// </summary>
/// <param name="args">program arguments</param>
static void ProcessArgs(string[] args)
{
for (int i = 0; i < args.Length; i++)
{
switch (args[i])
{
case "-c":
i++;
configurationFileLocation = args[i];
break;
default:
break;
}
}
}

/// <summary>
/// Initializes the workspace
/// </summary>
static void SetupWorkspace()
{
Configuration conf = Configuration.GetConfiguration(

Program.configurationFileLocation);
Program.configuration = conf;
}
}
 /// <summary>
/// Class ImageFilterMapper. Mapper implementation for the ImageFilter
/// application. The Map method reads the source images and performs
/// the rquired filtering. The output of the Map function is the filtered
/// image.
/// </summary>
public class ImageFilterMapper : Mapper<string, BytesWriteable>
{
/// <summary>
/// The Map function receives as input the name of the image and its
/// contents. The filtering is then performed on the contents before
/// writing the results back to the storage.
/// </summary>
/// <param name="input">A key-value pair representing the name of the
/// file and its contents.</param>
protected override void Map(IMapInput<string, BytesWriteable> input)
{
byte[] image = input.Value.GetBytes();

// Put your image filtering algorithm here

// ...
// ...

Emit(input.Key, image);
}
}

/// <summary>
/// Class ImageFilterReducer. Reducer implementation for the ImageFilter
/// application. The Reducer is an identity function which does no
processing.
/// </summary>
public class ImageFilterReducer : Reducer<string, BytesWriteable>
{
/// <summary>
/// The Reduce function is an identify function which does no further
/// processing on the contents.
/// </summary>
protected override void Reduce(IReduceInputEnumerator<BytesWriteable>
input)
{
// This is an identity function. No additional processing is
required.
}
}

Once you have written and compiled you code, run your application by varying first the input size
and then the number of nodes (for example: 2, 4, 8, 16, ..). Plot a single graph of execution time (y-
axis) versus input size (x-axis) for the different sets of nodes used, so that your final graph shows the
difference in execution time for each of the sets of nodes. Next plot a graph of speed-up (x-axis)
versus input size (y-axis) for the different sets of nodes used.
Problem 6.19: Using a Thread programming model supported in Aneka, develop a program for
parallel multiplication of two very large square matrices or order greater than 500. Conduct and
report results of scalability experiments by varying the order of matrix from 500 to 1000 in steps of
100 on computing nodes/workers varying from 10 to 50 in step of 10 in an Aneka-based enterprise
Cloud. Aneka (prof.Buyya)

Answer: The following program illustrates the use of Aneka’s Thread Programming Model for matrix
multiplication. The program takes as inputs two square matrices. Each AnekaThread instance is a
row-column multiplier, that is, a row from the first matrix is multiplied with the corresponding row
from the second matrix to produce the resulting cell for the final matrix. Each of these row-column
computations is performed independently on a Worker node. The results of the computations are
then put together by the client application.

/// <summary>
/// Class <i><b>MatrixMultiplier</b></i>. Multiplies two square matrices,
where
/// each element in the resulting matrix, C, is computed by multiplying
the
/// corresponding row and column vectors of matrix A and B. Each of these
/// computations are carried out by distict instances of AnekaThread.
Multiplying
/// two square matrices of dimension n would thus requires n*n
AnekaThread
/// instances.
/// </summary>
public class MatrixMultiplier
{
/// <summary>
/// The application configuration
/// </summary>
private Configuration configuration;

/// <summary>
/// Creates an instance of MatrixMultiplier
/// </summary>
/// <param name="schedulerUri">The uri to the Aneka scheduler</param>
public MatrixMultiplier(Uri schedulerUri)
{
configuration = new Configuration();
configuration.SchedulerUri = schedulerUri;
}

/// <summary>
/// Multiplies two matrices A and B and returns the resulting matrix
C. This
/// method creates a list of AnekaThread instances to compute each of
the
/// elements in Matrix C. These threads are submitted to the Aneka
runtime
/// for execution and the results of each of these executions are
used to
/// compose the resulting matrix C.
/// </summary>
/// <param name="matrixA">Matrix A</param>
/// <param name="matrixB">Matrix B</param>
/// <returns>The result, Matrix C</returns>
public Matrix Multiply(Matrix matrixA, Matrix matrixB)
{
// Create application and computation threads
 // execute threads on Aneka
this.ExecuteThreads(threads);

// gather results
Matrix matrixC = this.ComposeResult(threads, matrixA.Size);

// stop application
application.StopExecution();

return matrixC;
}

/// <summary>
/// Creates AnekaThread instances to compute each of the elements in the
resulting
/// matrix C. These threads are initialized to execute the
/// RowColumnMultiplier.DoMultiply method on the remote node.
/// </summary>
/// <param name="application">The AnekaApplication instance containing the
/// application configuration</param>
/// <param name="matrixA">Matrix A</param>
/// <param name="matrixB">Matrix B</param>
/// <returns>The result, Matrix C<</returns>
private IList<AnekaThread> CreateComputeThreads(AnekaApplication<AnekaThread,
ThreadManager> application, Matrix matrixA, Matrix matrixB)
{
IList<AnekaThread> threads = new List<AnekaThread>();

int dimension = matrixA.Size;

for (int row = 0; row < dimension; row++)

{
double[] rowData = this.ExtractRow(matrixA.Data, row, matrixA.Size);

for (int column = 0; column < dimension; column++)

{
double[] columnData = this.ExtractColumn(matrixB.Data, column,
matrixB.Size);

RowColumnMultiplier rcMultiplier = new

RowColumnMultiplier(rowData,
columnData);
AnekaThread anekaThread = new AnekaThread(new
ThreadStart(rcMultiplier.DoMultiply),
application);
threads.Add(anekaThread);
}
}

return threads;
}

/// <summary>
/// Executes the list of AnekaThread instances on the Aneka runtime
environment.
/// </summary>
/// <param name="threads">The list of AnekaThread instances to
execute</param>
private void ExecuteThreads(IList<AnekaThread> threads)
{
foreach (AnekaThread thread in threads)
{
thread.Start();
}
 /// <summary>
/// Composes the resulting matrix C. This method pauses until all elements
/// of matrix C have been computed. The results of each of these executions
/// are then used to compose matric C.
/// </summary>
/// <param name="threads">The list of AnekaThread instances that were
submitted for
/// execution</param>
/// <param name="size">The size of the resulting matrix C</param>
/// <returns>The result, Matrix C</returns>
private Matrix ComposeResult(IList<AnekaThread> threads, int size)
{

// wait till all threads complete..

foreach (AnekaThread thread in threads)
{
thread.Join();
}

// compose resultant matrix

Matrix matrixC = new Matrix(size);
for (int row = 0; row < size; row++)
{
for(int column = 0; column < size; column++)
{
AnekaThread thread = threads[(row * size) + column];
RowColumnMultiplier rcMultiplier =
(RowColumnMultiplier)thread.Target;
matrixC.Data[row, column] = rcMultiplier.Result;
}
}

return matrixC;
}

/// <summary>
/// Extracts a row from a two-dimensional array.
/// </summary>
/// <param name="array">The two-dimensional array</param>
/// <param name="rowIndex">The index of the row to extract</param>
/// <param name="length">The length of the row to extract</param>
/// <returns>A one-dimensional array</returns>
private double[] ExtractRow(double[,] array, int rowIndex, int length)
{
double[] row = new double[length];
for (int x = 0; x < length; x++)
{
row[x] = array[rowIndex, x];
}
return row;
}

/// <summary>
/// Extracts a column from a two-dimensional array.
/// </summary>
/// <param name="array">The two-dimensional array<</param>
/// <param name="columnIndex">The index of the column to extract</param>
/// <param name="length">The length of the column to extract</param>
/// <returns>A one-dimensional array</returns>
private double[] ExtractColumn(double[,] array, int columnIndex, int length)
{
 double[] column = new double[length];
for (int x = 0; x < length; x++)
{
column[x] = array[x, columnIndex];
}
return column;
}

/// <summary>
/// The main entry point to the application
/// </summary>
/// <param name="args"></param>
static void Main(string[] args)
{
Matrix matrixA = new Matrix(10);
Matrix matrixB = new Matrix(10);

matrixA.InitRandom();
matrixB.InitRandom();

Uri schedulerUri = new Uri("tcp://localhost:9090/Aneka");

MatrixMultiplier multiplier = new

MatrixMultiplier(schedulerUri);
Matrix matrixC = multiplier.Multiply(matrixA, matrixB);
matrixC.Print();

Console.ReadKey();
}
}

/// <summary>
/// Class <i><b>Matrix</b></i>. Represents a square matrix where
/// each element occupies a slot in a two-dimensional array.
/// </summary>
[Serializable]
public class Matrix
{
/// <summary>
/// Array of elements in the matrix.
/// </summary>
private double[,] data;

/// <summary>
/// Gets or sets the 2D array containing the elements in the matrix.
/// </summary>
public double[,] Data
{
get { return this.data; }
set { this.data = value; }
}

/// <summary>
/// The size of the square matrix.
/// </summary>
private int size;

/// <summary>
/// Gets or sets the size of the square matrix.
/// </summary>
public int Size
{
get { return this.size; }
set { this.size = value; }
 /// <summary>
/// Creates a new square matrix of dimension <paramref name="size"/>
/// </summary>
/// <param name="size">The dimension of square matrix. </param>
public Matrix(int size)
{
data = new double[size, size];
this.size = size;
}

/// <summary>
/// Initializes the matrix with random doubles between 0 to 10.
/// </summary>
public void InitRandom()
{
Random rand = new Random();

for (int x = 0; x < size; x++)

{
for (int y = 0; y < size; y++)
{
data[x, y] = rand.NextDouble() * 10;
}
}
}

/// <summary>
/// Prints the elements in the matrix to the console.
/// </summary>
public void Print()
{
for (int x = 0; x < size; x++)
{
for (int y = 0; y < size; y++)
{
Console.Write(data[x, y].ToString("0.00"));

if (y < size - 1)
{
Console.Write(", ");
}
}
Console.WriteLine();
}
}
}

/// <summary>
/// Class <i><b>RowColumnMultiplier</b></i>. Multiplies a row matrix with a
/// column matrix to produce a matrix with one element.
/// </summary>
[Serializable]
public class RowColumnMultiplier
{
/// <summary>
/// A row matrix to multiply
/// </summary>
private double[] row;

/// <summary>
/// A column matrix to multiply
/// </summary>
private double[] column;
 /// <summary>
/// The result of row-column multiplication
/// </summary>
private double result;

/// <summary>
/// Gets the result of the row-column multiplication
/// </summary>
public double Result
{
get { return this.result; }
}

/// <summary>
/// Creates a new RowColumnMultiplier
/// </summary>
/// <param name="row">The row to multiply</param>
/// <param name="column">The column to multiply</param>
public RowColumnMultiplier(double[] row, double[] column)
{
this.row = row;
this.column = column;
}

/// <summary>
/// Multiplies the row and column matrices
/// </summary>
public void DoMultiply()
{
// row and column are of the same dimension
for (int x = 0; x < row.Length; x++)
{
this.result += row[x] * column[x];
}
}
}

Once you have written and compiled you code, run your application by varying first the input size of
the matrix and then the number of nodes as specified in the question (for example: 10, 20, 30, ..).
Plot a single graph of execution time (y-axis) versus input size (x-axis) for the different sets of nodes
used, so that your final graph shows the difference in execution time for each of the sets of nodes.
Next plot a graph of speed-up (x-axis) versus input size (y-axis) for the different sets of nodes used.
Problem 6.20: Visit the Manjrasoft Aneka Software Center at web site: www.manjrasoft.com.
Download the Aneka software and create enterprise Cloud using LAN-connected PCs in your
university/student laboratory and run various example applications that come with Aneka. Write
simple parallel programs using Aneka’s Task, Thread, and MapReduce programming models and
carry out their execution on workers ranging from 2 to 20 in step of 2. Aneka (prof.Buyya)

Answer: The Aneka installation comes with a number of examples which you will find in the
installation’s destination directory. After installation take a look at the following directory (Note that
the directory will vary depending on where you installed Aneka.):

C:\Program Files\Manjrasoft\Aneka.2.1\Examples\Tutorials

Here you will find sample applications, including source code, for each of the different programming
models: MapReduce, Task and Thread. Open the Visual Studio solution for each of these examples,
compile and run them by varying the number of nodes as specified in the question. Observe that as
you increase the number of nodes, you’ll find that the execution time reduces (given that you have a
sufficiently large dataset to work with). Plot a graph showing the execution time (x-axis) versus the
input size (y-axis) for the different sets of nodes.