Using C# DLLs and Native Components

This tutorial is from the MBrace Starter Kit.

It is very simple to use C# DLLs, native DLLs and any nuget packages in your cloud computations. For C# DLLs you just download and reference the packages as normal in your F# scripting or other client application. The DLLs for the packages are automatically uploaded to the cloud workers as needed. In a sense, you don't need to do anything special.

In this tutorial, you first reference some C# DLLs from the Math.NET NuGet package. You also use native binaries from the Intel MKL library.

Using C# DLLs Locally

First, you reference and use the packages on the local machine:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11:

#load @"../packages/MathNet.Numerics.FSharp/MathNet.Numerics.fsx"

open MathNet.Numerics
open MathNet.Numerics.LinearAlgebra

let matrix1 = Matrix<double>.Build.Random(10,10)
let vector1 = Vector<double>.Build.Random(10)

let product = vector1 * matrix1 

let check = (matrix1 * matrix1.Inverse()).Determinant()

Using C# DLLs on the cluster

Next, run the code on MBrace. Note that the DLLs from the packages are uploaded automatically.

The following inverts 100 150x150 matrices using C# code on the cluster.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17:

let csharpMathJob = 
    [ for i in 1 .. 100 -> 
         local { 
            Control.UseManaged()
            Control.UseSingleThread()
            let m = Matrix<double>.Build.Random(250,250) 
            return (m * m.Inverse()).Determinant()
         } ]
    |> Cloud.ParallelBalanced
    |> cluster.CreateProcess

// Show the progress
csharpMathJob.ShowInfo()


// Await the result, we expect an array of numbers very close to 1.0
let csharpMathResults = csharpMathJob.Result

Running Native Code on the Cluster

Next, you run the same work using the Interl MKL native DLLs.

To upload native DLLs, register their paths as native dependencies. These will be included with all uploaded dependencies of the session.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18:

let contentDir = __SOURCE_DIRECTORY__ + "/../packages/MathNet.Numerics.MKL.Win-x64/content/"
cluster.RegisterNativeDependency (contentDir + "libiomp5md.dll")
cluster.RegisterNativeDependency (contentDir + "MathNet.Numerics.MKL.dll")

(** The first MKL job can take a while first time you run it, because 'MathNet.Numerics.MKL.dll' needs to be uploaded: *) 
let firstNativeJob = 
    cloud { 
        Control.UseNativeMKL()
        let m = Matrix<double>.Build.Random(250,250) 
        return (m * m.Inverse()).Determinant()
    }
    |> cluster.CreateProcess

// Check progress
firstNativeJob.ShowInfo()

// Wait for the result
firstNativeJob.Result

Now run a much larger job: 1000 250x250 matrices, inverted using Intel MKL:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22:

let nativeMathJob = 
    [ for i in 1 .. 1000 -> 
         local { 
            Control.UseNativeMKL()
            Control.UseSingleThread()
            let m = Matrix<double>.Build.Random(250,250) 
            return (m * m.Inverse()).Determinant() 
         } ]
    |> Cloud.ParallelBalanced
    |> cluster.CreateProcess

// Check progress
nativeMathJob.ShowInfo()

// Wait for the result
nativeMathJob.Result

(** Once complete, you can compare the execution times of the jobs to see if using native code has improved performance: *) 
let timeNative = nativeMathJob.ExecutionTime.Value.TotalSeconds / 1000.0 
let timeCSharp = csharpMathJob.ExecutionTime.Value.TotalSeconds / 100.0  

let perfRatio = timeCSharp/timeNative

Summary

In this tutorial, you've learned how to use C# DLLs, NuGet packages and native DLLs in your MBrace computations. You've also compared performance between native code and C# code running on your cluster for one particular example.
Continue with further samples to learn more about the MBrace programming model.

Note, you can use the above techniques from both scripts and compiled projects. To see the components referenced by this script, see ThespianCluster.fsx or AzureCluster.fsx.

namespace System

namespace System.IO

namespace MBrace

namespace MBrace.Core

namespace MBrace.Flow

val cluster : MBrace.Thespian.ThespianCluster

Full name: 5-using-csharp-and-native-dlls.cluster

module Config

val GetCluster : unit -> MBrace.Thespian.ThespianCluster

Full name: Config.GetCluster

Gets or creates a new Thespian cluster session.

namespace MathNet

namespace MathNet.Numerics

namespace MathNet.Numerics.LinearAlgebra

val matrix1 : Matrix<double>

Full name: 5-using-csharp-and-native-dlls.matrix1

type Matrix<'T (requires default constructor and value type and 'T :> IEquatable<'T> and 'T :> IFormattable and 'T :> ValueType)> =
  member Add : scalar:'T -> Matrix<'T> + 3 overloads
  member Append : right:Matrix<'T> -> Matrix<'T> + 1 overload
  member At : row:int * column:int -> 'T + 1 overload
  member Cholesky : unit -> Cholesky<'T>
  member Clear : unit -> unit
  member ClearColumn : columnIndex:int -> unit
  member ClearColumns : [<ParamArray>] columnIndices:int[] -> unit
  member ClearRow : rowIndex:int -> unit
  member ClearRows : [<ParamArray>] rowIndices:int[] -> unit
  member ClearSubMatrix : rowIndex:int * rowCount:int * columnIndex:int * columnCount:int -> unit
  ...

Full name: MathNet.Numerics.LinearAlgebra.Matrix<_>

Multiple items
val double : value:'T -> double (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.double

--------------------
type double = Double

Full name: Microsoft.FSharp.Core.double

Multiple items
type Random =
  new : unit -> Random + 1 overload
  member Next : unit -> int + 2 overloads
  member NextBytes : buffer:byte[] -> unit
  member NextDouble : unit -> float

Full name: System.Random

--------------------
Random() : unit
Random(Seed: int) : unit

val vector1 : Vector<double>

Full name: 5-using-csharp-and-native-dlls.vector1

type Vector<'T (requires default constructor and value type and 'T :> IEquatable<'T> and 'T :> IFormattable and 'T :> ValueType)> =
  member AbsoluteMaximum : unit -> 'T
  member AbsoluteMaximumIndex : unit -> int
  member AbsoluteMinimum : unit -> 'T
  member AbsoluteMinimumIndex : unit -> int
  member Add : scalar:'T -> Vector<'T> + 3 overloads
  member At : index:int -> 'T + 1 overload
  member Clear : unit -> unit
  member ClearSubVector : index:int * count:int -> unit
  member Clone : unit -> Vector<'T>
  member CoerceZero : threshold:float -> unit + 1 overload
  ...

Full name: MathNet.Numerics.LinearAlgebra.Vector<_>

val product : Vector<double>

Full name: 5-using-csharp-and-native-dlls.product

val check : double

Full name: 5-using-csharp-and-native-dlls.check

Matrix.Inverse() : Matrix<double>

val csharpMathJob : MBrace.Runtime.CloudProcess<obj>

Full name: 5-using-csharp-and-native-dlls.csharpMathJob

val i : int

type Control =
  static member BlockSize : int with get, set
  static member CheckDistributionParameters : bool with get, set
  static member ConfigureAuto : unit -> unit
  static member LinearAlgebraProvider : ILinearAlgebraProvider with get, set
  static member MaxDegreeOfParallelism : int with get, set
  static member NativeProviderPath : string with get, set
  static member TaskScheduler : TaskScheduler with get, set
  static member ThreadSafeRandomNumberGenerators : bool with get, set
  static member UseManaged : unit -> unit
  static member UseMultiThreading : unit -> unit
  ...

Full name: MathNet.Numerics.Control

Control.UseManaged() : unit

Control.UseSingleThread() : unit

member MBrace.Runtime.MBraceClient.CreateProcess : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> MBrace.Runtime.CloudProcess<'T>

member MBrace.Runtime.CloudProcess.ShowInfo : unit -> unit

val csharpMathResults : obj

Full name: 5-using-csharp-and-native-dlls.csharpMathResults

property MBrace.Runtime.CloudProcess.Result: obj

val contentDir : string

Full name: 5-using-csharp-and-native-dlls.contentDir

member MBrace.Runtime.MBraceClient.RegisterNativeDependency : assemblyPath:string -> unit

val firstNativeJob : MBrace.Runtime.CloudProcess<obj>

Full name: 5-using-csharp-and-native-dlls.firstNativeJob

Control.UseNativeMKL() : unit
Control.UseNativeMKL(?consistency: Providers.LinearAlgebra.Mkl.MklConsistency, ?precision: Providers.LinearAlgebra.Mkl.MklPrecision, ?accuracy: Providers.LinearAlgebra.Mkl.MklAccuracy) : unit

val nativeMathJob : MBrace.Runtime.CloudProcess<obj>

Full name: 5-using-csharp-and-native-dlls.nativeMathJob

val timeNative : float

Full name: 5-using-csharp-and-native-dlls.timeNative

property MBrace.Runtime.CloudProcess.ExecutionTime: TimeSpan option

property Option.Value: TimeSpan

property TimeSpan.TotalSeconds: float

val timeCSharp : float

Full name: 5-using-csharp-and-native-dlls.timeCSharp

val perfRatio : float

Full name: 5-using-csharp-and-native-dlls.perfRatio

MBrace.Core and MBrace.Azure

Using C# DLLs and Native Components

Using C# DLLs Locally

Using C# DLLs on the cluster

Running Native Code on the Cluster

Summary