MBrace.Core and MBrace.Azure

Introduction to Cloud Combinators

This tutorial is from the MBrace Starter Kit.

You now perform a very simple parallel distributed job on your MBrace cluster.

You now use Cloud.Parallel to run 10 cloud workflows in parallel using fork-join pattern.

1: 
2: 
3: 
4: 
5: 
6: 

let parallelTask = 
    [ for i in 1 .. 10 -> cloud { return sprintf "i'm work item %d" i } ]
    |> Cloud.Parallel
    |> cluster.CreateProcess

cluster.ShowProcesses()

Get the results

1: 

let results = parallelTask.Result

Cloud.Parallel is not the only combinator for parallelism and for some items doesn't use all multi-core capacity on machines. The Cloud.ParallelBalanced combinator defined in lib/utils.fsx is designed for use when you have many single-machine, single-core items to run on multiple machines using multi-core capacity.

When specifying individual work items that are constrained to only execute on a single machine you use local { ... }. These work items must commonly be used as inputs to parallelism combinators that use particular multi-core aware scheduling strategies.

1: 
2: 
3: 
4: 
5: 
6: 

let parallelTask2 = 
    [ for i in 1 .. 10 -> local { return sprintf "i'm work item %d" i } ]
    |> Cloud.ParallelBalanced
    |> cluster.CreateProcess

cluster.ShowProcesses()

Get the results

1: 

let results2 = parallelTask2.Result

Again, in shorthand

1: 
2: 
3: 
4: 

let quickResults =
    [ for i in 1 .. 50 -> local { return sprintf "i'm work item %d" i } ]
    |> Cloud.ParallelBalanced
    |> cluster.Run

Next you use Cloud.Choice: the first cloud workflow to return "Some" wins.

1: 
2: 
3: 
4: 
5: 
6: 

let searchTask =
    [ for i in 1 .. 50 -> cloud { if i % 10 = 0 then return Some i else return None } ]
    |> Cloud.Choice
    |> cluster.CreateProcess

searchTask.ShowInfo()

Await the result of the search:

1: 

let searchResult = searchTask.Result

Like Cloud.ParallelBalanced, you can also use Cloud.ChoiceBalanced, which is multi-core aware.

1: 
2: 
3: 
4: 
5: 
6: 

let searchTask2 =
    [ for i in 1 .. 50 -> local { if i % 10 = 0 then return Some i else return None } ]
    |> Cloud.ChoiceBalanced
    |> cluster.CreateProcess

searchTask2.ShowInfo()

Await the result of the search:

1: 

let searchResult2 = searchTask2.Result

Summary

In this tutorial, you've learned about Cloud.Parallel and Cloud.Choice as ways of composing cloud workflows. You've also learned about Cloud.ParallelBalanced and Cloud.ChoiceBalanced defined in lib/utils.fsx which utlize the multi-code capacity off your cluster. Continue with further samples to learn more about the MBrace programming model.

Note, you can use the above techniques from both scripts and compiled projects. To see the components referenced by this script, see ThespianCluster.fsx or AzureCluster.fsx.

namespace System
namespace System.IO
namespace MBrace
namespace MBrace.Core
namespace MBrace.Flow
val cluster : MBrace.Thespian.ThespianCluster

Full name: 2-cloud-parallel.cluster
module Config
val GetCluster : unit -> MBrace.Thespian.ThespianCluster

Full name: Config.GetCluster


 Gets or creates a new Thespian cluster session.
val parallelTask : MBrace.Runtime.CloudProcess<obj>

Full name: 2-cloud-parallel.parallelTask
val i : int
val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
member MBrace.Runtime.MBraceClient.CreateProcess : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> MBrace.Runtime.CloudProcess<'T>
member MBrace.Runtime.MBraceClient.ShowProcesses : unit -> unit
val results : obj

Full name: 2-cloud-parallel.results
property MBrace.Runtime.CloudProcess.Result: obj
val parallelTask2 : MBrace.Runtime.CloudProcess<obj>

Full name: 2-cloud-parallel.parallelTask2
val results2 : obj

Full name: 2-cloud-parallel.results2
val quickResults : obj

Full name: 2-cloud-parallel.quickResults
member MBrace.Runtime.MBraceClient.Run : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> 'T
val searchTask : MBrace.Runtime.CloudProcess<obj>

Full name: 2-cloud-parallel.searchTask
union case Option.Some: Value: 'T -> Option<'T>
union case Option.None: Option<'T>
Multiple items
type Choice<'T1,'T2> =
  | Choice1Of2 of 'T1
  | Choice2Of2 of 'T2

Full name: Microsoft.FSharp.Core.Choice<_,_>

--------------------
type Choice<'T1,'T2,'T3> =
  | Choice1Of3 of 'T1
  | Choice2Of3 of 'T2
  | Choice3Of3 of 'T3

Full name: Microsoft.FSharp.Core.Choice<_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4> =
  | Choice1Of4 of 'T1
  | Choice2Of4 of 'T2
  | Choice3Of4 of 'T3
  | Choice4Of4 of 'T4

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5> =
  | Choice1Of5 of 'T1
  | Choice2Of5 of 'T2
  | Choice3Of5 of 'T3
  | Choice4Of5 of 'T4
  | Choice5Of5 of 'T5

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5,'T6> =
  | Choice1Of6 of 'T1
  | Choice2Of6 of 'T2
  | Choice3Of6 of 'T3
  | Choice4Of6 of 'T4
  | Choice5Of6 of 'T5
  | Choice6Of6 of 'T6

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5,'T6,'T7> =
  | Choice1Of7 of 'T1
  | Choice2Of7 of 'T2
  | Choice3Of7 of 'T3
  | Choice4Of7 of 'T4
  | Choice5Of7 of 'T5
  | Choice6Of7 of 'T6
  | Choice7Of7 of 'T7

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_,_,_>
member MBrace.Runtime.CloudProcess.ShowInfo : unit -> unit
val searchResult : obj

Full name: 2-cloud-parallel.searchResult
val searchTask2 : MBrace.Runtime.CloudProcess<obj>

Full name: 2-cloud-parallel.searchTask2
val searchResult2 : obj

Full name: 2-cloud-parallel.searchResult2
F# Project
Fork me on GitHub