MBrace.Core and MBrace.Azure

Creating and Using Cloud Values

This tutorial is from the MBrace Starter Kit.

You now learn how to upload data to cloud storage using CloudValue and then process it using MBrace cloud tasks.

When using MBrace, data is implicitly uploaded if it is part of the closure of a cloud workflow - for example, if a value is referenced in a cloud { ... } block. That data is a transient part of the process specification. This is often the most convenient way to get small amounts (KB-MB) of data to the cloud: just use the data as part of a cloud workflow and run that work in the cloud.

If you wish to persist data in the cloud - for example, if it is too big to upload multiple times - then you can use one or more of the cloud data constructs that MBrace provides.

Note you can alternatively use any existing cloud storage APIs or SDKs you already have access to.

If using Azure, you can copy larger data to Azure using the AzCopy.exe command line tool, see Transfer data with the AzCopy Command-Line Utility. You can manage storage using the "azure" command line tool, see Install the Azure CLI Also, if you wish you can read/write using the .NET Azure storage SDKs directly rather than using MBrace primitives.

Here's some data (~1.0MB)

let mkData () = String.replicate 10000 "The quick brown fox jumped over the lazy dog\r\n" 

Generate the data, upload to blob storage and return a handle to the stored data

let persistedCloudData = 
    cloud { 
            let data = mkData()
            let! cell = CloudValue.New data 
            return cell }
    |> cluster.Run

Run a cloud job which reads the blob and processes the data

let lengthOfData = 
    cloud { let readData = persistedCloudData.Value
            return readData.Length }
    |> cluster.Run


In this tutorial, you've learned how to store values in cloud storage. Continue with further samples to learn more about the MBrace programming model.

Note, you can use the above techniques from both scripts and compiled projects. To see the components referenced by this script, see ThespianCluster.fsx or AzureCluster.fsx.

namespace System
namespace System.IO
namespace MBrace
namespace MBrace.Core
namespace MBrace.Flow
val cluster : MBrace.Thespian.ThespianCluster

Full name: 6-using-cloud-values.cluster
module Config
val GetCluster : unit -> MBrace.Thespian.ThespianCluster

Full name: Config.GetCluster

 Gets or creates a new Thespian cluster session.
val mkData : unit -> string

Full name: 6-using-cloud-values.mkData
Multiple items
type String =
  new : value:char -> string + 7 overloads
  member Chars : int -> char
  member Clone : unit -> obj
  member CompareTo : value:obj -> int + 1 overload
  member Contains : value:string -> bool
  member CopyTo : sourceIndex:int * destination:char[] * destinationIndex:int * count:int -> unit
  member EndsWith : value:string -> bool + 2 overloads
  member Equals : obj:obj -> bool + 2 overloads
  member GetEnumerator : unit -> CharEnumerator
  member GetHashCode : unit -> int

Full name: System.String

String(value: nativeptr<char>) : unit
String(value: nativeptr<sbyte>) : unit
String(value: char []) : unit
String(c: char, count: int) : unit
String(value: nativeptr<char>, startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int) : unit
String(value: char [], startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int, enc: Text.Encoding) : unit
val replicate : count:int -> str:string -> string

Full name: Microsoft.FSharp.Core.String.replicate
val persistedCloudData : obj

Full name: 6-using-cloud-values.persistedCloudData
member MBrace.Runtime.MBraceClient.Run : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> 'T
val lengthOfData : obj

Full name: 6-using-cloud-values.lengthOfData
Fork me on GitHub