Creating and Using Cloud Values

This tutorial is from the MBrace Starter Kit.

You now learn how to upload data to cloud storage using CloudValue and then process it using MBrace cloud tasks.

When using MBrace, data is implicitly uploaded if it is part of the closure of a cloud workflow - for example, if a value is referenced in a cloud { ... } block. That data is a transient part of the process specification. This is often the most convenient way to get small amounts (KB-MB) of data to the cloud: just use the data as part of a cloud workflow and run that work in the cloud.

If you wish to persist data in the cloud - for example, if it is too big to upload multiple times - then you can use one or more of the cloud data constructs that MBrace provides.

Note you can alternatively use any existing cloud storage APIs or SDKs you already have access to.

If using Azure, you can copy larger data to Azure using the AzCopy.exe command line tool, see Transfer data with the AzCopy Command-Line Utility. You can manage storage using the "azure" command line tool, see Install the Azure CLI Also, if you wish you can read/write using the .NET Azure storage SDKs directly rather than using MBrace primitives.

Here's some data (~1.0MB)

1:	`let mkData () = String.replicate 10000 "The quick brown fox jumped over the lazy dog\r\n"`

Generate the data, upload to blob storage and return a handle to the stored data

1: 
2: 
3: 
4: 
5: 
6:

let persistedCloudData = 
    cloud { 
            let data = mkData()
            let! cell = CloudValue.New data 
            return cell }
    |> cluster.Run

Run a cloud job which reads the blob and processes the data

1: 
2: 
3: 
4:

let lengthOfData = 
    cloud { let readData = persistedCloudData.Value
            return readData.Length }
    |> cluster.Run

Summary

In this tutorial, you've learned how to store values in cloud storage. Continue with further samples to learn more about the MBrace programming model.

Note, you can use the above techniques from both scripts and compiled projects. To see the components referenced by this script, see ThespianCluster.fsx or AzureCluster.fsx.

namespace System

namespace System.IO

namespace MBrace

namespace MBrace.Core

namespace MBrace.Flow

val cluster : MBrace.Thespian.ThespianCluster

Full name: 6-using-cloud-values.cluster

module Config

val GetCluster : unit -> MBrace.Thespian.ThespianCluster

Full name: Config.GetCluster

Gets or creates a new Thespian cluster session.

val mkData : unit -> string

Full name: 6-using-cloud-values.mkData

Multiple items
type String =
  new : value:char -> string + 7 overloads
  member Chars : int -> char
  member Clone : unit -> obj
  member CompareTo : value:obj -> int + 1 overload
  member Contains : value:string -> bool
  member CopyTo : sourceIndex:int * destination:char[] * destinationIndex:int * count:int -> unit
  member EndsWith : value:string -> bool + 2 overloads
  member Equals : obj:obj -> bool + 2 overloads
  member GetEnumerator : unit -> CharEnumerator
  member GetHashCode : unit -> int
  ...

Full name: System.String

--------------------
String(value: nativeptr<char>) : unit
String(value: nativeptr<sbyte>) : unit
String(value: char []) : unit
String(c: char, count: int) : unit
String(value: nativeptr<char>, startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int) : unit
String(value: char [], startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int, enc: Text.Encoding) : unit

val replicate : count:int -> str:string -> string

Full name: Microsoft.FSharp.Core.String.replicate

val persistedCloudData : obj

Full name: 6-using-cloud-values.persistedCloudData

member MBrace.Runtime.MBraceClient.Run : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> 'T

val lengthOfData : obj

Full name: 6-using-cloud-values.lengthOfData

MBrace.Core and MBrace.Azure

Creating and Using Cloud Values

Summary