managing your MBrace.Azure clusters using F# interactive

This tutorial is from the MBrace Starter Kit. If using a locally simulated cluster you can ignore this tutorial.

In this tutorial you learn how you can use the MBrace.Azure.Management library to provision, monitor and scale an MBrace cluster using Visual Studio and F# Interactive. In order to proceed, you will need to sign up for an Azure subscription. Once signed up, download your publication settings file , which contains all authentication information necessary to manage your Azure subscription(s).

1: 
2:

// replace with your local .publishsettings path
let pubSettingsFile = Config.pubSettingsFile

Next, read your publish settings file by calling

1:	`let pubSettings = PublishSettings.ParseFile pubSettingsFile`

Yielding

1: 
2:

val pubSettings : PublishSettings =
  [|"Visual Studio Premium with MSDN"; "Azure in Open"; "Azure Free Trial"|]

Now we can select our subscription of preference

1:	`let subscription = pubSettings.GetSubscriptionById "Azure Free Trial"`

Select your prefered Azure region. This specifies the default Azure data center to be used for your deployments. Choosing a data center close to your location is recommended.

1:	`let myRegion = Region.North_Europe`

Let's now instantiate our subscription manager instance

1:	`let manager = SubscriptionManager.Create(subscription, myRegion)`

You now provision an MBrace cluster as follows:

1:	`let deployment = manager.Provision(serviceName = "mbracetest", vmCount = 4, vmSize = VMSize.A3)`

This will provision a 4-worker MBrace cluster of Large (A3) instances Provisioning can take some time (approximately 5 minutes).

If you have already provisioned a cluster, reconnect to it using the following:

1: 
2: 
3:

let clusterName = "... cluster name ..."

let deployment = Deployment.GetDeployment(pubSettings, clusterName)

To reconnect to a cluster when you don't have access to a pubsettings file, use:

1: 
2: 
3: 
4:

let serviceBusConnection = " ... enter ServiceBusConnectionString here ..."
let storageConnection = "... enter StorageConnectionString here ... "
let config = Configuration(storageConnection, serviceBusConnection)
let cluster = AzureCluster.Connect(config, logger = ConsoleLogger(true), logLevel = LogLevel.Info)

You can track the status of the cluster as follows:

1:	`deployment.ShowInfo()`

Which yields

1: 
2: 
3: 
4: 
5:

Cloud Service "mbracetest"

Name        Region        VM size  #Instances  Service Status  Deployment Status   Last Modified
----        ------        -------  ----------  --------------  -----------------   -------------
mbracetest  North Europe  Large    4           Created         Provisioning 33.3%  19/11/2015 6:24:01 pm

Finally, when deployment has completed we can easily obtain a cluster instance by typing

1:	`let cluster = AzureCluster.Connect deployment`

and submit our first computation to the cluster

1:	`Cloud.CurrentWorker \|> Cloud.ParallelEverywhere \|> cluster.Run`

Resizing the cluster is as simple as calling

1:	`deployment.Resize(vmCount = 20)`

When done, it's always a good idea to dispose of our deployment

1:	`deployment.Delete()`

Managing storage accounts

The subscription object can be used to manage Azure storage accounts

1:	`let newAccount = manager.Storage.CreateAccount(accountName = "mynewaccount", region = Region.Southeast_Asia)`

or it can be used to retrieve already existing accounts

1:	`let existingAccount = manager.Storage.GetAccount(accountName = "existingaccount")`

which can be used in subsequent cluster deployments

1:	`let deployment' = manager.Provision(vmCount = 10, region = Region.Southeast_Asia, storageAccount = newAccount.AccountName)`

A list of all accounts can be viewed by writing

1:	`manager.Storage.ShowAccounts()`

Yielding

1: 
2: 
3: 
4: 
5: 
6:

Azure Storage Accounts for subscription "Azure Free Trial"          

Account Name    Region          Account Type  Status   Affinity Group    
------------    ------          ------------  ------   --------------    
mynewaccount    Southeast Asia  Standard_LRS  Created  N/A
existingaccount North Europe    Standard_LRS  Created  N/A

Summary

In this tutorial, you've learned about MBrace.Azure.Management and how it can be used to deploy and manage MBrace clusters running on Azure.

namespace MBrace

namespace MBrace.Core

namespace MBrace.Azure

namespace MBrace.Azure.Management

val pubSettingsFile : string

Full name: 200-managing-azure-clusters.pubSettingsFile

Multiple items
module Config

--------------------
type Config =
  private new : unit -> Config
  static member DefaultLogger : ISystemLogger
  static member DefaultLogger : ISystemLogger with set

Full name: MBrace.Azure.Management.Config

val pubSettingsFile : string

Full name: Config.pubSettingsFile

val pubSettings : PublishSettings

Full name: 200-managing-azure-clusters.pubSettings

type PublishSettings =
  {Subscriptions: Subscription [];}
  member GetSubscriptionById : subscriptionId:string -> Subscription
  member Item : index:int -> Subscription
  member SubscriptionNames : string []
  static member Parse : xml:string -> PublishSettings
  static member ParseFile : publishSettingsFile:string -> PublishSettings

Full name: MBrace.Azure.Management.PublishSettings

static member PublishSettings.ParseFile : publishSettingsFile:string -> PublishSettings

val subscription : Subscription

Full name: 200-managing-azure-clusters.subscription

member PublishSettings.GetSubscriptionById : subscriptionId:string -> Subscription

val myRegion : Region

Full name: 200-managing-azure-clusters.myRegion

type Region =
  private new : regionId:string -> Region
  override Equals : other:obj -> bool
  override GetHashCode : unit -> int
  override ToString : unit -> string
  member Id : string
  static member Define : regionId:string -> Region
  static member Central_US : Region
  static member East_Asia : Region
  static member East_US : Region
  static member East_US_2 : Region
  ...

Full name: MBrace.Azure.Management.Region

property Region.North_Europe: Region

val manager : SubscriptionManager

Full name: 200-managing-azure-clusters.manager

type SubscriptionManager =
  private new : client:SubscriptionClient * defaultRegion:Region * _logger:ISystemLogger * logLevel:LogLevel -> SubscriptionManager
  member AttachLogger : l:ISystemLogger -> IDisposable
  member DeleteDeployment : serviceName:string * ?deleteStorageAccount:bool * ?deleteServiceBusAccount:bool -> unit
  member DeleteDeploymentAsync : serviceName:string * ?deleteStorageAccount:bool * ?deleteServiceBusAccount:bool -> Async<unit>
  member GetDeployment : serviceName:string -> Deployment
  member GetDeploymentAsync : serviceName:string -> Async<Deployment>
  member GetDeployments : unit -> Deployment []
  member GetDeploymentsAsync : unit -> Async<Deployment []>
  member Provision : vmCount:int * ?serviceName:string * ?region:Region * ?vmSize:VMSize * ?mbraceVersion:string * ?storageAccount:string * ?serviceBusAccount:string * ?cloudServicePackage:string * ?clusterLabel:string * ?enableDiagnostics:bool * ?reuseAccounts:bool -> Deployment
  member ProvisionAsync : vmCount:int * ?serviceName:string * ?region:Region * ?vmSize:VMSize * ?mbraceVersion:string * ?storageAccount:string * ?serviceBusAccount:string * ?cloudServicePackage:string * ?clusterLabel:string * ?enableDiagnostics:bool * ?reuseAccounts:bool -> Async<Deployment>
  ...

Full name: MBrace.Azure.Management.SubscriptionManager

static member SubscriptionManager.Create : subscription:Subscription * defaultRegion:Region * ?logger:MBrace.Runtime.ISystemLogger * ?logLevel:LogLevel -> SubscriptionManager

val deployment : Deployment

Full name: 200-managing-azure-clusters.deployment

member SubscriptionManager.Provision : vmCount:int * ?serviceName:string * ?region:Region * ?vmSize:VMSize * ?mbraceVersion:string * ?storageAccount:string * ?serviceBusAccount:string * ?cloudServicePackage:string * ?clusterLabel:string * ?enableDiagnostics:bool * ?reuseAccounts:bool -> Deployment

type VMSize =
  private new : vmId:string -> VMSize
  override Equals : other:obj -> bool
  override GetHashCode : unit -> int
  override ToString : unit -> string
  member Id : string
  static member Define : vmId:string -> VMSize
  static member A1 : VMSize
  static member A10 : VMSize
  static member A11 : VMSize
  static member A2 : VMSize
  ...

Full name: MBrace.Azure.Management.VMSize

property VMSize.A3: VMSize

member Deployment.ShowInfo : ?showVmInstances:bool -> unit

val cluster : AzureCluster

Full name: 200-managing-azure-clusters.cluster

type AzureCluster =
  inherit MBraceClient
  private new : manager:ClusterManager * faultPolicy:FaultPolicy option -> AzureCluster
  member AttachLocalWorker : ?workerId:string * ?workingDirectory:string * ?maxWorkItems:int * ?logFile:string * ?logLevel:LogLevel * ?quiet:bool * ?heartbeatInterval:TimeSpan * ?heartbeatThreshold:TimeSpan * ?background:bool -> unit
  member AttachLocalWorkers : workerCount:int * ?maxWorkItems:int * ?logLevel:LogLevel * ?quiet:bool * ?heartbeatInterval:TimeSpan * ?heartbeatThreshold:TimeSpan * ?background:bool -> unit
  member CullNonResponsiveWorkers : heartbeatThreshold:TimeSpan -> unit
  member CullNonResponsiveWorkersAsync : heartbeatThreshold:TimeSpan -> Async<unit>
  member KillAllLocalWorkers : unit -> unit
  member KillLocalWorker : worker:IWorkerRef -> bool
  member Reset : ?deleteQueues:bool * ?deleteRuntimeState:bool * ?deleteLogs:bool * ?deleteUserData:bool * ?deleteAssemblyData:bool * ?force:bool * ?reactivate:bool -> unit
  member ResetAsync : ?deleteQueues:bool * ?deleteRuntimeState:bool * ?deleteLogs:bool * ?deleteUserData:bool * ?deleteAssemblyData:bool * ?force:bool * ?reactivate:bool -> Async<unit>
  ...

Full name: MBrace.Azure.AzureCluster

static member AzureCluster.Connect : deployment:Deployment * ?clientId:string * ?faultPolicy:MBrace.Core.FaultPolicy * ?logger:MBrace.Runtime.ISystemLogger * ?logLevel:LogLevel -> AzureCluster
static member AzureCluster.Connect : config:Configuration * ?clientId:string * ?faultPolicy:MBrace.Core.FaultPolicy * ?logger:MBrace.Runtime.ISystemLogger * ?logLevel:LogLevel -> AzureCluster
static member AzureCluster.Connect : storageConnectionString:string * serviceBusConnectionString:string * ?clientId:string * ?faultPolicy:MBrace.Core.FaultPolicy * ?logger:MBrace.Runtime.ISystemLogger * ?logLevel:LogLevel -> AzureCluster

member MBrace.Runtime.MBraceClient.Run : workflow:MBrace.Core.Cloud<'T> * ?cancellationToken:MBrace.Core.ICloudCancellationToken * ?faultPolicy:MBrace.Core.FaultPolicy * ?target:MBrace.Core.IWorkerRef * ?additionalResources:MBrace.Core.Internals.ResourceRegistry * ?taskName:string -> 'T

member Deployment.Resize : vmCount:int -> unit

member Deployment.Delete : ?deleteStorageAccount:bool * ?deleteServiceBusAccount:bool -> unit

val newAccount : StorageAccount

Full name: 200-managing-azure-clusters.newAccount

property SubscriptionManager.Storage: StorageManager

member StorageManager.CreateAccount : accountName:string * ?region:Region -> StorageAccount

property Region.Southeast_Asia: Region

val existingAccount : StorageAccount

Full name: 200-managing-azure-clusters.existingAccount

member StorageManager.GetAccount : accountName:string -> StorageAccount

val deployment' : Deployment

Full name: 200-managing-azure-clusters.deployment'

property StorageAccount.AccountName: string

member StorageManager.ShowAccounts : ?region:Region -> unit

MBrace.Core and MBrace.Azure

managing your MBrace.Azure clusters using F# interactive

Managing storage accounts

Summary