Wade Wegner

Significant updates to the SQL Azure Migration Wizard

George Huey has done it again!  He has just published some significant updates to the SQL Azure Migration Wizard.

Previously, I’ve described the SQL Azure Migration Wizard as a tool that helps you migrate your SQL Server database into SQL Azure.  This is still true, but now, thanks to updates made by George Huey, you can also migrate from SQL Azure-to-SQL Server and SQL Azure-to-SQL Azure.  These are significant updates to the tool!

Please watch the following video for an updated explanation of the tool:

As I said, the updates made by George enable all the following scenarios for database migration …

• SQL Server-to-SQL Azure
• SQL Azure-to-SQL Server
• SQL Azure-to-SQL Azure

These last two updates are significant!  Take a look at this thread on the SQL Azure Migration Wizard codeplex site – the user had a scenario where they wanted to migrate a 1 GB database in SQL Azure into a 10 GB database in SQL Azure.

Please take a look at the SQL Azure Migration Wizard up on Codeplex, where you can download the source code and/or binaries.

Leveraging WMI in an Azure Web Role

A few weeks ago I threw out a teaser on Twitter:

This got the attention of a few folks, and I promised I’d follow-up on this with some details on how I got this to work.  Of course, I completely forgot to do so, and was only recently reminded (thanks, Roger Jennings).

You can still take a look at that application here (if nothing else, I find it interesting to look at the specifications of the CTP machines for Windows Azure): http://wmi.cloudapp.net/

In actuality, the solution is really quite straightforward – I used WMI.

What’s WMI?  Well, it stands for Windows Management Instrumentation.  Here’s quick blurb from MSDN:

Windows Management Instrumentation (WMI) is the infrastructure for management data and operations on Windows-based operating systems. You can write WMI scripts or applications to automate administrative tasks on remote computers but WMI also supplies management data to other parts of the operating system and products, for example System Center Operations Manager, formerly Microsoft Operations Manager (MOM), or Windows Remote Management.

For more information on how to use WMI in .NET, take a look at the WMI .NET Overview.  Lots of great information.

Assuming you’ve read the overview, it should be clear that this is really pretty easy.  At the heart of this are the following two lines of code …

ManagementClass mgmt = new ManagementClass(managementClass);

ManagementObjectCollection objCol = mgmt.GetInstances();

… where managementClass represent one of the Win32 classes (i.e. Win32_NTLogEventLog).

Now, to make this work, you need to iterate through the collection of management objects.  Something like the following works:

foreach (var obj in objCol)

{

    foreach (var prop in obj.Properties)

    {

        ...

    }

}

Now, I did two additional things to make this easy.

1. I created a dropdown list with all the Win32 classes so that you can easily choose which class to review.
2. I defined a querystring value that, if it exists, loads a specific Win32 class.

There are a lot of Win32 classes, so here’s the array I used – feel free to leverage it yourself:

public static string[] Win32Classes =

{

    "Win32_1394Controller",

    "Win32_1394ControllerDevice",

    "Win32_Account",

    "Win32_AccountSID",

    "Win32_ACE",

    "Win32_ActionCheck",

    "Win32_ActiveRoute",

    "Win32_AllocatedResource",

    "Win32_ApplicationCommandLine",

    "Win32_ApplicationService",

    "Win32_AssociatedBattery",

    "Win32_AssociatedProcessorMemory",

    "Win32_AutochkSetting",

    "Win32_BaseBoard",

    "Win32_BaseService",

    "Win32_Battery",

    "Win32_Binary",

    "Win32_BindImageAction",

    "Win32_BIOS",

    "Win32_BootConfiguration",

    "Win32_Bus",

    "Win32_CacheMemory",

    "Win32_CDROMDrive",

    "Win32_CheckCheck",

    "Win32_CIMLogicalDeviceCIMDataFile",

    "Win32_ClassicCOMApplicationClasses",

    "Win32_ClassicCOMClass",

    "Win32_ClassicCOMClassSetting",

    "Win32_ClassicCOMClassSettings",

    "Win32_ClassInfoAction",

    "Win32_ClientApplicationSetting",

    "Win32_CodecFile",

    "Win32_CollectionStatistics",

    "Win32_COMApplication",

    "Win32_COMApplicationClasses",

    "Win32_COMApplicationSettings",

    "Win32_COMClass",

    "Win32_ComClassAutoEmulator",

    "Win32_ComClassEmulator",

    "Win32_CommandLineAccess",

    "Win32_ComponentCategory",

    "Win32_ComputerShutdownEvent",

    "Win32_ComputerSystem",

    "Win32_ComputerSystemEvent",

    "Win32_ComputerSystemProcessor",

    "Win32_ComputerSystemProduct",

    "Win32_ComputerSystemWindowsProductActivationSetting",

    "Win32_COMSetting",

    "Win32_Condition",

    "Win32_ConnectionShare",

    "Win32_ControllerHasHub",

    "Win32_CreateFolderAction",

    "Win32_CurrentProbe",

    "Win32_CurrentTime",

    "Win32_DCOMApplication",

    "Win32_DCOMApplicationAccessAllowedSetting",

    "Win32_DCOMApplicationLaunchAllowedSetting",

    "Win32_DCOMApplicationSetting",

    "Win32_DefragAnalysis",

    "Win32_DependentService",

    "Win32_Desktop",

    "Win32_DesktopMonitor",

    "Win32_DeviceBus",

    "Win32_DeviceChangeEvent",

    "Win32_DeviceMemoryAddress",

    "Win32_DeviceSettings",

    "Win32_DFSNode",

    "Win32_DFSNodeTarget",

    "Win32_DFSTarget",

    "Win32_Directory",

    "Win32_DirectorySpecification",

    "Win32_DiskDrive",

    "Win32_DiskDrivePhysicalMedia",

    "Win32_DiskDriveToDiskPartition",

    "Win32_DiskPartition",

    "Win32_DiskQuota",

    "Win32_DisplayConfiguration",

    "Win32_DisplayControllerConfiguration",

    "Win32_DMAChannel",

    "Win32_DriverForDevice",

    "Win32_DriverVXD",

    "Win32_DuplicateFileAction",

    "Win32_Environment",

    "Win32_EnvironmentSpecification",

    "Win32_ExtensionInfoAction",

    "Win32_Fan",

    "Win32_FileSpecification",

    "Win32_FloppyController",

    "Win32_FloppyDrive",

    "Win32_FontInfoAction",

    "Win32_Group",

    "Win32_GroupInDomain",

    "Win32_GroupUser",

    "Win32_HeatPipe",

    "Win32_IDEController",

    "Win32_IDEControllerDevice",

    "Win32_ImplementedCategory",

    "Win32_InfraredDevice",

    "Win32_IniFileSpecification",

    "Win32_InstalledSoftwareElement",

    "Win32_IP4PersistedRouteTable",

    "Win32_IP4RouteTable",

    "Win32_IP4RouteTableEvent",

    "Win32_IRQResource",

    "Win32_JobObjectStatus",

    "Win32_Keyboard",

    "Win32_LaunchCondition",

    "Win32_LoadOrderGroup",

    "Win32_LoadOrderGroupServiceDependencies",

    "Win32_LoadOrderGroupServiceMembers",

    "Win32_LocalTime",

    "Win32_LoggedOnUser",

    "Win32_LogicalDisk",

    "Win32_LogicalDiskRootDirectory",

    "Win32_LogicalDiskToPartition",

    "Win32_LogicalFileAccess",

    "Win32_LogicalFileAuditing",

    "Win32_LogicalFileGroup",

    "Win32_LogicalFileOwner",

    "Win32_LogicalFileSecuritySetting",

    "Win32_LogicalMemoryConfiguration",

    "Win32_LogicalProgramGroup",

    "Win32_LogicalProgramGroupDirectory",

    "Win32_LogicalProgramGroupItem",

    "Win32_LogicalProgramGroupItemDataFile",

    "Win32_LogicalShareAccess",

    "Win32_LogicalShareAuditing",

    "Win32_LogicalShareSecuritySetting",

    "Win32_LogonSession",

    "Win32_LogonSessionMappedDisk",

    "Win32_LUID",

    "Win32_LUIDandAttributes",

    "Win32_ManagedSystemElementResource",

    "Win32_MappedLogicalDisk",

    "Win32_MemoryArray",

    "Win32_MemoryArrayLocation",

    "Win32_MemoryDevice",

    "Win32_MemoryDeviceArray",

    "Win32_MemoryDeviceLocation",

    "Win32_MethodParameterClass",

    "Win32_MIMEInfoAction",

    "Win32_ModuleLoadTrace",

    "Win32_ModuleTrace",

    "Win32_MotherboardDevice",

    "Win32_MountPoint",

    "Win32_MoveFileAction",

    "Win32_MSIResource",

    "Win32_NamedJobObject",

    "Win32_NamedJobObjectActgInfo",

    "Win32_NamedJobObjectLimit",

    "Win32_NamedJobObjectLimitSetting",

    "Win32_NamedJobObjectProcess",

    "Win32_NamedJobObjectSecLimit",

    "Win32_NamedJobObjectSecLimitSetting",

    "Win32_NamedJobObjectStatistics",

    "Win32_NetworkAdapter",

    "Win32_NetworkAdapterConfiguration",

    "Win32_NetworkAdapterSetting",

    "Win32_NetworkClient",

    "Win32_NetworkConnection",

    "Win32_NetworkLoginProfile",

    "Win32_NetworkProtocol",

    "Win32_NTDomain",

    "Win32_NTEventlogFile",

    "Win32_NTLogEvent",

    "Win32_NTLogEventComputer",

    "Win32_NTLogEventLog",

    "Win32_NTLogEventUser",

    "Win32_ODBCAttribute",

    "Win32_ODBCDataSourceAttribute",

    "Win32_ODBCDataSourceSpecification",

    "Win32_ODBCDriverAttribute",

    "Win32_ODBCDriverSoftwareElement",

    "Win32_ODBCDriverSpecification",

    "Win32_ODBCSourceAttribute",

    "Win32_ODBCTranslatorSpecification",

    "Win32_OnBoardDevice",

    "Win32_OperatingSystem",

    "Win32_OperatingSystemAutochkSetting",

    "Win32_OperatingSystemQFE",

    "Win32_OptionalFeature",

    "Win32_OSRecoveryConfiguration",

    "Win32_PageFile",

    "Win32_PageFileElementSetting",

    "Win32_PageFileSetting",

    "Win32_PageFileUsage",

    "Win32_ParallelPort",

    "Win32_Patch",

    "Win32_PatchFile",

    "Win32_PatchPackage",

    "Win32_PCMCIAController",

    "Win32_Perf",

    "Win32_PerfFormattedData",

    "Win32_PerfFormattedData_ASP_ActiveServerPages",

    "Win32_PerfFormattedData_ContentFilter_IndexingServiceFilter",

    "Win32_PerfFormattedData_ContentIndex_IndexingService",

    "Win32_PerfFormattedData_InetInfo_InternetInformationServicesGlobal",

    "Win32_PerfFormattedData_ISAPISearch_HttpIndexingService",

    "Win32_PerfFormattedData_MSDTC_DistributedTransactionCoordinator",

    "Win32_PerfFormattedData_NTFSDRV_SMTPNTFSStoreDriver",

    "Win32_PerfFormattedData_PerfDisk_LogicalDisk",

    "Win32_PerfFormattedData_PerfDisk_PhysicalDisk",

    "Win32_PerfFormattedData_PerfNet_Browser",

    "Win32_PerfFormattedData_PerfNet_Redirector",

    "Win32_PerfFormattedData_PerfNet_Server",

    "Win32_PerfFormattedData_PerfNet_ServerWorkQueues",

    "Win32_PerfFormattedData_PerfOS_Cache",

    "Win32_PerfFormattedData_PerfOS_Memory",

    "Win32_PerfFormattedData_PerfOS_Objects",

    "Win32_PerfFormattedData_PerfOS_PagingFile",

    "Win32_PerfFormattedData_PerfOS_Processor",

    "Win32_PerfFormattedData_PerfOS_System",

    "Win32_PerfFormattedData_PerfProc_FullImage_Costly",

    "Win32_PerfFormattedData_PerfProc_Image_Costly",

    "Win32_PerfFormattedData_PerfProc_JobObject",

    "Win32_PerfFormattedData_PerfProc_JobObjectDetails",

    "Win32_PerfFormattedData_PerfProc_Process",

    "Win32_PerfFormattedData_PerfProc_ProcessAddressSpace_Costly",

    "Win32_PerfFormattedData_PerfProc_Thread",

    "Win32_PerfFormattedData_PerfProc_ThreadDetails_Costly",

    "Win32_PerfFormattedData_PSched_PSchedFlow",

    "Win32_PerfFormattedData_PSched_PSchedPipe",

    "Win32_PerfFormattedData_RemoteAccess_RASPort",

    "Win32_PerfFormattedData_RemoteAccess_RASTotal",

    "Win32_PerfFormattedData_RSVP_ACSRSVPInterfaces",

    "Win32_PerfFormattedData_RSVP_ACSRSVPService",

    "Win32_PerfFormattedData_SMTPSVC_SMTPServer",

    "Win32_PerfFormattedData_Spooler_PrintQueue",

    "Win32_PerfFormattedData_TapiSrv_Telephony",

    "Win32_PerfFormattedData_Tcpip_ICMP",

    "Win32_PerfFormattedData_Tcpip_IP",

    "Win32_PerfFormattedData_Tcpip_NBTConnection",

    "Win32_PerfFormattedData_Tcpip_NetworkInterface",

    "Win32_PerfFormattedData_Tcpip_TCP",

    "Win32_PerfFormattedData_Tcpip_UDP",

    "Win32_PerfFormattedData_TermService_TerminalServices",

    "Win32_PerfFormattedData_TermService_TerminalServicesSession",

    "Win32_PerfFormattedData_W3SVC_WebService",

    "Win32_PerfRawData",

    "Win32_PerfRawData_ASP_ActiveServerPages",

    "Win32_PerfRawData_ContentFilter_IndexingServiceFilter",

    "Win32_PerfRawData_ContentIndex_IndexingService",

    "Win32_PerfRawData_InetInfo_InternetInformationServicesGlobal",

    "Win32_PerfRawData_ISAPISearch_HttpIndexingService",

    "Win32_PerfRawData_MSDTC_DistributedTransactionCoordinator",

    "Win32_PerfRawData_NTFSDRV_SMTPNTFSStoreDriver",

    "Win32_PerfRawData_PerfDisk_LogicalDisk",

    "Win32_PerfRawData_PerfDisk_PhysicalDisk",

    "Win32_PerfRawData_PerfNet_Browser",

    "Win32_PerfRawData_PerfNet_Redirector",

    "Win32_PerfRawData_PerfNet_Server",

    "Win32_PerfRawData_PerfNet_ServerWorkQueues",

    "Win32_PerfRawData_PerfOS_Cache",

    "Win32_PerfRawData_PerfOS_Memory",

    "Win32_PerfRawData_PerfOS_Objects",

    "Win32_PerfRawData_PerfOS_PagingFile",

    "Win32_PerfRawData_PerfOS_Processor",

    "Win32_PerfRawData_PerfOS_System",

    "Win32_PerfRawData_PerfProc_FullImage_Costly",

    "Win32_PerfRawData_PerfProc_Image_Costly",

    "Win32_PerfRawData_PerfProc_JobObject",

    "Win32_PerfRawData_PerfProc_JobObjectDetails",

    "Win32_PerfRawData_PerfProc_Process",

    "Win32_PerfRawData_PerfProc_ProcessAddressSpace_Costly",

    "Win32_PerfRawData_PerfProc_Thread",

    "Win32_PerfRawData_PerfProc_ThreadDetails_Costly",

    "Win32_PerfRawData_PSched_PSchedFlow",

    "Win32_PerfRawData_PSched_PSchedPipe",

    "Win32_PerfRawData_RemoteAccess_RASPort",

    "Win32_PerfRawData_RemoteAccess_RASTotal",

    "Win32_PerfRawData_RSVP_ACSRSVPInterfaces",

    "Win32_PerfRawData_RSVP_ACSRSVPService",

    "Win32_PerfRawData_SMTPSVC_SMTPServer",

    "Win32_PerfRawData_Spooler_PrintQueue",

    "Win32_PerfRawData_TapiSrv_Telephony",

    "Win32_PerfRawData_Tcpip_ICMP",

    "Win32_PerfRawData_Tcpip_IP",

    "Win32_PerfRawData_Tcpip_NBTConnection",

    "Win32_PerfRawData_Tcpip_NetworkInterface",

    "Win32_PerfRawData_Tcpip_TCP",

    "Win32_PerfRawData_Tcpip_UDP",

    "Win32_PerfRawData_TermService_TerminalServices",

    "Win32_PerfRawData_TermService_TerminalServicesSession",

    "Win32_PerfRawData_W3SVC_WebService",

    "Win32_PhysicalMedia",

    "Win32_PhysicalMemory",

    "Win32_PhysicalMemoryArray",

    "Win32_PhysicalMemoryLocation",

    "Win32_PingStatus",

    "Win32_PnPAllocatedResource",

    "Win32_PnPDevice",

    "Win32_PnPEntity",

    "Win32_PnPSignedDriver",

    "Win32_PnPSignedDriverCIMDataFile",

    "Win32_PointingDevice",

    "Win32_PortableBattery",

    "Win32_PortConnector",

    "Win32_PortResource",

    "Win32_POTSModem",

    "Win32_POTSModemToSerialPort",

    "Win32_PowerManagementEvent",

    "Win32_Printer",

    "Win32_PrinterConfiguration",

    "Win32_PrinterController",

    "Win32_PrinterDriver",

    "Win32_PrinterDriverDll",

    "Win32_PrinterSetting",

    "Win32_PrinterShare",

    "Win32_PrintJob",

    "Win32_PrivilegesStatus",

    "Win32_Process",

    "Win32_Processor",

    "Win32_ProcessStartTrace",

    "Win32_ProcessStartup",

    "Win32_ProcessStopTrace",

    "Win32_ProcessTrace",

    "Win32_Product",

    "Win32_ProductCheck",

    "Win32_ProductResource",

    "Win32_ProductSoftwareFeatures",

    "Win32_ProgIDSpecification",

    "Win32_ProgramGroup",

    "Win32_ProgramGroupContents",

    "Win32_ProgramGroupOrItem",

    "Win32_Property",

    "Win32_ProtocolBinding",

    "Win32_Proxy",

    "Win32_PublishComponentAction",

    "Win32_QuickFixEngineering",

    "Win32_QuotaSetting",

    "Win32_Refrigeration",

    "Win32_Registry",

    "Win32_RegistryAction",

    "Win32_RemoveFileAction",

    "Win32_RemoveIniAction",

    "Win32_ReserveCost",

    "Win32_ScheduledJob",

    "Win32_SCSIController",

    "Win32_SCSIControllerDevice",

    "Win32_SecurityDescriptor",

    "Win32_SecurityDescriptorHelper",

    "Win32_SecuritySetting",

    "Win32_SecuritySettingAccess",

    "Win32_SecuritySettingAuditing",

    "Win32_SecuritySettingGroup",

    "Win32_SecuritySettingOfLogicalFile",

    "Win32_SecuritySettingOfLogicalShare",

    "Win32_SecuritySettingOfObject",

    "Win32_SecuritySettingOwner",

    "Win32_SelfRegModuleAction",

    "Win32_SerialPort",

    "Win32_SerialPortConfiguration",

    "Win32_SerialPortSetting",

    "Win32_ServerConnection",

    "Win32_ServerFeature",

    "Win32_ServerSession",

    "Win32_Service",

    "Win32_ServiceControl",

    "Win32_ServiceSpecification",

    "Win32_ServiceSpecificationService",

    "Win32_Session",

    "Win32_SessionConnection",

    "Win32_SessionProcess",

    "Win32_SettingCheck",

    "Win32_ShadowBy",

    "Win32_ShadowContext",

    "Win32_ShadowCopy",

    "Win32_ShadowDiffVolumeSupport",

    "Win32_ShadowFor",

    "Win32_ShadowOn",

    "Win32_ShadowProvider",

    "Win32_ShadowStorage",

    "Win32_ShadowVolumeSupport",

    "Win32_Share",

    "Win32_ShareToDirectory",

    "Win32_ShortcutAction",

    "Win32_ShortcutFile",

    "Win32_ShortcutSAP",

    "Win32_SID",

    "Win32_SIDandAttributes",

    "Win32_SMBIOSMemory",

    "Win32_SoftwareElement",

    "Win32_SoftwareElementAction",

    "Win32_SoftwareElementCheck",

    "Win32_SoftwareElementCondition",

    "Win32_SoftwareElementResource",

    "Win32_SoftwareFeature",

    "Win32_SoftwareFeatureAction",

    "Win32_SoftwareFeatureCheck",

    "Win32_SoftwareFeatureParent",

    "Win32_SoftwareFeatureSoftwareElements",

    "Win32_SoundDevice",

    "Win32_StartupCommand",

    "Win32_SubDirectory",

    "Win32_SystemAccount",

    "Win32_SystemBIOS",

    "Win32_SystemBootConfiguration",

    "Win32_SystemConfigurationChangeEvent",

    "Win32_SystemDesktop",

    "Win32_SystemDevices",

    "Win32_SystemDriver",

    "Win32_SystemDriverPnPEntity",

    "Win32_SystemEnclosure",

    "Win32_SystemLoadOrderGroups",

    "Win32_SystemLogicalMemoryConfiguration",

    "Win32_SystemMemoryResource",

    "Win32_SystemNetworkConnections",

    "Win32_SystemOperatingSystem",

    "Win32_SystemPartitions",

    "Win32_SystemProcesses",

    "Win32_SystemProgramGroups",

    "Win32_SystemResources",

    "Win32_SystemServices",

    "Win32_SystemSetting",

    "Win32_SystemSlot",

    "Win32_SystemSystemDriver",

    "Win32_SystemTimeZone",

    "Win32_SystemTrace",

    "Win32_SystemUsers",

    "Win32_TapeDrive",

    "Win32_TCPIPPrinterPort",

    "Win32_TemperatureProbe",

    "Win32_Thread",

    "Win32_ThreadStartTrace",

    "Win32_ThreadStopTrace",

    "Win32_ThreadTrace",

    "Win32_TimeZone",

    "Win32_TokenGroups",

    "Win32_TokenPrivileges",

    "Win32_Trustee",

    "Win32_TypeLibraryAction",

    "Win32_UninterruptiblePowerSupply",

    "Win32_USBController",

    "Win32_USBControllerDevice",

    "Win32_USBHub",

    "Win32_UserAccount",

    "Win32_UserDesktop",

    "Win32_UserInDomain",

    "Win32_UTCTime",

    "Win32_VideoConfiguration",

    "Win32_VideoController",

    "Win32_VideoSettings",

    "Win32_VoltageProbe",

    "Win32_Volume",

    "Win32_VolumeChangeEvent",

    "Win32_VolumeQuota",

    "Win32_VolumeQuotaSetting",

    "Win32_VolumeUserQuota",

    "Win32_WindowsProductActivation",

    "Win32_WMIElementSetting",

    "Win32_WMISetting"

};

Other than attaching the results of the WMI query to a generic list and binding to a DataList, that’s pretty much it.  The end result is a web page that you can use to view system details via WMI:

As I’ve said, this is a very simple demonstration of WMI in Windows Azure.  What would be more interesting is to see how someone might combine the ability to spin up asynchronous processes in Windows Azure (as described in my post How to Leverage the RoleEntryPoint in an Azure Web Role) that query data via WMI and centralize it in Azure storage (i.e. combining and centralizing all event logs) – that would be very cool!

In case anyone wants to take a look, I’ve uploaded the code to SkyDrive.

Enjoy!

How to Leverage the RoleEntryPoint in an Azure Web Role

One of the advantages to the approach our teams building the Windows Azure Platform have taken is flexibility.  Recently, when I spoke at the Day of Cloud presentation, I recall Don Schwarz from Google making these two points (you can see video of his talk here):

• You can’t spin up your own threads in Google App Engine.
• You build your applications according to how Google thinks your apps should be built (the argument being that Google knows how to run highly available services at scale, which I think is a fair statement).

Now to be fair, there are good reasons for this – the Google App Engine has a number of very good use cases (Don Schwarz demonstrated one of them when showing the audience a multiplayer game running on Google App Engine).

Note: I’d like state for the record that I mean no criticism of other cloud vendors (i.e. Amazon, Google, and SalesForce).  I think each of them have a place in the market, and exhibit various strengths.  That said, I do believe that the Windows Azure Platform stands out as the only real platform that can bridge the chasm between cloud services and on-premises software. (Note to self: back this statement up in a future blog post.)

I would argue, however, that most enterprise developers require a little more flexibility when building out enterprise class applications.  The Windows Azure Platform provides this flexibility (I mean, come on – sometimes you just want to execute some native code!).

I decided to test how far I could take this flexibility in Windows Azure.

I am a big fan of Worker Roles in Windows Azure.  I really like the idea of having an asynchronous, “headless” service that’s working for me in the background.  I wanted to see if I could spin up an equivalent to a Worker Role in a Web Role, so that in addition to having my Web application running in a Web Role I could also get a service running in the background.

Why would I want to do this?  Well, here are a few of reasons:

• Asynchronous logging service running in every Azure instance (whether a web or worker role); this service might take information from the event log and write it to an Azure table.
• Clean-up operations (i.e. temporary scratch writes, etc.)
• Opening and managing socket connections to the service bus (this is my favorite scenario).

The list could go on.  Hopefully you get the point.

So, how would you do this?  Again, given the flexibility of the platform, there are many ways you can do this.  Below you’ll find one simple way.

1. Create a new cloud services project, and add a web role.

2. Create a new class and inherit from Microsoft.ServiceHosting.ServiceRuntime.RoleEntryPoint.  You’ll have to override the Start and RoleStatus methods.

using Microsoft.ServiceHosting.ServiceRuntime;

using System.Threading;


namespace Web

{

    public class MyWebWorkerRole : RoleEntryPoint

    {

        public override void Start()

        {

            while (true)

            {

                RoleManager.WriteToLog("Information", "The worker role is running.");


                Thread.Sleep(5000);

            }

        }


        public override RoleStatus GetHealthStatus()

        {

            return RoleStatus.Healthy;

        }

    }

}

3. Create a Global.asax page.

4. Create a method where you initialize and start and initialize your class that inherits from RoleEntryPoint.

private void StartWorkerRole()

{

    MyWebWorkerRole myWebWorkerRole = new MyWebWorkerRole();

    myWebWorkerRole.Initialize();

    myWebWorkerRole.Start();

}

5. Create local threading variables that you will use to run your method.

public class Global : System.Web.HttpApplication

{

    System.Threading.ThreadStart ts;

    System.Threading.Thread t;


    ...

5. Update the Application_BeginRequest method to invoke your method on a new thread.  Be sure and check to see if the thread is already running, otherwise it will get started multiple times.

protected void Application_BeginRequest(object sender, EventArgs e)

{

    if (t == null)

    {

        ts = new System.Threading.ThreadStart(StartWorkerRole);

        t = new System.Threading.Thread(ts);

        t.Start();

    }

}

And that should do it.  Now you have a class that will run asynchronously in all your Web Role instances.  And since it inherits from the RoleEntryPoint, you can leverage this technique for both your Web and Worker Role instances.  I’d recommend placing this class in a separate class library and adding a reference to it from your various projects.

Note: It is highly probable that there will be some changes to the APIs and assemblies come PDC.  While these concepts will stay valid, the underlying code will probably change.  If this happens, I’ll be sure and provide an update.

I hope this helps!

Chunking BCP output to upload lots of data into SQL Azure

Note: This is a guest post from George Huey, Architect Evangelist in the Developer and Platform Evangelism group.

When you upload your data into SQL Azure, SQL Azure replicates your data to three different locations in order to provide triple redundancy.  Therefore, it needs a little more time to get the data in the proper places.

One of the things that we found out during a series of Windows Azure Platform Migration Labs held in the Chicago MTC is that you cannot upload hundreds of thousands of records without giving SQL Azure time to catch up.  Consequently, you have to chunk your data and give SQL Azure time to process each chunk before uploading the next chunk of data.

The tool that we used for migrating our customer databases to SQL Azure was the SQL Azure Migration Wizard.  The migration wizard uses BCP to download data from an on premise SQL Server database and then uses BCP to upload the data to SQL Azure.  BCP allows you to specify the first row (-F), the last row (-L), and the batch size (-b).  These options will allow you to chunk the data beginning uploaded to SQL Azure.  For example:

BCP MyDb.dbo.Transactions out Transactions.dat -E -q -n –T 

The above command extracts data from table Transactions in the database MyDb.  At the end of the BCP output, you will find the number of records copied to file (for example: 2,524,520 rows copied).

In order to upload in chunks, you would do something like this:

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F1 –L250000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F250001 –L500000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F500001 –L750000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F750001 –L1000000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F1000001 –L1250000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F1250001 –L1500000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F1500001 –L1750000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F1750001 –L2000000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F2250001 –L2500000 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

BCP  MyDb.dbo.Transactions in  Transactions.dat -E -q -n –b 5000 –F2500001 –L2524520 -S  tcp:azureserver.ctp.database.windows.net -U admin@azureserver -P password

Note, you will have to put some kind of delay between BCP commands to give SQL Azure time to store the data (say 15 seconds).  You will probably find that sometimes the 15 seconds is not enough time and that, during the upload of one of your BCP chunks, SQL Azure might shut it down.  If that happens you will see something like this happen:

5000 rows sent to SQL Server. Total sent: 145000

5000 rows sent to SQL Server. Total sent: 150000

5000 rows sent to SQL Server. Total sent: 155000

SQLState = S1000, NativeError = 21

Error = [Microsoft][SQL Server Native Client 10.0][SQL Server]Warning: Fatal error 40501 occurred at Oct 30 2009  4:15PM. Note the error and time, and contact your system administrator.

BCP copy in failed 

From the above BCP output, you will see that a total of 155,000 rows were sent before SQL Azure closed the connection.  Thus you would have to adjust your next BCP command to start at your –F value + 155000.

While this process works reasonably well, it can make the process of uploading data a little tedious if you have a large number of tables with a large number of records per table.  In order to simplify the process, we have modified the SQL Azure Migration Wizard to do all of this work for you.  It allows you to specify the chunk size, the batch size, and the time to wait between BCP chunks in SQLAzureMW.exe.Config.  It also catch BCP errors and adjust for records processed and then retry.

Try it out, review the source code, and be sure to provide some good feedback!

Taste of Chicago benefits from Windows Azure and Silverlight

As a native Illinoisan, I know that the Taste of Chicago is a big deal!  It is the largest outdoor food and music festival in the world, and is attended by more than 3 million people each year.  If you like food and music, this is a festival for you!

This past year, the City of Chicago needed to quickly build out an interactive map for the event – and when I say quickly, I mean it.  They needed an interactive mapping experience built and deployed in less than 10 days.  To help make this a reality, the City of Chicago turned to a very capable Microsoft partner, West Monroe Partners (WMP), for assistance. 

The challenges here were pretty clear:

• Build an interactive and immersive website in less than 10 days.
• The website needed to scale massively for the event, but once the event was over they wanted to scale the infrastructure down.
• WMP wanted to leverage their existing skills and tools.

WMP proposed the following – an immersive Silverlight Deep Zoom experience running on Windows Azure (if you’re not familiar with Deep Zoom, take a look at the Hard Rock memorabilia site).  You can still visit the Taste of Chicago interactive map – check it out:

The Windows Azure Platform – which I’ve discussed many times on my blog – is Microsoft’s cloud platform.  It is comprised of the following: Windows Azure, an operating system as a service; SQL Azure, a fully relational database in the cloud; and .NET Services, consumable web-based services that provide both secure connectivity and federated access control for applications.

The City of Chicago’s decision to go with Silverlight and Windows Azure provided a host of very tangible benefits, including:

• Fast, cost-effective development
• Immediate customer satisfaction
• Increased agility
• Dynamic user experience

In fact, this is broken down in detail in the Taste of Chicago case study with WMP and the City of Chicago.

Additionally, Erik Brown – technical lead at WMP – spoke to Microsoft about their experience building the interactive map in Silverlight and on Windows Azure.  I definitely recommend you take a look:

A really great story about how a capable partner can leverage innovative technology to not only dazzle a customer, but provide a valuable service for millions of users.

I hope this helps!