2013-11-11

Windows Azure Management Script

When I first started writing these scripts the Windows Azure Management Portal was less than stable. Many times just starting a VM would cause it to get stuck in a starting state, disks could get lease blocked, if you tried to start more than one VM at a time you'd get an error saying ~ x-ms-requestid that requires exclusive access... so in brief using the Windows Azure Management Portal was challenging for production IaaS resources.

So I wrote some PowerShell functions to help me out. The most challenging part was the lack of working procedures and the constant flux of the Azure SDK, which presented itself in changed properties, differing methods and unexpected behaviour...
The management portal is significantly more stable now.
The script has a IaaS focus and consequently will only be of help in managing Virtual Machines and their Storage, but with a little more work can be extended to .

This script is on GitHub (Manage-Azure) and I encourage any with an interest to contribute to it's further development.

Anyway as of October 2013 the script works, so lets go in to their setup and use case.


  1. Ensure you have a Microsoft Azure account.
  2. Download the scripts from: Manage-Azure
  3. Extract them to a path on your local machine.
  4. Open a PowerShell window > import the script ". C:\Your-Location\Manage-Azure.ps1":

    Here you can see my PS execution policy doesn't allow me to run the script, and after I chnage the policy the script tells me I need to download the Azure Cmd line tools (bcos I'm testing on a completely vanilla Win7 VM).
  5. Unfortunately there's no way around downloading the SDK/Cmd line tools dependencies :(.
  6. Once the script is imported you have 2 functions available to you:
    1. Manage-AStorage
    2. Manage-AVM
  7. Type in the function and a list of operations will popup.
  8. If a "publishsettings" file isn't in the script location, it will ask you to choose the directory where it's located... download yours if you haven't already.
  9. Choose your subscription if you have more than 1.


Manage-AStorage

This function provides Storage operations.

  1. Copy Blob.
    This operation enables copying blobs from container to container, and from storage account to storage account. If the storage accounts are in different data centers the copy operation can take a significant amount of time... eg: 127GB from Ireland to Hong Kong ~ 35minutes.


  2. Delete Blob.
    Delete 1 or more blobs.

  3. Add Disk
    Just like the 
    Windows Azure Management Portal, it creates a disk from a blog reference.
  4. Remove Disk
    Just like the 
    Windows Azure Management Portal
  5. Break Lease
    Sometimes a blob will have a lease by an object that doesn't exist, this used to be a big problem before October 2013 when VM's would frequently get stuck on starting, now not so much.
  6. New Snapshot
    Takes a blob snapshot.
  7. Restore Snapshot
    Restores a blob snapshot.


Manage-AVM

This function provides VM operations.

  1. Start
    Starts 1 or more VM's; this operation does one at a time and waits for them to start.

    Note: They don't always start at the same speed.
  2. Stop
    Stops 1 or more VM's; 
    this operation does one at a time and waits for them to stop.
  3. Backup
    This operation will stop a VM if started, copy the disk blobs attached to the VM to the backup container specified in the PS script ($BackupPath) and Start the VM if it was originally started.

    Note: In this screenshot you can observe iios-1 was "stopped" while iios-2 was "started" and " vhd-backups" didn't exist, so was created.

    Note: Here iios-2 has 2 attached disks.
  4. Restore
    This operation will stop a VM if started, Export the VM config, detach all disks, delete the disk blobs, remove the VM, let you select a blob to restore from the backup container, copy it to the vhds container, create disks for the blobs, recreate the VM from the config file and Start the VM if it was originally started.

  5. Export VM XML
    This just saves the VM configuration to an XML file.
  6. New VM XML
    This creates a VM from an XML configuration file. This comes in handy if a Restore operation fails for some reason, or you Delete the VM while keeping the attached disks.


Known Issues - Improvements

  1. Operation behaviour is inconsistent; you can perform a restore and it will fail at some point, one time and then do exactly the same operation and it works the next. The seeming randomness of it also makes it challenging to resolve.

    Note: Here you can see while trying to perform a Restore the script fails when recreating the VM from the config file. It's easy to recover from this point by just calling the "New VM XML" operation, but then this also fails when starting the VM even though the VM starts successfully... pretty annoying.
  2. Reuse of functions; I've been a little lax in reusing functions like copy/delete blob are used by multiple operations in Manage-AVM, but I don't use the Manage-AStorage functions.
  3. Better error handling; most Azure operations have an Id with returned properties indicating success/failure in most cases I don't use these.
  4. Logging needs improving, I capture Start and Stop VM logs but that's about it.

    2013-03-14

    SharePoint Foundation 2013 customize Stand-alone server installation

    SharePoint installs have two server type install options; either Complete, or Stand-alone.

    The Complete option lets you specify SQL server, accounts, and a couple other options. The problem with this option is that it requires a domain.
    The Stand-alone option doesn't let you specify anything and will install and configure the Service Apps using the built-in SQL Express 2008.

    What if you want to setup a Single Server, specify a local SQL Server, say SQL 2012 Dev edition and manually configure the service apps?


    1. Select Complete.
    2. Wait for it to finish, but don't launch SharePoint Products Configuration Wizard 
    3. Run the SharePoint 2013 Management Shell: C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe  -NoExit  " & ' C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\CONFIG\POWERSHELL\Registration\\sharepoint.ps1 ' "
    4. Paste the following command using names appropriate for your environment: New-SPConfigurationDatabase -DatabaseName "SPF2013_Config" -DatabaseServer "SPF2013SQL" -AdministrationContentDatabaseName "Content_CA"
    5. Run the SharePoint Products Configuration Wizard: "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN\psconfigui.exe"
    6. Let it complete.
    7. Now the problem is you will not be able to add any Managed Accounts through the UI, to fix this you need to edit the following registry key: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS]
    8. Change "ServerRole"="APPLICATION" to "ServerRole"="SINGLESERVER". Making this change will also break anything that uses psconfig, unless you have setup the SQL server to have a SharePoint instance, eg: server\SharePoint, that's how the Stand-alone SQL instance is configured so the psconfig checks if that instance is running. You'll only need psconfig to repair SP or install new service packs, and it's easy to change ServerRole back.
    This should also work with SPF 2010.

    2013-03-13

    Top 20 Replies by Programmers when their programs don't work

    20. That's weird... 
    19. It's never done that before. 
    18. It worked yesterday.
    17. How is that possible? 
    18. It must be a hardware problem. 
    15. What did you type in wrong to get it to crash? 
    14. There has to be something funky in your data. 
    13. I haven't touched that module in weeks! 
    12. You must have the wrong version. 
    11. It's just some unlucky coincidence. 
    10. I can't test everything! 
    9. THIS can't be the source of THAT. 
    8. It works, but it hasn't been tested. 
    7. Somebody must have changed my code. 
    6. Did you check for a virus on your system? 
    5. Even though it doesn't work, how does it feel? 
    4. You can’t use that version on your system. 
    3. Why do you want to do it that way? 
    2. Where were you when the program blew up? 
    1. It works on my machine.

    2013-02-22

    SharePoint 2010 upgrade to 2013 FBA MachineKey validation attribute invalid

    Issue:
    You've upgraded a Forms Based Authentication SharePoint 2010 web application to SP2013, and implemented FBA on the SharePoint 2013 Web Application.
    After creating the AspNetSQL membership DB you add the Membership Provider, use passwordFormat="Encrypted". None of the passwords work. You test the AspNetDB on a standalone .net site and get the following:


    Cause:
    • SP 2013 uses .net 4.0, which out of the gates defaults to Hash-based Message Authentication Code (HMAC) using the SHA256 hash function for the web.config machineKey.
    • SP 2010 uses .net 2.0 which defaults to SHA1.
    • IIS7.5 IIS manager doesn't even seem to support .net 4.0 properly.

    Fix:
    • Don't use Encrypted or Hash passwords... I know, I know that's not a fix, it's a lazy work around.
    • Use the SP 2010 web.config machineKey's or ...
    • Generate a new machineKey using SHA1: Go to http://www.developmentnow.com/articles/machinekey_generator.aspx generate the key and replace the existing ones in all the SP web.config files or C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config, and the SecurityTokenServiceApplication web.config
    • Regenerate passwords for all users.
    If the machineKey changes, then you may need to reprovision the Security Token Service Application.

    2012-11-08

    Web Application Navigator easy cross site navigation

    If you want to architect a functional and easy to maintain SharePoint logical architecture you're going to have to use multiple site collections, and even web applications.

    Using multiple site collections is easy enough, but then you'll encounter navigation issues. There are a number of cross site navigation web parts, but I didn't like any of them.
    Most  inherit some for sitemap provider and require you to edit the layouts.sitemap... anyway I figured I could do better so I did, and I'm sharing it with you.

    Introducing the Web Application Navigator
    Get source and solution file from: http://wan.codeplex.com

    Features:
    1. List based navigation.
    2. Multiple navigation list support.
    3. Menu or tree view.
    4. Security trimming.
    5. Display, or hide without deleting navigation item.
    Installation:
    1. Add the IIOS.SP.WebParts.Navigation.wsp to your farm solutions.
    2. Deploy the Solution to a Web Application.
    3. Enable the Feature in Site Collection.
    4. Add the web part to your master page or web part page.
     Activation:
    1. When you activate the Feature it will create a Navigation list in the root site, go to the Web Application and loop through all the Site Collections, and their Sites the user who is activating the feature has access to (so probably a good idea for a Farm Admin to activate it) and save the URL, Position and Site name in the Navigation list.
    2. The Feature will create a Custom Action on the settings page under Look and Feel, so you can access the Navigation list
    3. And finally adds the web part definition to the web part gallery.
    Usage:
    1. All navigation links are stored in a Navigation list. Activating the feature will create one, but you can create more if required. The list implements a simple Parent> Child relationship to render a hierarchy.  You can have n level children. You can also add as many navigation links as required.
    2. The list contains the metadata to render the navigation. All the fields are self explanatory maybe with the exception of...
      Position: This is the position the link will have in relation to it's siblings .
      Display: This shows or hides the link.
      Security Trim: If this is disabled, it won't check if the user has access to the site and will render the link regardless. This is useful if you want to give users access to a child site, but not the parent site.
    3. To render the navigation menu, you need to add the web part to a page, or a master page.
    4. Once the web part has 3 properties you can edit.
      List Name: this is the name of the list that contains the Navigation items. If you create another list just enter the name here.
      Display Tree: If enabled it will render the navigation as a tree. This is implemented with the help of GTreeView by Per GĂ„rdebrink with his permission.
      Use Full Width: If you want the menu to cover the full width select this, otherwise it will only use the minimum required.
    Use case:
    You have a web application with 10 Site Collections, these Site Collections have a combined 30 subsites.
    You want to implement consistent navigation across all sites.

    1. Deploy the Solution to the web app.
    2. Activate the Feature on the root Site Collection of the web app.
    3. Modify all the Site Collection master pages by adding the web part in
      or modify the SharePoint hive master page by adding the web part in
      or add the web part in to a page.
    4. If you modified the master page/s, then the web part will be visible on all Site Collections, and there's no further action required.
      If you added it to a page, then you'll need to export the web part definition and import it to every Site Collection web part Gallery.
    FAQ:
    1. Q: What if I activate the feature in other Site Collections of the same web app?
      A: The root Site Collection list will be ignored. The web part first tries to get the Navigation list from the current site, then works it's way to the root.
    2. Q: What happens when I deactivate the Feature?
      A: It will delete the default Navigation list and remove the web part definition from the Gallery.
    3. Q: People see different links, why is that?
      A: Because Security Trim is enabled on that link and the person does/doesn't have access.
    4. Q: Does it work on SharePoint 2013?
      A: I've only done minimal testing, and it works except the Feature icon doesn't deploy properly, but you could copy it over manually.



    2012-11-03

    SP 2013 App Foundation

    2012-09-27

    European Commision releases their final European Cloud Computing Strategy document

    The document can be view here cloud computing strategy.
    What does it mean? Well it's a strategy document, so if you're a CIO/CTO or Enterprise Architect it gives another research stream.
    The 16 page document at a high (very) level aims to lay the foundations to address Interoperability, Portability, Security, Copyright, Licensing, Standards, Compliance, Certification, Adoption, Risks and Service Level Agreements.
    The document is fairly meaningless, but it gives you an idea how Cloud computing is growing in significance to warrant a European Commission level Cloud policy framework, and a strategy to realize it.
    I recommend reading the Data Protection Working Party's Opinion 05/2012 on Cloud Computing for a more in-depth look at some of Cloud Computing's considerations.