PowerShell – 2 Huge Announcements

PowerShell is like Semantic Bash. Tweeted by SwiftOnSecurity

That’s right, PowerShell is now a cross-platform (Windows, Linux and OS X) command-line shell and associated scripting language.

Here it is in Action

Running on Windows 10, in Hyper-V on Ubuntu 16.04 Desktop.

  • Showing $PSVersionTable displaying information about the version, build and compatibility
  • Using Get-Command to get the number of PowerShell cmdlets currently shipping
  • Launching Visual Studio Code
  • Finally, calling python code inline in a PowerShell script

Wait there’s more

Microsoft’s Visual Studio Code Editor runs on Linux too (and on Windows). Plus, there’s a PowerShell Code extension. Your editing experience is now the same across platforms (and check out the VS Code marketplace for even more extensions )

This extension provides rich PowerShell language support for Visual Studio Code.
Now you can write and debug PowerShell scripts using the excellent IDE-like interface that VS Code provides.

Features

  • Syntax highlighting
  • Code snippets
  • IntelliSense for cmdlets and more
  • Rule-based analysis provided by PowerShell Script Analyzer
  • Go to Definition of cmdlets and variables
  • Find References of cmdlets and variables
  • Document and workspace symbol discovery
  • Run selected selection of PowerShell code using F8
  • Launch online help for the symbol under the cursor using Ctrl+F1
    Local script debugging and basic interactive console support!

PowerShell is Open source

You can read it, and you can contribute. It’s going to be interesting to watch the pull requests. I’m looking at some of the source on some cmdlets for the first time without using a disassembler.

PowerShell Package Management

Package managers are central to promoting code discovery and re-use. The PowerShellGallery (announced in 2015) is hosted by Microsoft.

The PowerShell Gallery is the central repository for PowerShell content. You can find new PowerShell commands or Desired State Configuration (DSC) resources in the Gallery

There’s been over 2 million downloads of the packages and tremendous community engagement. It’s about to get bigger. Give Find-Module and Install-Module a try.

What can you make with these building blocks?

Plus, there is Find-Package and Install-Package, these integrate with Nuget and Chocolatey package feeds.

New variables

This is just a sampling showing the next release of PowerShell is positioned to truly work across platforms.

Part 3 – The Troika : PowerShell, YAML and PowerShell Classes

  • In Part 1, we took YAML, converted it to PowerShell objects and then used PowerShell v5.0 to cast it to a class. This gave us runtime checking to see if we fat fingered entries in the YAML. This required very little coding, just some simple scaffold setup.

  • In Part 2, we took it further, easily making the variables know if they integers or not (for example) and with no work on our part, report an error if the YAML we typed was wrong. Plus, we saw how we could supply default values for fields not specified in the YAML. Streamling the authoring of the YAML information.

Checking for required data

In Part 3 we’ll take another step. We’ll see how we can make data required in the YAML that has been authored. Sometimes we forget it cause we move quickly other times we don’t specify because we don’t know about. In PowerShell we use [Parameter(Mandatory)] on parameters in a function to make this happen for example.

Let’s look again at the Purchaser example. Notice for Jane, the Age property is blank, because Age was not specified in the YAML.

Prints

This information may be critical for processing, so catching it and reporting it early is important. A fast feeback loop helps you correct problems more quickly. Plus, the sooner the missing data is detected, the less you have to undo downstream. Maybe this data is being inserted into a database. Name may get inserted first, then age. When the Jane record gets inserted the database my throw an error that age is null. Now you have to delete that record before running again.

Determining the missing age data early is good all around.

Added a Verification Method to the Data Class

Here’s one approach to check for data integrity. You’ll loop through all the data, here, the $Purchasers variable and check anything you is important. In this example, we’ll check for any purchasers age that is null, and if we find one, add a ‘message’ to the $dataError array. If we don’t find any missing data, $dataError is empty, we return it and use that to indicated no errors were found. This can be as sophisticated as you’d like to make it.

Using the Verification Method

After we cast the converted YAML into the $records variable. Call the Verify() method, capture the results in $verification and then check it. If its $null all is good, otherwise print out the errors.

Again, this can be as sophisticated as you want to make it. Use Write-Verbose, throw the error count and return the messages, or even log them.

Also, you can implement verification rules beyond checking for $null. You can check to see if data is in certain ranges, you can look up data in other data stores and make comparisons. Lots of flexibility.

Putting this all together, it will print this.

YAML Comments

One of the many things I like about YAML is that I can use comments. You can use comments to expalain parts or to tell ConvertFrom-YAML to skip over information. I much prefer that to deleting and re-adding it later.

In Closing

Here in Part 3 we see we can use PowerShell class methods to verify the data specified in the YAML. It’s a great way to enable a fast feeback loop for problems and keeping bad data out of downstream processes. We also see we can comments in YAML to add explanations and have data skipped. Tough to do in XML and JSON.

We’re not done yet, I’ll post some more about this approach.

Part 2 – The Troika : PowerShell, YAML and PowerShell Classes

In the last post The Troika : PowerShell, YAML and PowerShell Classes, we took YAML, converted it to PowerShell objects and then used PowerShell v5.0 to cast it to a class. This gave us runtime checking to see if we fat fingered entries in the YAML. This required very little coding, just some simple scaffold setup.

Once you do this, you’re a couple of steps from even more capability.

Note: Notice that using PowerShell v5.0 classes to do this work gives you intellisense in the console and editors.

Defaults and Types

In this example, the YAML has two Purchasers, and we left out Jane’s Age. When we convert, cast and print it, is shows up blank.

Prints

Lets set the type the $Age property in the Purchaser class to int.

Now, when we covert and cast, Jane’s age is now 0.

Prints

There’s another benfit to typing $Age. What if in the YAML we type a string for Jane’s age?

PowerShell handles checking the type and reporting a very useful error.

Doing Defaults

Going back the PowerShell class, you can set defaults for properties not defined in the YAML (Jane’s Age) and introduce new ones.

Note: Adding $DateCreated to the Purchaser class now makes it legal to use and overridden from the YAML.

Prints

Next steps

Now we’ve seen how we can convert YAML from a string (or file) into PowerShell and have a simple way to catch errors, correctly type the data, create defaults and easily introduce new properties.

In upcoming posts we’ll look at how to validate the YAML, for example, what if certain properties are required but not set?

The Troika : PowerShell, YAML and PowerShell Classes

YAML Ain’t Markup Language, its purpose is a data-oriented, rather than document markup. Combining this with PowerShell is potent, adding PowerShell classes, well its like adding bacon bits to bacon and eggs.

In this post, we’ll take it around the block.

Hello YAML

Key value pairs are easy, and we get a PowerShell object with property names.

Prints

Arrays

Now add an items property that creates an array. Use to create a collection of ordered value.

Note: YAML uses white space to indicate scoping of items

Prints

Let’s Get Complicated

Multi-nesting. Here we have a Purchaser with a MailingAddress and a State. Try to represent that using PSCustomObject and a HashTable.

Prints

Wait, there’s more

Now, what happens if you use these structures in your PowerShell scripts and someone fat fngers Age with AgeX. If you tried $r.Purchaser.Age, PowerShell would print nothing. This is where we add the bacon bits.

PowerShell Classes

Let’s introduce the Data class, cast $r to it, and print out the Purchaser.Age.

This buys us some intellisense when we type ([Data]$r). but we want some more saftey.

Let’s create a Purchaser class, add another purchaser and make an error by changing Age to AgeX.

Add the Purchaser Classe

Notice, we type the $Purchaser variable in the Data class as an array of Purchaser.

We also added Jane as and incorrectly specifed AgeX instead of Age.

Using a PowerShell class lets us find errors early in the process and without any coding logic on our part.

Prints

A lot of Potential

This can be applied to many scenarios. Having a data oriented human-readble file that can be easily used in PowerShell and have it’s structure easily validated is tremendous.

In future posts we’ll kick it up a few notches.

Add Fuzzy Search to PowerShell

I’ve tried many text editors over time. One of the features I really like in Sublime Text and Visual Studio Code is the fuzzy search they implement in many places.

Here’s Visual Studio Code’s file picker in action.

I’ve wanted this feature in PowerShell for a while and I came across this post How is the fuzzy search algorithm in Sublime Text designed? and ported it to PowerShell.

Fuzzy Search in PowerShell

Here’s fuzzy search in action, applied to a list of strings, countries. It’s not limited to lists and can be applied to many other things in PowerShell.

One example

Get-Command | sfs invokepr

Grab the Script

Here’s my GitHub repo Select-FuzzySearch applies fuzzy searching to text.

Try it out and do a PR to improve it.

Getting Started with Azure Functions in PowerShell

Azure Functions is an event driven, compute-on-demand experience that extends the existing Azure application platform with capabilities to implement code triggered by events occurring in virtually any Azure or 3rd party service as well as on-premises systems. Azure Functions allows developers to take action by connecting to data sources or messaging solutions, thus making it easy to process and react to events. Azure Functions scale based on demand and you pay only for the resources you consume.

You can create functions in the Azure portal, but manual intervention isn’t good for scaling continuous delivery.

Create an Azure Function App

This link walks you through creating an Azure Function App. Be sure to grab the Resource Group Name and the SiteName so you can plug it into the PowerShell script.

Automate it with PowerShell

Using the New-AzureRmResource Azure cmdlet, you’ll setup the required parameters. $props is a PowerShell hashtable that holds the config information for the function. Here you’ll create a timerTrigger and put it on a schedule. In the files hashtable, the key value pair are the name of the PowerShell file to be created and the value is valid PowerShell.

In this example. I’ll leverage the .NET framework, creating a Net.WebClient, so I can download finance data from the Yahoo Finance web service.

This is a quick tour. You can also upload JavaScript code to run as a function. Plus, you could read the PowerShell from a file rather then hard coding it in a script.

Lastly

This is just scratching the surface. For example, you can create functions in many different languages and there other Trigger types as well. Plus, we only outputted information to the log, there’s much more to explore.

Full-Text Search: PowerShell meet Lucene

Bruce Payette, co-founder of PowerShell, gave a talk on integrating full-text search using Lucene with PowerShell at the PowerShell Summit.

Refactoring

So I did a little refactoring and wrapped it in GUI, this is all PowerShell.

In Action

It indexed the content of 2500+ files in ~2 seconds.

Index and Search

Type in the name of the directory to be searched, including a filter, c:\posh\*.ps1, press enter and it will recursively search the directory for all ps1 files and index the contents, keeping the index in memory. You can also search multiple directories with different filters. E.g. c:\temp\*.cs,c:\test\*.ts,c:\arm\*.json

Then, you can search for a term across everything that was indexed by typing in the Query box and pressing enter.

What is Lucene?

Apache Lucene is a free and open-source information retrieval software library, originally written in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License.

Who uses Lucene

  • Apache SOLR (Used by OMS Operational Insights)
  • Elastic Search (everybody uses this)
  • CIteSeerX
  • Apple
  • 7digital (digital media)
  • Comcast
  • Disney

On GitHub

Grab it all from my GitHub Repo.

Paste JSON as a PowerShell Class in Visual Studio Code

The PowerShell extension for Visual Studio Code just got way cool.

First, we got a $profile. It lets you customize your environment and add session-specific elements to your PowerShell Visual Studio Code session.

Second, we got a way to interact from PowerShell directly with the vs code editor.

You use Register-EditorCommand to register a PowerShell function|scriptblock. In the PowerShell script you can set it up to a take a “context” parameter that gets passed in. This parameter contains a reference to the vs code editor (in the future this reference will surface more features), you can now do things like InsertText into the active file.

ConvertTo-Class

I took my existing PowerShell module and added this handful of code to connect it to Visual Studio. ConvertTo-Class takes a JSON string, figures out it structure plus data types and code generates a PowerShell v5.0 Class.

In Action

If you have PowerShell v5.0 installed, you can grab the ConvertTo-Class module from the command line like this Install-Module ConvertToClass

Edit your PowerShell VS Code $profile and add Import-Module ConvertToClass.

Start VS Code. In VS Code you can edit both a JSON file and PowerShell file and have a rich editing experience for each.

Here, I’m copying a JSON snippet of an Azure Resource Management (ARM) Template, then switching to a PowerShell file and pasting it as a PowerShell Class.

That’s the quick tour. PowerShell module authors just got some really cool integration tools for Microsoft’s new modern editor, Visual Studio Code.

I can’t wait to see what the community starts to add to the ecosystem.

Resources

Check out these links for more info.