PowerShell Hacker #11

by Doug Finke on August 6, 2010

in Blackhat,CouchDB,Defcon,Opalis,PowerShell,PowerShell Hacker,Python,Ruby,Splunk,Xml

PowerShell lets me leverage .Net to do heavy lifting from the command-line

This talk was about the powerful ability to perform advanced tasks through PowerShell and how you can utilize PowerShell on penetration tests.

{ 1 comment }

Zoom.it from LiveLabs

by Doug Finke on August 5, 2010

in LiveLabs,Windows Azure,Zoom.it

Create your own. Just give them the link to an image on the web.

I grabbed the Sydney Harbor Bridge, pasted it in the site and embedded the script here.

Zoom.it runs on Windows Azure. Plus, they have and API.

{ 0 comments }

Microsoft – From Research to Product

by Doug Finke on August 5, 2010

in DGML,GLEE,MSAGL,PowerShell,Visual Studio 2010,Visualizations,Visualizing Data

Chris Lovett posts From Research to Product

I really love taking the coolest stuff that comes out of research, and finding ways to productize that and ship it in our products…

Three years ago I blogged about the Microsoft Research tool PowerShell + GLEE – Graph Layout Execution Engine.

Since then it

Other posts I did on this

{ 0 comments }

OData PowerShell Explorer

by Doug Finke on August 4, 2010

in CodePlex,Data Services,OData,PowerShell,Sql Server,WPF,WPK

I’ve been working with OData and decided to build, and make available on CodePlex, an OData PowerShell Explorer that let’s you access and drill down on a service from either the command line or the GUI.

The Open Data Protocol (OData) is a Web protocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today.

Open Data Protocol Home

One of the many interesting parts of this open source module is, it is a GUI written in PowerShell built on top of  PowerShell code. You can either use the GUI to navigate the data or do it directly at the command line. Both are driven by the OData PowerShell module.

Another interesting note is the GUI’s ‘Drill Down’ buttons dynamically change depending on which service and collection you are viewing.

Same goes for the command line objects. The objects methods are dynamically built on the fly as you navigate the details of a collection and drill down.

Using the PowerShell WPK GUI with OData Services

image

Using the PowerShell Command Line with OData Services

image

Great Automation Story

Using the OData PowerShell module at the command line blends the power of Open Data Protocol for sharing and PowerShell’s great range of cmdlets to slice, dice, transform and store data.

In a few lines of PowerShell you can fetch information from several data services and then export them to Excel, Databases, XML and more.

Query Options

The PowerShell OData module also supports OData query options like $expand, $filter, $top, $skip and more. So for example you could do this:  

Import-Module C:\CodePlex\psodata\trunk\modules\OData.psm1            

$netflix = New-ODataService "http://odata.netflix.com/Catalog"
$netflix.People("`$filter=endswith(Name, 'Abbott')")            

# Results
Id       Name
--       ----
189      Bruce Abbott
190      Bud Abbott
196      George Abbott
20006674 Mark Abbott
20013692 Philip Abbott
20017929 John Abbott
20018086 David Abbott
20031575 Abdul Malik Abbott
20038388 Diahnne Abbott
20045891 Annie Abbott
20057657 Jennifer Abbott
20057988 Norman Abbott
.....

There are many ways to subset the data. This is faster because there is less data returned from the remote server. An alternative would eliminate the $filter query option and use PowerShell’s Where-Object {$_.name.EndsWith(‘Abbott’)}. The tradeoff being that all the data is fetched from the server and then subsetted.

Summary

I’ll continue to grow the PowerShell module to encompass additional features and surface them in the GUI as well.

Meanwhile grab the code and let me know what you think.

Resources

{ 1 comment }

Another Reason I like PowerShell

by Doug Finke on August 3, 2010

in .Net,C#,PowerShell,Ruby

I enjoyed reading Rob Conery’s post The Reasons I Like Ruby. He has a nice example where he wants to import a list of countries from an online file.

Version that follows the Ruby one

$url = "http://openconcept.ca/sites/openconcept.ca/files/country_code_drupal_0.txt"            

$wc = New-Object Net.WebClient            

($wc.DownloadString($url)).Split("`r`n") |
    ForEach {
        $countryCode, $countryName = $_.Split('|')
        New-Object PSObject -Property @{
            CountryCode = $countryCode
            CountryName = $countryName
        }
    }

Shorter version using PowerShell’s ConvertFrom-Csv

$url = "http://openconcept.ca/sites/openconcept.ca/files/country_code_drupal_0.txt"            

$wc = New-Object Net.WebClient            

$wc.DownloadString($url) |
    ConvertFrom-Csv -Delimiter "|" -Header CountryCode, CountryName

Results

  • This is all out of the box PowerShell
  • Note the New-Object Net.WebClient. Tapping into the .NET framework
  • Notice the 5th line of the first version, the one with the .Split(‘|’), showing off PowerShell’s ability to assign variables in succession

Both versions return an array of objects (.NET objects), each object has two properties. CountryCode, CountryName

# Results
CountryName CountryCode
----------- -----------
AF          Afghanistan
AL          Albania
DZ          Algeria
AS          American Samoa
AD          Andorra
AO          Angola
AI          Anguilla
AQ          Antarctica

It’s safe to say that with C# and System.Net it’s just a bit longer. Not that much longer – but you’d have to have a pretty practiced hand to get through the "machine noise" that begins to creep in.  – Rob Conery

{ 2 comments }

Why I Like PowerShell, Part 2 : ScriptBlocks

by Doug Finke on August 3, 2010

in PowerShell,Ruby

I am enjoying Rob Conery’s posts, his latest is Why I Like Ruby, Part 2: Blocks.

Code blocks and closures are nothing new to programming – you can find them in most OO (object oriented) languages – Rob Conery

PowerShell has them too

What we call scriptblocks in PowerShell are called anonymous functions or sometimes lambda expressions in other languages. A number of languages, including Python and dialects of LISP, still use lambda as a language keyword. 

Windows PowerShell in Action – Bruce Payette

This example shows how to use PowerShell scriptblocks to reduce repetitive code. It is also the application of the DRY Principle, Don’t Repeat Yourself.

Download the Chinook Database for the sample XML.

Repetitive

$xml = [xml][system.io.file]::ReadAllText( "C:\ChinookDatabase1.1\SampleData\ChinookData.xml" )            

Write-Host "Loading artists"
$xml.ChinookDataSet.Artist |
  foreach {
    "{0} {1}" -f [int]$_.ArtistId, $_.Name
  }            

Write-Host "Loading genres"
$xml.ChinookDataSet.Genre |
  foreach {
    "{0} {1}" -f [int]$_.GenreId, $_.Name
  }

The code works but it’s not pretty and repeats itself. Also, say you were to change the name of the dataset from ChinookDataSet to TempDataset you’d have to do it in two places. Let’s improve it.

ScriptBlocks and the DRY Principle

$xml = [xml][system.io.file]::ReadAllText( "C:\ChinookDatabase1.1\SampleData\ChinookData.xml" )                        

function Parse-Key ($key, $code){
    $xml.ChinookDataSet.$key |
      foreach { & $code }
}                        

Parse-Key Artist { "{0} {1}" -f [int]$_.ArtistId, $_.Name }
Parse-Key Genre  { "{0} {1}" -f [int]$_.GenreId, $_.Name  }

Here we are calling the Parse-Key function and passing in the name of the key we want, Artist. We are also passing in a scriptblock. In the ForEach, we use the call operator & to execute that scriptblock and leverage closures. The $_is evaluated in the context of the ForEach in Parse-key function.

A closure is a first-class function with free variables that are bound in the lexical environment.

Wikipedia

{ 2 comments }

Using the Congress API with Spreadsheets or PowerShell

by Doug Finke on August 3, 2010

in Congress,NY Times,NY Times API,PowerShell

The New York Times Developer Network released the Congress API back in January, Announcing the Congress API.

Lets developers access information about Congressional representatives and their votes

Here is their write up Using the Congress API With Spreadsheets.

PowerShell and the Congress API

First, you’ll need and API key. Using the .NET Webclient namespace makes easy work of grabbing the XML and displaying it in Out-GridView.

$url = "http://api.nytimes.com/svc/politics/v3/us/legislative/congress/111/house/members/leaving?api-key=<YOUR KEY>"
$wc = New-Object Net.Webclient
$xml = [xml] ($wc.DownloadString($url))            

$xml.result_set.results.members.member |
    select first_name,last_name, party, state, status, note |
    sort last_name | Out-GridView

Members Leaving Office

image

How many Democrats v. Republicans are leaving?

$url = "http://api.nytimes.com/svc/politics/v3/us/legislative/congress/111/house/members/leaving?api-key=$apiKey"
$wc = New-Object Net.Webclient
$xml = [xml] ($wc.DownloadString($url))            

$xml.result_set.results.members.member | group party -NoElement            

# Results
Count Name
----- ----
   26 D
   22 R

{ 1 comment }

Up and running with a new web hosting company

by Doug Finke on August 3, 2010

in ARVIXE,Blogging

The company I was with for the last 3 years went bananas. Seems they merged with a new company and migrated their client base.

The migration was less than optimal. Slower network and they lost a bunch of my data. Every interaction with customer support was turned around saying I had a browser problem, cookie issues or network delays on my side.

image

is my new home. Their sales people responded to all my inquires and their performance/customer support is great so far. They responded in under 5 minutes on a Sunday morning, accurately pointing me to information.

{ 0 comments }

PowerShell Hacker #10

by Doug Finke on August 3, 2010

in PowerGUI,PowerShell,PowerShell Hacker,Web

Into double digits.

I heard the same thing about VBScript years ago, about C++ a decade earlier, and being aware that I am dating myself here, about C, Pascal, Windows Programming, VI, Emacs, punch cards, VT100 terminals and about rubbing two sticks together to start a fire.

{ 0 comments }

Researchers from Harvard and MIT create Programmable Origami

by Doug Finke on June 29, 2010

in Harvard,MIT,Origami,Programmable

Based on the ancient art of origami, the sheets are edged by foil actuators–thin, solid-state motors–that contract or expand when they receive an electric current from flexible electronic circuits embedded in the sheets. After they achieve their preprogrammed shape, the sheets are held in place by tiny magnets on the edges of the fold joints.

Adam Flaherty

{ 0 comments }

← Previous Entries

Next Entries →