Skip to main content

PowerShell Scripting Games 2013 - Advanced Event 1 - My Submission and Learnings

Please review my submitted code and vote on what you think, also please leave a comment.

While the below script isn't exactly what I submitted, it is close, and what I would have ended up with if I had been a bit more patient and didn't feel the need to submit before it was too late. But I digress...

I want to explain why I think this script in general is good, if not an excellent example of how to accomplish the events goals.

The event goals are summarized as having to take files from source X, Y, Z and move to A if the file age is greater than # days old.

Honestly, the requirements left a lot of ambiguity, that I believe, having voted on several entries, produced a variety of styles. The script I ended up with below is based on my original submission and the adaptions I made based on what I learned from others examples, but in the most simplified manner I could envision with PowerShell 3.0 and that fit well within the framework of the original script I submitted.

Like I did the practice event, I want to break down the decisions and why I wrote it like this so that others can hopefully learn how to think in PowerShell.
  1. #Requires -Version 3.0  
  3. function Move-Files {  
  4.     <#  
  5.     .SYNOPSIS  
  6.     Moves old files.  
  8.     .DESCRIPTION  
  9.     Moves files contained in the source directories to the destination directory if the age of the file (Today - CreationTime) is greater than the rention.  
  11.     .PARAMETER Path  
  12.     The path(s) to scan for files that have exceeded the retention.  
  14.     .PARAMETER Destination  
  15.     The path to move all files to that have exceeded the retention.  
  17.     .PARAMETER Days  
  18.     The number of days old the file must be to be moved.  
  20.     .PARAMETER TimeSpan  
  21.     The age restriction of files that are to be moved.  
  23.     .PARAMETER DateTime  
  24.     The date that the file must be created before in order to qualify for the move.  
  26.     .EXAMPLE  
  27.     Move-Files 'C:\Application\Log\*\*.log' '\\NASServer\Archive'  
  29.     .EXAMPLE  
  30.     Move-Files 'C:\Application\Log\*\*.log' '\\NASServer\Archive' -TimeSpan 24.23:59:59.999  
  32.     .EXAMPLE  
  33.     Move-Files 'C:\Application\Log\*\*.log' '\\NASServer\Archive' -DateTime '01/01/2013 12:00:00 AM'  
  35.     .EXAMPLE  
  36.     Get-ChildItem 'C:\Application\Log' -Directory | Move-Files '\\NASServer\Archive'  
  37.     #>  
  38.     [CmdletBinding(ConfirmImpact = 'Medium', SupportsShouldProcess, DefaultParameterSetName='Days')]  
  39.     param (  
  40.         [Parameter(Position=0, Mandatory, ValueFromPipeline)]  
  41.         [SupportsWildcards()]  
  42.         [ValidateNotNullOrEmpty()]  
  43.         [string[]] $Path,  
  45.         [Parameter(Position=1, Mandatory)]  
  46.         [ValidateNotNullOrEmpty()]  
  47.         [string] $Destination,  
  49.         [Parameter(ParameterSetName='Days')]  
  50.         [PSDefaultValue(Help='90 Days')]  
  51.         [int] $Days = 90,  
  53.         [Parameter(Mandatory, ParameterSetName='TimeSpan')]  
  54.         [TimeSpan] $TimeSpan = [TimeSpan]::FromDays($Days),  
  56.         [Parameter(Mandatory, ParameterSetName='DateTime')]  
  57.         [DateTime] $DateTime = ((Get-Date) - $TimeSpan),  
  59.         [ValidateSet('CreationTime''LastWriteTime''LastAccessTime')]  
  60.         [string] $FileProperty = 'CreationTime',  
  62.         [switch$Force  
  63.     )  
  65.     begin {  
  66.         Set-StrictMode -Version Latest  
  68.         Write-Verbose "Files created before $DateTime will be moved to $Destination"  
  69.     }  
  71.     process {  
  72.         $Files = Get-ChildItem -Path $Path -File | Where-Object $FileProperty -lt $DateTime  
  74.         $Files | Select-Object -ExpandProperty Directory -Unique |  
  75.                     ForEach-Object { Join-Path -Path $Destination -ChildPath $_.Name } |   
  76.                         Where-Object { !(Test-Path $_ -PathType Container) } |   
  77.                             New-Item -Path {$_} -ItemType Container -Force |   
  78.                                 Out-Null  
  80.         $Files | Move-Item  -Destination { Join-Path $Destination "$($_.Directory.Name)\$($_.Name)" } -Force:$Force  
  81.     }  
  82. }  
PowerShell 3.0 (Line 1)
I started writing this and immediately wanted to leverage the simplified syntax of PowerShell 3.0. If you haven't read about it, or played with those enhancements, you are definitely missing out.

Comment Based Help (Line 4-37) 
One of the first comments I received on my submission was the lack of examples, I really don't get the need for that based on the built-in help being pretty sufficient, but with the parameter sets, it was easier to show examples of how it might be used so I added them.

CmdletBinding (Line 38)
ConfirmImpact = 'Medium' is the default, but I included it to show why the SupportsShouldProcess is not being invoked within my script as I rely on that flowing down to New-Item and Move-Item to do the prompting, no need for me to prompt when PowerShell will do it for me. The default parameter set is Days as that is the default value for the age of the files. SupportsShouldProcess also gets me native -WhatIf support, no coding needed, New-Item and Move-Item handle it automagically. Just more magical PowerShell <3.

Parameters (Line 40-62)
The Path parameter is required, one thing I love about PowerShell 3.0 parameters don't have to have = $true to enable them. It also takes values from the pipeline and it notes that wildcards are supported in the values, again another nifty PowerShell 3.0 feature.

The Destination is obvious, no validation here, I let PowerShell cmdlets do the validation for me when I should. This is questionable, I could add a simple ValidateScript attribute with Test-Path; one feature I wish PowerShell had was a custom error message if ValidateScript fails (maybe there is I just don't know how currently), so I tend to avoid ValidateScript.

The Days parameter is the default value for the age of files, being 90 days. This is where the beauty of cascading values comes into play and parameter sets. If Days is specified, TimeSpan default value is the number of days, oops, my C# hat is showing, that should be (New-TimeSpan -Days $Days). Also, notice the new PSDefaultValue attribute, this lets me customized the default value help message, very handy and helpful.

The TimeSpan parameter can be specified and if it is, Days is ignored and invalid anyway, so then the DateTime parameter will take the current Date and subtract the TimeSpan parameter from it for the actual maximum age of the file.

The DateTime parameter can be specified and if it is, Days and TimeSpan are ignored and invalid anyway. I believe I took this syntax structure from Search-ADObject or some form of that. I like it, it provides easy command line parameter syntax but variety.

The FileProperty parameter is used to contract which property is compared for the file, CreationTime, LastWriteTime, or LastAccessed time. I used CreationTime as the default because the event requirements stated that after the file is created it is not accessed again. I saw the majority of scripts use LastWriteTime. I don't think there is a write or wrong answer there, but having the option to change it is very valuable.

The Force parameter is for the Move-Item, shouldn't be needed, but provided it as a pass through.

Begin (Line 65-69)
I used the Set-StrictMode -Version Latest to make sure I am writing good PowerShell 3.0 script. Reminds me of Option Explicit from VBScript. That was always a best practice to use, I'm sure this will become one too.

Identifying the Culprits (Line 72)
You have to remember the Path parameter takes values from the Pipeline, so that will auto populate if we are passed values from something like Get-ChildItem. The beauty also of Get-ChildItem is that it supports multiple Paths to the Paths parameter and PowerShell 3.0 lets us just ask for files to be returned with the Files parameter.

Next we need to find the files that have exceeded the age using the FileProperty specified (default is CreationTime) and the DateTime calculated or provided value. The Where-Object cmdlet is part of the simplified syntax in PowerShell 3.0. Prior to that we had to use a ScriptBlock to calculate true/false for each item, now if we are just comparing a property and value(s) the cmdlet supports almost every comparison operator. Check out the full help on the Where-Object cmdlet for more information.

Overcoming Deficiencies (Line 74-78)
If only Move-Item had a parameter to create the parent container if it doesn't exist New-Item for would do it automatically with -Force then this could literally be line 72 and 80 combined into one pipeline. But I digress, it cannot so we must accommodate a small annoyance...

We have the list of files, so we need to maintain the directory structure in the destination so if the file was moved from C:\Application\Log\App1 to \\NASServer\Archive the files should end up in \\NASServer\Archive\App1. So, take the list of files and get the list of parent directories, oh and make it unique I only need to check them once total, not once per item.

Then ForEach-Object I take the destination path and join it to the directory name of the file, so \\NASServer\Archive + App1 becomes \\NASServer\Archive\App1.

Then Test-Path to only gather the ones that don't exist and have New-Item create it. You should notice the {$_} on the New-Item, this is a calculated parameter, I use it instead of |%{} or ForEach-Object to keep it in the pipeline. Finally, Out-Null because the script should produce no output except for errors.

Error Handling (Line ???)
This is a perfect time to address error handling. I couldn't believe how many Try..Catch statements I saw in the scripts that were submitted. Sometimes, well most of the times, PowerShell will provide this for you. Get-ChildItem will error if the Path contains invalid paths, New-Item will error if the Destination is not valid or can't be created, Move-Item will error if source can't be read or destination can't be created/written.

What about custom error messages? Simply, they are not needed, let PowerShell do the complaining, not your script. When are custom error messages appropriate, when you need to add additional context to the error message, such as variable values. PowerShell will do this for the built-in cmdlets, utilize PowerShell so you don't reinvent the wheel, because I saw A LOT of that going on in the advanced event entries.

PowerShell, Move Those Items (Line 80)
I saw ROBOCOPY wrappers, CMD.EXE wrappers, all sorts of interesting things in some of the entries, maybe I'll blog about some of the most interesting, we'll see. But really, PowerShell does this nicely, except the whole not creating the parent container automagically.

I originally had this all on one pipeline, thus I was still dealing with the file. On Move-Item is actually where I started using pipeline scipt based parameters. This calculated the value of Destination for each item and as you can see, it would take the information it knew about the File (Directory and Name) and make the destination value. I could have had two Join-Path statements, but that just felt odd, so I compromised.

The bulk of my submission is in the script with the exceptions of the enhanced comment based help, additional parameters instead of just a single TimeSpan $Retention parameter, and the addition of selecting which property to compare for age.

I hope this explanation helps you learn more about why I did some of the things I did, maybe provides some insight in how I think in PowerShell and in the end, you walk away with ideas on how to make your scripts more natural in PowerShell.

Feedback is appreciated, both negative and positive, I can only learn by failing or just rocking it like nobody else can.

I can't wait for event #2 ... event 1 really reminded me why I <3 PowerShell.


Popular posts from this blog

PowerShell SupportsShouldProcess Worst & Best Practices

This has been a very big discussion within the Scripting Games 2013 community and I want to add my two cents in an official blog post.

I've left several people comments on how they might be misunderstanding how SupportsShouldProcess works, but I also realize, everyone of these individuals has given me more insight into its use and perhaps, how it should best be utilized.

For those of you that don't know, SupportsShouldProcess is a parameter on the CmdletBinding attribute you can place on your cmdlets that automatically adds the -WhatIf and -Confirm parameters. These will naturally flow into other cmdlets you use that also SupportsShouldProcess, e.g. New-Item, Move-Item.

The major discussion has been around, should you just let the other cmdlets handle the $PSCmdlet.ShouldProcess feature, and if not how should you implement it. ShouldProcess has the following definitions.

Generate Random SecureString Key

Ever need to encrypt a SecureString that can be used across multiple servers? I suggest storing this BASE64 value in a secure location only accessible by the account(s) that need to decrypt the SecureString.
$secret = 'secret1234'$key    = [Convert]::ToBase64String((1..32 |% { [byte](Get-Random -Minimum 0 -Maximum 255) }))  $encryptedSecret = ConvertTo-SecureString -AsPlainText -Force -String $secret | ConvertFrom-SecureString -Key ([Convert]::FromBase64String($key))  $encryptedSecret | Select-Object @{Name='Key';Expression={$key}},@{Name='EncryptedSecret';Expression={$encryptedSecret}} | fl  $ss = ConvertTo-SecureString -Key ([Convert]::FromBase64String($key)) -String $encryptedSecret(New-Object System.Management.Automation.PSCredential 'SECURESTRING',$ss).GetNetworkCredential().Password

PowerShell Error Handling Behavior Debunked

Note: I am using simple error messages as an example, please reference the best practices and guidelines I outlined on when to use custom error messages.

I have been churning in my mind for the last few days all the entries in the 2013 Scripting Games and how they handle errors, or lack thereof.

I am coming to the conclusion through some testing that the simple fact of seeing a try..catch or throw statements does not mean there is proper error handling.

I've been testing several variations and forms of error handling, so lets start with the basics.
function Test-WriteError {      [CmdletBinding()] param()  "Test-WriteError::ErrorActionPreference = $ErrorActionPreference"Move-Item -Path 'C:\Does\Not\Exists.log' -Destination 'C:\No\Where'"Test-WriteError::End"}   Test-WriteError::ErrorActionPreference = Continue
Move-Item : Cannot find path 'C:\Does\Not\Exists.log' because it does not exist.
At line:6 char:5
+     Move-Item -Path 'C:\Does\N…