Skip to main content

Throttling PowerShell Jobs

I needed to multi-thread a task that had a thousand records to process and each one needed a 15 second sleep. Well as I found out, there is no built-in throttling to Start-Job, so I had 1000 powershell.exe try to launch. Not good to say the least. Anyway, after looking online I found throttling code and adapted it to something simple and sweet. All I need to do is inject a call to Throttle-Jobs before I do a Start-Job and it'll wait for the number of running jobs to drop below the maximum.
  1. function Throttle-Jobs {  
  2.     param( [int] $maximum = 25, [TimeSpan] $interval = [TimeSpan]::FromSeconds(1) )  
  3.       
  4.     while ( (Get-Job -State Running | Measure-Object).Count -gt $maximum ) {  
  5.         Start-Sleep -Milliseconds $interval.TotalMilliseconds  
  6.     }  
  7. }  
Labels:

Comments

  1. Wicky NilliamsApril 24, 2013 at 8:08 AM

    This is fantastic :) I've been looking for something like this and all the other examples I've found online are massively over complex!

    THANKS!

    ReplyDelete
  2. AnonymousJuly 10, 2013 at 6:00 PM

    Ditto to wicky's comment. This was easily to implement in an already fleshed-out framework for job management I had. The other blogs are great for starting from scratch but this elegant solution let's you just use the function as you said, just before running the start-job cmdlet. Well done! Thank you! :)

    ReplyDelete

Post a Comment

Popular posts from this blog

PowerShell SupportsShouldProcess Worst & Best Practices

This has been a very big discussion within the Scripting Games 2013 community and I want to add my two cents in an official blog post.

I've left several people comments on how they might be misunderstanding how SupportsShouldProcess works, but I also realize, everyone of these individuals has given me more insight into its use and perhaps, how it should best be utilized.

For those of you that don't know, SupportsShouldProcess is a parameter on the CmdletBinding attribute you can place on your cmdlets that automatically adds the -WhatIf and -Confirm parameters. These will naturally flow into other cmdlets you use that also SupportsShouldProcess, e.g. New-Item, Move-Item.

The major discussion has been around, should you just let the other cmdlets handle the $PSCmdlet.ShouldProcess feature, and if not how should you implement it. ShouldProcess has the following definitions.
OverloadDefinitions�����������������������������������������������������������������������������������������…
1 comment
Read more

Generate Random SecureString Key

Ever need to encrypt a SecureString that can be used across multiple servers? I suggest storing this BASE64 value in a secure location only accessible by the account(s) that need to decrypt the SecureString.
$secret = 'secret1234'$key    = [Convert]::ToBase64String((1..32 |% { [byte](Get-Random -Minimum 0 -Maximum 255) }))  $encryptedSecret = ConvertTo-SecureString -AsPlainText -Force -String $secret | ConvertFrom-SecureString -Key ([Convert]::FromBase64String($key))  $encryptedSecret | Select-Object @{Name='Key';Expression={$key}},@{Name='EncryptedSecret';Expression={$encryptedSecret}} | fl  $ss = ConvertTo-SecureString -Key ([Convert]::FromBase64String($key)) -String $encryptedSecret(New-Object System.Management.Automation.PSCredential 'SECURESTRING',$ss).GetNetworkCredential().Password
1 comment
Read more

PowerShell Error Handling Behavior Debunked

Note: I am using simple error messages as an example, please reference the best practices and guidelines I outlined on when to use custom error messages.

I have been churning in my mind for the last few days all the entries in the 2013 Scripting Games and how they handle errors, or lack thereof.

I am coming to the conclusion through some testing that the simple fact of seeing a try..catch or throw statements does not mean there is proper error handling.

I've been testing several variations and forms of error handling, so lets start with the basics.
function Test-WriteError {      [CmdletBinding()] param()  "Test-WriteError::ErrorActionPreference = $ErrorActionPreference"Move-Item -Path 'C:\Does\Not\Exists.log' -Destination 'C:\No\Where'"Test-WriteError::End"}   Test-WriteError::ErrorActionPreference = Continue
Move-Item : Cannot find path 'C:\Does\Not\Exists.log' because it does not exist.
At line:6 char:5
+     Move-Item -Path 'C:\Does\N…
Post a Comment
Read more