Skip to main content

Throttling PowerShell Jobs

I needed to multi-thread a task that had a thousand records to process and each one needed a 15 second sleep. Well as I found out, there is no built-in throttling to Start-Job, so I had 1000 powershell.exe try to launch. Not good to say the least. Anyway, after looking online I found throttling code and adapted it to something simple and sweet. All I need to do is inject a call to Throttle-Jobs before I do a Start-Job and it'll wait for the number of running jobs to drop below the maximum.
  1. function Throttle-Jobs {  
  2.     param( [int] $maximum = 25, [TimeSpan] $interval = [TimeSpan]::FromSeconds(1) )  
  3.       
  4.     while ( (Get-Job -State Running | Measure-Object).Count -gt $maximum ) {  
  5.         Start-Sleep -Milliseconds $interval.TotalMilliseconds  
  6.     }  
  7. }  
Labels:

Comments

  1. This is fantastic :) I've been looking for something like this and all the other examples I've found online are massively over complex!

    THANKS!

    ReplyDelete

  2. Ditto to wicky's comment. This was easily to implement in an already fleshed-out framework for job management I had. The other blogs are great for starting from scratch but this elegant solution let's you just use the function as you said, just before running the start-job cmdlet. Well done! Thank you! :)

    ReplyDelete

Post a Comment

Popular posts from this blog

PowerShell SupportsShouldProcess Worst & Best Practices

This has been a very big discussion within the Scripting Games 2013 community and I want to add my two cents in an official blog post.

I've left several people comments on how they might be misunderstanding how SupportsShouldProcess works, but I also realize, everyone of these individuals has given me more insight into its use and perhaps, how it should best be utilized.

For those of you that don't know, SupportsShouldProcess is a parameter on the CmdletBinding attribute you can place on your cmdlets that automatically adds the -WhatIf and -Confirm parameters. These will naturally flow into other cmdlets you use that also SupportsShouldProcess, e.g. New-Item, Move-Item.

The major discussion has been around, should you just let the other cmdlets handle the $PSCmdlet.ShouldProcess feature, and if not how should you implement it. ShouldProcess has the following definitions.
OverloadDefinitions�����������������������������������������������������������������������������������������…
1 comment
Read more

FIM 2010 R2 Self-Service Password Reset Auto-Registration

I have been working on our FIM 2010 R2 SP1 lab environment looking for ways to simplify some of the overly complicated scenarios we had to implement to workaround the limitations in FIM 2010. One of those workarounds was the auto-registration of SSPR for new employees. When we onboard a new employee, we want to create a simple SSPR for them to get their first-time password reset.

Prior to the R2 release we were using the http://fim2010client.codeplex.com/ client and PowerShell to complete the registration process on a daily schedule. I was working on converting this to use the R2 registration cmdlets and combined it with PowerShell 3.0 Workflows to get to a solution similar to this.
[CmdletBinding()]  param()  begin {      workflow Invoke-RegisterSSPR {          InlineScript {              Import-Module FIM          }  $workflow = InlineScript { Get-FIMResource -Filter '/WorkflowDefinition[DisplayName = "Password Reset AuthN Workflow for New Employees"]' }  $members  …
Post a Comment
Read more

PowerShell Error Handling Behavior Debunked

Note: I am using simple error messages as an example, please reference the best practices and guidelines I outlined on when to use custom error messages.

I have been churning in my mind for the last few days all the entries in the 2013 Scripting Games and how they handle errors, or lack thereof.

I am coming to the conclusion through some testing that the simple fact of seeing a try..catch or throw statements does not mean there is proper error handling.

I've been testing several variations and forms of error handling, so lets start with the basics.
function Test-WriteError {      [CmdletBinding()] param()  "Test-WriteError::ErrorActionPreference = $ErrorActionPreference"Move-Item -Path 'C:\Does\Not\Exists.log' -Destination 'C:\No\Where'"Test-WriteError::End"}   Test-WriteError::ErrorActionPreference = Continue
Move-Item : Cannot find path 'C:\Does\Not\Exists.log' because it does not exist.
At line:6 char:5
+     Move-Item -Path 'C:\Does\N…
Post a Comment
Read more