Certificate recieving ends after 30 minutes with "The handle is invalid"

Oct 28, 2013 at 7:38 AM
I'm trying to receive issued valid certificates from sub CA DB on 2003 r2 & DB contains over 150000 certificates. It always ends after 30 minutes and 819 certificates with "The handle is invalid. 0x80070006 (WIN32: 6)". Changing ViewAgeMinutes give no effect.
Main question is how to change this 30-minutes value.
Additional question is there anything thal'll make this certificate recieving more faster (I've already compacted my CA DB).
Coordinator
Oct 28, 2013 at 7:47 AM
can you, please, provide exact command you are using to dump CA database?
BTW, 819 certs in 30 minutes is abnormally slow.
Oct 28, 2013 at 8:10 AM
Edited Oct 28, 2013 at 8:10 AM
Yep, and it's so slow on any hardware i've tried (i can't improve only harddrive features). At the beginning all CA DB loads into memory (one half into physical, one half into VM) (it takes near 5 minutes), then the search begins. Wherein queue to logical disc with CA DB length increase up to 100 percent. CA DB is near 985 Mb after compacting. For processor scheduling & memory usage best performance adjust of programs. Total paging file size is 2048 Mb.
I've tried the same at CA on 2008, and the performance is near 10000 in 16 minutes.

i'm using this code to receive issued valid certificates:
$CAView=New-Object -ComObject CertificateAuthority.View
$CAView.OpenConnection($CNDN)
$properties="RequestID","NotAfter","NotBefore","SerialNumber"
$CAView.SetResultColumnCount($properties.Count)
$properties|%{$CAView.SetResultColumn($CAView.GetColumnIndex($False, $_))}
$RColumn=$CAView.GetColumnIndex($False, "Disposition")
$CAView.SetRestriction($RColumn,1,0,20)
$RColumn=$CAView.GetColumnIndex($False, "NotAfter")
$CAView.SetRestriction($RColumn,0x10,0,$DT)
$RColumn=$CAView.GetColumnIndex($False, "NotBefore")
$CAView.SetRestriction($RColumn,0x10,0,$lastDT)
$Row=$CAView.OpenView()
while ($Row.Next() -ne -1) {
$cert=New-Object psobject
$Column=$Row.EnumCertViewColumn()
while ($Column.Next() -ne -1) {
$current=$Column.GetName()
$Cert|Add-Member -MemberType NoteProperty $($Column.GetDisplayName()) -Value $($Column.GetValue(1)) -Force
}
$SN=$Cert.{Serial Number}
$ID=$Cert.{Issued Request ID}
Connect-CertificationAuthority -ComputerName $CN|Get-IssuedRequest -Filter "SerialNumber -eq $SN"|Receive-Certificate -Path $pathCertIssuedExport -Force
$certNameFull=[string]$certNameFirstPart+[string]$id+[string]$certNameExt
$certNameNewFull=[string]$SN+[string]$certNameExt
if((Test-Path -path $pathCertIssuedExport"\"$certNameFull) -eq $True) {Get-ChildItem $pathCertIssuedExport"\"$certNameFull|rename-item -newname $certNameNewFull}
$Column.Reset()
}
$Row.Reset()

The $Row.Next() method throw error "The handle is invalid. 0x80070006 (WIN32: 6)".
On 2008 it throws this error too, but it happens when the time that is determined by ViewAgeMinutes is reached.
Coordinator
Oct 28, 2013 at 10:25 AM
Ok, I see where is your problem with performance, your code is inefficient at all and there are alot of redundant calls. Why are you calling Connect-CA command inside a loop? This call consumes a lot of time. If you wish to save all issued certs to files, then you should run:
Connect-CA -comp $computername | Get-IssuedRequest -filter "NotBefore -ge $DT","NotAfter -le $lastDT" -Property "RawCertificate" | %{[IO.File]::WriteAllText("$pathCertIssuedExport\$($_.SerialNumber).cer", $_.RawCertificate)}
Here we add "RawCertificate" property which contains raw certificate in request row. So we add this property and retrieve certificates along the query call. And finally we save RawCertificate property to a file. Note that $pathCertIssuedExport must exist. I'm using [IO.File]::WriteAllText method instead of standard Set-Content cmdlet because it provides better performance than cmdlet.
Oct 28, 2013 at 11:11 AM
I'm calling it inside a loop, cause i'm just learning PowerShell) Thank you very much for showing me such easy way to do it. I've tried this and it works really fast.
But what do you think about, why my loop works so fast on 2008?
Coordinator
Oct 28, 2013 at 11:24 AM
Edited Oct 28, 2013 at 11:25 AM
There can be various factors, like caching and other underlying API improvements. But in any way, you should consider to make as fewer calls as possible.
Oct 29, 2013 at 11:37 AM
Is there any way to avoid making many calls if it's neсessary to do smth like this:
Get-ChildItem $pathCertSuspendedValidExport -Name|ForEach-Object -Process {$SN=$_ -replace ".cer"; Connect-CertificationAuthority -ComputerName $CN|Get-IssuedRequest -Filter "Disposition -eq 20","NotAfter -ge $DT","SerialNumber -eq $SN" -Property "RawCertificate"| %{[IO.File]::WriteAllText("$pathCertIssuedExport\$($_.SerialNumber).cer", $_.RawCertificate)}}
$pathCertSuspendedValidExport contains all valid suspended certificates, so i'm truing to find out if there is any of them that become unrevoked

I've tried to put all filenames in $pathCertSuspendedValidExport in massive by this code:
$SN = @(Get-ChildItem $pathCertSuspendedValidExport -Name -exclude "DT.txt"|ForEach-Object -process {$_ -replace ".cer"})
But don't know how to compare SerialNumber in filter with every massive value without using foreach.
Coordinator
Oct 29, 2013 at 1:06 PM
I don't understand what is your final goal. And what do you mean under "suspended certificates"?
Oct 29, 2013 at 1:19 PM
Suspeneded certificates are those that were revoked with reason "Certificate Hold". I'm using this code to do it:
Connect-CertificationAuthority -ComputerName $CN|Get-RevokedRequest -Filter "Disposition -eq 21","NotAfter -ge $DT","RevokedEffectiveWhen -ge $lastDT","RevokedReason -eq 6" -Property "RawCertificate" | %{[IO.File]::WriteAllText("$pathCertSuspendedValidExport\$($_.SerialNumber).cer", $_.RawCertificate)}

My final goal is to get all certificates from database sorted in four folders: issued, lapsed, revoked & suspended. And then refresh them every day.
So certificates, that were placed in Suspended folder, could become unrevoked, so i should place them in Issued folder and remove from Revoked & Suspended folders.
Coordinator
Oct 29, 2013 at 2:00 PM
when querying revoked certificates, you should use 2 queries:
Get-RevokedRequest -Filter "NotAfter -ge $DT","RevokedEffectiveWhen -ge $lastDT","RevokedReason -eq 6" -Property "RawCertificate"
Get-RevokedRequest -Filter "RevokedReason -lt 6","NotAfter -ge $DT","RevokedEffectiveWhen -ge $lastDT","RevokedReason -eq 6" -Property "RawCertificate"
use first query to save revoked certs with "Certificate Hold" reason to a desired folder. The second query dumps all revoked certificates except ones retrieved from previous query and save them to a desired folder.
Oct 29, 2013 at 2:22 PM
Edited Oct 29, 2013 at 2:22 PM
I'm already using 2 queries. I think there are some misunderstanding. My plan is:
  1. Valid issued certificates -> Issued folder
  2. Lapsed issued certificates -> Lapsed folder with removing from Issued folder
  3. Valid revoked certificates -> Revoked folder with removing from Issued folder
  4. Valid suspended certificates -> Suspended folder
  5. Lapsed revoked certificates -> Lapsed folder with removing from Revoked & Suspended folders
  6. Renewed certificates -> Issued folder with removing from Revoked & Suspended folders
I'm talking about point 6.
I'm doing it using this code:
Get-ChildItem $pathCertSuspendedValidExport -Name|ForEach-Object -Process {$SN=$_ -replace ".cer"; Connect-CertificationAuthority -ComputerName $CN|Get-IssuedRequest -Filter "Disposition -eq 20","NotAfter -ge $DT","SerialNumber -eq $SN" -Property "RawCertificate"| %{[IO.File]::WriteAllText("$pathCertIssuedExport\$($_.SerialNumber).cer", $_.RawCertificate)}}
but it causes many calls (number equals to files in Suspended folder quantity. I've asked about how to avoid using many calls in this situation.
Coordinator
Oct 29, 2013 at 2:30 PM
from what I see, you have to use 6 calls in overall and restrict each call to meet your plan., say:
1) call all valid certificates, except lapsed and revoked certs. Move them to Issued certs folder.
2) call all lapsed issued certs
3) call all revoked certs

and so on. Basically each query should return only certs that should be placed in a specified folder, so you don't have to manage the folder content in further calls.
Oct 29, 2013 at 4:07 PM
Edited Oct 29, 2013 at 8:30 PM
It's impossible avoid managing folders, cause this "export certificates" script starts exporting every N hour only new information: certificates that were issued, revoked, lapsed or renewed since last exporting. After managing this four folders indicates current database state. I can't overwrite whole folders cause another programmer analyze them, it's the wish of the customer(.