Pair each rule with evidence

A rule file defines metadata and evaluation behavior. Self-test folders declare the source fixture, intended use, threshold, expected status, expected result code, findings that must appear, and findings that must not appear. Every active rule should have a positive self-test; rules with false-positive risk should also have negative regression tests.

Start from the included templates

The public Rule and Test Authoring Guide contains the maintained source contract. Copy the template, then rename the new file and metadata together.

src/PSRafScan/Templates/RuleTemplate.ps1
src/PSRafScan/Templates/SelfTestTemplate.psd1

Create the rule file

The file name must match RuleName. Use direct, user-facing Issue and Fix text. Choose Pattern for line or document text, Token for lexical tokens, Ast for syntax-tree structure, or Hybrid when more than one evidence source is required.

src/PSRafScan/Rules/CustomRules/<RuleName>.rule.ps1
@{
    SchemaVersion = '1.0'
    RuleName = 'ExampleRuleName'
    DisplayName = 'Example Rule Name'
    Category = 'Safety'
    RuleType = 'Ast'
    Severity = 'Warning'
    Enabled = $true
    IntendedUse = @('TerminalPaste', 'Ps1File')
    Tags = @('safety', 'example')
    Description = 'Detects Example-Command calls.'
    Issue = 'The script calls Example-Command.'
    Fix = 'Replace Example-Command with the approved command.'
    Version = '1.0.0'
    Author = 'PSRafScan'
    Aliases = @()
    Deprecated = $false
    DeprecatedSince = $null
}

Create positive and negative self-tests

  • A positive fixture should trigger the exact rule and expected severity.
  • A negative fixture should resemble the trigger but demonstrate an allowed case.
  • ExpectedFindings names findings that must appear.
  • NotExpectedFindings records false-positive protection.
  • RuleNames must contain only current rule names.
src/PSRafScan/SelfTests/Cases/<TestName>/Script.ps1
src/PSRafScan/SelfTests/Cases/<TestName>/<TestName>.expected.psd1

Search and validate

Run catalog, self-test, and quality gates in the project development environment. A new rule is incomplete if metadata search cannot find it, positive evidence is missing, stale self-test mappings remain, or the full suite regresses.

Get-PSRafScanRule -RuleName ExampleRuleName
Get-PSRafScanSelfTest -RuleName ExampleRuleName
RSX -SelfTest
pwsh -ExecutionPolicy Bypass -File ./tools/Test-PSRafScanMetadataCatalog.ps1
pwsh -ExecutionPolicy Bypass -File ./tools/Test-PSRafScanQualityGate.ps1

Manage the lifecycle

  • Set Enabled = $false to omit a rule from scanning and default listings without deleting it.
  • Set Deprecated = $true and record DeprecatedSince when replacing a rule while preserving discovery.
  • When deleting a rule, remove or update every self-test reference in RuleNames, ExpectedFindings, and NotExpectedFindings.
  • Update rule and self-test evidence in the same source change.