Maintenance & Operations SOPs

PI System Maintenance Philosophy

Consistent, documented maintenance is the foundation of a reliable PI System. This module provides SOP-style procedures for daily and monthly tasks, with automation scripts to reduce manual effort.

Daily Maintenance Checklist

Morning Health Check (15 minutes)

[ ] 1. Check PI System dashboard in PI Vision
      - All interfaces showing green (connected)
      - No active alarms on system health display
      - Archive fill % < 80%

[ ] 2. Review overnight alerts
      - Check email/SMS alerts from previous night
      - Review PI Notifications inbox
      - Document any anomalies in shift log

[ ] 3. Verify interface status
      - PI SMT → Interfaces → All interfaces "Running"
      - Event rates match expected values
      - No "No Data" or "Shutdown" system states

[ ] 4. Check buffer subsystem
      - PI SMT → Buffering → Queue depth < 1000
      - No buffer overflow warnings in pipc.log

[ ] 5. Verify backup completed
      - Check backup server for last night's backup
      - Confirm backup file size is consistent
      - Log backup status in maintenance record

Monthly Maintenance Checklist

Archive Management (Week 1)

[ ] 1. Archive capacity planning
      - Calculate current fill rate (GB/day)
      - Project when current archive will fill
      - Pre-create next archive if needed (at 70% fill)

[ ] 2. Archive grooming
      - Review archives older than retention policy
      - Verify old archives are backed up before deletion
      - Remove archives beyond retention period

[ ] 3. Archive backup validation
      - Restore a test archive to validation server
      - Verify data integrity (spot-check 10 tags)
      - Document validation results

Security Audit (Week 2)

[ ] 4. Review PI user accounts
      - Disable accounts for departed employees
      - Review service account passwords (rotate if > 90 days)

[ ] 5. Review PI Identity mappings
      - Verify AD group memberships are current
      - Remove mappings for decommissioned systems

[ ] 6. PI Trust review
      - Review PI Interface trusts
      - Remove trusts for decommissioned interfaces
      - Verify trust IP addresses are still valid

Incident Response Procedures

P1: Interface Disconnection

Severity: P1 (immediate response required)
SLA: Acknowledge < 15 min, Resolve < 2 hours

Step 1: Confirm the issue
  - PI SMT → Interfaces → Verify interface status
  - Check event rate (should be > 0)

Step 2: Check interface node
  - RDP to interface node
  - Check interface service status
  - Review interface log: C:\PI\Interfaces\<name>\<name>.log

Step 3: Common resolutions
  - Service stopped: Start-Service <interface-name>
  - Network issue: ping DCS/SCADA source system
  - Authentication: Check PI Trust configuration

Step 4: Verify recovery
  - Confirm event rate returns to normal
  - Check for data backfill from buffer
  - Document incident and resolution

P2: Archive Approaching Full

Severity: P2 (respond within 1 hour)
Trigger: Archive fill > 85%

Step 1: Check current fill level
  piartool -al

Step 2: Create new archive
  piartool -createarchive <path>\<name>.arc <size_MB>
  piartool -al  (verify new archive is "Current")

Step 3: Document
  - Log archive creation in maintenance record
  - Update capacity planning spreadsheet