Task Workflow Validation Pipeline

Task Workflow Validation Pipeline

5-Stage Validation Pipeline for Badlucksbane Lab Automation

This document describes the end-to-end validation pipeline that ensures quality, security, and authenticity across all automated tasks in the Planner-Worker system.

Pipeline Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                         TASK LIFECYCLE                                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                 │
│  │   STAGE 1    │───▶│   STAGE 2    │───▶│   STAGE 3    │                 │
│  │ Input         │    │ Process       │    │ Output       │                 │
│  │ Validation    │    │ Validation    │    │ Validation    │                 │
│  └──────────────┘    └──────────────┘    └──────────────┘                 │
│           │                   │                   │                           │
│           ▼                   ▼                   ▼                           │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                 │
│  │ Planner       │    │ Worker        │    │ Post-Worker  │                 │
│  │ (Observer)    │    │ (Nurturer)    │    │ Quality      │                 │
│  └──────────────┘    └──────────────┘    └──────────────┘                 │
│                                                                              │
│           │                   │                   │                           │
│           ▼                   ▼                   ▼                           │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                 │
│  │ validate-     │    │ validate-     │    │ validate-     │                 │
│  │ planner-      │    │ worker-       │    │ deployment   │                 │
│  │ task.sh       │    │ output.sh     │    │ .sh          │                 │
│  └──────────────┘    └──────────────┘    └──────────────┘                 │
│                                                                              │
│           │                   │                   │                           │
│           ▼                   ▼                   ▼                           │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                        STAGE 4 & 5                                │   │
│  │              Deployment & System Verification                      │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Stage-by-Stage Details

Stage 1: Input Validation (Planner)

Purpose: Prevent bad tasks from entering the system at the source.

Implementation: [LOCAL_BIN_PATH]validate-planner-task.sh

Checks:

• ✅ Title validation (minimum 10 characters)
• ✅ Description validation (minimum 50 characters)
• ✅ Actionable verbs present (publish, create, fix, update, implement, etc.)
• ✅ Clear intent/acceptance criteria present
• ✅ Task type validation (known types only)
• ✅ Queue capacity check (backpressure at 10 tasks)
• ✅ Duplicate detection (basic exact match)

When: Before task creation in planner.sh

Result: Tasks that fail validation are logged and rejected; Planner continues without creating the task.

Stage 2: Process Validation (Worker)

Purpose: Ensure task execution produced quality output.

Implementation: [LOCAL_BIN_PATH]validate-worker-output.sh

Checks:

• ✅ Forbidden patterns (no system paths, external repositories, env files, credentials)
• ✅ Git state (no unpushed commits if content changed)
• ✅ Task-specific quality:
  • Notebook: minimum 500 bytes, 200 words, date in filename, no placeholders
  • Content: forbidden pattern scan
• ✅ Site accessibility (HTTP 200 from localhost:8080)

When: After LLM returns TASK_COMPLETED but before closing the task in worker.sh

Result: Tasks that fail validation remain open and are retried in the next worker run.

Stage 3: Output Validation (Post-Worker)

Purpose: Verify the output meets quality standards.

Implementation: Integrated into validate-worker-output.sh

Checks:

• ✅ Notebook quality (size, word count, structure)
• ✅ Content link integrity
• ✅ Markdown syntax validation

When: Immediately after task execution, as part of process validation

Result: Output that fails is flagged; task remains open for correction.

Stage 4: Deployment Validation (CI/CD)

Purpose: Ensure deployments succeed and content reaches production.

Implementation: .gitea/workflows/deploy.yml

Checks:

• ✅ Build artifacts exist (index.html, index.xml)
• ✅ Files deployed to /var/www/html/badlucksbane/public/
• ✅ Symlinks recreated (e.g., /work → [LOCAL_WORK_PATH])
• ✅ Nginx configuration test passes
• ✅ Nginx reload succeeds

When: Automatically triggered on git push via Gitea Actions

Result: Failed deployments prevent task closure and trigger alerts.

Stage 5: System-Level Validation (Safeguard)

Purpose: Monitor overall system health and auto-repair critical issues.

Implementation: [LOCAL_BIN_PATH]safeguard-check.sh

Checks:

• ✅ Content directory symlink integrity
• ✅ Git repository sync status
• ✅ Notebook authenticity (daily entries exist and are substantive)
• ✅ Beads database health
• ✅ CI/CD health (Gitea Runner, Nginx running)
• ✅ Script integrity (planner.sh, worker.sh exist and are executable)

When:

• Pre-flight check before every Planner run (every 30 minutes)
• Pre-flight check before every Worker run (every hour)
• Auto-repair: Creates beads issues for self-healing when problems detected

Result: Critical issues are auto-detected and auto-repaired; system remains healthy without manual intervention.

Validation Scripts Reference

Hand-off Criteria Between Stages

Planner → Worker

• Task must pass all Stage 1 (Input) validations
• Task ID must be unique
• Task description must be actionable
• Queue must have capacity (≤ 10 ready tasks)

Worker → Post-Worker

• LLM must return TASK_COMPLETED
• All Stage 2 (Process) validations must pass
• No forbidden patterns in output
• Git state must be clean (if content was modified)

Post-Worker → Deployment

• All Stage 3 (Output) validations must pass
• Site must be accessible (HTTP 200)

Deployment → System

• Deploy.yml workflow must complete successfully
• All Stage 4 (Deployment) validations must pass

System → Archive

• Task must remain closed with no validation failures
• All logs must be written
• Safeguard must confirm system health

Pipeline Visibility and Auditability

Logs

All validation results are logged to:

• [LOCAL_LOGS_PATH]planner-validation.log - Stage 1 validation results
• [LOCAL_LOGS_PATH]worker-validation.log - Stage 2/3 validation results
• [LOCAL_LOGS_PATH]safeguard.log - Stage 5 system health checks

Metrics

Tracked in beads database:

• Tasks created, closed, rejected
• Validation pass/fail rates
• Retry counts
• Auto-repair events

Alerts

Failed validations trigger:

1. Log entries with specific error messages
2. Task remains open for retry
3. After 3 consecutive failures: escalation to P0 priority

Failed Stage Handling

Related Tasks

• ✅ benbrown-0nh - Implement Input Validation in Planner
• ✅ benbrown-tmy - Implement Process Validation in Worker
• 🎯 benbrown-9lq - Restructure Task Workflow (this document)
• ⏳ benbrown-2nq - Add Deployment Validation to Task Workflow
• ⏳ benbrown-qri - Implement System-Level Validation - Safeguard Enhancement

Success Metrics

The pipeline is considered healthy when:

• ✅ 95%+ of tasks pass all validations on first attempt
• ✅ Zero tasks closed without passing all validations
• ✅ All validation logs show PASS status
• ✅ No critical safeguard alerts in 24 hours
• ✅ Site remains accessible and up-to-date

Last updated: 2026-08-02 This pipeline ensures the Lab Authenticity Principle: No claims without proof, no placeholder content.