wp-plugin-starter-template-for-ai-coding/.agents/error-checking-feedback-loops.md

# Error Checking and Feedback Loops

This document outlines the processes for error checking, debugging, and establishing feedback loops between AI assistants and various systems in the development workflow. The goal is to create a seamless, autonomous CI/CD pipeline where the AI can identify, diagnose, and fix issues with minimal human intervention.

## Table of Contents

* [GitHub Actions Workflow Monitoring](#github-actions-workflow-monitoring)
* [Local Build and Test Feedback](#local-build-and-test-feedback)
* [Code Quality Tool Integration](#code-quality-tool-integration)
* [Automated Error Resolution](#automated-error-resolution)
* [Feedback Loop Architecture](#feedback-loop-architecture)
* [When to Consult Humans](#when-to-consult-humans)

## GitHub Actions Workflow Monitoring

### Checking Workflow Status via GitHub API

AI assistants can directly monitor GitHub Actions workflows using the GitHub API to identify failures and diagnose issues:

```
github-api /repos/{owner}/{repo}/actions/runs
```

#### Step-by-Step Process

1. **Get Recent Workflow Runs**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs
   ```

2. **Filter for Failed Runs**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs?status=failure
   ```

3. **Get Details for a Specific Run**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs/{run_id}
   ```

4. **Get Jobs for a Workflow Run**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs/{run_id}/jobs
   ```

5. **Analyze Job Logs** (if accessible via API):
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/jobs/{job_id}/logs
   ```

### Common GitHub Actions Errors and Solutions

#### Missing or Outdated Action Versions

**Error**: `Missing download info for actions/upload-artifact@v3`

**Solution**: Update to the latest version of the action:
```yaml
uses: actions/upload-artifact@v4
```

#### Port Configuration Issues for WordPress Multisite

**Error**: `The current host is 127.0.0.1:8888, but WordPress multisites do not support custom ports.`

**Solution**: Use port 80 for multisite environments:
```yaml
npx @wp-playground/cli server --blueprint playground/multisite-blueprint.json --port 80 --login &
```

#### Artifact Path Syntax Issues

**Error**: Invalid path syntax for artifacts

**Solution**: Use multi-line format for better readability:
```yaml
path: |
  cypress/videos
  cypress/screenshots
```

#### Concurrency Control

**Problem**: Redundant workflow runs when multiple commits land quickly

**Solution**: Add concurrency control to cancel in-progress runs:
```yaml
concurrency:
  group: playground-tests-${{ github.ref }}
  cancel-in-progress: true
```

## Local Build and Test Feedback

### Monitoring Local Test Runs

AI assistants can monitor local test runs by analyzing the output of test commands:

#### PHP Unit Tests

```bash
composer run phpunit
```

#### Cypress Tests

```bash
npm run test:single
npm run test:multisite
```

#### WordPress Playground Tests

```bash
npm run test:playground:single
npm run test:playground:multisite
```

### Capturing and Analyzing Test Output

1. **Run Tests with Output Capture**:
   ```bash
   npm run test:single > test-output.log 2>&1
   ```

2. **Analyze Output for Errors**:
   ```bash
   launch-process command="cat test-output.log | grep -i 'error\|fail\|exception'" wait=true max_wait_seconds=10
   ```

3. **Parse Structured Test Results** (if available):
   ```bash
   launch-process command="cat cypress/results/results.json" wait=true max_wait_seconds=10
   ```

### Common Local Test Errors and Solutions

#### WordPress Playground Port Issues

**Error**: `Error: The current host is 127.0.0.1:8888, but WordPress multisites do not support custom ports.`

**Solution**: Modify the port in the blueprint or test configuration:
```json
{
  "features": {
    "networking": true
  }
}
```

#### Cypress Selector Errors

**Error**: `Timed out retrying after 4000ms: expected '<body.login.js.login-action-login.wp-core-ui.locale-en-us>' to have class 'wp-admin'`

**Solution**: Update selectors to be more robust and handle login states:
```javascript
cy.get('body').then(($body) => {
  if ($body.hasClass('login')) {
    cy.get('#user_login').type('admin');
    cy.get('#user_pass').type('password');
    cy.get('#wp-submit').click();
  }
});

// Check for admin bar instead of body class
cy.get('#wpadminbar').should('exist');
```

## Code Quality Tool Integration

### Automated Code Quality Checks

AI assistants can integrate with various code quality tools to identify and fix issues:

#### PHPCS (PHP CodeSniffer)

```bash
composer run phpcs
```

#### ESLint (JavaScript)

```bash
npm run lint:js
```

#### Stylelint (CSS)

```bash
npm run lint:css
```

### Parsing Code Quality Tool Output

1. **Run Code Quality Check**:
   ```bash
   composer run phpcs > phpcs-output.log 2>&1
   ```

2. **Analyze Output for Errors**:
   ```bash
   launch-process command="cat phpcs-output.log | grep -i 'ERROR\|WARNING'" wait=true max_wait_seconds=10
   ```

3. **Automatically Fix Issues** (when possible):
   ```bash
   composer run phpcbf
   ```

### Monitoring Code Quality Feedback in Pull Requests

Automated code quality tools often provide feedback directly in pull requests. AI assistants can check these comments to identify and address issues:

#### Accessing PR Comments via GitHub API

```
github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/comments
```

#### Accessing PR Review Comments

```
github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/reviews
```

#### Checking CodeRabbit Feedback

CodeRabbit provides AI-powered code review comments that can be accessed via the GitHub API:

1. **Get PR Comments**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/comments
   ```

2. **Filter for CodeRabbit Comments**:
   Look for comments from the `coderabbitai` user.

3. **Parse Actionable Feedback**:
   CodeRabbit comments typically include:
   * Code quality issues
   * Suggested improvements
   * Best practice recommendations
   * Specific code snippets to fix

#### Checking Codacy and CodeFactor Feedback

These tools provide automated code quality checks and post results as PR comments or status checks:

1. **Check PR Status Checks**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/commits/{commit_sha}/check-runs
   ```

2. **Get Detailed Reports** (if available via API):
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/commits/{commit_sha}/check-runs/{check_run_id}
   ```

3. **Parse Common Issues**:
   * Code style violations
   * Potential bugs
   * Security vulnerabilities
   * Performance issues
   * Duplication

#### Checking SonarCloud Analysis

SonarCloud provides detailed code quality and security analysis:

1. **Check SonarCloud Status**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/commits/{commit_sha}/check-runs?check_name=SonarCloud
   ```

2. **Parse SonarCloud Issues**:
   * Code smells
   * Bugs
   * Vulnerabilities
   * Security hotspots
   * Coverage gaps

### Common Code Quality Issues and Solutions

#### WordPress Coding Standards

**Error**: `ERROR: Expected snake_case for function name, but found camelCase`

**Solution**: Rename functions to follow snake_case convention:
```php
// Before
function getPluginVersion() { ... }

// After
function get_plugin_version() { ... }
```

#### Missing Docblocks

**Error**: `ERROR: Missing doc comment for function`

**Solution**: Add proper docblocks:
```php
/**
 * Get the plugin version.
 *
 * @since 1.0.0
 * @return string The plugin version.
 */
function get_plugin_version() { ... }
```

## Automated Error Resolution

### Error Resolution Workflow

1. **Identify Error**: Use GitHub API or local test output to identify errors
2. **Categorize Error**: Determine error type and severity
3. **Search for Solution**: Look for patterns in known solutions
4. **Apply Fix**: Make necessary code changes
5. **Verify Fix**: Run tests again to confirm the issue is resolved
6. **Document Solution**: Update documentation with the solution

### Processing Code Quality Feedback

#### Extracting Actionable Items from PR Comments

1. **Collect All Feedback**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/comments
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/reviews
   ```

2. **Categorize Issues**:
   * Critical: Security vulnerabilities, breaking bugs
   * High: Code quality violations, potential bugs
   * Medium: Style issues, best practices
   * Low: Documentation, minor improvements

3. **Prioritize Fixes**:
   * Address critical issues first
   * Group related issues for efficient fixing
   * Consider dependencies between issues

4. **Create Fix Plan**:
   * Document files that need changes
   * Outline specific changes needed
   * Note any potential side effects

#### Responding to Code Quality Tool Comments

1. **Acknowledge Feedback**:
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/issues/comments/{comment_id}/reactions
   ```

2. **Implement Fixes**:
   ```
   str-replace-editor command="str_replace" path="path/to/file.php" str_replace_entries=[...]
   ```

3. **Explain Changes** (if needed):
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/comments
   ```

4. **Request Review** (if needed):
   ```
   github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/pulls/{pull_number}/requested_reviewers
   ```

### Example: Fixing GitHub Actions Workflow

```
# 1. Identify the error
github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs?status=failure

# 2. Get details of the failing job
github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs/{run_id}/jobs

# 3. Fix the issue in the workflow file
str-replace-editor command="str_replace" path=".github/workflows/playground-tests.yml" str_replace_entries=[...]

# 4. Commit and push the changes
launch-process command="git add .github/workflows/playground-tests.yml" wait=true max_wait_seconds=30
launch-process command="git commit -m 'Fix GitHub Actions workflow: update upload-artifact to v4'" wait=true max_wait_seconds=60
launch-process command="git push" wait=true max_wait_seconds=60

# 5. Verify the fix
github-api /repos/wpallstars/wp-plugin-starter-template-for-ai-coding/actions/runs
```

## Feedback Loop Architecture

### Complete Feedback Loop System

```
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│                 │     │                 │     │                 │
│  Code Changes   │────▶│  Local Testing  │────▶│  GitHub Actions │
│                 │     │                 │     │                 │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         │                       │                       │
         │                       │                       │
┌────────▼────────┐     ┌────────▼────────┐     ┌────────▼────────┐
│                 │     │                 │     │                 │
│  AI Assistant   │◀────│  Error Analysis │◀────│  Status Check   │
│                 │     │                 │     │                 │
└────────┬────────┘     └─────────────────┘     └─────────────────┘
         │
         │
         │
┌────────▼────────┐     ┌─────────────────┐
│                 │     │                 │
│  Fix Generation │────▶│  Human Review   │ (only when necessary)
│                 │     │                 │
└─────────────────┘     └─────────────────┘
```

### Key Components

1. **Code Changes**: Initial code modifications
2. **Local Testing**: Immediate feedback on local environment
3. **GitHub Actions**: Remote CI/CD pipeline validation
4. **Status Check**: Monitoring workflow status via GitHub API
5. **Error Analysis**: Parsing and categorizing errors
6. **AI Assistant**: Central intelligence for error resolution
7. **Fix Generation**: Creating and implementing solutions
8. **Human Review**: Optional step for complex decisions

## Handling Direct Feedback from Code Review Tools

### Accessing and Processing CodeRabbit Feedback

CodeRabbit provides detailed AI-powered code reviews that can be directly accessed and processed:

#### Example CodeRabbit Feedback

```
coderabbitai bot left a comment
Actionable comments posted: 1

🧹 Nitpick comments (3)
.github/workflows/playground-tests-fix.yml (3)
9-13: Add concurrency control to avoid redundant runs.
When multiple commits land in quick succession, you may end up with overlapping Playground test jobs. Adding a concurrency block will cancel in‑progress runs for the same ref and reduce CI load:

concurrency:
  group: playground-tests-${{ github.ref }}
  cancel-in-progress: true
```

#### Processing Steps

1. **Extract Specific Recommendations**:
   * Identify file paths and line numbers
   * Parse suggested code changes
   * Understand the rationale for changes

2. **Implement Recommendations**:
   ```
   str-replace-editor command="str_replace" path=".github/workflows/playground-tests-fix.yml" str_replace_entries=[{"old_str": "name: WordPress Playground Tests Fix\n\non:\n  push:\n    branches: [ main, feature/*, bugfix/* ]\n  pull_request:\n    branches: [ main ]", "new_str": "name: WordPress Playground Tests Fix\n\non:\n  push:\n    branches: [ main, feature/*, bugfix/* ]\n  pull_request:\n    branches: [ main ]\n\nconcurrency:\n  group: playground-tests-${{ github.ref }}\n  cancel-in-progress: true", "old_str_start_line_number": 1, "old_str_end_line_number": 6}]
   ```

3. **Verify Implementation**:
   * Run local tests if applicable
   * Commit changes with descriptive message
   * Monitor CI/CD pipeline for success

### Handling SonarCloud and Codacy Feedback

These tools provide structured feedback that can be systematically addressed:

#### Example SonarCloud Feedback

```
SonarCloud Quality Gate failed
- 3 Bugs
- 5 Code Smells
- 1 Security Hotspot
```

#### Processing Steps

1. **Access Detailed Reports**:
   * Use the SonarCloud API or web interface
   * Categorize issues by severity and type

2. **Address Issues Systematically**:
   * Fix bugs first
   * Address security hotspots
   * Resolve code smells

3. **Document Resolutions**:
   * Note patterns of issues for future prevention
   * Update coding guidelines if necessary

## When to Consult Humans

While the goal is to create an autonomous system, there are scenarios where human input is necessary:

### Scenarios Requiring Human Consultation

1. **Product Design Decisions**: Features, UX, and strategic direction
2. **Security-Critical Changes**: Changes that could impact security posture
3. **Architectural Decisions**: Major structural changes to the codebase
4. **Deployment Approvals**: Final approval for production releases
5. **Access Requirements**: When additional permissions are needed
6. **Ambiguous Errors**: When errors have multiple possible interpretations
7. **Novel Problems**: Issues without precedent or documented solutions
8. **External Service Issues**: Problems with third-party services

### Effective Human Consultation

When consulting humans, provide:

1. **Clear Context**: Explain what you were trying to accomplish
2. **Error Details**: Provide specific error messages and logs
3. **Attempted Solutions**: Document what you've already tried
4. **Specific Questions**: Ask targeted questions rather than open-ended ones
5. **Recommendations**: Suggest possible solutions for approval

## Conclusion

This error checking and feedback loop system creates a comprehensive framework for AI-driven development with minimal human intervention. By systematically monitoring, analyzing, and resolving errors across local and remote environments, the AI assistant can maintain high code quality and ensure smooth CI/CD processes.

For specific workflows related to feature development, bug fixing, and releases, refer to the other documents in the `.ai-workflows/` directory.