Auto-Documentation Workflows

Overview

This project uses automated workflows to maintain documentation quality and detect documentation gaps. The system consists of two complementary workflows that work together to ensure documentation stays current and comprehensive.

Complete System Guide: For the full picture of how all documentation automation works together, see the Documentation Workflows Hub.

System Integration

This document focuses on the quality monitoring and evolution tracking aspects of the documentation system. It works together with:

Workflows Overview

1. Documentation Check Workflow

  • File: .github/workflows/documentation-check.yml
  • Purpose: Detects documentation gaps and creates GitHub issues for missing documentation
  • Triggers: Pushes to dev only, daily at 2 AM UTC, manual dispatch

2. Documentation Evolution Workflow

  • File: .github/workflows/documentation-evolution.yml
  • Purpose: Analyzes code changes and maintains documentation quality over time
  • Triggers: Pushes to dev only, daily at 2 AM UTC, manual dispatch

Documentation Check Workflow

Features

Stub File Detection

Automatically detects documentation stubs with these markers:

  • STUB: - Comprehensive documentation needed
  • TODO: - Specific tasks to complete
  • PLACEHOLDER - Content that needs to be filled in
  • TBD: - To be determined sections

Gap Analysis

Identifies missing documentation by:

  • Scanning for undocumented components
  • Checking for outdated documentation
  • Finding broken internal links
  • Detecting incomplete documentation sections

Issue Creation

Creates GitHub issues with:

  • auto-documentation label for automated tracking
  • Priority levels based on file location
  • Detailed descriptions of what documentation is needed
  • Links to related code files

Usage

Creating Stub Files

When implementing new features, create stub files to trigger documentation:

# NEW_FEATURE_NAME.md

STUB: This feature needs comprehensive documentation

## Overview
TODO: Add detailed feature overview

## API Reference
PLACEHOLDER: Document all public methods and interfaces

## Examples
TBD: Add real-world usage examples

## Configuration
CRITICAL: Document configuration options and defaults

Priority Markers

Use these keywords to set documentation priority:

  • CRITICAL: - Essential documentation (development/API docs)
  • URGENT: - Time-sensitive documentation needs
  • IMPORTANT: - High-impact feature documentation

File Location Priority

The workflow prioritizes documentation based on location:

  • Critical Priority: docs/development/ (build, deployment, API docs)
  • Moderate Priority: docs/features/, docs/testing/, docs/troubleshooting/
  • Minor Priority: Other documentation directories

Documentation Evolution Workflow

Features

Commit-Based Analysis

When code is modified, the workflow:

  1. Detects changed files in commits and PRs
  2. Identifies which files need documentation updates
  3. Generates specific suggestions based on code analysis
  4. Creates GitHub issues or PR comments with actionable recommendations

Daily Quality Review

Runs daily to:

  • Select documentation files for quality review (oldest, random, or smart selection)
  • Analyze documentation for improvement opportunities
  • Check for redundant or overly complex content
  • Generate quality metrics and reports

Meta-Documentation Generation

Automatically creates and updates system-level documentation:

  • System architecture diagrams
  • Component interaction maps
  • Data flow documentation

Quality Markers

The workflow recognizes these improvement markers in documentation:

# Component Documentation

## Overview
IMPROVEMENT: This section needs more detail about component lifecycle

## Usage Examples  
MISSING_EXAMPLE: Add real-world usage scenarios

## Configuration
SIMPLIFY: This section is too complex, break into smaller parts

## API Reference
OUTDATED: This API has changed since last update

## Related Components
REDUNDANT: This information is duplicated in COMPONENT_GUIDE.md

Supported markers:

  • IMPROVEMENT: - Marks sections needing enhancement
  • SIMPLIFY: - Identifies overly complex content
  • MISSING_EXAMPLE: - Flags missing code examples
  • OUTDATED: - Marks potentially outdated content
  • REDUNDANT: - Identifies duplicate content

Code Analysis Features

Component Analysis

For TypeScript/React components, the workflow detects:

  • Exported functions and interfaces
  • Component props and their types
  • Usage patterns and dependencies

API Analysis

For Python backend code, the workflow identifies:

  • Function signatures and docstrings
  • Class definitions and methods
  • API endpoints and parameters

Documentation Suggestions

The workflow generates specific suggestions based on code analysis:

For Component Changes

// This code will trigger suggestions for:
// - Props documentation
// - Usage examples
// - Integration patterns

export interface ToolSelectorProps {
  selectedTools: string[];
  onChange: (tools: string[]) => void;
  toolConfigs?: Record<string, Record<string, any>>; 
}

export function ToolSelector(props: ToolSelectorProps) {
  // Implementation details
}

For API Changes

# This code will trigger suggestions for:
# - Parameter documentation
# - Return value examples
# - Error handling documentation

def process_tool_config(config: Dict[str, Any]) -> ToolResult:
    """Process tool configuration and return result."""
    pass

Meta-Documentation

The workflow can automatically generate these system documents:

System Architecture (docs/meta/SYSTEM_ARCHITECTURE.md)

  • High-level system overview
  • Technology stack documentation
  • Deployment architecture
  • Core component relationships

Component Interactions (docs/meta/COMPONENT_INTERACTIONS.md)

  • Component dependency maps
  • Communication patterns
  • Data flow between components
  • Integration points

Data Flow (docs/meta/DATA_FLOW.md)

  • Request/response flows
  • State management patterns
  • Database interactions
  • External service integrations

Configuration

Environment Variables

Both workflows use these environment variables:

  • DOCS_BOT_NAME: “claude” - Bot name for issue assignments
  • DOCS_LABEL: “documentation” - Main documentation label
  • AUTO_DOCS_LABEL: “auto-documentation” - Automated workflow label
  • EVOLUTION_LABEL: “doc-evolution” - Evolution workflow label
  • META_DOCS_LABEL: “meta-documentation” - Meta-documentation label

Scripts Used

The workflows depend on these scripts in the scripts/ directory:

  • analyze-documentation-gaps.sh - Finds documentation gaps
  • detect-stub-files.sh - Identifies stub files
  • check-documentation-freshness.sh - Checks document age
  • check-documentation-for-changed-files.sh - Analyzes file changes
  • generate-documentation-suggestions.sh - Creates suggestions
  • select-documentation-review-files.sh - Selects files for review
  • review-documentation-quality.sh - Reviews documentation quality
  • analyze-system-interactions.sh - Analyzes system relationships
  • find-redundant-documentation.sh - Finds duplicate content

Documentation Principles

Both workflows enforce these documentation principles:

1. Simplicity

  • Use clear, concise language
  • Avoid jargon and overly technical terms
  • Break complex topics into digestible sections

2. Real Examples

  • Include working code examples from the actual codebase
  • Show practical usage scenarios
  • Provide copy-paste ready code snippets

3. Context Connection

  • Explain how components fit into the larger system
  • Reference related components and workflows
  • Show integration patterns

4. Developer Focus

  • Document what developers need to know to use the code
  • Focus on practical usage over implementation details
  • Provide troubleshooting information

Best Practices

For Developers

When Writing Code

  1. Use clear naming conventions that match documentation
  2. Include TypeScript interfaces for better analysis
  3. Write descriptive docstrings for Python functions
  4. Create stub files for new features immediately

When Modifying Existing Code

  1. Check if documentation exists for the component
  2. Update related documentation when changing APIs
  3. Add markers (OUTDATED:, IMPROVEMENT:) to flag needed updates
  4. Review generated suggestions in PR comments

When Creating Documentation

  1. Use the appropriate docs/ subdirectory
  2. Follow the CAPS_WITH_UNDERSCORES.md naming convention
  3. Include quality markers to help the evolution workflow
  4. Reference related code files and components

For Documentation Maintainers

Monitor Workflow Activity

  • Check auto-generated issues regularly
  • Review PR comments for documentation suggestions
  • Use the daily quality reports to prioritize improvements

Manage Documentation Quality

  • Close resolved auto-documentation issues
  • Update outdated documentation marked by the workflow
  • Consolidate redundant documentation identified by analysis

Evolve the System

  • Adjust priority markers based on project needs
  • Update workflow configurations as the project grows
  • Add new quality markers for emerging documentation patterns

Troubleshooting

Common Issues

Workflow Not Triggering

  • Check that scripts have execute permissions
  • Verify GitHub token permissions for issue creation
  • Ensure workflow files are in the correct location

False Positive Detections

  • Review the detection scripts for edge cases
  • Update filtering patterns in the analysis scripts
  • Adjust quality markers to reduce noise

Missing Documentation Suggestions

  • Check that changed files are being detected properly
  • Verify that the analysis scripts can parse the code
  • Ensure documentation naming conventions are followed

Debugging

Enable Verbose Logging

The workflows include debugging output that shows:

  • Which files are being analyzed
  • What patterns are being detected
  • Why specific suggestions are generated

Manual Workflow Triggers

Both workflows can be triggered manually for testing:

  • Go to Actions tab in GitHub
  • Select the workflow
  • Use “Run workflow” with custom parameters

Future Enhancements

Planned Features

  • Integration with code review tools
  • Automated documentation testing
  • AI-powered content suggestions
  • Documentation coverage metrics
  • Integration with external documentation tools

Contributing

To improve the documentation workflows:

  1. Test changes on a fork first
  2. Update related scripts in the scripts/ directory
  3. Add tests for new detection patterns
  4. Update this documentation with new features

Core Documentation System

Development Integration

Workflow Scripts and Implementation

Quality Assurance

Component Documentation Examples