API Integration Tests

This document describes the integration tests for the backend API. These tests are designed to run after deployment to verify that the API is functioning correctly in the deployed environment.

Overview

The integration tests are divided into two categories:

  1. Basic Integration Tests - Tests that verify the basic functionality of the API without requiring authentication.
  2. Authenticated Tests - Tests that verify functionality that requires authentication.

Both types of tests are designed to be run against a deployed instance of the API, either in a development, test, or production environment.

Running the Tests

Environment Configuration

The tests can be configured with the following environment variables:

  • API_BASE_URL - The base URL of the API to test. If not specified, it will be derived using PROJECT_NUMBER and REGION.
  • PROJECT_NUMBER - The GCP project number for generating the base URL.
  • REGION - The GCP region for generating the base URL (default: “europe-west1”).
  • TEST_ASSISTANT_ID - The ID of a test assistant to use for testing (default: echo assistant ID).
  • TEST_EMAIL - The email to use for non-authenticated test requests.
  • AUTH_TEST_EMAIL - The email to use for authenticated test requests.
  • AUTH_TEST_TOKEN - The authentication token to use for authenticated test requests.
  • PRIVATE_ASSISTANT_ID - The ID of a private assistant that requires authentication (for testing access control).
  • SKIP_INTEGRATION_TESTS - Set to “true” to skip the non-authenticated integration tests.
  • SKIP_AUTH_TESTS - Set to “true” to skip the authenticated tests.

Running Locally

To run the tests locally against a deployed API:

# Set the required environment variables
export API_BASE_URL="https://backend-api-{project_number}.europe-west1.run.app"
export TEST_ASSISTANT_ID="360762b1-5f00-4b3c-847c-83959e36aad7"  # Echo assistant

# Run the basic integration tests
python -m pytest tests/test_api_integration.py -v

# For authenticated tests, set the required credentials
export AUTH_TEST_EMAIL="your-email@example.com"
export AUTH_TEST_TOKEN="your-auth-token"
export PRIVATE_ASSISTANT_ID="private-assistant-id"

# Run the authenticated tests
python -m pytest tests/test_api_authenticated.py -v

Running in CI/CD

The integration tests are integrated into the cloudbuild.yaml file and run automatically after deployment. The CI/CD pipeline sets the appropriate environment variables based on the deployment environment.

Test Coverage

Basic Integration Tests (test_api_integration.py)

These tests cover:

  • Health endpoint functionality
  • Basic assistant endpoint functionality
  • Request validation
  • Error handling for invalid requests
  • Retry logic for service initialization
  • Basic streaming endpoint functionality

Authenticated Tests (test_api_authenticated.py)

These tests cover:

  • Authenticated requests to the assistant endpoint
  • Access control for endpoints that require authentication
  • Tool permissions enforcement

Adding New Tests

When adding new integration tests, follow these guidelines:

  1. Use the existing helpers - Utilize the existing helper functions for creating headers and payloads.
  2. Handle retries - Include retry logic for tests that might fail temporarily due to service initialization.
  3. Skip tests gracefully - Use pytest.skip() to skip tests when the necessary environment isn’t available.
  4. Keep tests independent - Each test should be independent and not rely on the state from previous tests.
  5. Handle authentication properly - For authenticated tests, always check if the required credentials are available.

Test Implementation Notes

Network Reliability

Since these tests run over the network against a deployed service, they include retry logic to handle transient failures. Each test attempts to connect to the API multiple times before failing.

Authentication

The authenticated tests are skipped by default unless the required credentials are provided through environment variables. This ensures that the tests can run in both authenticated and non-authenticated modes.

Skip Flags

The tests include skip flags that can be set through environment variables. This allows selectively running subsets of tests based on the environment or test requirements.

CI/CD Integration

The integration tests are run as part of the CI/CD pipeline after the API is deployed. The pipeline:

  1. Deploys the backend API
  2. Waits for the service to be available
  3. Runs the basic integration tests
  4. Runs the authenticated tests if credentials are available
  5. Reports the test results

If the tests fail, the pipeline will still complete successfully but will log the test failures for review.