Query Builder Field Validation

This document describes the field validation capabilities added to the QueryBuilder module.

Overview

The QueryBuilder now includes comprehensive field validation that automatically checks if all Europe PMC API fields are properly defined in the FieldType literal. This ensures that the query builder stays up-to-date with the latest API changes.

Features

1. Complete Field Coverage

The FieldType literal now includes all 142 fields from the Europe PMC API (as of 2025-11-05), plus:

2. API Field Fetching

from pyeuropepmc.query_builder import get_available_fields

# Fetch current fields from Europe PMC API
fields = get_available_fields()
print(f"Available fields: {len(fields)}")

3. Automatic Validation

from pyeuropepmc.query_builder import validate_field_coverage

# Check if all API fields are covered
result = validate_field_coverage(verbose=True)

if result['up_to_date']:
    print("✅ All API fields are covered!")
else:
    print(f"❌ Missing {len(result['missing_in_code'])} fields")

4. Command-Line Tool

# Check field coverage from command line
python scripts/check_fields.py

Field Categories

Core Bibliographic Fields

Date Fields

Author & Affiliation

Scientific Metadata

Funding

Full Text Availability

Database Cross-References

Citations

Field Aliases

Europe PMC supports both full and abbreviated field names:

Full Name Abbreviated Usage
author auth Author names
affiliation aff Author affiliations
language lang Publication language
keyword kw Keywords
chemical chem Chemical substances
source src Data source (MED, PMC, etc.)
editor ed Book editors

Both forms are accepted by the API and included in FieldType.

Examples

Basic Usage

from pyeuropepmc import QueryBuilder

# Use full field names
qb = QueryBuilder()
query = qb.keyword("Smith", field="author").build()
# Result: Smith:AUTHOR

# Or use abbreviations
qb = QueryBuilder()
query = qb.keyword("Smith", field="auth").build()
# Result: Smith:AUTH

Validation in CI/CD

# Add to your CI pipeline to ensure fields stay up-to-date
python scripts/check_fields.py || exit 1

Checking for Updates

from pyeuropepmc.query_builder import validate_field_coverage

result = validate_field_coverage(verbose=False)

if not result['up_to_date']:
    print("⚠️  Field definitions need updating!")
    print(f"Missing fields: {result['missing_in_code']}")

Maintenance

The field list is validated against the Europe PMC API:

Updating Fields

When the Europe PMC API adds new fields:

  1. Run validation: 
    python scripts/check_fields.py

  2. If new fields are found, add them to FieldType in src/pyeuropepmc/query_builder.py

  3. Update the “Last Updated” comment in the code

  4. Re-run tests and validation

API Reference

get_available_fields(api_url=None) -> list[str]

Fetches the current list of searchable fields from Europe PMC API.

Returns: List of field names (lowercase, sorted)

Raises:

validate_field_coverage(verbose=False) -> dict

Validates that FieldType includes all API fields.

Returns: Dictionary with:

Parameters:

Notes

Documented but Not in API Response

Some fields are documented in the Europe PMC reference guide but don’t appear in the /fields API endpoint:

These are intentionally kept in FieldType as they’re documented and functional.

System Fields

Some fields are for internal use and rarely needed in queries:

These are included for completeness but typically not used in user queries.

See Also