Test Suite

This document describes the test suite structure for the QueryTranslator library, focusing on test organization by component, data-driven testing with providers, the fixture override mechanism, and PHPUnit configuration. For information about CI/CD configuration, see page 7.2.

Overview

The QueryTranslator library uses PHPUnit as its testing framework. Tests are organized by component type: Tokenizer, Parser, and Generators. The test suite employs data providers for parametric testing and inheritance-based fixture overrides to minimize code duplication.

Test Suite Organization

Sources: phpunit.xml1-20 tests/Galach/Tokenizer/FullTokenizerTest.php1-20 tests/Galach/Tokenizer/TextTokenizerTest.php1-17 tests/Galach/Tokenizer/TokenExtractorTest.php1-15

PHPUnit Configuration

The phpunit.xml file defines test suite settings, bootstrap configuration, and code coverage parameters.

Configuration Element	Value	Purpose
Bootstrap	./tests/bootstrap.php	Initialize test environment
Test Suite Name	"Query parser tests"	Test suite identifier
Test Directory	./tests	Root directory for test discovery
Test File Suffix	Test.php	File naming convention for test classes
Coverage Directory	./lib	Source code to include in coverage reports
Coverage Processing	processUncoveredFiles="true"	Include files not executed in coverage analysis

Additional settings:

• convertErrorsToExceptions="true" - Convert PHP errors to exceptions
• convertNoticesToExceptions="true" - Convert PHP notices to exceptions
• convertWarningsToExceptions="true" - Convert PHP warnings to exceptions
• Symfony deprecation helper is disabled via environment variable SYMFONY_DEPRECATIONS_HELPER

Sources: phpunit.xml1-20

Test Organization by Component

Tests are organized into directories corresponding to the main components of the query processing pipeline:

Tokenizer Tests

Located in tests/Galach/Tokenizer/, these tests validate lexical analysis:

Test Class	Component Under Test	Purpose
TokenExtractorTest	TokenExtractor abstract class	Validates error handling in token extraction
FullTokenizerTest	Tokenizer with TokenExtractor\Full	Tests full token extractor recognizing all token types
TextTokenizerTest	Tokenizer with TokenExtractor\Text	Tests text token extractor (inherits from FullTokenizerTest)

Sources: tests/Galach/Tokenizer/TokenExtractorTest.php12-75 tests/Galach/Tokenizer/FullTokenizerTest.php17-20 tests/Galach/Tokenizer/TextTokenizerTest.php12-17

Parser Tests

Located in tests/Galach/Parser/, these tests validate syntax analysis and SyntaxTree construction.

Generator Tests

Located in tests/Galach/Generators/, these tests validate code generation for different backends:

Test Class	Generator Under Test	Output Format
Native Generator Tests	Generators\Native\Generator	Galach query string
ExtendedDisMaxTest	Generators\ExtendedDisMax\Generator	Solr Extended DisMax query
QueryStringTest	Generators\QueryString\Generator	Elasticsearch QueryString query

The QueryStringTest inherits from ExtendedDisMaxTest to reuse test fixtures with backend-specific differences.

Sources: tests/Galach/Tokenizer/FullTokenizerTest.php1-20 tests/Galach/Tokenizer/TextTokenizerTest.php1-17

Data-Driven Testing with Providers

The test suite uses PHPUnit data providers extensively to enable parametric testing. Each data provider method returns an array of test cases, where each test case is an array containing input parameters and expected results.

Data Provider Pattern in FullTokenizerTest

The providerForTestTokenize() method in FullTokenizerTest returns 200+ test cases covering:

• Word tokens with various characters (alphanumeric, Unicode, escaped)
• Phrase tokens with double quotes
• Tag tokens (#tag)
• User tokens (@user)
• Domain-prefixed tokens (domain:term)
• Logical operators (AND, OR, NOT, &&, ||)
• Unary operators (+, -, !)
• Grouping delimiters ((, ))
• Whitespace handling
• Escape sequences

Each test case is structured as:

Sources: tests/Galach/Tokenizer/FullTokenizerTest.php22-1232 tests/Galach/Tokenizer/FullTokenizerTest.php1240-1250

Data Provider Example: Logical Operators

The data provider includes comprehensive test cases for operator tokenization:

Input String	Expected Tokens
'one AND two'	[WordToken('one'), WHITESPACE, LOGICAL_AND('AND'), WHITESPACE, WordToken('two')]
'one && two'	[WordToken('one'), WHITESPACE, LOGICAL_AND('&&'), WHITESPACE, WordToken('two')]
'AND'	[LOGICAL_AND('AND')]
'ANDword'	[WordToken('ANDword')]
'AND+'	[LOGICAL_AND('AND'), MANDATORY('+')]

Sources: tests/Galach/Tokenizer/FullTokenizerTest.php521-593

Fixture Override Mechanism

The TextTokenizerTest class demonstrates the fixture override mechanism, which allows a child test class to inherit test cases from a parent while overriding specific fixtures where behavior differs.

TextTokenizerTest Inheritance Pattern

Sources: tests/Galach/Tokenizer/TextTokenizerTest.php17-184 tests/Galach/Tokenizer/FullTokenizerTest.php1240-1250

Override Implementation Details

The TextTokenizerTest overrides the testTokenize() method to apply fixture overrides before delegating to the parent implementation:

1. Method Override: The testTokenize() method is overridden in the child class tests/Galach/Tokenizer/TextTokenizerTest.php30-34
2. Fixture Lookup: getExpectedFixtureWithOverride() checks if an override exists for the input string tests/Galach/Tokenizer/TextTokenizerTest.php42-51
3. Override Storage: setFixtureOverride() populates the static $fixtureOverride array on first use tests/Galach/Tokenizer/TextTokenizerTest.php53-175
4. Parent Delegation: If no override exists, the original expected tokens are used tests/Galach/Tokenizer/TextTokenizerTest.php50

Overridden Fixtures Example

The Text token extractor does not recognize tag and user tokens, treating them as regular words instead:

Input	Full Extractor Result	Text Extractor Result (Override)
'#tag'	TagToken('#tag', 0, '#', 'tag')	WordToken('#tag', 0, '', '#tag')
'@user'	UserToken('@user', 0, '@', 'user')	WordToken('@user', 0, '', '@user')
'domain:word'	WordToken('domain:word', 0, 'domain', 'word')	WordToken('domain:word', 0, '', 'domain:word')

The $fixtureOverride array contains 43 overridden test cases for inputs where the Text extractor behaves differently from the Full extractor.

Sources: tests/Galach/Tokenizer/TextTokenizerTest.php56-173

TokenExtractor Selection

The fixture override mechanism works in conjunction with the getTokenExtractor() factory method:

• FullTokenizerTest::getTokenExtractor() returns new TokenExtractor\Full() tests/Galach/Tokenizer/FullTokenizerTest.php1343-1346
• TextTokenizerTest::getTokenExtractor() returns new TokenExtractor\Text() tests/Galach/Tokenizer/TextTokenizerTest.php180-183

This allows the same test logic to execute against different TokenExtractor implementations.

Sources: tests/Galach/Tokenizer/FullTokenizerTest.php1343-1346 tests/Galach/Tokenizer/TextTokenizerTest.php180-183

TokenExtractor Error Handling Tests

The TokenExtractorTest class validates error handling in the TokenExtractor abstract class. These tests use PHPUnit's mock objects to inject invalid configurations and verify that appropriate exceptions are thrown.

Test Cases

Test Method	Scenario	Expected Exception
testExtractThrowsExceptionPCRE	Invalid PCRE regex pattern causes backtrack limit error	RuntimeException with "PCRE regex error code: 2"
testFullExtractTermTokenThrowsException	Full extractor cannot extract valid term token from matched lexeme	RuntimeException with "Could not extract term token"
testTextExtractTermTokenThrowsException	Text extractor cannot extract valid term token from matched lexeme	RuntimeException with "Could not extract term token"

The tests use reflection to modify the protected $expressionTypeMap property, simulating invalid configurations that would not occur in normal usage.

Sources: tests/Galach/Tokenizer/TokenExtractorTest.php17-74

Test Inheritance Pattern

The test suite uses inheritance to reduce code duplication. For example:

1. TextTokenizerTest extends FullTokenizerTest to inherit base test cases
2. QueryStringTest extends ExtendedDisMaxTest to reuse test providers

This allows specialized behavior to be implemented while reusing common test fixtures.

Sources: tests/Galach/Tokenizer/TextTokenizerTest.php17-18 tests/Galach/Tokenizer/TextTokenizerTest.php30-34 tests/Galach/Generators/QueryStringTest.php11-12 tests/Galach/Generators/QueryStringTest.php13-32

Best Practices

1. Data Provider Usage: Use data providers to test multiple input-output scenarios
2. Test Class Inheritance: Extend existing test classes to reuse test cases
3. Component Integration: Test how components interact in the pipeline
4. Edge Cases: Include tests for edge cases (escaped characters, special tokens)
5. Test Isolation: Ensure each test is independent and can run in isolation

Conclusion

The QueryTranslator test suite provides comprehensive validation of all components in the query processing pipeline. By understanding the test structure and patterns, developers can effectively maintain existing tests and add new ones as the system evolves.