Magento Forums
Help
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Migrating PHPUnit 6 to PHPUnit 9.1 in Magento 2.4

lbajsarowiczpl
Core Maintainer
‎06-16-2020 10:27 AM
‎06-16-2020 10:27 AM

Magento 2.4.0 comes with a major upgrade of the PHPUnit framework, which is used for all types of tests: Static, Unit, Integration, API Functional, and the Magento Functional Testing Framework. Magento Architects decided to make an upgrade to the latest PHPUnit version, which is 9.1.

Please be aware that the upgrade of PHPUnit is backwards incompatible.

 

Migration process

 

The first approach started during Magento Commerce Global Contribution Day on April 4th. Together with the community, in scope of issue #27500, we started the migration. One week after, not even a half of work was done. The clock was ticking, as the 2.4 feature freeze was coming.

I tried to encourage Magento contributors to undertake the challenge.

 

"Use Rector"

appeared quite often instead. Out of curiosity, I tried to use existing Rectors. The first issue I noticed was the fact that Rector PHP expects correct PHP syntax (including annotations) and additionally, expects classes to exist. Otherwise, code refactoring fails.

 

Migration Challenges

 

We had to contend with:

  1. Over 30.000 Unit Tests to migrate in very limited time
  2. Invalid PHP code syntax, missing or incorrect annotations
  3. Backward compatibility requirements
  4. Required to pass Static Analysis

Note: The first approach assumed that we need to migrate PHPUnit 6 to PHPUnit 8 first, as it was backwards-compatible, and then in a further release, upgrade to PHPUnit 9. Finally the Core Team said that we are allowed to migrate all the way to PHPUnit 9.

 

Step 1: Fix failing Unit Tests

 

The Magento codebase includes over 30.000 unit tests (including Magento 2 Commerce, B2B and projects such as MSI). Some of them were just flaky (eg. environment-dependent) or invalid. The first step was to fix them and make sure these tests pass on all supported PHP versions.

 

Step 2: Migration to PHPUnit 8

 

All contributions to Magento 2 must follow the Definition of Done:

  • Pass Integrity Tests which include Static Code Analysis

    Every single file modified in the Magento codebase has to follow coding standards. Most of the issues were fixed automatically by PHP Code Beautifier and PHP CS Fixer

  • Pass Unit Tests

    Migration of unit tests included not only compatibility with PHPUnit 8, but also introduction of Strict Typing. This step revealed many, many hidden issues caused by Type Juggling. For example:

    $this->assertTrue($validator->isValid($data))

    is actually a tautology, because both true and array with an error was returning a true value:

    return [
    'status' => 'error',
    'message' => __('Error message')
];

    There was also an issue with handling Phrase object with \PHPUnit\Framework\Assert::assertEquals or \PHPUnit\Framework\MockObject\InvocationMocker::with. Declaration of strict_types=1 "turns off" the Type Juggling, though ->__toString has to be called explicitly on a Phrase object, otherwise assertEquals('Expected Error Message', $phraseObject) failed.

    There are two ways to fix this: assertEquals('Expected Error Message', $phraseObject->__toString()) or the preferred method: assertEquals('Expected Error Message', (string)$phraseObject)

The most time-consuming manual job was related to line length: the limit of 120 characters could not be fixed automatically. We had to edit the exception messages manually, wrapping them into multiple lines.

No one is capable of reviewing 30,000 of files submitted in scope of single PR. The decision was made to split the changes in scope of their module. Every single Module was separately reviewed and stabilized.

At this stage, PHPUnit Warnings and Notices were allowed.

 

Step 3: Migration to PHPUnit 9

 

Contributors who decided to face the challenge were given a separate Slack channel for synchronizing with the core team, as well as separate PHP 7.4-based branch. The goals were:

  1. Unit Tests passing with PHP 7.3 and PHP 7.4
  2. Compliance with PHPUnit 9.1
  3. Drop deprecated methods and structures

At the end of that stage, PHPUnit Warnings and Notices were not allowed.

PHPUnit 9 comes with very strict rules for Mocking objects. Invalid usage of \PHPUnit\Framework\TestCase::createPartialMock for Interfaces and Abstract classes had to be replaced with getMockBuilder and then configured. The \PHPUnit\Framework\MockObject\MockBuilder::setMethods method was replaced with onlyMethods and addMethods.

What is the difference?

You need to explicitly define it if you mock an existing or non-existing method of a class or interface.

 

Step 4: Upgrade & Merge

 

Magento internal teams were working on the API functional tests and integration tests, while the Community worked on unit tests. The final step was to synchronize the results and stabilize the outcome. Having magento/2.4-develop74 stabilized with PHP 7.4, these changes were merged to the mainline magento/2.4-develop branch.

Our journey was done!

 

Tooling

 

Such a big undertaking could not be done manually. That is why, step by step, we used automated tools and scripts to make the upgrade possible. This code was not expected to be published, which is why this is not a first-class example.

 

Script: Per-module automation

 

The natural way of delivering units of work is splitting the results by Module. That is why the shell scripts were focused on these scopes:

./migration.sh {Module name} [Custom directory]

 

Command $TESTS_PATH
./migration.sh Catalog /var/www/html/app/code/Magento/Catalog/Test/Unit
./migration.sh Staging .ee /var/www/html/.ee/app/code/Staging/Test/Unit

 

Where [Custom directory] was optional and was used to handle different products:

  • Magento Open Source
  • Magento Commerce
  • Magento B2B
  • PageBuilder.
#!/bin/bash
set -e

BASE_DIR="/var/www/html/${2:-}"
TESTS_PATH="$BASE_DIR/app/code/Magento/$1/Test/Unit"

Working with different projects, we had to handle multiple Git repositories:

LocalGit () {
    git --work-tree="$BASE_DIR" --git-dir="$BASE_DIR/.git" "$@"
}

Not all modules have unit tests, so our script must verify test availability:

test -d "$TESTS_PATH" || (echo "Module $1 does not contain Unit Tests in $TESTS_PATH" && exit 1)

and if the existing tests were not failing PHPUnit 6 before our modifications:

/var/www/html/phpunit8/vendor/bin/phpunit -c dev/tests/unit/phpunit6.xml $TESTS_PATH

 

Script: Fix known issues

 

There were plenty of issues with the PHP syntax, which is easy to fix with sed:

  • Invalid arguments order for PHPDoc: $variable \Type instead of \Type $variable

    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/@var\s+(\$[^\s]+)\s+([\\|A-Za-z_]+)/@var \2 \1/g' {} \;

  • Introduce correct DOCBlock opening (replace /* with /**)

    find $TESTS_PATH -name "*.php" -exec sed 's/\/\*$/\/\*\*/g' {} \; \

  • Remove redundant use section alias

    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/use \\?(PHPUnit[^;]+) as MockObject/use \1/g' {} \;

  • Remove redundant empty lines

    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i ':a;N;$!ba;s/\n\n\n/\n\n/g' {} \;

  • Replace Phrase with string for non LocalizedException

    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/( \\[A-Za-z]+)\(__\(([^)]+)\)\)/\1(\2)/g' {} \;

  • Move declare() introduced by PHP CS Fixer below Magento Copyright

    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i ':a;N;$!ba;s/(declare\(strict_types=1\);)\n(\/\*\*[^\/]+\/)/\n\2\n\1\n/g' {} \;

 

PHP Rector

 

Once we had the PHP code ready for migration, we were finally able to use "The right tools for the job right". As recommended by the community, I decided to introduce PHPRector to the project using the Composer command composer require rector/rector --dev.

This tool offers predefined set of rectors (scenarios), just like the PHPUnit 8.0 migration set. The backwards-compatibility with PHPUnit 6 had to be kept, which is why some rectors were excluded in our rector.yml configuration file:

parameters:
auto_import_names: true
import_short_classes: false
import_doc_blocks: true
sets:
  - 'phpunit80'
exclude_rectors:
  - 'Rector\PHPUnit\Rector\MethodCall\SpecificAssertInternalTypeRector'
  - 'Rector\PHPUnit\Rector\MethodCall\SpecificAssertContainsRector'

During the second stage of migration (PHPUnit 8 to PHPUnit 9 and drop for backwards-compatibility) the rector.yml file was much more advanced:

parameters:
    auto_import_names: true
    import_short_classes: false
    import_doc_blocks: true
    sets:
        - 'phpunit70'
        - 'phpunit80'
        - 'phpunit90'
        - 'phpunit91'
        - 'phpunit-specific-method'
        - 'phpunit-mock'
services:
    M2Coach\Rector\ReplacePartialMockRector: null
    Rector\Renaming\Rector\MethodCall\RenameMethodCallRector:
        $oldToNewMethodsByClass:
            PHPUnit\Framework\Assert:
                assertRegExp: assertMatchesRegularExpression
                expectExceptionMessageRegExp: expectExceptionMessageMatches
            PHPUnit\Framework\TestCase:
                assertRegExp: assertMatchesRegularExpression
                expectExceptionMessageRegExp: expectExceptionMessageMatches
    Rector\Renaming\Rector\Class_\RenameClassRector:
        $oldToNewClasses:
            # https://github.com/sebastianbergmann/phpunit/issues/3123
            PHPUnit\Framework\MockObject\Matcher\InvokedCount: 'PHPUnit\Framework\MockObject\Rule\InvokedCount'
            PHPUnit\Framework\MockObject\Matcher\Invocation: 'PHPUnit\Framework\MockObject\Invocation'
            PHPUnit_Framework_MockObject_MockObject: 'PHPUnit\Framework\MockObject\MockObject'

And that actually did most of the work.

 

Code Style

 

Once we have the code ready for PHPUnit 9, we had to adjust the Code Style to follow Magento Coding Standard.

  • PHP CS Fixer (.php_cs)

    ->setRules([
    '@PSR2' => true,
    'array_indentation' => true,
    'array_syntax' => ['syntax' => 'short'],
    'concat_space' => ['spacing' => 'one'],
    'declare_strict_types' => true,
    'hash_to_slash_comment' => true,
    'include' => true,
    'method_chaining_indentation' => true,
    'method_argument_space' => true,
    'modernize_types_casting' => true,
    'new_with_braces' => true,
    'no_empty_statement' => true,
    'no_empty_comment' => true,
    'no_empty_phpdoc' => true,
    'no_extra_consecutive_blank_lines' => true,
    'no_leading_import_slash' => true,
    'no_leading_namespace_whitespace' => true,
    'no_multiline_whitespace_around_double_arrow' => true,
    'no_multiline_whitespace_before_semicolons' => true,
    'no_singleline_whitespace_before_semicolons' => true,
    'no_short_bool_cast' => true,
    'no_trailing_comma_in_singleline_array' => true,
    'no_unused_imports' => true,
    'no_whitespace_in_blank_line' => true,
    'object_operator_without_whitespace' => true,
    'ordered_imports' => true,
    'php_unit_set_up_tear_down_visibility' => true,
    'standardize_not_equals' => true,
    'ternary_operator_spaces' => true,
]);

  • PHP Mess Detector (default dev/tests/static/testsuite/Magento/Test/Php/_files/phpmd/ruleset.xml)

  • PHP Code Beautifier (config shared with Mess Detector)

 

Script: PHP Mess Detector fixer

 

Some of the issues reported by Mess Detector were fixed automatically with a script

#!/bin/bash
set -e

TOFIX="$(/var/www/html/vendor/bin/phpmd "$1" text /var/www/html/dev/tests/static/testsuite/Magento/Test/Php/_files/phpmd/ruleset.xml)" || echo "Found $(echo $TOFIX | wc -l)"

[[ "$TOFIX" == "" ]] && exit 0

sed -i ':a;N;$!ba;s/\nclass/\n\/\*\*\n \* \@SuppressWarnings\(PHPMD.CouplingBetweenObjects\)\n \*\/\nclass/' $(echo "$TOFIX" | grep 'number of dependencies' | grep -oE '^([^:]+)' | tr '\n' ' ') || true
sed -i ':a;N;$!ba;s/\nclass/\n\/\*\*\n \* \@SuppressWarnings\(PHPMD.AllPurposeAction\)\n \*\/\nclass/' $(echo "$TOFIX" | grep 'processed HTTP methods' | grep -oE '^([^:]+)' | tr '\n' ' ') || true
sed -i ':a;N;$!ba;s/\n \*\/\n\/\*\*//' $(echo "$TOFIX" | grep -oE '^([^:]+)' | tr '\n' ' ') || true

/var/www/html/vendor/bin/phpmd "$1" text dev/tests/static/testsuite/Magento/Test/Php/_files/phpmd/ruleset.xml || exit 1

 

Wrap up

 

The automated part of migration was wrapped into a find command that iterated through the module directories, executing Bash scripts. If the procedure failed, the information was written to a todo-manual.txt file, to review manually.

 

#!/bin/bash
set -e

LocalGit () {
    git --work-tree="$BASE_DIR" --git-dir="$BASE_DIR/.git" "$@"
}

BASE_DIR="/var/www/html/${2:-}"
TESTS_PATH="$BASE_DIR/app/code/$1/Test/Unit"

test -d "$TESTS_PATH" || (echo "Module $1 does not contains Unit Tests in $TESTS_PATH" && exit 1)

echo "=== Performing actions on $1 module ==="

(
    find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/@var\s+(\$[^\s]+)\s+([\\|A-Za-z_]+)/@var \2 \1/g' {} \; \
    && find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i ':a;N;$!ba;s/\n\n\n/\n\n/g' {} \; \
    && /var/www/html/vendor/bin/rector --config=/var/www/html/rector.yml process $TESTS_PATH \
    && find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/use \\?(PHPUnit[^;]+) as MockObject/use \1/g' {} \; \
    && find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i "s/expectException\('\\?(.*?)'\)/expectException(\\\\\1::class)/g" {} \; \
    && find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i 's/( \\[A-Za-z]+)\(__\(([^)]+)\)\)/\1(\2)/g' {} \; \
    && (/var/www/html/vendor/bin/phpcbf --standard=/var/www/html/dev/tests/static/framework/Magento/ruleset.xml $TESTS_PATH; test $? -le 1 ) \
    && /var/www/html/vendor/bin/phpcs --standard=/var/www/html/dev/tests/static/framework/Magento/ruleset.xml $TESTS_PATH \
    && /var/www/html/vendor/bin/php-cs-fixer fix --config=/var/www/html/.php_cs $TESTS_PATH \
    && ( /var/www/html/fix-mess.sh "$TESTS_PATH" || echo "Mess fixer failed") \
    && find $TESTS_PATH -name "*.php" -exec sed --regexp-extended -i ':a;N;$!ba;s/(declare\(strict_types=1\);)\n(\/\*\*[^\/]+\/)/\n\2\n\1\n/g' {} \; \
    && /var/www/html/vendor/bin/phpunit --fail-on-warning -c dev/tests/unit/phpunit9.xml $TESTS_PATH \
    && LocalGit commit -m "#27500 PHPUnit9 for $1 module" $TESTS_PATH && git push lbajsarowicz
) || ( echo "$TESTS_PATH" >> todo-manual.txt && LocalGit reset --hard)

The code was not expected to be shared, so it is not "pretty". However, this way we were able to accomplish the entire PHPUnit 6 to PHPUnit 9 migration in a few weeks, and having them compatible with Magento coding standards.

 

Extension Developers

 

As an extension developer, you can easily adjust the provided scripts to migrate existing PHPUnit tests to the latest version semi-automatically. If you followed the PHP coding standards during extension development, that should be enough to use PHP Rector with the configuration provided.

 

Thank you!

 

This project could not be accomplished without huge support of Slava Mankivski, Lena Orobei, Igor Sviziev. Community contributions could not be merged without the help of Oleksii Korshenko and Igor Miniailo.

I really appreciate the huge impact of Mediotype, who supported me the whole way! Special thanks to my competent colleagues, who shared their experience and expertise. Thank you, David Alger, for the Warden Development Environment and introducing PHP 7.4 support after my request.