[BUGFIX] Dependencies between unit tests 35/50435/5
authorChristian Kuhn <lolli@schwarzbu.ch>
Sat, 29 Oct 2016 20:42:44 +0000 (22:42 +0200)
committerWouter Wolters <typo3@wouterwolters.nl>
Sun, 30 Oct 2016 18:13:20 +0000 (19:13 +0100)
commit7d067bb2a1892980d25e0ac652dffcc516135b11
treeedbba2df809a8cc4b94171fc5ffeac8395017671
parenta135a5dde169423a3b9356c00d7f93dd941b9fa9
[BUGFIX] Dependencies between unit tests

Our unit tests have an annoying history of dependencies between
each other: There are tests that rely on global state changes from
other tests. Those are hard and time consuming to track down and
lead to erratic test results if not fixed.

The patch resolves the hopefully last occurences of those cross
dependencies, all of these issues were recently introduced:

* A bug in extbase DebuggerUtility lets a test in core DebugUtilityTest fail
* SchemaColumnDefinitionListenerTest is changed to cope with a possible
  global change by a different test that we can not prevent otherwise
* felogin FrontendLoginControllerTest properly handles registered
  global objects to prevent side effects on TemplateServiceTest

To prevent such issues in the future, the new require-dev dependency
"phpunit-randomizer" is added. It allows running tests in random order
to reveal cross dependencies:

./bin/phpunit-randomizer -c typo3/sysext/core/Build/UnitTests.xml --order rand

Each run echos out a specific seed that can be used to replay this specific
order of tests:

./bin/phpunit-randomizer -c typo3/sysext/core/Build/UnitTests.xml --order rand:1234

The randomizer will be added to the pre-merge tests with a couple of
runs to find those cross dependencies more quickly.

Change-Id: Id08eda19df98f1a8adf25259f852928020acd810
Resolves: #78499
Releases: master
Reviewed-on: https://review.typo3.org/50435
Tested-by: TYPO3com <no-reply@typo3.com>
Reviewed-by: Markus Klein <markus.klein@typo3.org>
Reviewed-by: Daniel Lorenz <daniel.lorenz@extco.de>
Tested-by: Daniel Lorenz <daniel.lorenz@extco.de>
Reviewed-by: Wouter Wolters <typo3@wouterwolters.nl>
Tested-by: Wouter Wolters <typo3@wouterwolters.nl>
composer.json
composer.lock
typo3/sysext/core/Tests/Unit/Database/Schema/EventListener/SchemaColumnDefinitionListenerTest.php
typo3/sysext/core/Tests/Unit/FormProtection/FormProtectionFactoryTest.php
typo3/sysext/core/Tests/Unit/Utility/DebugUtilityTest.php
typo3/sysext/extbase/Classes/Utility/DebuggerUtility.php
typo3/sysext/extbase/Tests/Unit/Utility/DebuggerUtilityTest.php
typo3/sysext/felogin/Tests/Unit/Controller/FrontendLoginControllerTest.php