Organizing your Symfony project tests
Published on 2021-12-22 • Modified on 2021-12-22
In this post, we see how to organize your Symfony project tests. We check all the available test types and create consistent and robust test suites. Let's go! ๐
» Published in "A week of Symfony 782" (20-26 December 2021).
Prerequisite
I will assume you have at least a basic knowledge of Symfony and know what automated tests are.
Configuration
- PHP 8.4
- Symfony 6.4.16
- PHPUnit 9.5.26
Introduction
When you start adding tests, sometimes you add files at the root of the tests
folder. But, as you add more and more tests, it can quickly be a mess if you continue to do so. Let's see how to do it the right way.
Goal
We will review all the tests types with an example covering each case. I'll show what I put in place in most of my projects and the resulting folder hierarchy
Test cases?
First, let's pass in revue all the differents tests types. But what Symfony says about the subject? We don't even have to read the documentation; for now, let's use the make:test
command:
bin/console make:test
We get the following output:
Which test type would you like?: [TestCase ] basic PHPUnit tests [KernelTestCase ] basic tests that have access to Symfony services [WebTestCase ] to run browser-like scenarios, but that don't execute JavaScript code [ApiTestCase ] to run API-oriented scenarios [PantherTestCase] to run e2e scenarios, using a real-browser or HTTP client and a real web server
Great! That's a good summary. Let's check each of these "TestCase".
TestCase > Unit tests (PHPUnit tests)
โA unit test is an automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behaviour of that unit of work. โ
I took this definition from this article of Royo Sherove, which sounds straightforward enough. Let's see an example from this project:
<?php
// tests/Unit/Helper/StringHelperTest.php
declare(strict_types=1);
namespace App\Tests\Unit\Helper;
use App\Helper\String\AppUnicodeString;
use App\Helper\String\StringHelper;
use PHPUnit\Framework\TestCase;
/**
* @see StringHelper
*/
final class StringHelperTest extends TestCase
{
/**
* @return iterable<int, array{0:?string, 1:string}>
*/
public function provideTestFirstCharForUnicode(): iterable
{
yield ['ABCD', 'A'];
yield ['รคBCD', 'รค'];
yield [null, ''];
yield ['', ''];
yield ['123456789', '1'];
yield ['เคจเคฎเคธเฅเคคเฅ', 'เคจ'];
yield ['ใใใชใ', 'ใ'];
yield ['๐๐ป๐ฑ๐ณ๐', '๐'];
yield ['๐ซ๐ท๐ฌ๐ง๐บ๐ธ๐ง๐ช', '๐ซ๐ท'];
yield ['ำำาพัพัฎ', 'ำ'];
}
/**
* @dataProvider provideTestFirstCharForUnicode
*
* @see StringHelper::s
*/
public function testFirstCharFromS(?string $string, string $expected): void
{
$appString = new StringHelper();
self::assertSame($expected, $appString->s($string)->firstChar());
}
/**
* @dataProvider provideTestFirstCharForUnicode
*
* @see StringHelper::u
* @see AppUnicodeString::firstChar
*/
public function testFirstCharFromU(?string $string, string $expected): void
{
$stringHelper = new StringHelper();
self::assertSame($expected, $stringHelper->u($string)->firstChar());
}
/**
* @return iterable<int, array{0:?string, 1:string}>
*/
public function provideTestFirstCharForByte(): iterable
{
yield ['ABCD', 'A'];
yield ['abcd', 'a'];
yield [null, ''];
yield ['', ''];
yield ['123456789', '1'];
}
/**
* @dataProvider provideTestFirstCharForByte
*
* @see StringHelper::b
* @see AppByteString::firstChar
*/
public function testFirstCharFromB(?string $string, string $expected): void
{
$stringHelper = new StringHelper();
self::assertSame($expected, $stringHelper->b($string)->firstChar());
}
}
This test checks the firstChar()
function of the StringHelper
class. We instantiate the class then we run assertions with different data thanks to the PHPUnit dataProvider
. I put these tests in the tests/Unit
folder. Notice that it extends the PHPUnit\Framework\TestCase
class. That means we aren't in a Symfony context; we use plain PHP classes.
Let's run it:
./vendor/bin/phpunit --filter=StringHelperTest
PHPUnit 9.5.10 by Sebastian Bergmann and contributors. Testing ......................... 25 / 25 (100%) Time: 00:00.035, Memory: 20.00 MB OK (25 tests, 25 assertions)
As we don't even use Symfony, these tests are very fast. You can use the mocking tools provided by PHPUnit (getMockBuilder()
) or use your favourite mocking frameworks like Mockery or Prophecy.
KernelTestCase > Integration tests
โIntegration testing is the phase in software testing in which individual software modules are combined and tested as a group. โ
This definition comes from Wikipedia. This time we are in a Symfony context. The name of the case is self explicit: we have access to the kernel, which means that we also have access to all services. To avoid extra configuration, all services are marked as "public" in the test environment to get them with the ContainerInterface::get()
function. Let's see an example:
<?php
declare(strict_types=1);
namespace App\Tests\Integration\Twig\Extension;
use App\Twig\Extension\EnvExtension;
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
/**
* @see EnvExtension
*/
final class EnvExtensionTest extends KernelTestCase
{
private EnvExtension $envExtension;
protected function setUp(): void
{
$this->envExtension = self::getContainer()->get(EnvExtension::class);
}
/**
* @return iterable<int, array{0: string}>
*/
public function provideGetGlobals(): iterable
{
yield ['php_minor_version'];
yield ['php_version'];
yield ['sf_major_version'];
yield ['sf_minor_version'];
yield ['sf_version'];
}
/**
* @dataProvider provideGetGlobals
*
* @see EnvExtension::getGlobals
*/
public function testGetGlobals(string $global): void
{
$globals = $this->envExtension->getGlobals();
self::assertArrayHasKey($global, $globals);
self::assertNotEmpty($globals[$global]);
}
}
The setup()
function retrieves the service we want to test and stores it as a property. Then we can use it in our test. This one test a custom Twig extension that dynamically adds some global variables for the current environment (PHP version, Symfony version...).
Let's run it:
./vendor/bin/phpunit --filter=EnvExtensionTest
PHPUnit 9.5.10 by Sebastian Bergmann and contributors. Testing . 1 / 1 (100%) Time: 00:00.267, Memory: 44.50 MB OK (1 test, 10 assertions)
We notice the test is about ten times slower than the unit test! Why? Remember, we are now using the kernel, and it has to be booted. It is automatically done when calling self::getContainer()
. The great thing is that it has only to be booted once. After, all the following scenarios are much faster.
ApiTestCase > API-oriented scenarios
These tests are a particular type of functional tests. I put them before because they are simpler than the first ones, and we generally only deal with JSON requests and responses. The ApiTestCase
file comes from API Platform; that's a good reason to install it even if you don't use it already! Here is an example:
<?php
declare(strict_types=1);
namespace App\Tests\Api\Security;
use ApiPlatform\Symfony\Bundle\Test\ApiTestCase;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Contracts\HttpClient\ResponseInterface;
use function Symfony\Component\String\u;
final class JsonLoginTest extends ApiTestCase
{
public function testLoginOK(): void
{
$response = $this->login('reader', 'test');
self::assertResponseIsSuccessful();
$arrayResponse = $response->toArray();
self::assertArrayHasKey('token', $arrayResponse);
self::assertNotTrue(u($response->toArray()['token'])->isEmpty());
}
public function testLoginNOK(): void
{
$this->login('reader', 'wrong password');
self::assertResponseStatusCodeSame(Response::HTTP_UNAUTHORIZED);
self::assertResponseHeaderSame('content-type', 'application/json');
self::assertJsonEquals([
'code' => Response::HTTP_UNAUTHORIZED,
'message' => 'Invalid credentials.',
]);
self::assertJsonContains([
'message' => 'Invalid credentials.',
]);
}
/**
* JSON Login try with a given email and password.
*/
public function login(string $username, string $password): ResponseInterface
{
return self::createClient()->request('POST', '/api/login_check', [
'json' => compact('username', 'password'),
]);
}
}
Let's run it:
./vendor/bin/phpunit --filter=JsonLoginTest
PHPUnit 9.5.10 by Sebastian Bergmann and contributors. Testing .. 2 / 2 (100%) Time: 00:01.184, Memory: 56.50 MB OK (2 tests, 6 assertions)
This time we are above a second even we only have two tests in this scenario. Why so slow? Because this time, we use actual Symfony requests and responses, and everything needs to be booted at the first test. The following are much faster. In this test, we use the assertJsonEquals
assertion, which is very practical to test the content of a response. We also have the assertJsonContains
that allows checking partial content. Imagine we have dynamic content in the response like a creation date: we can't test a given value as it would change at each fixtures loading.
WebTestCase > functional or application tests
โWikipedia: Functional testing is a quality assurance (QA) process and a type of black-box testing that bases its test cases on the specifications of the software component under test. โ
This kind of test allows testing a full feature of your application. This time, we can use the UI, fill and post forms. It also includes smoke tests, which guarantee that a page "work" means that it returns at least a 200 status code and not an error 500. Let's check an example:
<?php
declare(strict_types=1);
namespace App\Tests\Functional\Controller;
use App\Subscriber\NotFoundExceptionSubscriber;
use App\Tests\WebTestCase;
use Symfony\Component\ErrorHandler\ErrorHandler;
use Symfony\Component\HttpFoundation\Response;
final class AppControllerTest extends WebTestCase
{
/**
* @see AppController::root
*/
public function testRootAction(): void
{
$client = self::createClient();
$this->assertUrlIsRedirectedTo($client, '/', '/en');
}
/**
* @see AppController::homepage
*/
public function testHomepage(): void
{
$client = self::createClient();
$this->assertResponseIsOk($client, '/en');
$this->assertResponseIsOk($client, '/fr');
}
/**
* @see NotFoundExceptionSubscriber
*/
public function testNotFoundExceptionSubscriber(): void
{
$client = self::createClient();
$client->request('GET', '/404');
self::assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
self::assertSelectorTextContains('body', '404 found');
}
/**
* @see ErrorHandler::handleException
*/
public function test404(): void
{
$client = self::createClient();
$client->request('GET', '/not-found');
self::assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
}
}
Let's run it:
./vendor/bin/phpunit --filter=AppControllerTest
PHPUnit 9.5.10 by Sebastian Bergmann and contributors. Testing ... 3 / 3 (100%) Time: 00:00.354, Memory: 62.50 MB OK (3 tests, 5 assertions)
Once again, the kernel needs to be booted. These tests are very straightforward. We check that the root URL is redirected to the correct localized path depending on your prefered language. After, we check status codes. And finally, we test the 404 page. Why this last one? Isn't it something handled by Symfony? Yes, but one can customize the 404 page, and if there is an error in your Twig template, it will return an error 500. So, let's be sure. assertUrlIsRedirectedTo
and assertResponseIsOk
are not native assertions but shortcuts to simplify the tests.
External tests
I have regrouped some tests into a particular category, "external". These tests are also functional, but they do network access; for example, one test checks that the https://www.strangebuzz.com/index.php
URL is not accessible and redirects. The specificity of these tests is that they need to be run online, or they won't work.
PantherTestCase > end-to-end tests
We can take the definition of the maker bundle:
โEnd-to-end tests use a real-browser or HTTP client and a real webserver where we can test JavaScript. โ
This time, an actual browser like Firefox is used and run. You see it open itself and call the URL to test. I've already made an entire blog post on the subject. You can find it here. These tests are the slower ones as the browser need to be started. This is the only type of test case that allows testing JavaScript thoroughly. Here is an example:
<?php
declare(strict_types=1);
namespace App\Tests\E2E;
use Symfony\Component\Panther\PantherTestCase;
final class BlogPost138Test extends PantherTestCase
{
private const BUTTON_SELECTOR = '#subscribe_button_panther';
private const ERROR_MESSAGE_SELECTOR = '#error_msg_panther';
private const FORM_SELECTOR = '#account_create';
/**
* @debug make test filter=BlogPost138Test
*/
public function testPost138(): void
{
$client = self::createPantherClient([
'browser' => PantherTestCase::FIREFOX,
]);
$crawler = $client->request('GET', '/en/blog/end-to-end-testing-with-symfony-and-panther');
self::assertSelectorTextContains('h1', 'End-to-end testing with Symfony and Panther');
// At first load, the error message is shown and the button isn't there
self::assertSelectorExists(self::ERROR_MESSAGE_SELECTOR);
self::assertSelectorNotExists(self::BUTTON_SELECTOR);
// Fill the form so the subscribe button appears
$crawler->filter(self::FORM_SELECTOR)->form([
'account_create[login]' => 'Les',
'account_create[password]' => 'Tilleuls',
]);
$client->waitForVisibility(self::BUTTON_SELECTOR); // wait for the button to appear!
// Ok, now the button is visble and the error message should be removed from the DOM!
self::assertSelectorNotExists(self::ERROR_MESSAGE_SELECTOR);
self::assertSelectorExists(self::BUTTON_SELECTOR);
}
}
Let's run it:
./vendor/bin/phpunit --filter=BlogPost138Test
[07:32:37] coil@mac-mini.home:/Users/coil/Sites/strangebuzz.com$ ./vendor/bin/phpunit --filter=BlogPost138Test PHPUnit 9.5.10 by Sebastian Bergmann and contributors. Testing . 1 / 1 (100%) Time: 00:09.270, Memory: 107.00 MB OK (1 test, 5 assertions)
As you can see, these tests are indeed very slow: this is almost ten seconds for a single scenario. Of course, the following tests are faster once the browser is open.
That's it; we saw the different tests types. Now, let's see how to run all this. We will use tests suites.
Test suites
Here is what the tests
directory looks like now:
ll tests/
0 drwxr-xr-x 5 coil staff 160B 4 dรฉc 15:44 Api
0 drwxr-xr-x 4 coil staff 128B 15 dรฉc 08:25 E2E
0 drwxr-xr-x 9 coil staff 288B 5 dรฉc 07:44 External
0 drwxr-xr-x 4 coil staff 128B 4 dรฉc 17:44 Functional
0 drwxr-xr-x 7 coil staff 224B 4 dรฉc 17:43 Integration
0 drwxr-xr-x 6 coil staff 192B 4 dรฉc 17:35 Unit
8 -rw-r--r-- 1 coil staff 3,6K 10 mai 2021 WebTestCase.php
All the tests are in the folder corresponding to their test case, easy. Everything is already stored correctly. It allows us to execute each test case separately from the other. For example, if you work on unit tests, want only to run all the tests of this type.
Creating test suites
First, we are going to create test suites; in the phpunit.xml
file, we can apply the following configuration for the testsuites
node:
<testsuite name="unit">
<directory>tests/Unit</directory>
</testsuite>
<testsuite name="integration">
<directory>tests/Integration</directory>
</testsuite>
<testsuite name="api">
<directory>tests/Api</directory>
</testsuite>
<testsuite name="functional">
<directory>tests/Functional</directory>
</testsuite>
<testsuite name="main">
<directory>tests/</directory>
<exclude>tests/External</exclude>
<exclude>tests/E2E</exclude>
</testsuite>
<testsuite name="external">
<directory>tests/External/</directory>
</testsuite>
<testsuite name="e2e">
<directory>tests/E2E</directory>
</testsuite>
</testsuites>
<listeners>
<listener class="Symfony\Bridge\PhpUnit\SymfonyTestsListener"/>
</listeners>
<extensions>
<extension class="Symfony\Component\Panther\ServerExtension"/>
We have to specify the folder we want to use. Now, to execute the unit tests, we can run:
./vendor/bin/phpunit --testsuite=unit
This would also work:
./vendor/bin/phpunit tests/Unit
The "all" test suite isn't mandatory because we can run all tests by calling PHPUnit without arguments, but I use it in my makefile
, where I have a target that helps me filter the test I want to run. You can find my whole makefile
here. There is also a suite "main" that excludes external and e2e tests, the slower ones.
## โโ Tests โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
test: phpunit.xml check ## Run tests with optionnal suite and filter
@$(eval testsuite ?= 'all')
@$(eval filter ?= '.')
@$(PHPUNIT) --testsuite=$(testsuite) --filter=$(filter) --stop-on-failure
I use this target most of the time to run a specific file or scenario.
Summary
We can summarize the features of the tests with the following matrix:
Name | TestCase | Directory | Can be run offline? | Test JavaScript? | Test is fast? |
---|---|---|---|---|---|
Unit tests | TestCase | Unit | โ | โ | โกโก |
Integration tests | KernelTestCase | Integration | โ | โ | โก |
Api tests | ApiTestCase | Api | โ | โ | ๐ถ |
Functional tests | WebTestCase | Functional | โ | โ | ๐ถ |
End-to-end tests | PantherTestCase | E2E | โ | โ | ๐ |
External tests | WebTestCase | External | โ | โ | ๐ |
Test is fast: โกโก = very fast, โก = fast,๐ถ = OK, ๐ = slow
Now that our tests are well organized, we can also run them in parallel. That's not the subject of this blog post but click on the "more on the web" button (at the end of this article) to read the "Improve Symfony Tests Performance" article by Maks Rafalko; you will find many good tips to optimize your test suite. Spoiler alert: Depending on your application, you can speed your test suite up to a 10x ratio!
One important thing to notice is that within a given folder, let's say Functional
; the sub-directories takes the exact structure we have in src/
. For example, the AppContolerTest
file is located here tests/Functional/Controller/AppControllerTest.php
, and it tests src/Controller/AppController.php
. This way, it's easy to check what is tested or not (of course it doesn't replace a full code coverage report).
We have the following final directory structure:
tests/
โโโ Api
โย ย โโโ Controller
โย ย โย ย โโโ ApiControllerTest.php
โย ย โโโ Entity
โย ย โย ย โโโ ArticleTest.php
โย ย โย ย โโโ BookTest.php
โย ย โโโ Security
โย ย โโโ JsonLoginTest.php
โโโ E2E
โย ย โโโ BlogPost138Test.php
โย ย โโโ BlogPost138ZenstruckTest.php
โโโ External
โย ย โโโ Controller
โย ย โย ย โโโ FrontControllerTest.php
โย ย โย ย โโโ Post
โย ย โย ย โย ย โโโ Post26TraitTest.php
โย ย โย ย โโโ Snippet
โย ย โย ย โย ย โโโ Snippet99Test.php
โย ย โย ย โย ย โโโ SnippetsControllerTest.php
โย ย โย ย โโโ ToolsControllerTest.php
โย ย โโโ FeedburnerTest.php
โย ย โโโ SslTest.php
โโโ Functional
โย ย โโโ Controller
โย ย โย ย โโโ Admin
โย ย โย ย โย ย โโโ DashboardControllerTest.php
โย ย โย ย โโโ AppControllerTest.php
โย ย โย ย โโโ BlogControllerTest.php
โย ย โย ย โโโ GameControllerTest.php
โย ย โย ย โโโ LegacyControllerTest.php
โย ย โย ย โโโ Post
โย ย โย ย โย ย โโโ Post13TraitTest.php
โย ย โย ย โย ย โโโ Post59TraitTest.php
โย ย โย ย โโโ RedirectControllerTest.php
โย ย โย ย โโโ SearchControllerTest.php
โย ย โย ย โโโ SearchPart1ControllerTest.php
โย ย โย ย โโโ SearchPart2ControllerTest.php
โย ย โย ย โโโ SitemapControllerTest.php
โย ย โย ย โโโ SnippetsControllerTest.php
โย ย โย ย โโโ StaticControllerTest.php
โย ย โย ย โโโ SuggestControllerTest.php
โย ย โย ย โโโ TagControllerTest.php
โย ย โย ย โโโ ToolsControllerTest.php
โย ย โโโ Entity
โย ย โโโ ArticleTypeTest.php
โโโ Integration
โย ย โโโ Command
โย ย โย ย โโโ ShowVersionCommandTest.php
โย ย โโโ Controller
โย ย โย ย โโโ Snippets
โย ย โย ย โโโ Snippet100Test.php
โย ย โย ย โโโ Snippet105Test.php
โย ย โย ย โโโ Snippet107Test.php
โย ย โย ย โโโ Snippet108Test.php
โย ย โย ย โโโ Snippet114Test.php
โย ย โย ย โโโ Snippet115Test.php
โย ย โย ย โโโ Snippet116Test.php
โย ย โย ย โโโ Snippet12Test.php
โย ย โย ย โโโ Snippet131Test.php
โย ย โย ย โโโ Snippet132Test.php
โย ย โย ย โโโ Snippet142Test.php
โย ย โย ย โโโ Snippet147Test.php
โย ย โย ย โโโ Snippet14Test.php
โย ย โย ย โโโ Snippet157Test.php
โย ย โย ย โโโ Snippet158Test.php
โย ย โย ย โโโ Snippet160Test.php
โย ย โย ย โโโ Snippet168Test.php
โย ย โย ย โโโ Snippet173Test.php
โย ย โย ย โโโ Snippet176Test.php
โย ย โย ย โโโ Snippet177Test.php
โย ย โย ย โโโ Snippet20Test.php
โย ย โย ย โโโ Snippet2Test.php
โย ย โย ย โโโ Snippet30Test.php
โย ย โย ย โโโ Snippet32Test.php
โย ย โย ย โโโ Snippet33Test.php
โย ย โย ย โโโ Snippet42Test.php
โย ย โย ย โโโ Snippet49Test.php
โย ย โย ย โโโ Snippet50Test.php
โย ย โย ย โโโ Snippet58Test.php
โย ย โย ย โโโ Snippet61Test.php
โย ย โย ย โโโ Snippet6Test.php
โย ย โย ย โโโ Snippet70Test.php
โย ย โย ย โโโ Snippet71Test.php
โย ย โย ย โโโ Snippet74Test.php
โย ย โย ย โโโ Snippet76Test.php
โย ย โย ย โโโ Snippet7Test.php
โย ย โย ย โโโ Snippet8Test.php
โย ย โโโ Repository
โย ย โย ย โโโ BaseRepositoryTraitTest.php
โย ย โโโ Twig
โย ย โย ย โโโ Extension
โย ย โย ย โย ย โโโ EnvExtensionTest.php
โย ย โย ย โย ย โโโ SeoExtensionTest.php
โย ย โย ย โย ย โโโ TypeExtensionTest.php
โย ย โย ย โย ย โโโ UrlExtensionTest.php
โย ย โย ย โโโ Snippet152Test.php
โโโ Unit
โย ย โโโ Helper
โย ย โย ย โโโ StringHelperTest.php
โย ย โโโ Log
โย ย โย ย โโโ Processor
โย ย โย ย โโโ EnvProcessorTest.php
โย ย โโโ Tools
โย ย โย ย โโโ FilesystemTest.php
โย ย โโโ Utility
โย ย โโโ SpamCheckerTest.php
โโโ WebTestCase.php
28 directories, 81 files
Conclusion
Tests are essentials. You can always write dirty code and take shortcuts if you have a good test suite because it allows you to improve and refactor your code with confidence without fearing breaking everything. So, your test suite is probably the most critical part of your application. Make it shine, optimize it, spoil it , make it stable (and boring). That's the guarantee of an application growing healthily.
That's it! I hope you like it. Check out the links below to have additional information related to the post. As always, feedback, likes and retweets are welcome. (see the box below) See you! COil. ๐
Call to action
Did you like this post? You can help me back in several ways: (use the "reply" link on the right to comment or to contact me )
- Report any error/typo.
- Report something that could be improved.
- Like and repost!
- Follow me on Bluesky ๐ฆ
- Subscribe to the RSS feed.
- Click on the More on Stackoverflow buttons to make me win "Announcer" badges ๐ .
Thank you for reading! And see you soon on Strangebuzz! ๐
