I have always struggled with how packages are versioned and published to the npm registry.
My main concern is that you probably want to also add a git tag to your source code when you are releasing a new version, but npm packages have their own versioning system, and include a &quot;version&quot; field in the package.json
Because of this, most of the workflows I have seen, usually involve:
<ol>
<li>Running <code>npm version ...</code>. This will update the &quot;version&quot; field in the package.json file.</li>
<li>Publishing the package to the npm registry, using the version set in previous step.</li>
<li>Committing to git, so that we track the change in package.json file.</li>
<li>Adding a git tag.</li>
<li>Pushing to git remote to track new commit and new tag.</li>
</ol>
<blockquote>
Steps 3 and 4 are frequently done automatically as part of <code>npm version</code>.
</blockquote>
For me this flow has two main problems:
<ul>
<li>
My personal preference is that everything is driven by git tags, making them the single source of truth.
This would allow to just have one manual step -&gt; <code>git tag -a v1.0.0 -m &quot;...&quot; &amp;&amp; git push origin --tags</code>. Then, this would trigger a number of potential automations, including publishing to the npm registry, but also others like creating a release with the appropriate release notes, building release artifacts, etc.
</li>
<li>
The &quot;version&quot; field in the package.json file can very easily become outdated, if you git-tag by mistake before publishing the package, or if pushing the changes fails after <code>npm version ...</code> has been run.
I&#x27;ve seen this happening more often than it seems.
</li>
</ul>
Because of this, my current take when publishing npm packages for my personal projects is:
<ol>
<li>
Get rid of the &quot;version&quot; field from the package.json file. It&#x27;s misleading in source code, as it can very easily be out of date.
The only case where it&#x27;s useful is when the package has been installed and is inside node_modules, as other packages will assume this field to exist and have the right value on it.
In this case it&#x27;s always going to be correct due to the<code>npm version ...</code> + <code>npm publish</code> tandem (see next point).
</li>
<li>
Creating the git tag is the only &quot;manual step&quot;. This triggers an automation running <code>npm version ${GIT_TAG} --git-tag-version=false &amp;&amp; npm publish</code> (this is simplified. You probably also need to build your project and other potential extra steps).
The version to publish is inferred from the git tag (<code>${GIT_TAG}</code> is just a placeholder), and only at this stage we will set the &quot;version&quot; inside package.json, just before the package is published.
</li>
<li>
Other automations that need to happen during releases, can be also triggered now, like publishing release notes, building artifacts, etc.
</li>
</ol>
<blockquote>
As an example, you can see <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/github-actions/blob/24de7a2b89a9eb11b0acfd184de7187698b051b6/.github/workflows/js-lib-publish.yml">this GitHub actions workflow</a>, which follows this approach.
</blockquote>

I have always struggled with how packages are versioned and published to the npm registry. My main concern is that you probably want to also add a git tag to your source code when you are releasing a new version, but npm packages have their own versioning system, and include a "version" field in…

My current take when publishing packages to the npm registry

A couple of weeks ago I migrated a React project using redux to <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/">Redux Toolkit</a>.
Redux has always been my preferred state management tool for React (which doesn&#x27;t mean I &quot;love&quot; it), but one of its main problems is the amount of boilerplate code it requires.
Because of that I had a couple of helper functions, but there was still some things not properly handled, like proper type inference on reducers and actions (I use TypeScript).
A bit by accident (I think it was on Twitter), I found out about Redux Toolkit (RTK from now on), an official library from the redux team, which provides some opinionated helpers to reduce the amount of boilerplate, and I decided to give it a chance.
These are my impressions.
<h3>TL;DR</h3>
I first started to use it in a proof of concept, and I quickly loved it.
RTK does not &quot;replace&quot; any of the redux principles, so you can progressively adopt it, by replacing small bits of code. My plan was to use it just for some reducers, but I liked it so much that I ended up migrating the whole project in a series of consecutive pull requests.
<h3>A bit of context</h3>
Before the migration, I used to follow the <a target="_blank" rel="noopener noreferrer" href="https://github.com/erikras/ducks-modular-redux">ducks modular pattern</a>, which promotes putting all your action types, action creators and the reducer handling those actions on the same module.
This fits pretty well with RTK, as in most of the cases you will use <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/api/createSlice">slices</a>, and end up exporting the reducer and action creators returned by it, from the same module.
If you use <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/api/createAsyncThunk">async thunks</a>, they usually need to be used by that same slice, so it&#x27;s not a bad idea to export them also from there.
<h3>Why I love it</h3>
There are a couple of reasons for which I loved RTK right away. The main one is that they make an impressive work on making type inference in TypeScript work properly.
<ul>
<li>Slices detect the type of the initial state in order to know what is expected to be passed as <code>state</code> to reducers.</li>
<li>You can infer the type of your store from what is generated by <code>configureStore</code> or <code>combineReducers</code> functions, allowing you to also properly type the <code>dispatch</code> function.</li>
<li>Action creators are generated with the proper param types based on the reducers provided to <code>createSlice</code> or the callback provided to <code>createAsyncThunk</code>.</li>
<li><code>createSlice</code> can consume all the action creators generated by <code>createAsyncThunk</code>, and infer the proper payload for each one of them.</li>
</ul>
Thanks to this, the code is super predictable and potential bugs can be caught sooner.
<blockquote>
More details on how to use RTK with typescript https://redux-toolkit.js.org/usage/usage-with-typescript
</blockquote>
But this is not the only benefit. Other reasons that make it very nice to work with are:
<ul>
<li>It keeps the same concepts from Redux, but removing the boilerplate or your own helper functions if you have them (yes, I didn&#x27;t want to keep all those <code>switch</code> statements everywhere).</li>
<li>It is mostly transparent for the components dispatching the actions or consuming the state.</li>
<li>All the tests covering your reducers/actions/components should keep passing, as the outcome from RTK is the same as if you built everything manually.</li>
<li>It introduces a couple of extra helpers, like <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/rtk-query/overview">RTK query</a> or the <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/api/createListenerMiddleware">listener middleware</a>, which is very useful to dispatch actions as a result of other actions being dispatched.</li>
</ul>
<h3>Challenges I found</h3>
Of course, adopting a new tool always comes with some challenges.
The main &quot;problem&quot; I found is that RTK is a bit opinionated, but that&#x27;s precisely how they manage to reduce boilerplate code, and it&#x27;s also &quot;advertised&quot; in their website, so not a big surprise.
I usually prefer to be in control of everything (configuration over convention), but that&#x27;s just a personal preference, and sometimes it&#x27;s ok to accept a bit of magic for the sake of simplicity.
This caused the next challenges for my particular case:
<ul>
<li>
RTK assumes your project follows a strict <a target="_blank" rel="noopener noreferrer" href="https://redux.js.org/tutorials/fundamentals/part-7-standard-patterns#flux-standard-actions">Flux pattern</a>, meaning your actions always have the properties <code>type</code>, <code>payload</code> and optionally <code>meta</code> and <code>error</code>.
I was not following this pattern (my actions had the payload &quot;mixed&quot; on the first object level), so I had to adapt all my actions first, fix the tests, then migrate the reducer to RTK and make sure the tests kept passing.
<div></div>
</li>
<li>
Immutability is no longer promoted in reducers. Instead, the library requires <a target="_blank" rel="noopener noreferrer" href="https://immerjs.github.io/immer/">immer</a> internally, and using proxies they automatically ensure immutability, even if you &quot;mutate&quot; the state.
This feels a bit counterintuitive at first, because redux always promoted an immutable approach, and the use of immer is not obvious.
However, they have <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/usage/immer-reducers#why-immer-is-built-in">good reasons</a> to do it, and if your reducers return a state object, you can still use the immutable approach. It&#x27;s up to you.
<div></div>
</li>
<li>
Action creators resulting of <code>createAsyncThunk</code> can only have one argument.
This is because the async thunk callback receives a second argument containing the <code>dispatch</code> and <code>getState</code> functions, as well as a couple of other goodies, enforcing this convention to avoid human errors.
<div></div>
</li>
<li>
You cannot dispatch other actions after the callback passed to <code>createAsyncThunk</code>, because it has to return the payload and RTK dispatches the action for you.
However, this is probably something you usually should not be doing, and in my case, I resolved it with the <a target="_blank" rel="noopener noreferrer" href="https://redux-toolkit.js.org/api/createListenerMiddleware">listener middleware</a>, listening for the second action in order to dispatch the second one.
</li>
</ul>
<h3>Conclusion</h3>
My conclusion is that RTK is definitely worth it if you use redux.
It makes code simpler, more predictable and better typed. However, it still requires knowing how redux works internally and what are its patterns, in order to understand why the library exposes the helpers it exposes and why.
For existing redux projects, you can migrate progressively, addressing the challenges bit by bit and not all at once, which simplifies its adoption.
For new projects in which you want to use redux, I would recommend starting right away with RTK.

A couple of weeks ago I migrated a React project using redux to Redux Toolkit. Redux has always been my preferred state management tool for React (which doesn't mean I "love" it), but one of its main problems is the amount of boilerplate code it requires.

Because of that I had a couple of helper…

My experience migrating to Redux Toolkit (RTK)

In 2019, GitHub published their own solution to run automated workflows called GitHub Actions, which allowed those hosting their code in GitHub, to be able to define and run their CI/CD pipelines in the same platform. When it was released, one of the main pain points to use it was that defining…

How to reduce duplication in your GitHub Actions workflows

Recently ReactJS v18 was released. I started to look into its improvements, and checked which of my projects could benefit from them.
The main react-based project I maintain at the moment of writing this article is <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlink-web-client</a>, so I naturally created a new branch and started the process.
The update was quite smooth, as most of the other dependencies I was using from the react ecosystem were already updated to support v18. All but one, <a target="_blank" rel="noopener noreferrer" href="https://enzymejs.github.io/enzyme/">enzyme</a>, the testing framework I was using to unit-test react components.
To be precise, I was able to continue running my tests, but some of them (those using <code>mount</code> instead of <code>shallow</code>) were still rendering components with the &quot;old&quot; React v17 approach, causing them to behave as in that version and a warning being printed in the console.
<h3>First fix attempt</h3>
The way enzyme integrates with the react version you are using is through an adapter library. The last official one supports only React v16, but someone released an <a target="_blank" rel="noopener noreferrer" href="https://github.com/wojtekmaj/enzyme-adapter-react-17">unofficial adapter for v17</a> which, with close to 650k weekly downloads, quickly became the way to go.
I checked if there was one for React 18, but sadly there wasn&#x27;t.
Also, the author of the adapter for v17 posted an article indicating <a target="_blank" rel="noopener noreferrer" href="https://dev.to/wojtekmaj/enzyme-is-dead-now-what-ekl">enzyme is dead</a>, he was not going to release one for v18, and publishing one for v17 was a mistake.
With those facts in mind, I started to face what was probably going to be the moment to migrate to <a target="_blank" rel="noopener noreferrer" href="https://testing-library.com/docs/react-testing-library/intro">react testing library</a>.
<h3>Migration strategy</h3>
I decided to follow a migration strategy similar to the one suggested in the article above, but with a couple of modifications:
<ul>
<li>I update to React 18 anyway, as tests still pass.</li>
<li>Migrate right away all tests using <code>mount</code>, as those are the ones behaving differently.</li>
<li>New tests should be written directly with react testing library (RTL from now on).</li>
<li>When an existing test needs to be changed, consider migrating it to RTL, and do it if there&#x27;s time.</li>
<li>Introduce some PR from time to time just migrating a handful of tests.</li>
<li>Once everything is migrated, get rid of everything related with enzyme.</li>
</ul>
<h3>First contact and impressions</h3>
I was a bit sceptical with RTL, as it was focused more on integration tests, instead of unit tests, which meant I had to completely change the mindset to write tests.
It also didn&#x27;t support shallow rendering, meaning that you will end up sometimes rendering a bigger component tree when testing container-like components, potentially causing side effects that could affect your test.
On the other hand there&#x27;s more and more people stating that <a target="_blank" rel="noopener noreferrer" href="https://kentcdodds.com/blog/why-i-never-use-shallow-rendering">shallow rendering is an anti-pattern</a>, and that&#x27;s probably the reason of enzyme getting &quot;discontinued&quot;.
Also, RTL has become the <a target="_blank" rel="noopener noreferrer" href="https://reactjs.org/docs/testing.html#tools">officially recommended framework</a> to test react components, it has become very popular, and it has even joined a wider group of testing libraries under the <a target="_blank" rel="noopener noreferrer" href="https://testing-library.com/"><code>@testing-library</code></a> organization.
The thing is that I started to use it, and I was quickly surprised with how it felt to work with it, and all the benefits it was bringing to my tests.
<h3>Benefits I have experienced</h3>
Once I started migrating the first tests I noticed some benefits:
<ul>
<li>Changing my mindset was actually easier than I thought. The docs are very well written, so it&#x27;s relatively easy to start using RTL.</li>
<li>My tests got simpler, and required less of those nasty global mocks that jest allows (I&#x27;m looking at you, <a target="_blank" rel="noopener noreferrer" href="https://jestjs.io/es-ES/docs/mock-functions#mocking-modules"><code>jest.mock</code></a>).</li>
<li>Tests were way less coupled with implementation details. I moved from testing the component props, to test what the user would see on the screen.</li>
<li>Related with previous point, I started to write my tests by looking at the screen instead of the component&#x27;s implementation.</li>
<li>Changes in source code led to fewer tests requiring to be changed.</li>
<li>A progressive migration was possible, where some tests still use enzyme and others are already using RTL.</li>
</ul>
And overall, it felt good and I have already dedicated a couple of PRs just to migrate more tests.
<h3>Challenges I faced</h3>
But of course, migrating to a new tool always comes with its own challenges, and this was not an exception. I&#x27;ll explain the main ones I have faced so far and how I tackled them:
<ul>
<li>
RTL lets you do some things in a couple of different ways, and the docs may not be super clear about what&#x27;s the recommended option.
For this I recommend reading <a target="_blank" rel="noopener noreferrer" href="https://acel.me/DCMb1">this article</a> from Kent C. Dodds, RTL&#x27;s author, which explains some common mistakes when using the library.
</li>
<li>
At first, I struggled a bit with asynchronous side effects and state changes in components, which can easily end up printing a warning like <code>Warning: An update to MyComponent inside a test was not wrapped in act</code>.
The warning also explains how you are supposed to solve this by wrapping your code in <code>act()</code>, but this warning comes from React, not RTL, and what it is not telling you is that RTL already wraps all needed operations in <code>act()</code>.
What this error usually mean is that you are not waiting for something to happen. <a target="_blank" rel="noopener noreferrer" href="https://acel.me/jnUdT">This other article</a>, also from Kent C. Dodds, explains everything you need to know about the things that can lead to this warning, and how to solve them.
</li>
<li>
Since you start testing from the user point of view, some use cases can be challenging to test without coupling with implementation details. For example, when you have a component which dynamically renders different images or icons, I used to directly check the component properties.
For this I found the best solution was to use <a target="_blank" rel="noopener noreferrer" href="https://jestjs.io/docs/snapshot-testing">jest snapshots</a>. I didn&#x27;t want to manually check which SVG was being rendered in the DOM, just that different ones were being used in every case.
<pre class="language-tsx"><code class="language-tsx">// I went from this...
test(&#x27;...&#x27;, () =&gt; {
 const wrapper = shallow(&lt;MyComp dynamicValue=&quot;foo&quot; /&gt;);
 expect(wrapper.find(FontAwesomeIcon).prop(&#x27;icon&#x27;)).toEqual(someIcon);
})

// To this
test(&#x27;...&#x27;, () =&gt; {
 const { container } = render(&lt;MyComp dynamicValue=&quot;foo&quot; /&gt;);
 expect(container.firstChild).toMatchSnapshot();
})
</code></pre>
</li>
<li>
Something very similar happens if you use some library that renders canvas. There&#x27;s no way to test that from a DOM point of view, which is what RTL does.
In my case I was using <a target="_blank" rel="noopener noreferrer" href="https://www.chartjs.org/">Chart.js</a> to render some charts, so I used <a target="_blank" rel="noopener noreferrer" href="https://github.com/hustcc/jest-canvas-mock">jest-canvas-mock</a>, which exposes a method on the canvas to see which are all the events invoked from the library.
Then I used snapshots again to verify they had the expected &quot;shape&quot;.
<pre class="language-tsx"><code class="language-tsx">// I went from this...
test(&#x27;...&#x27;, () =&gt; {
 const wrapper = shallow(&lt;MyCompWithCanvas /&gt;);
 expect(wrapper.find(Chart).prop(&#x27;data&#x27;).datasets).toEqual(...);
})

// To this
test(&#x27;...&#x27;, () =&gt; {
 const { container } = render(&lt;MyCompWithCanvas /&gt;);
 const events = container.querySelector(&#x27;canvas&#x27;)?.getContext(&#x27;2d&#x27;)?.__getEvents();

 expect(events).toBeTruthy();
 expect(events).toMatchSnapshot();
})
</code></pre>
</li>
</ul>
<h3>Conclusion</h3>
For better or worst, enzyme should no longer be considered an option. If you are starting a new project or are using it on small ones, consider migrating to RTL as soon as possible.
If you are using it in bigger projects, following the process described in this article will probably help you.
In any case, you will get surprised. React testing library is a great tool and provides many benefits over enzyme.
Its main issue is that it&#x27;s not a drop-in replacement for enzyme, so you will have to change a bit your mindset.

Recently ReactJS v18 was released. I started to look into its improvements, and checked which of my projects could benefit from them. The main react-based project I maintain at the moment of writing this article is shlink-web-client, so I naturally created a new branch and started the process.

The…

My experience migrating from enzyme to react testing library

Capturing code coverage for a test suite is a very useful way to know which parts of your source code are actually getting executed by tests. This is useful not only to know if you need to add more tests to your project (in case the coverage is too low), but also, if you want to apply technics like…

Capturing remote code coverage in API tests with PHPUnit

For a long time, I have been trying to include tests in every project in which I&#x27;ve worked on.
There are several types of automated tests (or what should actually be called <a target="_blank" rel="noopener noreferrer" href="https://abstracta.us/insights/guide-continuous-testing/automated-checks">automated checks</a>). From unit tests, integration and functional tests, to end-to-end tests.
Each one of them differs from the rest by the scope they try to cover. From small units of code which are tested completely detached from external dependencies, to a whole app which is tested as a black box like if an end consumer was using it.
I&#x27;m not going to deeply discuss the semantics behind this, or where an integration test ends and an end-to-end test starts, because it&#x27;s quite frequently a blurry line.
I want to focus on unit tests, and a frequently used practice (even in my own tests), which I have read (and felt) several times, it might not be the best approach.
<blockquote>
This article is not focused on any particular technology, since I have found this in a lot of different projects.
</blockquote>
<h3>What&#x27;s a unit test</h3>
According to <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Unit_testing">Wikipedia</a>, unit testing is a &quot;method by which individual units of source code [...] are tested to determine whether they are fit for use&quot;.
While this might look like a clear definition, it&#x27;s actually quite open because, what&#x27;s a &quot;unit&quot; in the context of unit testing?
Very often, in object oriented programming projects, people take a class as a unit to test, and create a unit test for every class. In other paradigms, like functional programming, you might consider a function as the unit to be tested, or a set of functions which you have decided to put together in a module.
This is not really important for the purpose of this article. What&#x27;s really important is to take into account that, in order to properly &quot;unit test&quot; a &quot;unit of source code&quot;, you have to do it by isolating it from any dependency of that unit.
One of the better ways to achieve this is by following the <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Injection Principle</a>. Without it you will probably end up trying to hack yourself in order to detach a &quot;unit&quot; from its dependencies.
<blockquote>
You can find other articles about <a href="/tag/dependency-injection/">Dependency Injection</a> in my blog.
</blockquote>
<h3>Replacing dependencies</h3>
A very frequent practice when unit testing a unit of source code is to replace its dependencies with <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Test_double">test doubles</a>, in order to properly isolate every unit when testing it.
Test doubles are basically objects compatible with the dependencies of the subject under test, that are created just for the purpose of the test and have some useful capabilities, like capturing how they have been called, or what should they return when called.
When you use one of those test doubles to assert how they were called as part of the test itself, you are using them as &quot;spies&quot;.
<h3>Over-trusting spies</h3>
All this introduction and contextualization had a single purpose.
I find myself (and others) in the situation in which you prepare a test double to behave in a certain way, then you call your subject under test, and then assert that your expectations on how the test double was called are correct, by spying on it.
This sometimes feels wrong, since it seems you are testing the test double/s instead of the subject under test.
I have read many times that, in a unit test, we should not actually know the internal implementation of the subject under test, but only test that, based on certain input, the expected output is returned (so, trust its public API).
However, there are cases in which some function/method does not really have any output, and we can only verify everything is going right by spying on the test doubles.
So, at the end of the day, I have mixed feelings about this practice.
<h3>Conclusion</h3>
Sadly, I cannot state a clear conclusion. This is actually an open article, and I would love to get your point of view in the form of a <a href="#comments-box">comment</a>.
It would be great to see how others face this, if you use other kinds of test doubles instead of spies, or if you are just ok with the implications.

For a long time, I have been trying to include tests in every project in which I've worked on. There are several types of automated tests (or what should actually be called automated checks). From unit tests, integration and functional tests, to end-to-end tests.

Each one of them differs from the…

Is spying on mocks a bad practice for unit tests?

There&#x27;s no doubt that having tests in a project allows you to find potential bugs earlier and more easily.
Lots of OSS projects require a minimum code coverage in order to accept new pull requests from contributors, and proprietary projects also tend to have some sort of continuous integration workflow which requires certain metrics to be fulfilled in order to get builds passing.
However, the code coverage can lead to a false sense of security, which makes you think that if certain class has a 100% code coverage, it is also 100% bug-free.
This is not always true, since you could be calling a method and yet not being properly testing its output or its real behavior. The code coverage will mark it as covered, but you might introduce a bug and still have a green test.
This is where mutation testing comes in.
<h3>Mutation testing</h3>
According to <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Mutation_testing">wikipedia</a>, mutation testing involves applying small modifications to a software. These modifications are called mutations.
Then, if you pass your tests and any of them fails because of this &quot;mutation&quot;, then you have &quot;killed the mutant&quot;. This is what should happen in code which is covered by tests. If tests keep passing after applying this modification, then your tests covering that part of the project are not so good, and you should improve them to really test what they are covering.
So in other words, mutation testing is a quality assurance practice for your tests.
In PHP projects, there used to be one library which purpose was working with the mutation testing approach, <a target="_blank" rel="noopener noreferrer" href="https://github.com/humbug/humbug">humbug/humbug</a>. However, it is mostly abandoned now, and it has motivated the appearance of another project, <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/">infection/infection</a>.
<h3>Infection</h3>
Infection is a mutation testing framework for PHP.
It is easy to start using it in your project, since you just need to install it (<code>composer require infection/infection --dev</code>), and create a configuration file. Then, it uses your existing test suite in order to apply mutations to your code and check your tests.
It currently integrates with <a target="_blank" rel="noopener noreferrer" href="https://github.com/sebastianbergmann/phpunit/">phpunit</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/phpspec/phpspec">phpspec</a>, but I suppose there will be compatibility with other testing frameworks in the future.
<h4>Configuring infection</h4>
It consumes an <code>infection.json</code> configuration file which could have a structure like this:
<pre class="language-json"><code class="language-json">{
 &quot;source&quot;: {
 &quot;directories&quot;: [
 &quot;module/*/src&quot;
 ]
 },
 &quot;timeout&quot;: 10,
 &quot;logs&quot;: {
 &quot;text&quot;: &quot;build/infection/infection-log.txt&quot;,
 &quot;summary&quot;: &quot;build/infection/summary-log.txt&quot;,
 &quot;debug&quot;: &quot;build/infection/debug-log.txt&quot;
 },
 &quot;tmpDir&quot;: &quot;build/infection/temp&quot;,
 &quot;phpUnit&quot;: {
 &quot;configDir&quot;: &quot;.&quot;
 }
}
</code></pre>
This is the configuration of one of my company&#x27;s projects. A modular application where every module is located inside the <code>module</code> directory and has its own <code>src</code> folder with source files.
This config file basically defines where the sources are located, where infection&#x27;s logs should be placed, a temporary folder which will be used instead of using system&#x27;s default tmp folder, and where is the <code>phpunit.xml[.dist]</code> file.
That&#x27;s all you need to get infection working.
<blockquote>
You can find detailed information of all the configuration options <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Configuration">here</a>
</blockquote>
<h4>Running infection</h4>
Now that we have the configuration file, we just need to run <code>vendor/bin/infection</code>, and it will execute our test suite and then apply mutations to our source files.
Then it will display the result, telling how many mutants have been killed.
Infection supports lots of mutators, like replacing <code>++</code> by <code>--</code>, <code>true</code> by <code>false</code>, changing method visibilities, return values, etc. You can find the whole list <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/mutators.html">here</a>.
The command line tool supports a list of modifiers, some of which are very interesting. This is how I would recommend you to run it if you want to address the whole project.
<code>vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2</code>.
<ul>
<li><code>--threads</code>: It runs tests and mutators in parallel processes, so you can take advantage of all the cores in your machine. Set a value that makes sense your you. If you have 8 cores, maybe you want to set it with value 8.</li>
<li><code>--min-msi</code>: This flag makes the process fail if the number of killed mutants is below certain percentage. In this case we require 70% of mutants to be killed. Adapt the value to your needs, and try to gradually improve your tests and increase this value (MSI stands for Mutation Score Indicator). This flag is really useful in continuous integration environments.</li>
<li><code>--only-covered</code>: This makes infection address only covered code. We know mutants won&#x27;t be killed in uncovered code, so I think it makes no sense to apply mutators there. I prefer to separate the required code coverage metric from the required mutation score indicator metric.</li>
<li><code>--log-verbosity</code>: By default, infection logs all the mutations it has applied to your code. This could lead to a log with thousands of lines if the project is big, so I prefer to set the log verbosity to 2, which makes it log only non-killed mutants.</li>
</ul>
And that&#x27;s mainly it. You can now automate this task in your continuous integration pipeline, or just run it locally, but at least you will discover some parts both in your source files and your tests that can be improved.
<h3>Troubleshooting</h3>
While this could be enough for an introduction to infection, the title of this article says something about big projects, so I&#x27;m going to explain how we integrated infection in one of my company&#x27;s biggest projects and how we addressed some inherent problems.
<h4>Already unstable</h4>
The first problem while working with infection is that, at the moment of writing this article, it&#x27;s not in a stable version yet. That means you could find bugs.
We found one in which infection wasn&#x27;t properly parsing our phpunit config file and it was excluding almost the whole project from the code coverage report. This caused all mutations to be skipped.
We reported it and they fixed it very fast, but they haven&#x27;t tagged a version including the fix yet, so we require one specific commit from master while we wait for v0.8 to be released.
<h4>Time consuming</h4>
Another problem is that infection consumes a lot of memory and CPU, and takes a lot of time to be run when you have a lot of classes and tests.
For example, our project has around 1500 unit tests, which take about 1&#x27;5min-2min to be run. Our build process also passes some coding style checks and functional repository tests. It usually takes about 3 minutes.
The first time we run infection for the whole project, it took more than 16 minutes. That would make a build which is taking 3 minutes to take almost 20 minutes from now on, which is not an option because our build is run a lot of times every day.
Our solution was making the build run infection only on changed sources, taking advantage of infection&#x27;s <code>--filter</code> flag, which lets you pass a comma-separated list of files and it gets applied only to them.
The way in which we find which files have changed from previous build is by making a <code>git diff</code> between a commit from previous successful build and the new commit after updating the branch in which the build is being run (the commit identifiers are defined by jenkins&#x27; GIT plugin as environment variables).
If it is the first time the build is run for this branch, we make the diff with <code>origin/develop</code> instead.
The result is a bash script like this:
<pre class="language-bash"><code class="language-bash"># Get files which have changed from latest processed commit, including only those inside sources
# If there&#x27;s no previous commit, diff with develop
INFECTION_FILTER=$(git diff ${GIT_PREVIOUS_SUCCESSFUL_COMMIT:-origin/develop} $GIT_COMMIT --name-only | grep /src/ | paste -sd &quot;,&quot; -)

# Check mutations over those files, if any, and require a 70% MSI
if [ -n &quot;$INFECTION_FILTER&quot; ]; then
 vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;$INFECTION_FILTER&quot;
else
 echo &quot;No source files affected. Infection skipped.&quot;
fi
</code></pre>
As you can see, we perform a diff by including only file names, then we exclude those which do not contain &quot;/src/&quot; as part of the name, and then we convert them into a comma-separated list.
With this approach, the build usually takes 1 extra minute in the worst case scenario, which can be assumed.
However, I&#x27;m sure the performance of this project will be improved as it grows and gets more stable.
<h4>Test run duplication</h4>
While previous approach filters the files where mutations are applied, it is still running all tests beforehand, which makes all tests to be run twice on every build.
<ul>
<li>First, our own execution, which generates a code coverage report to be published later, and makes the build fail if tests do not pass.</li>
<li>Then, infection runs tests again, on its own &quot;conditions&quot; (they take your phpunit config file and generate a new one, which has some custom configuration entries).</li>
</ul>
<strike>If it were possible to pass infection an already generated code coverage, we could prevent this duplication, but, as far as I know, it does not support that.</strike> It does now.
Another solution would be being able to run only tests which affect changed sources, instead of running all tests suites, but phpunit does not allow to filter by a list of files, only a specific file or folder.
<strike>This is the only problem for which we have not yet found a solution, so If you know any workaround, a comment will be very welcome :-)</strike>
Update 2018-02-22
Following what Maks suggested in this <a href="#comment-3762900802">comment</a>, we have changed the process a little bit, and using a more recent commit from <code>dev-master</code>, we can take advantage of a feature which allows an existing code coverage to be passed to infection.
Now, our phpunit execution includes these two flags: <code>--coverage-xml=build/coverage-xml --log-junit=build/phpunit.junit.xml</code>
And then, infection is executed with <code>--coverage=build</code>.
That&#x27;s it, no need to run tests twice anymore :-)
<h4>Xdebug vs phpdbg</h4>
Since infection needs access to a code coverage report, you need some tool which is capable of generating it.
People usually use xdebug, since it is the most extended and feature-rich debugging tool.
However, there&#x27;s a simpler tool, which is designed to debug php console executions only, and comes bundled with php. It is <a target="_blank" rel="noopener noreferrer" href="https://github.com/krakjoe/phpdbg">phpdbg</a>.
In our project, it usually takes 30%-50% less time to generate a code coverage report than xdebug. The result is not 100% the same, but it is very similar, and it&#x27;s worth it.
So, instead of running infection like this:
vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;$INFECTION_FILTER&quot;
We have to do it like this
phpdbg -qrr vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;$INFECTION_FILTER&quot;
It is explained in <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Running-with-phpdbg">infection&#x27;s docs</a>.
We have even started to use it to run our own phpunit execution.
<h3>Conclusion</h3>
We have seen how to use infection, and how to integrate it in a continuous integration pipeline, even for big projects.
I think it is a very useful tool, and the team behind it is doing a great job. It is also not hard to start using it, since you don&#x27;t really need to write new code, just add a config file and run the CLI tool.
Finally, I recommend you to read this article, from infection&#x27;s author. It explains how it internally works, and how to use it in a development workflow <a target="_blank" rel="noopener noreferrer" href="https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1">https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1</a>.

There's no doubt that having tests in a project allows you to find potential bugs earlier and more easily. Lots of OSS projects require a minimum code coverage in order to accept new pull requests from contributors, and proprietary projects also tend to have some sort of continuous integration…

Mutation testing with infection in big PHP projects

Docker is, without any doubt, the trending tool these days. Everybody wants to use it, because it is very useful, allowing to easily generate development environments for any kind of application.
A couple months ago I started working with docker myself (it has taken me a while, I know), and now I can&#x27;t imagine working without it.
I started using it at work, but now I&#x27;m migrating all of my OSS projects too.
<h3>The first stopper</h3>
Regardless docker is very cool, there is a problem when you start using it.
I had my development environment perfectly configured, and all my tools properly integrated. I was able, among other things, to run any project tests from within PhpStorm, just by right-clicking any test class, test method or phpunit.xml file, and selecting the &quot;Run&quot; option.
However, once you start using docker in the project, it is not that simple.
<h3>Integration with PhpStorm</h3>
Gladly, since PhpStorm is a very powerful tool, it supports integration with docker, which can be then used, among other things, to automatically run tests inside a container.
<blockquote>
From this point, I&#x27;m going to explain the process, assuming you have already installed docker in your system. If not, you need to install at least <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/engine/installation/">docker engine</a> and <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/compose/install/">docker compose</a>.
</blockquote>
The first thing you need to do is configure the connection with docker engine. Go to Settings -&gt; Build, Execution, Deployment -&gt; Docker.
Click the green plus button and a new docker connection will be created. The only value you should need to change is the API URL:
<ul>
<li>Linux: unix:///var/run/docker.sock</li>
<li>Windows and Mac: http://127.0.0.1:2376 (The URL of the docker machine. If you are using the deprecated docker toolbox with boot2docker, you have to make sure which is the IP address of the virtual machine)</li>
</ul>
<img src="/assets/img/phpstorm-docker/phpstorm-docker.png" alt="Docker in PhpStorm"/>
Apply changes and we are done here.
<h3>Create the remote interpreter</h3>
Now let&#x27;s create a remote interpreter. It will be used so that tools that need a php runtime are executed inside a docker container.
Go to Settings -&gt; Languages &amp; Frameworks -&gt; PHP. In here, you select the default interpreter for your project.
<img src="/assets/img/phpstorm-docker/interpreter.png" alt="Interpreter"/>
Click the button with the three dots, which will open the dialog to create a new interpreter.
<img src="/assets/img/phpstorm-docker/interpreter-dialog.png" alt="Interpreter dialog"/>
In that dialog you will likely have a local interpreter. Click the green plus button, at the top-left corner, and select Remote.
That will open the dialog to create a new remote interpreter. One of the options is to use a docker image from which PhpStorm will create a docker container that should contain a php interpreter.
<img src="/assets/img/phpstorm-docker/docker-interpreter.png" alt="Docker iterpreter"/>
In this dialog, just select the server (this is the connection with the docker engine, the one we created in the first step), then select the docker image you want to use. The dropdown will be filled with all the images in your machine.
Finally select the interpreter path. Usually, <a target="_blank" rel="noopener noreferrer" href="https://hub.docker.com/_/php/">PHP-based docker images</a> have the php interpreter in the PATH, so just setting <code>php</code> should be enough.
Save everything and we are done configuring the interpreter.
<h3>Create the PHPUnit config</h3>
Now that the interpreter is created, we have to define a phpunit configuration that uses it.
Go to Settings -&gt; Languages &amp; Frameworks -&gt; PHP -&gt; PHPUnit, Use the green plus button at the top-left corner, select By Remote interpreter and a dialog window will be prompt.
It only contains a dropdown where you have to select the remote interpreter we created in previous step.
Select it and you&#x27;ll see a configuration similar to this.
<img src="/assets/img/phpstorm-docker/phpunit.png" alt="PHPUnit configuration"/>
As you can see, PhpStorm mounts the project dir as a volume inside the container, in the <code>/opt/project</code> dir, and based on that path, selects the path to composer&#x27;s autoloader script and the phpunit.xml script.
If any of those params is not correct in your project, you can (and should) change it.
And that&#x27;s it!
Now, you can right-click any test class, test method or phpunit.xml file and select &quot;Run&quot;. PhpStorm will automatically connect with docker engine, set up a temporary container and run the tests inside of it.
You&#x27;ll know it is working because of this:
<img src="/assets/img/phpstorm-docker/tests-in-container.png" alt="Tests inside container"/>
<h3>Conclusion</h3>
PhpStorm&#x27;s team is always working to add integrations with the latest development tools, making it the most powerful IDE for PHP.
Now go, configure your environment and start using docker. It&#x27;s amazing!

Docker is, without any doubt, the trending tool these days. Everybody wants to use it, because it is very useful, allowing to easily generate development environments for any kind of application. A couple months ago I started working with docker myself (it has taken me a while, I know), and now I…

Alejandro Celaya | Blog

Software development, agile methodologies and open source software.

tag Archive — "tests"

— 2022 —

— 2019 —

— 2018 —

— 2017 —