How to reduce duplication in your GitHub Actions workflows

Recently ReactJS v18 was released. I started to look into its improvements, and checked which of my projects could benefit from them.The main react-based project I maintain at the moment of writing this article is <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlink-web-client</a>, so I naturally created a new branch and started the process.The update was quite smooth, as most of the other dependencies I was using from the react ecosystem were already updated to support v18. All but one, <a target="_blank" rel="noopener noreferrer" href="https://enzymejs.github.io/enzyme/">enzyme</a>, the testing framework I was using to unit-test react components.To be precise, I was able to continue running my tests, but some of them (those using <code>mount</code> instead of <code>shallow</code>) were still rendering components with the &quot;old&quot; React v17 approach, causing them to behave as in that version and a warning being printed in the console.<h3>First fix attempt</h3>The way enzyme integrates with the react version you are using is through an adapter library. The last official one supports only React v16, but someone released an <a target="_blank" rel="noopener noreferrer" href="https://github.com/wojtekmaj/enzyme-adapter-react-17">unofficial adapter for v17</a> which, with close to 650k weekly downloads, quickly became the way to go.I checked if there was one for React 18, but sadly there wasn&#x27;t.Also, the author of the adapter for v17 posted an article indicating <a target="_blank" rel="noopener noreferrer" href="https://dev.to/wojtekmaj/enzyme-is-dead-now-what-ekl">enzyme is dead</a>, he was not going to release one for v18, and publishing one for v17 was a mistake.With those facts in mind, I started to face what was probably going to be the moment to migrate to <a target="_blank" rel="noopener noreferrer" href="https://testing-library.com/docs/react-testing-library/intro">react testing library</a>.<h3>Migration strategy</h3>I decided to follow a migration strategy similar to the one suggested in the article above, but with a couple of modifications:<ul><li>I update to React 18 anyway, as tests still pass.</li><li>Migrate right away all tests using <code>mount</code>, as those are the ones behaving differently.</li><li>New tests should be written directly with react testing library (RTL from now on).</li><li>When an existing test needs to be changed, consider migrating it to RTL, and do it if there&#x27;s time.</li><li>Introduce some PR from time to time just migrating a handful of tests.</li><li>Once everything is migrated, get rid of everything related with enzyme.</li></ul><h3>First contact and impressions</h3>I was a bit sceptical with RTL, as it was focused more on integration tests, instead of unit tests, which meant I had to completely change the mindset to write tests.It also didn&#x27;t support shallow rendering, meaning that you will end up sometimes rendering a bigger component tree when testing container-like components, potentially causing side effects that could affect your test.On the other hand there&#x27;s more and more people stating that <a target="_blank" rel="noopener noreferrer" href="https://kentcdodds.com/blog/why-i-never-use-shallow-rendering">shallow rendering is an anti-pattern</a>, and that&#x27;s probably the reason of enzyme getting &quot;discontinued&quot;.Also, RTL has become the <a target="_blank" rel="noopener noreferrer" href="https://reactjs.org/docs/testing.html#tools">officially recommended framework</a> to test react components, it has become very popular, and it has even joined a wider group of testing libraries under the <a target="_blank" rel="noopener noreferrer" href="https://testing-library.com/"><code>@testing-library</code></a> organization.The thing is that I started to use it, and I was quickly surprised with how it felt to work with it, and all the benefits it was bringing to my tests.<h3>Benefits I have experienced</h3>Once I started migrating the first tests I noticed some benefits:<ul><li>Changing my mindset was actually easier than I thought. The docs are very well written, so it&#x27;s relatively easy to start using RTL.</li><li>My tests got simpler, and required less of those nasty global mocks that jest allows (I&#x27;m looking at you, <a target="_blank" rel="noopener noreferrer" href="https://jestjs.io/es-ES/docs/mock-functions#mocking-modules"><code>jest.mock</code></a>).</li><li>Tests were way less coupled with implementation details. I moved from testing the component props, to test what the user would see on the screen.</li><li>Related with previous point, I started to write my tests by looking at the screen instead of the component&#x27;s implementation.</li><li>Changes in source code led to fewer tests requiring to be changed.</li><li>A progressive migration was possible, where some tests still use enzyme and others are already using RTL.</li></ul>And overall, it felt good and I have already dedicated a couple of PRs just to migrate more tests.<h3>Challenges I faced</h3>But of course, migrating to a new tool always comes with its own challenges, and this was not an exception. I&#x27;ll explain the main ones I have faced so far and how I tackled them:<ul><li>RTL lets you do some things in a couple of different ways, and the docs may not be super clear about what&#x27;s the recommended option.For this I recommend reading <a target="_blank" rel="noopener noreferrer" href="https://acel.me/DCMb1">this article</a> from Kent C. Dodds, RTL&#x27;s author, which explains some common mistakes when using the library.</li><li>At first, I struggled a bit with asynchronous side effects and state changes in components, which can easily end up printing a warning like <code>Warning: An update to MyComponent inside a test was not wrapped in act</code>.The warning also explains how you are supposed to solve this by wrapping your code in <code>act()</code>, but this warning comes from React, not RTL, and what it is not telling you is that RTL already wraps all needed operations in <code>act()</code>.What this error usually mean is that you are not waiting for something to happen. <a target="_blank" rel="noopener noreferrer" href="https://acel.me/jnUdT">This other article</a>, also from Kent C. Dodds, explains everything you need to know about the things that can lead to this warning, and how to solve them.</li><li>Since you start testing from the user point of view, some use cases can be challenging to test without coupling with implementation details. For example, when you have a component which dynamically renders different images or icons, I used to directly check the component properties.For this I found the best solution was to use <a target="_blank" rel="noopener noreferrer" href="https://jestjs.io/docs/snapshot-testing">jest snapshots</a>. I didn&#x27;t want to manually check which SVG was being rendered in the DOM, just that different ones were being used in every case.<pre><pre><code class="language-tsx">// I went from this...
test(&#x27;...&#x27;, () =&gt; {
 const wrapper = shallow(&lt;MyComp dynamicValue=&quot;foo&quot; /&gt;);
 expect(wrapper.find(FontAwesomeIcon).prop(&#x27;icon&#x27;)).toEqual(someIcon);
})

// To this
test(&#x27;...&#x27;, () =&gt; {
 const { container } = render(&lt;MyComp dynamicValue=&quot;foo&quot; /&gt;);
 expect(container.firstChild).toMatchSnapshot();
})
</code></pre></pre></li><li>Something very similar happens if you use some library that renders canvas. There&#x27;s no way to test that from a DOM point of view, which is what RTL does.In my case I was using <a target="_blank" rel="noopener noreferrer" href="https://www.chartjs.org/">Chart.js</a> to render some charts, so I used <a target="_blank" rel="noopener noreferrer" href="https://github.com/hustcc/jest-canvas-mock">jest-canvas-mock</a>, which exposes a method on the canvas to see which are all the events invoked from the library.Then I used snapshots again to verify they had the expected &quot;shape&quot;.<pre><pre><code class="language-tsx">// I went from this...
test(&#x27;...&#x27;, () =&gt; {
 const wrapper = shallow(&lt;MyCompWithCanvas /&gt;);
 expect(wrapper.find(Chart).prop(&#x27;data&#x27;).datasets).toEqual(...);
})

// To this
test(&#x27;...&#x27;, () =&gt; {
 const { container } = render(&lt;MyCompWithCanvas /&gt;);
 const events = container.querySelector(&#x27;canvas&#x27;)?.getContext(&#x27;2d&#x27;)?.__getEvents();

 expect(events).toBeTruthy();
 expect(events).toMatchSnapshot();
})
</code></pre></pre></li></ul><h3>Conclusion</h3>For better or worst, enzyme should no longer be considered an option. If you are starting a new project or are using it on small ones, consider migrating to RTL as soon as possible.If you are using it in bigger projects, following the process described in this article will probably help you.In any case, you will get surprised. React testing library is a great tool and provides many benefits over enzyme.Its main issue is that it&#x27;s not a drop-in replacement for enzyme, so you will have to change a bit your mindset.

My experience migrating from enzyme to react testing library

Capturing code coverage for a test suite is a very useful way to know which parts of your source code are actually getting executed by tests.This is useful not only to know if you need to add more tests to your project (in case the coverage is too low), but also, if you want to apply technics like mutation testing, having a properly generated code coverage report will dramatically speed up te execution and help you know if a non-killed mutant is relevant.<h3>Generating the code coverage</h3>Capturing code coverage is usually easy in the most common types of tests, like unit tests, where the source code is executed in the same process.Tools like <a target="_blank" rel="noopener noreferrer" href="https://phpunit.de/">PHPUnit</a>, one of the most extended PHP testing frameworks, ships with tools like <a target="_blank" rel="noopener noreferrer" href="https://github.com/sebastianbergmann/php-code-coverage">php-code-coverage</a>, designed to capture the code under execution, just by providing a couple of lines of configuration.However, generating a reliable code coverage report is not always that easy for other types of tests.In contexts like E2E tests, where you may spin-up an API on a different process, and make your tests just call this API via HTTP and assert on the responses, PHPUnit will not be able to capture the code coverage, because the source code is actually being loaded on a different process.<h3>Remote code coverage</h3>A couple of weeks ago, I faced the issue described above. I had a web service for which I had a decent API test suite, but I wanted to capture code coverage for two main reasons:<ul><li>Be able to merge this coverage with the one generated by other tests suites, to know what&#x27;s the &quot;total&quot; code coverage, regardless the test producing it.</li><li>Apply mutation testing to the API tests, as I was already doing that for the rest of the test suites in the project. For this, I also needed to know exactly what tests were covering every line of the source code.</li></ul>After some investigation, I couldn&#x27;t find a built-in feature, where PHPUnit itself was able to &quot;remotely&quot; capture te code under test, so I had to think on a way to manually use <code>php-code-coverage</code> by myself.<h3>Code coverage reports</h3>In order to be able to capture code coverage, I had to make sure the instances of the objects provided by <code>php-code-coverage</code> where &quot;spin-up&quot; with the server, and then, the code coverage report was dumped at the end.Also, you usually define the code to include/exclude from coverage in PHPUnit&#x27;s config, but since this time we are the ones creating the coverage objects, this was not possible.I tried a couple of different approaches (more on this at the end), but the most accurate one for me, was to define two routes in my server, that were dynamically configured only when running the server in the context of these tests, and were responsible for creating and dumping the code coverage.<pre><pre><code class="language-php">use PHPUnit\Runner\Version;
use SebastianBergmann\CodeCoverage\CodeCoverage;
use SebastianBergmann\CodeCoverage\Driver\Selector;
use SebastianBergmann\CodeCoverage\Filter;
use SebastianBergmann\CodeCoverage\Report\Html\Facade as Html;
use SebastianBergmann\CodeCoverage\Report\PHP;
use SebastianBergmann\CodeCoverage\Report\Xml\Facade as Xml;

// Define directories containing the code you want to cover
$filter = new Filter();
$filter-&gt;includeDirectory(...);
$filter-&gt;includeDirectory(...);
$coverage = new CodeCoverage((new Selector())-&gt;forLineCoverage($filter), $filter);

// This is &quot;pseudo-code&quot;. Defining the routes depends on the framework you use
$app-&gt;get(&#x27;/api-tests/start-coverage&#x27;, function () use (&amp;$coverage) {
 $coverage-&gt;start(&#x27;API tests&#x27;);
 return new EmptyResponse();
});
$app-&gt;get(&#x27;/api-tests/stop-coverage&#x27;, function () use (&amp;$coverage) {
 $basePath = &#x27;...&#x27;; // Wherever you want the reports to be dumped
 $coverage-&gt;stop();

 // I generated coverage reports in a couple of different formats
 (new PHP())-&gt;process($coverage, $basePath . &#x27;/coverage.cov&#x27;);
 (new Xml(Version::getVersionString()))-&gt;process($coverage, $basePath . &#x27;/coverage-xml&#x27;);
 (new Html())-&gt;process($coverage, $basePath . &#x27;/coverage-html&#x27;);

 return new EmptyResponse();
});
</code></pre></pre>Then, I added the next snippet to my PHPUnit <code>bootstrap.php</code> file, to make sure the endpoints configured above are invoked at the proper times:<pre><pre><code class="language-php">// when starting up the test suite, call the endpoint to start collecting coverage
$httpClient-&gt;request(&#x27;GET&#x27;, &#x27;http://localhost:8080/api-tests/start-coverage&#x27;);

// When the tests process shuts down, call the endpoint to stop the coverage
register_shutdown_function(function () use ($httpClient): void {
 $httpClient-&gt;request(&#x27;GET&#x27;, &#x27;http://localhost:8080/api-tests/stop-coverage&#x27;);
});
</code></pre></pre><blockquote>Learn more about how to configure PHPUnit&#x27;s <a target="_blank" rel="noopener noreferrer" href="https://phpunit.readthedocs.io/en/9.5/configuration.html#the-bootstrap-attribute">bootstrap script</a>.</blockquote>This was fine from a pure code coverage point of view, but if you try it yourself, you will notice the report says all the code has been covered by <code>API tests</code>, which is what we provide when calling <code>$coverage-&gt;start(&#x27;API tests&#x27;);</code><img src="/assets/img/remote-coverage/invalid-api-coverage.png" alt="Invalid API coverage"/>To properly do mutation testing we need to know which are the exact tests that cover every part of the code. That requirement took me to the next approach.<h3>Supporting mutation testing</h3>To be more precise with the generated reports, I needed some way to notify the server about the specific test performing the request.One way to do this is by sending a custom header when performing a request from your tests, and registering some event handler or middleware which starts the coverage for that request with the value coming from that header.With that, I evolved the configuration above, and added a middleware handler (the framework I use is <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-15/">PSR-15</a> based, but the framework of your choice will probably have a way to do this).<pre><pre><code class="language-php">use ...

$coverage = new CodeCoverage();

// This ensures the coverage is started and stopped on every request, using the &quot;x-coverage-id&quot; header as the ID.
$app-&gt;middleware(function (ServerRequestInterface $req, RequestHandlerInterface $handler) use (&amp;$coverage) {
$coverage?-&gt;start(&#x27;API Tests&#x27;);
 $coverage-&gt;start($req-&gt;getHeaderLine(&#x27;x-coverage-id&#x27;));

 try {
 return $handler-&gt;handle($req);
 } finally {
 $coverage-&gt;stop();
 }
});

// This route is no longer needed. You can also remove the request from your bootstrap script
// $app-&gt;get(&#x27;/api-tests/start-coverage&#x27;, ...);
$app-&gt;get(&#x27;/api-tests/stop-coverage&#x27;, ...);
</code></pre></pre>Then, inside your test you can do something like this:<pre><pre><code class="language-php">use PHPUnit\Framework\TestCase;

class MyApiTest extends TestCase
{
 private ClientInterface $httpClient;

 public function setUp(): void
 {
 $this-&gt;httpClient = ...;
 }

 public function testMyEndpoint(): void
 {
 $resp = $this-&gt;httpClient-&gt;request(&#x27;POST&#x27;, &#x27;...&#x27;, [
 &#x27;headers&#x27; =&gt; [
 &#x27;x-coverage-id&#x27; =&gt; static::class,
 ],
 ])

 // Asserts go here...
 }
}
</code></pre></pre>After this, the code coverage reports will properly show the actual tests covering them.<img src="/assets/img/remote-coverage/valid-api-coverage.png" alt="Valid API coverage"/><h3>Alternative approach</h3>The approach explained here is functional, but it&#x27;s a bit hacky. I had to do it like this because of limitations with how I serve the API with <a target="_blank" rel="noopener noreferrer" href="https://openswoole.com">openswoole</a>.However, instead of defining routes that need to be actively called to start/stop the code coverage (or just to dump the report, as in the second approach), you could try to register the shutdown function directly in the server side.If that works for you, it&#x27;s much more straightforward, and you won&#x27;t need to bother changing your tests bootstrap script.<h3>Conclusion</h3>Even though there&#x27;s room for improvement, this allows to generate code coverage reports for code that&#x27;s running on a different process.I plan to apply something similar for E2E tests for a command line app, so maybe I extend this article at some point.Also, I decided to write this article, because at the moment of having to implement this myself, I didn&#x27;t find too much useful information out there.I hope this helped you, and now you can collect code coverage reports on your API tests.

Capturing remote code coverage in API tests with PHPUnit

In the last couple of days I have been playing around with the idea of combining GitHub Actions and GitHub Pages to be able to dynamically build preview environments every time a pull request is opened against a repository hosting a client-side web app.The idea is that you can quickly preview the changes made by some contributor, without having to clone their branch and running everything locally, something specially useful for open source projects.In this article I will explain how to set everything up, and what are some considerations and problems you may face during the process.<h3>Enable github pages</h3>The first thing you need to do is enable your project to publish pages on some branch. I recommend creating a new branch for this, let&#x27;s say <code>preview-env</code>, and then enabling publishing on it.In order to do that go to your repository and then Settings -&gt; Pages, look for the Source section, and you should see something like this:<img src="/assets/img/preview-envs/github-pages-before-enabling.png" alt="GitHub pages before enabling"/>Click in the dropdown and select your new branch, <code>preview-env</code>. Then click Save.<img src="/assets/img/preview-envs/github-pages-after-enabling.png" alt="GitHub pages after enabling"/>After saving you will see a message at the top of this section telling you what&#x27;s the URL in which the pages are published. It always follows the same pattern. If your organization is <code>my-org</code> and your repository is <code>my-repo</code>, the url will be <code>https://my-org.github.io/my-repo/</code>.<h3>Configure the workflow</h3>Now we need to configure a workflow that will &quot;deploy&quot; the preview environment for the branch every time a new pull request is created or updated:Let&#x27;s imagine we have a React web app that is built with node.js. The configuration could look like this.<pre><pre><code class="language-yml"># .github/workflows/deploy-preview.yml
name: Deploy preview

on:
 pull_request: null

jobs:
 deploy:
 runs-on: ubuntu-20.04
 continue-on-error: true
 steps:
 - name: Checkout code
 uses: actions/checkout@v2
 - name: Use node.js 14.15
 uses: actions/setup-node@v1
 with:
 node-version: 14.15
 - name: Generate slug
 id: generate_slug
 run: echo &quot;##[set-output name=slug;]$(echo ${GITHUB_HEAD_REF#refs/heads/} | sed -r &#x27;s/[~\^]+//g&#x27; | sed -r &#x27;s/[^a-zA-Z0-9]+/-/g&#x27; | sed -r &#x27;s/^-+\|-+$//g&#x27; | tr A-Z a-z)&quot;
 - name: Build
 run: npm ci &amp;&amp; npm run build
 - name: Deploy
 uses: JamesIves/github-pages-deploy-action@4.1.1
 with:
 branch: preview-env
 folder: build
 target-folder: ${{ steps.generate_slug.outputs.slug }}
 - name: Publish env
 uses: marocchino/sticky-pull-request-comment@v2
 with:
 header: Preview environment
 message: |
 ## Preview environment
 https://my-org.github.io/my-repo/${{ steps.generate_slug.outputs.slug }}/
</code></pre></pre><blockquote>If you are not familiar with GitHub Actions yet, take a look at the <a target="_blank" rel="noopener noreferrer" href="https://docs.github.com/en/actions">official documentation</a>.</blockquote>This workflow will follow the next steps:<ul><li>Checkout the code from the repository.</li><li>Set-up node.js 14.15</li><li>Based on the branch name, generate a slug that we will use to have separated deployment folders for every pull request we receive. The slug is exposed to next steps as an output.</li><li>Build the project. This step will obviously depend on the needs of your project.</li><li>Using <a target="_blank" rel="noopener noreferrer" href="https://github.com/JamesIves/github-pages-deploy-action">JamesIves/github-pages-deploy-action</a>, deploy the contents we previously built in the GitHub page, under a sub-folder matching the slug we generated a few steps above, and under the branch <code>preview-env</code> (remember we enabled GitHub pages for this branch).</li><li>Finally, using <a target="_blank" rel="noopener noreferrer" href="https://github.com/marocchino/sticky-pull-request-comment">marocchino/sticky-pull-request-comment</a>, post a comment in the pull request with a link to the preview environment, so that you can just click on it and see the deployment.</li></ul>The steps may change depending on your needs and how you build your project. For example, I have an extra step to update a property in the <code>package.json</code> that is then used to allow serving the app in a sub-path instead of the root of the domain.You may also not build your site with node.js, as you could be using any of the multiple existing static site generators out there, but you get the idea.Finally, you have probably noticed the config includes <code>continue-on-error: true</code>. This will allow the pull request to be merged even if this process fails for some reason.<h3>Considerations when using forks</h3>While the configuration above will work perfectly when creating pull requests inside the same repository, there are some limitations when the pull request comes from a fork (which is the usual in OSS projects).For security reasons, the <code>GITHUB_TOKEN</code> injected in those cases doesn&#x27;t have write permissions, which will make the &quot;deploy&quot; step fail.Also, you cannot define your own token with more permissions and inject it as a secret, as the secrets are not exposed to workflows running on forks.However, with a couple small config changes, we can work around these limitations:<h4>Change event</h4>The first thing we need to do is use the <code>pull_request_target</code> event instead of the <code>pull_request</code> one.<pre><pre><code class="language-diff">on:
- pull_request: null
+ pull_request_target: null
</code></pre></pre>According to the documentation, this is equivalent to the <code>pull_request</code> event, but it runs in the scope of the base repository.Thanks to this, the <code>GITHUB_TOKEN</code> will come with write permissions, and we will have access to secrets if needed.<h4>Checkout the fork</h4>The second thing we are going to change is that we now need to specify the repository to checkout.In order to do that, we need to add two inputs to the &quot;Checkout code&quot; step, like this:<pre><pre><code class="language-diff"> steps:
 - name: Checkout code
 uses: actions/checkout@v2
+ with:
+ repository: ${{ github.event.pull_request.head.repo.full_name }}
+ ref: ${{ github.event.pull_request.head.ref }}
</code></pre></pre>This will ensure we checkout from the fork repository, instead of the base one, which is the default, as it&#x27;s the one where the <code>pull_request_target</code> event is scoped to.And that&#x27;s it. With these two changes you can now deploy preview environments for pull requests coming from forks.<blockquote>IMPORTANT! The limitations of the <code>pull_request</code> event are there for a reason. People could potentially get access to your secrets or manipulate your code. Proceed at your own risk.</blockquote><h3>Other considerations</h3>Other than the limitations for forks, there are a few other minor things to take into consideration.<ul><li>Changes in GitHub Pages do not reflect immediately. It may take up to a couple of minutes for the deployment to be available, but considering you will usually not review a PR right after it has been created, this should not be a big issue.</li><li>As you have seen, every time a pull request is created, the deployment will generate a new folder in your GitHub Pages branch. This may result in a lot of old folders to pile-up, so you may want to set up some way to clean them up periodically or after the pull request has been merged.</li><li>Due to the fact that GitHub Pages only allows static pages to be served, this process cannot be used with apps that require some kind of back-end or server-side logic.</li><li>If your project is already using GitHub Pages for something else (serve the project&#x27;s website or docs), that may conflict with this, preventing you from having this kind of preview envs.</li></ul>

Setting-up a preview environment for your client-side apps with GitHub Actions and GitHub Pages

I remember many years ago, when I released my first open source project, that every time I had a new release ready, I was never sure what should be the new version number.Versioning software is not as straightforward as it might look when you don&#x27;t have a lot of experience.In this article I&#x27;ll try to explain the <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">SemVer</a> approach to version software, and some things you might want to take into account on top of that.<h3>Semantic Versioning</h3>The first thing you need is a solid set of rules that help you decide which should your next version be.I remember thinking &quot;is this version big enough as to jump from 1.0 to 2.0?&quot; or &quot;I have seen projects using three numbers for the version, X.Y.Z, should I do the same?&quot;Those questions are easier to answer if you stick to some standard that tells you what to do, and that will help others more easily understand your intention.My recommendation is to use <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">Semantic Versioning</a>, as it defines a solid starting point, and it&#x27;s quite adopted by the community.Semantic Versioning states that you should use three numbers to identify versions, <code>MAJOR.MINOR.PATCH</code>, and follow these rules in order to decide which one to increase:<ul><li><code>PATCH</code>: the new release includes only bug fixes. People are encouraged to update to this release and no changes are needed in their code bases.</li><li><code>MINOR</code>: the new release includes new backwards-compatible features. People can update to this version without changing their code bases, and make use of the new features afterwards if they wish.</li><li><code>MAJOR</code>: the new release includes <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Backward_compatibility">backward compatibility</a> breaks. Changes in the code base will be needed when updating to this version.</li></ul><blockquote>These rules are easier to apply to libraries and frameworks than it is for apps and services. More about this later.</blockquote><h3>Try to keep backwards-compatibility</h3>Ok, we have some rules now. However, I have seen very frequently people mistaking what can be done with what should be done.Yes, if you introduce a breaking change, you can update the major version and that&#x27;s it. However, if you do it too frequently, people won&#x27;t be happy, and you will end up with a fragmented user base, that stop updating to newer versions.I remember using a library to deploy stuff that had a new major version every couple of months, changing things like &quot;The <code>host</code> function is now called <code>server</code>&quot;. Yes, ok, but why do I have to change my stuff so frequently.It&#x27;s usually better to try to implement things in a backwards-compatible way, and keep track of the things you want to change and do it all at a time.You can also deprecate things, and document the way you want those things to be done moving forward. That way you give people the chance to start changing their code and simplify a hypothetical upcoming major release.<h3>Provide clear upgrade paths</h3>As mentioned above, too frequent major version changes are not desirable, but sooner or later you will want to release a new major version for different reasons.<ul><li>Getting rid of that feature which is hard to maintain, introduces lots of bugs, and provides little value.</li><li>Supporting something new which is very hard to implement without breaking backwards compatibility.</li><li>Removing all those deprecated methods you have been dragging for months.</li><li>Fix design issues, where something was architected in the wrong way, and only spotted after some time.</li></ul>At the time this happens you will want your users to upgrade to the newer version, as you don&#x27;t want people to report bugs that only affect older versions, or to expect you to backport fixes to older versions because they haven&#x27;t updated to the new one.The best way to achieve this is to provide clear and simple upgrade paths:<ul><li>Prepare your releases for the things to come: If you want to introduce new ways of doing things, do it before the major release, and deprecate the older ones. For example, you can have a new method that gets called by an older one, while you mark the old one as deprecated.People will be more willing to update and change their code afterwards. By the time the major release is published, their code will be ready to upgrade.</li><li>Include a CHANGELOG file in your project, explaining the changes introduced in every version. A very good way to document a change log is the <a target="_blank" rel="noopener noreferrer" href="https://keepachangelog.com/">Keep a changelog</a> initiative.You can also use some other platform to document the change log, like GitHub releases.</li><li>Include documentation on how to upgrade to a major release: This is probably the most important one. Document all the steps that need to be followed and all the required changes to update from a release to the next major one.A good way to do this is by including an UPGRADE file in your project. You can see <a target="_blank" rel="noopener noreferrer" href="https://github.com/acmailer/acmailer/blob/main/UPGRADE.md">this example</a>.</li></ul><h3>Versioning libraries vs apps</h3>Semantic Versioning is a good standard to version software, but it plays much better with libraries and frameworks that are installed using some kind of package manager, than it does with apps and services that are not directly depended on.For example, how do you identify a backward compatibility break on a web user interface?With this kind of software there are more grey areas, and you will end up using your intuition.For example, not long ago I released a <a href="/2020/12/24/released-shlink-web-client-v3-0-0/">new major version</a> for a web app I maintain, and it was completely compatible with the previous one. However, it introduced a larger amount of features than usual, and big UI changes, so I decided it was worth the bump.Because of this there are projects that use different approaches to version apps. For example:<ul><li>Firefox and Chrome have 6-week cycles, and after each one of them they release a new major version no matter what.</li><li>Jetbrains publishes a few releases of their IDEs every year (I think 3) which always start with the year they were released on, like <code>2020.3.0</code>, and they increase the patch number for bugfixes.</li><li>Ubuntu releases 2 versions every year, one in April and one in October, and they are always identified by the year and month they were released on, like <code>19.04</code> or <code>20.10</code>.</li></ul><h3>Special versions</h3>There&#x27;s also a set of special versions that can be used to publish the so-called pre-release versions. These <a target="_blank" rel="noopener noreferrer" href="https://semver.org/#spec-item-9">are covered</a> by Semantic Versioning, but sometimes they are not so easy to understand.It is also very likely that you never have to use these, as only relatively big projects will need this level of granularity, but it&#x27;s still good to know what&#x27;s their purpose and what do they imply.<h4>0.x.y</h4>It is possible to release a version of your software where the major number is <code>0</code>. When this happens, it means your software is still in its early stages.It does not mean it&#x27;s necessarily unstable, but it means you are still introducing frequent changes on it, and therefore, it might be a bit more fragile than a version starting with a number greater than <code>0</code>.<h4>Alpha</h4>The intention of an alpha release is that a subset of people con test some new functionality, but you don&#x27;t want people to use it in productionAlpha versions are on their very early stages. Their public API is very likely to change, and they probably contain bugs.Semantic Versioning states that alpha versions are identified by this pattern: <code>MAJOR.MINOR.PATCH-alpha.NUMBER</code>, for example <code>2.0.0-alpha.1</code>.<h4>Beta</h4>Similar to alpha versions, beta versions are also prone to contain bugs, but their API will in theory not change. That means that you could potentially adapt your software to start using a beta version, and when the corresponding stable release is published, you shouldn&#x27;t need that many changes.Semantic Versioning defines a pattern similar to the one for alpha versions: <code>MAJOR.MINOR.PATCH-beta.NUMBER</code>, for example <code>2.0.0-beta.1</code>.<h4>Release candidate</h4>Release candidates (RC) are pre-release versions that you will want to publish for the general user-base to test an upcoming release. You want to publish a RC so that it&#x27;s battle tested by as many people as possible, in order to find possible bugs and edge cases.They should not contain many bugs, but some could be found. The public API should not change anymore.According to Semantic Versioning, the pattern to identify a RC is <code>MAJOR.MINOR.PATCH-rc.NUMBER</code>, for example <code>2.0.0-rc.1</code>.<h3>Conclusion</h3>We have seen some approaches and considerations to properly version your software.Versioning is not so straightforward when you don&#x27;t have a lot of experience, but as you have seen, you just need to know a small set of rules and use a bit your intuition.With those tools, you will be able to properly version your software.

What to take into account when versioning software

I&#x27;ve been wanting to write this article for a while, but it is a subject complex to approach.Lately I&#x27;ve had some &quot;conflicts&quot; with users in some of the open source software (OSS) projects I maintain, and I have also seen some of the people I follow on Twitter dealing with the same.Because of that I wanted to share my point of view on what should be the attitude and considerations when using OSS.<h3>Company vs personal</h3>The first thing you have to take into account when using OSS, is that there are two kinds of projects which usually have different goals.<h4>Company projects</h4>It is very common that companies which activity is related with software or make a deep use of technology, publish their own OSS projects.These projects are usually born from some need the company has had which is not strictly part of their domain.Thanks to publishing these projects they can find new talent (among contributors), or position themselves in the industry.It&#x27;s also quite common that companies provide paid services which are related with these OSS projects, like specific hostings or deployment services, or just consultancy on using them.<h4>Personal projects</h4>There&#x27;s also projects which are usually maintained by one person, or a very small team, during their own free time.These projects are created by people who do it mainly because they love to program and want to share something they have done.The goal here is to get some visibility, show some skills, learn something new and contribute to the open source community.While the main purpose is not usually getting money, they frequently accept donations and sponsorships.<hr/>Now we know the two main groups of OSS projects. Sometimes we think they are the same, but company projects are usually maintained by people which is paid by a company, and do it during their working hours.On the other hand, personal projects cannot get that much attention and resources, so they cannot progress as fast as the company ones.<h3>Considerations</h3>That said, when we want to use an open source project or contribute to it, there are a few things to take into account.<h4>Always be polite</h4>While using a project, you will probably end up finding something is not working as you expected, or you&#x27;ll develop a new need for which the project does not provide a solution.When reporting bugs or asking for new features, always try to be polite. Provide as many details as possible, be descriptive and help maintainers reproduce the issue.If you are asked to do some tests and get back with the result, try to find the time to do it. You may think that&#x27;s not your work, but the issue will get solved much earlier and you will avoid frustrations.There&#x27;s a formula I like to use when reporting issues, which consists in filling this template:<ul><li>I did [action].</li><li>I expected [behavior] to happen.</li><li>Instead [other_behavior] happened.</li></ul>This will simplify maintainers see the user&#x27;s intention, and detect confusions which are not actually bugs. Helping the user use the tool will be easier this way.<h4>Don&#x27;t take anything for granted</h4>Just because you open a ticket, does not mean the team is going to work on it right away. There are several reasons that could delay it.<ul><li>The project has other priorities.</li><li>It&#x27;s hard to reproduce.</li><li>There&#x27;s no one available to work on it.</li><li>Maintainers just want to spend their time in something else (specially in the case of personal projects).</li></ul>The people working in the project is usually dealing with a much wider scope than you might think, so you have to be patient and understand this.It could also happen that they don&#x27;t really want to maintain what you proposed, even though it&#x27;s a supper critical feature for you, but a project will never be able to cover the 100% of use cases.<h4>Consider contributing</h4>One of the main principles of the open source philosophy is collaboration.If you have been using an OSS project, the best way to show some appreciation and collaborate with its continuity is by contributing to it.There are several ways to contribute.<ul><li>You could improve the documentation,</li><li>You could provide some PR fixing some bug or improving the code.</li><li>You could provide some PR with some new feature.</li><li>You could make some economic donation or sponsorship.</li></ul>The last one is very rare, specially among individual users, but if your company is taking advantage of an OSS project, you should consider it.For the other three cases, if there&#x27;s something you think you can help with, there&#x27;s something to take into account in relation with previous point. You should always ask first.Maybe someone is already working on something similar, or the team has some ideas in mind that could be shared before starting any implementation, or, as I said before, maybe they just don&#x27;t want to maintain it.If you ask first you will save yourself some time and frustrations.<h4>Follow the project rules</h4>All projects have rules, and I&#x27;m talking about any kind of rule. How to write issues, the code of conduct, the coding standards, the amount of people that need to approve a PR before merging it...You have to always follow these rules. Don&#x27;t try to enforce something just because you consider it better.If the project requires the use of spaces for indentation but you used tabs because you like them better, don&#x27;t get mad if maintainers don&#x27;t want to merge your PR.<h4>Don&#x27;t take it personally</h4>And finally, because of the reasons exposed in previous points, there will be decisions you don&#x27;t like.Don&#x27;t take it personally, because it&#x27;s not in most of the cases.Your issue could be closed because you didn&#x27;t follow the issue template, or because they asked you to check something and you didn&#x27;t give any feedback after some days.Understand this can happen, and you can always be polite and open a discussion, but never get mad and accuse the team to be unfair or treat you in the wrong way.<h3>Conclusion</h3>This article describes a set of rules that should help people using OSS projects and people maintaining them to understand each other, but focusing on the former.I will try to write something similar but focusing on how maintainers should behave. We obviously forget some times that there&#x27;s people trusting on us when using the software we write.

Considerations to properly use open source software projects

Some of you know that I work now as a full stack javascript developer, and I have interacted with a few different projects, both in front-end javascript and in nodejs.My main concern about javascript has been that, apparently, the community has not adopted one of the practices that, for me, has been the most game changing of all: Dependency Injection.<blockquote>Take into account that this is my relative point of view, from a community which is still quite new to me, so if you find that something I say makes no sense at all, you are more than welcome to <a href="/#comments-box">leave a comment</a>.</blockquote><h3>Current state of DI in JS</h3>At the moment of writing this article, I have made some research, and my feeling is that only very strict object-oriented projects based on <code>Typescript</code> introduce the concept of Dependency Injection.Frameworks like <a target="_blank" rel="noopener noreferrer" href="https://angular.io/"><code>Angular</code></a> and <a target="_blank" rel="noopener noreferrer" href="https://nestjs.com/"><code>NestJS</code></a> include dependency injection containers which allow us solving the dependency injection problem in a more than decent way.However, these solutions force us to write a code which feels too coupled with the framework.I have also found what seems to be a pretty popular framework-agnostic dependency injection container, <a target="_blank" rel="noopener noreferrer" href="http://inversify.io/"><code>InversifyJS</code></a>, which again, seems to be more <code>Typescript</code> oriented (despite the fact that it says it&#x27;s for plain javascript as well).From my point of view, the main problem with this one is that its too complex to get it up and running, which makes people get tired of even trying dependency injection.Other than this, I have not seen much adoption in plain javascript projects (maybe more functional than object-oriented), where developers just import or require the dependencies to be used from other modules and use them, which couples the code and makes it really hard to test (at least unit test).<h3>Good dependency injection</h3>I want to start by trying to define how I feel dependency injection should be done, so that it&#x27;s flexible and useful, and not scary to use:<ul><li>We should be able to write code in javascript modules in a way we don&#x27;t care where our dependencies come from, or how are they going to be created. I have seen some attempts in which dependencies are imported or required in the code, but then the module exports some way to override the dependency at runtime (with the intention to do it in tests). This is a sort of working approach, but it feels a little bit hacky. It&#x27;s also always importing external code which could have other side effects.</li><li>We should never be forced to couple our own code with the dependency injection framework. At some point we could want to change the implementation, and that shouldn&#x27;t make us change our code.</li><li>The dependency injection container (which is the object which encapsulates how dependencies are created and injected into objects, DIC from now on), should never leak into our code. It belongs to bootstrapping scripts only. It is also acceptable to get it in code which only purpose is creating objects in the context of dependency injection itself. By importing or requiring the DIC inside modules, we are hiding the module&#x27;s dependencies, and ending up with the ugly service locator anti pattern.</li><li>Configuring the DIC should always be an easy task. We should be able to just provide simple configuration or simple factories for corner cases in which a basic config is not enough. When the DIC is hard to configure or too feature-bloated, people end up with the feeling that dependency injection is not worth it.</li><li>The DIC should be able to recursively resolve dependencies. We shouldn&#x27;t need to know what are the dependencies of our dependencies. When creating an object, it should be enough to just pull its direct dependencies from the container, and it should know how to recursively resolve the rest.</li></ul>With all this in mind, I have created a proof of concept <a target="_blank" rel="noopener noreferrer" href="https://expressjs.com/"><code>express.js</code></a>-based project which makes use of a very simple custom DIC to solve dependency injection.You can find the project <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/dependency-injection-container-js">here</a>.<h3>Explaining the proof of concept</h3>The project is pretty simple. You can just clone it and then run <code>npm install &amp;&amp; node server.js</code>. It will serve the app in <a target="_blank" rel="noopener noreferrer" href="http://localhost:3000">localhost:3000</a>.The default path just displays a &quot;Hello World&quot; message, and it has a second <a target="_blank" rel="noopener noreferrer" href="http://localhost:3000/users">/users</a> path which returns a JSON response where some data is dynamically generated. You can refresh the page in order to see it.The main characteristic of this project is that, if you open any of the files in <code>controllers</code> or <code>services</code>, you&#x27;ll see they don&#x27;t import or require anything.Instead they just expose a builder function (or factory function, or construction function, or whatever you want to name it), which in turn returns the actual object which is going to be used at runtime (remember that, in javascript, functions are first-class objects).Let&#x27;s take the <code>listUsersController</code> as an example:<pre><pre><code class="language-js">const listUsersController = getUsers =&gt; (req, res, next) =&gt; {
 const users = getUsers();
 res.send(JSON.stringify(users));
};

module.exports = listUsersController;
</code></pre></pre>What this module exports is a function where a dependency has to be injected. However, when implementing the controller, we don&#x27;t care where it is going to come from. We just expose the fact that the controller needs that <code>getUsers</code> dependency in order to properly work.When that function is called with the dependency, it will return the actual controller (a simple express middleware).The only place where actual modules are imported is the <code>config/services.js</code> file, which is a simple map where we basically define which are the dependencies for every object.<pre><pre><code class="language-js">const homeControllerFactory = require(&#x27;../controllers/homeController&#x27;);
const listUsersControllerFactory = require(&#x27;../controllers/listUsersController&#x27;);

const utils = require(&#x27;../services/utils&#x27;);
const getUsersFactory = require(&#x27;../services/getUsers&#x27;);

const services = {
 mainController: container =&gt; homeControllerFactory(),
 listUsersController: container =&gt; {
 const getUsers = container.get(&#x27;getUsers&#x27;);
 return listUsersControllerFactory(getUsers);
 },

 utilsService: container =&gt; utils,
 getUsers: container =&gt; {
 const utils = container.get(&#x27;utilsService&#x27;);
 return getUsersFactory(utils);
 }
};

module.exports = services;
</code></pre></pre>This file only defines how the builder/creator/factory functions for every object have to be invoked, by first pulling their direct dependencies from the container, if any.This file is then imported in the <code>server.js</code> file, which is our entry point. Together with the <code>container/container.js</code> file, they make a super basic dependency injection container.Since this DIC is capable of recursively resolving dependencies, our <code>server.js</code> file is the only place where we manually pull objects from the container (the controllers in this case).<pre><pre><code class="language-js">const express = require(&#x27;express&#x27;);
const services = require(&#x27;./config/services&#x27;);
const container = require(&#x27;./container/cotainer&#x27;)(services);

const app = express();
const port = 3000;

// We pull the controllers from the container here
app.get(&#x27;/&#x27;, container.get(&#x27;mainController&#x27;));
app.get(&#x27;/users&#x27;, container.get(&#x27;listUsersController&#x27;));

app.listen(port, () =&gt; console.log(`Example app listening on port ${port}!`));
</code></pre></pre><h3>Evolving the proof of concept</h3>We have defined a pretty simple approach where we have been able to fulfil all the requirements for a good dependency injection.The code is easy to maintain, completely decoupled from the container and from other modules, and registering objects just takes two or three lines of code.The main problem is that we are now forced to maintain a too basic DIC which also lacks some basic features, like lazy loading or avoiding the same service to be created more than once.In order to solve that, we are going to introduce a project dependency on the <a target="_blank" rel="noopener noreferrer" href="https://github.com/young-steveo/bottlejs"><code>BottleJS</code></a> package, which is a dependency injection micro container.Thanks to that package, we can evolve and merge our <code>config/services.js</code> and <code>container/container.js</code> files into a single one which looks like this:<pre><pre><code class="language-js">const Bottle = require(&#x27;bottlejs&#x27;);
 
const homeControllerFactory = require(&#x27;../controllers/homeController&#x27;);
const listUsersControllerFactory = require(&#x27;../controllers/listUsersController&#x27;);

const utils = require(&#x27;../services/utils&#x27;);
const getUsersFactory = require(&#x27;../services/getUsers&#x27;);

const bottle = new Bottle();

bottle.constant(&#x27;utilsService&#x27;, utils);
bottle.serviceFactory(&#x27;getUsers&#x27;, getUsersFactory, &#x27;utilsService&#x27;);

bottle.serviceFactory(&#x27;mainController&#x27;, homeControllerFactory);
bottle.serviceFactory(&#x27;listUsersController&#x27;, listUsersControllerFactory, &#x27;getUsers&#x27;);

module.exports = bottle.container;
</code></pre></pre>It&#x27;s basically the same, but this time, we just need to pass the list of dependencies for every object by its name to a <code>Bottle</code> instance, and then export the bottle container.This container has also a few interesting extra features which might be handy in larger projects.<blockquote>The amount of stars in this package makes me think that maybe my previous conclusion was wrong, and dependency injection is more extended in javascript projects than I thought.</blockquote>You can find the whole proof of concept project using <code>BottleJS</code> in the same repository, under the <code>with-bottle</code> branch, <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/dependency-injection-container-js/tree/with-bottle">here</a>.<h3>Conclusion</h3>My conclusion is that dependency injection in plain javascript and even non object-oriented projects is, not only possible, but desirable.Having a really decoupled codebase, where true <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">dependency inversion</a> is achieved thanks to dependency injection, will make our projects much easier to maintain in the long term, and easier to unit test.<blockquote>I have not covered other scenarios in this article, like front-end application with <a target="_blank" rel="noopener noreferrer" href="https://reactjs.org/">ReactJS</a> or <a target="_blank" rel="noopener noreferrer" href="https://vuejs.org/">VueJS</a>. I will try to do it in the future, when I have become wiser :P</blockquote><blockquote>Update 2018-12-16: I have found an article where <a target="_blank" rel="noopener noreferrer" href="https://medium.com/@magnusjt">Magnus Tovslid</a> proposes a very similar approach but for ReactJS and Redux front-end applications. Take a look at it, it&#x27;s very interesting. <a target="_blank" rel="noopener noreferrer" href="https://medium.com/@magnusjt/inversion-of-control-and-di-in-reactjs-and-redux-35161fcef847">Inversion of Control and DI in Reactjs and Redux</a></blockquote>

Dependency injection in nodejs projects

A couple years ago I wrote a <a href="/2016/06/12/using-service-manager-3-lazy-services-to-improve-your-php-application-performance/">post</a> about how to improve PHP applications performance by using <code>zend-servicemanager</code> lazy services.In that article I explained how the <code>ServiceManager</code> takes advantage of the proxy design pattern to delay the creation of services, when they are marked as lazy.That can improve performance if the object is resource consuming, but that is not the only advantage behind proxies.<h3>A use case</h3>Some days ago I was working on a new feature for <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">Shlink</a>, an open source project I maintain.The feature consisted on adding support to geolocate IP addresses by using <a target="_blank" rel="noopener noreferrer" href="https://dev.maxmind.com/geoip/geoip2/geolite2/">GeoLite2</a>, an IP address database which is updated every month.They have an official <a target="_blank" rel="noopener noreferrer" href="https://github.com/maxmind/GeoIP2-php">PHP library</a>, which exposes a <code>GeoIp2\Database\Reader</code> service where you need to inject the path to the GeoLite2 library file.Since Shlink uses the <code>zend-servicemanager</code> to manage dependency injection, I created a factory like this (these examples are pseudocode):<pre><pre><code class="language-php">&lt;?php
declare(strict_types=1);

namespace Shlinkio\Shlink\Factory;

use GeoIp2\Database\Reader;
use Interop\Container\ContainerInterface;
use Zend\ServiceManager\Factory\FactoryInterface;

class GeoLiteReaderFactory implements FactoryInterface
{
 public function __invoke(ContainerInterface $container, $requestedName, array $options = null)
 {
 $geoLiteDbLocation = $container-&gt;get(&#x27;config&#x27;)[&#x27;geolite2&#x27;][&#x27;db_location&#x27;] ?? &#x27;&#x27;;
 return new Reader($geoLiteDbLocation);
 }
}
</code></pre></pre>And then registered it like this:<pre><pre><code class="language-php">&lt;?php
declare(strict_types=1);

namespace Shlinkio\Shlink\Common;

use GeoIp2\Database\Reader;
use Shlinkio\Shlink\Factory\GeoLiteReaderFactory;

return [

 &#x27;dependencies&#x27; =&gt; [
 &#x27;factories&#x27; =&gt; [
 Reader::class =&gt; GeoLiteReaderFactory::class,
 ],
 ],

];
</code></pre></pre>Then I registered other services which were depending on this one, and everything seemed to be ok.<h3>The issue</h3>My intention was to download the GeoLite2 database during Shlink&#x27;s installation.The problem is that the <code>GeoIp2\Database\Reader</code> service throws an exception if the file you specified does not exist, and the first time Shlink is executed, it does not exist.The way I designed my service graph was making the <code>GeoIp2\Database\Reader</code> object to be instantiated, and throw an error before being able to download the file.I started to think on ways to workaround this issue, but none of them seemed clean enough.Then I thought &quot;maybe I can wrap the object somehow, so that it is not created until it is going to be actually used&quot;, and then I realized that was exactly what a Proxy does, and also, that the <code>ServiceManager</code> includes its own implementation to do this.<h3>The solution</h3>The solution was pretty straightforward. I just had to mark the service as lazy, changing the configuration to this:<pre><pre><code class="language-php">&lt;?php
declare(strict_types=1);

namespace Shlinkio\Shlink\Common;

use GeoIp2\Database\Reader;
use Shlinkio\Shlink\Factory\GeoLiteReaderFactory;
use Zend\ServiceManager\Proxy\LazyServiceFactory;

return [

 &#x27;dependencies&#x27; =&gt; [
 &#x27;factories&#x27; =&gt; [
 Reader::class =&gt; GeoLiteReaderFactory::class,
 ],
 ],
 
 &#x27;delegators&#x27; =&gt; [
 Reader::class =&gt; [
 LazyServiceFactory::class,
 ],
 ],
 
 &#x27;lazy_services&#x27; =&gt; [
 &#x27;class_map&#x27; =&gt; [
 Reader::class =&gt; Reader::class,
 ],
 &#x27;proxies_target_dir&#x27; =&gt; &#x27;data/proxies&#x27;,
 &#x27;proxies_namespace&#x27; =&gt; &#x27;ShlinkProxy&#x27;,
 &#x27;write_proxy_files&#x27; =&gt; true,
 ],

];
</code></pre></pre>And that&#x27;s it. No further code changes were needed, only configuration.With this change, the constructor of the <code>GeoIp2\Database\Reader</code> is no longer run until the object is going to be actually used, which allows Shlink to be installed, and the database file to be downloaded.<blockquote>If you want to more deeply understand what the new config actually means, it is explained in detail in my <a href="/2016/06/12/using-service-manager-3-lazy-services-to-improve-your-php-application-performance/">previous article</a> about lazy services.</blockquote><h3>Conclusion</h3>With this article I just wanted to share how to solve a specific issue introduced by third party code, where you don&#x27;t have the &quot;power&quot; to change the implementation.I also wanted to expose a real use case, which will probably remind you to some real situation in which you have faced a similar problem in the past.

Delay constructor execution by using ServiceManager lazy services

I have written a lot of posts about Zend Framework in general and Zend Expressive in particular, but I have noticed that I have never talked about one of the things that, from my point of view, makes Expressive so game-changing, Interoperability.<h3>Some context</h3>In the past, PHP frameworks used to be very big libraries, which tried to provide solutions to any possible problem in order to retain users.At that time, you had to decide which framework you wanted to use, by weighing pros and cons. People ended up saying &quot;I prefer framework foo, because it has a better templating system&quot;, &quot;ok, but framework bar has a better performance and its dependency injection approach is delightful&quot;.It wasn&#x27;t easy to pick the best part of each framework (or the parts you liked the most) and be able to use them in the same project. You couldn&#x27;t just take the templating system from foo and make it work with the dependency injection approach from bar.Because of this, the <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/">PHP Framework Interop Group</a> was born (PHP-FIG from now on). Its goal was to define standard recommendations that frameworks and libraries could adhere to.At some point this would mean that frameworks would use compatible APIs in their different components and people would be able to easily change one specific component by another.<h3>Expressive&#x27;s interoperability</h3>Mostly all frameworks which are part of the PHP-FIG have more or less adopted these standard recommendations. If you have been working with PHP in the last 3-4 years, you know it is now much easier to &quot;mix&quot; libraries and frameworks in the same project.However, the one which has been capable of making interoperability part of its essence is <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-expressive/">Zend Expressive</a> (probably because it was born in that period and interoperability was one of its main design goals).Expressive itself does not provide solutions to almost anything. Instead, it relies in a series of interfaces and abstractions in order to get everything working.Some of those abstractions are PHP Standard Recommendations (PSRs) from the PHP-FIG.<ul><li>It is a middleware-based framework, so it expects your middlewares and actions (request handlers) to implement <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-15/">PSR-15</a> interfaces. This way, your own code is not coupled with Expressive at all.</li><li>In order to dispatch this middlewares they use a middleware dispatcher, <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-stratigility/">Zend Stratigility</a> in this case, which also implements PSR-15&#x27;s request handler interface.</li><li>It is also a web framework, so it needs to dispatch HTTP requests. For this, it expects you to include a library implementing <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-7/">PSR-7</a>. They provide one implementation, <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-diactoros/">Zend Diactoros</a>, but you can use whatever you want.</li><li>Expressive promotes Dependency Injection, so it expects a dependency injection container to be used. Once again, they expect any container implementing <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-11/">PSR-11</a>, so you can use the container of your choice and everything will work.</li></ul>The PHP-FIG still has some work to do in different areas. For those cases in which there&#x27;s no standard recommendation, Zend&#x27;s team has defined a few own interfaces and abstractions Expressive relies on in order to ease decoupling and extendability.<ul><li>Routing: You can pick one of provided implementations or define your own (<a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/expressive-slim-router">I&#x27;ve done it</a> and it wasn&#x27;t that hard)</li><li>Templating: If you need to render templates, expressive has official support for three templating systems, but as in previous case, you can use other engines with a minimal effort. 
However, you are not forced to use a templating system if you don&#x27;t need it, which is great.</li><li>Error handling: You shouldn&#x27;t handle errors the same way in production and development environments. Expressive&#x27;s abstractions let you use different approaches, with great integration to use <a target="_blank" rel="noopener noreferrer" href="http://filp.github.io/whoops/">Whoops</a> in development, but you could use something like <a target="_blank" rel="noopener noreferrer" href="https://tracy.nette.org/en/">Tracy</a> too.</li></ul>All of these abstractions are defined in separated packages, so in the future, if the PHP-FIG accepts a PSR for any of these things, Zend Expressive will probably start supporting them and drop these packages&#x27; interfaces as far as possible.<h3>Expressive&#x27;s extendability</h3>There are a lot of things a real project needs which have been left out of expressive&#x27;s scope, which is in fact good.You will probably need some way to handle configuration. Expressive doesn&#x27;t tell you what to use, because usually config is consumed before the request is even dispatched, so there&#x27;s no need for any kind of integration.You will also need something to connect to a database, or consume external services. Once again, use whatever you want.If you need to be able to run your app from the command line, expressive and the HTTP layer won&#x27;t even come in place. For that purpose, Zend&#x27;s team recommends using <a target="_blank" rel="noopener noreferrer" href="http://symfony.com/doc/current/components/console.html">symfony/console</a> these days. In fact, there are a few CLI utilities for expressive which have been built with that component, like <a target="_blank" rel="noopener noreferrer" href="https://github.com/zendframework/zend-expressive-tooling">zend-expressive-tooling</a>.Of course, all of this flexibility is great for experienced developers, but newcomers need a little bit of help to begin with. That&#x27;s why a <a target="_blank" rel="noopener noreferrer" href="https://github.com/zendframework/zend-expressive-skeleton">skeleton project</a> was created.The skeleton is installed using an interactive process which let&#x27;s you choose the different components to be used, and the project approach you want to take.This wide range focus makes expressive fit any project. Also, despite the fact that it can be considered a microframework, I can tell you it <a href="/2016/07/21/project-scalability-with-zend-expressive/">scales quite well</a>.<h3>Proof of concept</h3>Ok, perfect, interoperability runs in expressive veins, but I don&#x27;t want you to just take my word. That&#x27;s why I have built an example project where I use a variety of libraries and components for every task.In order to build the project, I have used the skeleton mentioned before, and then tweaked it a little bit. You can find it here: <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/expressive-interoperability-proof-of-concept">https://github.com/acelaya-blog/expressive-interoperability-proof-of-concept</a> These are the components I have used:<ul><li>DI Container: <a target="_blank" rel="noopener noreferrer" href="http://php-di.org/">PHP-DI</a></li><li>Router: <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/expressive-slim-router">Slim framework v2</a></li><li>Template renderer: <a target="_blank" rel="noopener noreferrer" href="https://twig.symfony.com/">Twig</a></li><li>HTTP abstraction: <a target="_blank" rel="noopener noreferrer" href="https://github.com/guzzle/psr7">Guzzle PSR-7 HTTP message library</a></li><li>Error handling: <a target="_blank" rel="noopener noreferrer" href="http://filp.github.io/whoops/">Whoops</a></li></ul>The result is that very few of the code is actually from <code>Zend</code>&#x27;s namespace.The project includes the instructions to run it and see how everything is in fact working. It also includes one option in the menu to see an error page.<h3>Conclusion</h3>From my point of view, full-stack and coupled frameworks are something from the past. I like to be free to choose the tools I want to use in a project for every task, and to be able to use the best ones or the ones I like the most.For me, it makes no sense these days to ask &quot;What PHP framework do you use?&quot;, because thanks to the PHP-FIG and initiatives like Expressive it&#x27;s more and more hard to answer.Of course, if the full-stack approach is the one that fits you and makes you be more productive, I&#x27;m nobody to say otherwise :-)

Demonstrating the interoperability and decoupling of Zend Expressive

There&#x27;s no doubt that having tests in a project allows you to find potential bugs earlier and more easily.Lots of OSS projects require a minimum code coverage in order to accept new pull requests from contributors, and proprietary projects also tend to have some sort of continuous integration workflow which requires certain metrics to be fulfilled in order to get builds passing.However, the code coverage can lead to a false sense of security, which makes you think that if certain class has a 100% code coverage, it is also 100% bug-free.This is not always true, since you could be calling a method and yet not being properly testing its output or its real behavior. The code coverage will mark it as covered, but you might introduce a bug and still have a green test.This is where mutation testing comes in.<h3>Mutation testing</h3>According to <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Mutation_testing">wikipedia</a>, mutation testing involves applying small modifications to a software. These modifications are called mutations.Then, if you pass your tests and any of them fails because of this &quot;mutation&quot;, then you have &quot;killed the mutant&quot;. This is what should happen in code which is covered by tests. If tests keep passing after applying this modification, then your tests covering that part of the project are not so good, and you should improve them to really test what they are covering.So in other words, mutation testing is a quality assurance practice for your tests.In PHP projects, there used to be one library which purpose was working with the mutation testing approach, <a target="_blank" rel="noopener noreferrer" href="https://github.com/humbug/humbug">humbug/humbug</a>. However, it is mostly abandoned now, and it has motivated the appearance of another project, <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/">infection/infection</a>.<h3>Infection</h3>Infection is a mutation testing framework for PHP.It is easy to start using it in your project, since you just need to install it (<code>composer require infection/infection --dev</code>), and create a configuration file. Then, it uses your existing test suite in order to apply mutations to your code and check your tests.It currently integrates with <a target="_blank" rel="noopener noreferrer" href="https://github.com/sebastianbergmann/phpunit/">phpunit</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/phpspec/phpspec">phpspec</a>, but I suppose there will be compatibility with other testing frameworks in the future.<h4>Configuring infection</h4>It consumes an <code>infection.json</code> configuration file which could have a structure like this:<pre><pre><code class="language-json">{
 &quot;source&quot;: {
 &quot;directories&quot;: [
 &quot;module/*/src&quot;
 ]
 },
 &quot;timeout&quot;: 10,
 &quot;logs&quot;: {
 &quot;text&quot;: &quot;build/infection/infection-log.txt&quot;,
 &quot;summary&quot;: &quot;build/infection/summary-log.txt&quot;,
 &quot;debug&quot;: &quot;build/infection/debug-log.txt&quot;
 },
 &quot;tmpDir&quot;: &quot;build/infection/temp&quot;,
 &quot;phpUnit&quot;: {
 &quot;configDir&quot;: &quot;.&quot;
 }
}
</code></pre></pre>This is the configuration of one of my company&#x27;s projects. A modular application where every module is located inside the <code>module</code> directory and has its own <code>src</code> folder with source files.This config file basically defines where the sources are located, where infection&#x27;s logs should be placed, a temporary folder which will be used instead of using system&#x27;s default tmp folder, and where is the <code>phpunit.xml[.dist]</code> file.That&#x27;s all you need to get infection working.<blockquote>You can find detailed information of all the configuration options <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Configuration">here</a></blockquote><h4>Running infection</h4>Now that we have the configuration file, we just need to run <code>vendor/bin/infection</code>, and it will execute our test suite and then apply mutations to our source files.Then it will display the result, telling how many mutants have been killed.Infection supports lots of mutators, like replacing <code>++</code> by <code>--</code>, <code>true</code> by <code>false</code>, changing method visibilities, return values, etc. You can find the whole list <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/mutators.html">here</a>.The command line tool supports a list of modifiers, some of which are very interesting. This is how I would recommend you to run it if you want to address the whole project.<code>vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2</code>.<ul><li><code>--threads</code>: It runs tests and mutators in parallel processes, so you can take advantage of all the cores in your machine. Set a value that makes sense your you. If you have 8 cores, maybe you want to set it with value 8.</li><li><code>--min-msi</code>: This flag makes the process fail if the number of killed mutants is below certain percentage. In this case we require 70% of mutants to be killed. Adapt the value to your needs, and try to gradually improve your tests and increase this value (MSI stands for Mutation Score Indicator). This flag is really useful in continuous integration environments.</li><li><code>--only-covered</code>: This makes infection address only covered code. We know mutants won&#x27;t be killed in uncovered code, so I think it makes no sense to apply mutators there. I prefer to separate the required code coverage metric from the required mutation score indicator metric.</li><li><code>--log-verbosity</code>: By default, infection logs all the mutations it has applied to your code. This could lead to a log with thousands of lines if the project is big, so I prefer to set the log verbosity to 2, which makes it log only non-killed mutants.</li></ul>And that&#x27;s mainly it. You can now automate this task in your continuous integration pipeline, or just run it locally, but at least you will discover some parts both in your source files and your tests that can be improved.<h3>Troubleshooting</h3>While this could be enough for an introduction to infection, the title of this article says something about big projects, so I&#x27;m going to explain how we integrated infection in one of my company&#x27;s biggest projects and how we addressed some inherent problems.<h4>Already unstable</h4>The first problem while working with infection is that, at the moment of writing this article, it&#x27;s not in a stable version yet. That means you could find bugs.We found one in which infection wasn&#x27;t properly parsing our phpunit config file and it was excluding almost the whole project from the code coverage report. This caused all mutations to be skipped.We reported it and they fixed it very fast, but they haven&#x27;t tagged a version including the fix yet, so we require one specific commit from master while we wait for v0.8 to be released.<h4>Time consuming</h4>Another problem is that infection consumes a lot of memory and CPU, and takes a lot of time to be run when you have a lot of classes and tests.For example, our project has around 1500 unit tests, which take about 1&#x27;5min-2min to be run. Our build process also passes some coding style checks and functional repository tests. It usually takes about 3 minutes.The first time we run infection for the whole project, it took more than 16 minutes. That would make a build which is taking 3 minutes to take almost 20 minutes from now on, which is not an option because our build is run a lot of times every day.Our solution was making the build run infection only on changed sources, taking advantage of infection&#x27;s <code>--filter</code> flag, which lets you pass a comma-separated list of files and it gets applied only to them.The way in which we find which files have changed from previous build is by making a <code>git diff</code> between a commit from previous successful build and the new commit after updating the branch in which the build is being run (the commit identifiers are defined by jenkins&#x27; GIT plugin as environment variables).If it is the first time the build is run for this branch, we make the diff with <code>origin/develop</code> instead.The result is a bash script like this:<pre><pre><code class="language-bash"># Get files which have changed from latest processed commit, including only those inside sources
# If there&#x27;s no previous commit, diff with develop
INFECTION_FILTER=$(git diff ${GIT_PREVIOUS_SUCCESSFUL_COMMIT:-origin/develop} $GIT_COMMIT --name-only | grep /src/ | paste -sd &quot;,&quot; -)

# Check mutations over those files, if any, and require a 70% MSI
if [ -n &quot;$INFECTION_FILTER&quot; ]; then
 vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
else
 echo &quot;No source files affected. Infection skipped.&quot;
fi
</code></pre></pre>As you can see, we perform a diff by including only file names, then we exclude those which do not contain &quot;/src/&quot; as part of the name, and then we convert them into a comma-separated list.With this approach, the build usually takes 1 extra minute in the worst case scenario, which can be assumed.However, I&#x27;m sure the performance of this project will be improved as it grows and gets more stable.<h4>Test run duplication</h4>While previous approach filters the files where mutations are applied, it is still running all tests beforehand, which makes all tests to be run twice on every build.<ul><li>First, our own execution, which generates a code coverage report to be published later, and makes the build fail if tests do not pass.</li><li>Then, infection runs tests again, on its own &quot;conditions&quot; (they take your phpunit config file and generate a new one, which has some custom configuration entries).</li></ul><strike>If it were possible to pass infection an already generated code coverage, we could prevent this duplication, but, as far as I know, it does not support that.</strike> **It does now**.Another solution would be being able to run only tests which affect changed sources, instead of running all tests suites, but phpunit does not allow to filter by a list of files, only a specific file or folder.<strike>This is the only problem for which we have not yet found a solution, so If you know any workaround, a comment will be very welcome :-)</strike>Update 2018-02-22Following what Maks suggested in this <a href="/#comment-3762900802">comment</a>, we have changed the process a little bit, and using a more recent commit from <code>dev-master</code>, we can take advantage of a feature which allows an existing code coverage to be passed to infection.Now, our phpunit execution includes these two flags: <code>--coverage-xml=build/coverage-xml --log-junit=build/phpunit.junit.xml</code>And then, infection is executed with <code>--coverage=build</code>.That&#x27;s it, no need to run tests twice anymore :-)<h4>Xdebug vs phpdbg</h4>Since infection needs access to a code coverage report, you need some tool which is capable of generating it.People usually use xdebug, since it is the most extended and feature-rich debugging tool.However, there&#x27;s a simpler tool, which is designed to debug php console executions only, and comes bundled with php. It is <a target="_blank" rel="noopener noreferrer" href="https://github.com/krakjoe/phpdbg">phpdbg</a>.In our project, it usually takes 30%-50% less time to generate a code coverage report than xdebug. The result is not 100% the same, but it is very similar, and it&#x27;s worth it.So, instead of running infection like this:<pre><pre><code>vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
</code></pre></pre>We have to do it like this<pre><pre><code>phpdbg -qrr vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
</code></pre></pre>It is explained in <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Running-with-phpdbg">infection&#x27;s docs</a>.We have even started to use it to run our own phpunit execution.<h3>Conclusion</h3>We have seen how to use infection, and how to integrate it in a continuous integration pipeline, even for big projects.I think it is a very useful tool, and the team behind it is doing a great job. It is also not hard to start using it, since you don&#x27;t really need to write new code, just add a config file and run the CLI tool.Finally, I recommend you to read this article, from infection&#x27;s author. It explains how it internally works, and how to use it in a development workflow <a target="_blank" rel="noopener noreferrer" href="https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1">https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1</a>.

Alejandro Celaya | Blog

Software development, agile methodologies and open source projects.

Considerations to properly use open source software projects

Dependency injection in nodejs projects

Delay constructor execution by using ServiceManager lazy services

Demonstrating the interoperability and decoupling of Zend Expressive

Mutation testing with infection in big PHP projects