I remember many years ago, when I released my first open source project, that every time I had a new release ready, I was never sure what should be the new version number.Versioning software is not as straightforward as it might look when you don&#x27;t have a lot of experience.In this article I&#x27;ll try to explain the <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">SemVer</a> approach to version software, and some things you might want to take into account on top of that.<h3>Semantic Versioning</h3>The first thing you need is a solid set of rules that help you decide which should your next version be.I remember thinking &quot;is this version big enough as to jump from 1.0 to 2.0?&quot; or &quot;I have seen projects using three numbers for the version, X.Y.Z, should I do the same?&quot;Those questions are easier to answer if you stick to some standard that tells you what to do, and that will help others more easily understand your intention.My recommendation is to use <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">Semantic Versioning</a>, as it defines a solid starting point, and it&#x27;s quite adopted by the community.Semantic Versioning states that you should use three numbers to identify versions, <code>MAJOR.MINOR.PATCH</code>, and follow these rules in order to decide which one to increase:<ul><li><code>PATCH</code>: the new release includes only bug fixes. People are encouraged to update to this release and no changes are needed in their code bases.</li><li><code>MINOR</code>: the new release includes new backwards-compatible features. People can update to this version without changing their code bases, and make use of the new features afterwards if they wish.</li><li><code>MAJOR</code>: the new release includes <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Backward_compatibility">backward compatibility</a> breaks. Changes in the code base will be needed when updating to this version.</li></ul><blockquote>These rules are easier to apply to libraries and frameworks than it is for apps and services. More about this later.</blockquote><h3>Try to keep backwards-compatibility</h3>Ok, we have some rules now. However, I have seen very frequently people mistaking what can be done with what should be done.Yes, if you introduce a breaking change, you can update the major version and that&#x27;s it. However, if you do it too frequently, people won&#x27;t be happy, and you will end up with a fragmented user base, that stop updating to newer versions.I remember using a library to deploy stuff that had a new major version every couple of months, changing things like &quot;The <code>host</code> function is now called <code>server</code>&quot;. Yes, ok, but why do I have to change my stuff so frequently.It&#x27;s usually better to try to implement things in a backwards-compatible way, and keep track of the things you want to change and do it all at a time.You can also deprecate things, and document the way you want those things to be done moving forward. That way you give people the chance to start changing their code and simplify a hypothetical upcoming major release.<h3>Provide clear upgrade paths</h3>As mentioned above, too frequent major version changes are not desirable, but sooner or later you will want to release a new major version for different reasons.<ul><li>Getting rid of that feature which is hard to maintain, introduces lots of bugs, and provides little value.</li><li>Supporting something new which is very hard to implement without breaking backwards compatibility.</li><li>Removing all those deprecated methods you have been dragging for months.</li><li>Fix design issues, where something was architected in the wrong way, and only spotted after some time.</li></ul>At the time this happens you will want your users to upgrade to the newer version, as you don&#x27;t want people to report bugs that only affect older versions, or to expect you to backport fixes to older versions because they haven&#x27;t updated to the new one.The best way to achieve this is to provide clear and simple upgrade paths:<ul><li>Prepare your releases for the things to come: If you want to introduce new ways of doing things, do it before the major release, and deprecate the older ones. For example, you can have a new method that gets called by an older one, while you mark the old one as deprecated.People will be more willing to update and change their code afterwards. By the time the major release is published, their code will be ready to upgrade.</li><li>Include a CHANGELOG file in your project, explaining the changes introduced in every version. A very good way to document a change log is the <a target="_blank" rel="noopener noreferrer" href="https://keepachangelog.com/">Keep a changelog</a> initiative.You can also use some other platform to document the change log, like GitHub releases.</li><li>Include documentation on how to upgrade to a major release: This is probably the most important one. Document all the steps that need to be followed and all the required changes to update from a release to the next major one.A good way to do this is by including an UPGRADE file in your project. You can see <a target="_blank" rel="noopener noreferrer" href="https://github.com/acmailer/acmailer/blob/main/UPGRADE.md">this example</a>.</li></ul><h3>Versioning libraries vs apps</h3>Semantic Versioning is a good standard to version software, but it plays much better with libraries and frameworks that are installed using some kind of package manager, than it does with apps and services that are not directly depended on.For example, how do you identify a backward compatibility break on a web user interface?With this kind of software there are more grey areas, and you will end up using your intuition.For example, not long ago I released a <a href="/2020/12/24/released-shlink-web-client-v3-0-0/">new major version</a> for a web app I maintain, and it was completely compatible with the previous one. However, it introduced a larger amount of features than usual, and big UI changes, so I decided it was worth the bump.Because of this there are projects that use different approaches to version apps. For example:<ul><li>Firefox and Chrome have 6-week cycles, and after each one of them they release a new major version no matter what.</li><li>Jetbrains publishes a few releases of their IDEs every year (I think 3) which always start with the year they were released on, like <code>2020.3.0</code>, and they increase the patch number for bugfixes.</li><li>Ubuntu releases 2 versions every year, one in April and one in October, and they are always identified by the year and month they were released on, like <code>19.04</code> or <code>20.10</code>.</li></ul><h3>Special versions</h3>There&#x27;s also a set of special versions that can be used to publish the so-called pre-release versions. These <a target="_blank" rel="noopener noreferrer" href="https://semver.org/#spec-item-9">are covered</a> by Semantic Versioning, but sometimes they are not so easy to understand.It is also very likely that you never have to use these, as only relatively big projects will need this level of granularity, but it&#x27;s still good to know what&#x27;s their purpose and what do they imply.<h4>0.x.y</h4>It is possible to release a version of your software where the major number is <code>0</code>. When this happens, it means your software is still in its early stages.It does not mean it&#x27;s necessarily unstable, but it means you are still introducing frequent changes on it, and therefore, it might be a bit more fragile than a version starting with a number greater than <code>0</code>.<h4>Alpha</h4>The intention of an alpha release is that a subset of people con test some new functionality, but you don&#x27;t want people to use it in productionAlpha versions are on their very early stages. Their public API is very likely to change, and they probably contain bugs.Semantic Versioning states that alpha versions are identified by this pattern: <code>MAJOR.MINOR.PATCH-alpha.NUMBER</code>, for example <code>2.0.0-alpha.1</code>.<h4>Beta</h4>Similar to alpha versions, beta versions are also prone to contain bugs, but their API will in theory not change. That means that you could potentially adapt your software to start using a beta version, and when the corresponding stable release is published, you shouldn&#x27;t need that many changes.Semantic Versioning defines a pattern similar to the one for alpha versions: <code>MAJOR.MINOR.PATCH-beta.NUMBER</code>, for example <code>2.0.0-beta.1</code>.<h4>Release candidate</h4>Release candidates (RC) are pre-release versions that you will want to publish for the general user-base to test an upcoming release. You want to publish a RC so that it&#x27;s battle tested by as many people as possible, in order to find possible bugs and edge cases.They should not contain many bugs, but some could be found. The public API should not change anymore.According to Semantic Versioning, the pattern to identify a RC is <code>MAJOR.MINOR.PATCH-rc.NUMBER</code>, for example <code>2.0.0-rc.1</code>.<h3>Conclusion</h3>We have seen some approaches and considerations to properly version your software.Versioning is not so straightforward when you don&#x27;t have a lot of experience, but as you have seen, you just need to know a small set of rules and use a bit your intuition.With those tools, you will be able to properly version your software.

What to take into account when versioning software

Two days ago I released the first major version in almost 2 years for <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlink-web-client</a>, a web UI for <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">shlink</a>, my self-hosted URL shortener.This new version introduces several improvements to the project, and I thought it was worth writing a post to explain all of them in depth.<h3>New design</h3>The project has been using the same design almost since the beginning. It was clean, but it laked a bit of contrast and consistency in many sections.Now the main background is light-gray, with components composed in white cards with a slight box shadow, that makes them easier to spot inside the app.Also, the main area has now a max-width that prevents components from stretching too much on wide screens.<div class="row"><div class="col-md-6 "><img alt="Before" src="/assets/img/shlink-web-client-3/design-before.png" class="gallery-item"/>Before</div><div class="col-md-6 "><img alt="After" src="/assets/img/shlink-web-client-3/design-after.png" class="gallery-item"/>After</div></div><h3>Overview page</h3>Up until now, after connecting to a Shlink server, you went directly to the list of short URLs.With this new version, you now see an overview page first, which displays some general stats, a quick short URL creation form, and the latest 5 short URLs.From there, you can navigate to the full list of short URLs, with filtering capabilities, or go to the advanced short URL creation form.<div class="row"><div class="col-md-8 col-md-offset-2 "><img alt="Overview" src="/assets/img/shlink-web-client-3/overview.png" class="gallery-item"/>Overview</div></div><h3>Improved short URL creation form</h3>One of the sections with more room for improvement was the short URL creation form. It had all the inputs mixed, displaying just the &quot;long url&quot; one, with the option to un-collapsing the rest at will.The new design groups the inputs by context, which makes it more clear. It also displays the inputs all the time, since the &quot;quick form&quot; is now located in the overview page.<div class="row"><div class="col-md-6 "><img alt="Before" src="/assets/img/shlink-web-client-3/creation-form-before.png" class="gallery-item"/>Before</div><div class="col-md-6 "><img alt="After" src="/assets/img/shlink-web-client-3/creation-form-after.png" class="gallery-item"/>After</div></div><h3>Enhanced visits page</h3>The visits page had a problem, it was trying to display too much information in a single place.<ul><li>Visits over time</li><li>Visits from every platform</li><li>Visits from every location</li><li>A list of visits as a table</li></ul>The new approach splits those in subsections with their own sub-route each (<code>/by-location</code>, <code>/by-context</code>, etc). This helps to render a smaller amount of charts at once, since they are a bit CPU greedy.Also, only the visits from the last 30 days are loaded by default, making it load way faster for URLs with a lot of visits.<div class="row"><div class="col-md-6 "><img alt="Before" src="/assets/img/shlink-web-client-3/visits-before.png" class="gallery-item"/>Before</div><div class="col-md-6 "><img alt="After" src="/assets/img/shlink-web-client-3/visits-after.png" class="gallery-item"/>After</div></div><h3>Outer sections</h3>The app has some sections which are not related to a specific server, like creating/editing servers, or the settings page.These sections have also been adapted to the new style, and they also have a max-width for wide screens.<div class="row"><div class="col-md-6 "><img alt="Before" src="/assets/img/shlink-web-client-3/create-server-before.png" class="gallery-item"/>Before</div><div class="col-md-6 "><img alt="After" src="/assets/img/shlink-web-client-3/create-server-after.png" class="gallery-item"/>After</div></div><h3>Date range selector</h3>A new component allows selecting date ranges either by relative time ranges (last 30 days, last 90 days...) or by selecting absolute start and end dates.This component is used to filter both short URL lists and visits lists.<div class="row"><div class="col-md-6 col-md-offset-3"><img alt="Date range selector" src="/assets/img/shlink-web-client-3/date-range-selector.png" class=""/>Date range selector</div></div><h3>Domain selector</h3>Shlink has had multi-domain support for some versions, but changing the domain from the web client required to set the value manually, which is a bit error-prone.This version introduces a new component which allows selecting from domains already used in the past (which is what you will want in most of the cases), preventing accidental typos.It also allows setting new values in case you want to use a new domain that you never used before. This domain will then appear in the list the next time.<div class="row"><div class="col-md-6 "><img alt="Existing domains" src="/assets/img/shlink-web-client-3/domains-existing.png" class=""/>Existing domains</div><div class="col-md-6 "><img alt="New domain" src="/assets/img/shlink-web-client-3/domains-new.png" class=""/>New domain</div></div><h3>Other improvements</h3>The changes described so far were the major ones. However, v3.0.0 includes a few other minor improvements.<ul><li>Result and loading messages have been standardized: They are now more consistent and use the same set of components, which also ensures consistency in the future.</li><li>Error messages are now more descriptive: Instead of showing generic error messages when API calls fail, the actual error message is now always displayed, so that it&#x27;s easier to know what failed.</li><li>Dropped support for Shlink v1: All major versions come with some breaking changes that improve code maintainability. This one drops Shlink v1 support, which allows removing some conditional paths here and there.</li></ul>Other than the last point, Shlink v3.0.0 is mostly backwards compatible, so it&#x27;s safe to upgrade if you are already using Shlink v2.x<h3>Next steps</h3>Of course this doesn&#x27;t end here, and I have a lot of ideas to improve it even further.<ul><li>Dark theme: It&#x27;s what the cool kids do these days, and I have already done some experiments around it.</li><li>Shlink dashboard: Next gen web app for Shlink. You can read the whole reasoning in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client/issues/338">this issue</a>.</li><li>Filtering in lists added to URL: Many sections allow filtering lists, but the state is saved in memory, which means it gets lost if the page is refreshed, and it cannot be bookmarked. The intention is to make all filtering params to become part of the URL.</li><li>Exporting visits in CSV: It&#x27;s cool to see charts full of info, but sometimes admins need to feed other services with the data, and consuming the original API is not always an option. Exporting visits to a CSV file will be supported soon by shlink-web-client.</li></ul>

Released shlink-web-client v3.0.0

Let&#x27;s not deny that <a target="_blank" rel="noopener noreferrer" href="https://github.com/features/actions">GitHub actions</a> are hitting hard in the tech community, and many projects are transitioning from other continuous integration systems like <a target="_blank" rel="noopener noreferrer" href="https://circleci.com/">Circle CI</a> or <a target="_blank" rel="noopener noreferrer" href="https://travis-ci.org/">Travis CI</a>.Personally I have been using Travis a lot during the past years, for all my OSS projects, and I have gotten very used to it.However, people seem to be &quot;abandoning&quot; travis in favor of GitHub Actions, for different reasons. &quot;It&#x27;s faster&quot;, &quot;I can run smaller tasks in parallel&quot;, etc.In my case, I have moved a few very specific tasks there, where travis was becoming a bottleneck, but I still have mixed feelings about the very-verbose yaml-based config needed to configure a few GitHub Actions, compared to the more concise travis one.In this article I&#x27;ll explain how I improved a travis build to run several smaller tasks in parallel jobs, without having to completely move away from it.<h3>Initial state</h3>This project&#x27;s travis config is relatively simple. It is used to build a JS project for the web browser, and it has to run the next tasks:<ul><li>Lint the code, to make sure it fulfils the project&#x27;s coding standards.</li><li>Run unit tests.</li><li>Run mutation tests.</li><li>Build the docker image, to make sure none of the new changes broke it.</li><li>Conditionally publish a release on GitHub, with a dist file attached to it, if the build is being run for a git tag.</li></ul><pre><pre><code class="language-yaml">dist: bionic

language: node_js

branches:
 only:
 - /.*/

cache:
 directories:
 - node_modules

services:
 - docker

node_js:
 - &#x27;12.16.3&#x27;

install: npm ci

before_script:
 - echo &quot;Building commit range ${TRAVIS_COMMIT_RANGE}&quot;
 - export MUTATION_FILES=$(git diff ${TRAVIS_COMMIT_RANGE:-origin/main} --name-only | grep -E &#x27;src\/(.*).(ts|tsx)$&#x27; | paste -sd &quot;,&quot;)

script:
 - npm run lint
 - npm run test:ci
 - npm run mutate:ci
 - docker build -t project:test .

after_success:
 - node_modules/.bin/ocular coverage/clover.xml

before_deploy: npm run build ${TRAVIS_TAG#?}

deploy:
 - provider: releases
 api_key:
 secure: &lt;key&gt;
 file: &quot;./dist/project_${TRAVIS_TAG#?}_dist.zip&quot;
 skip_cleanup: true
 on:
 tags: true
</code></pre></pre><h3>Problems with this approach</h3>This is more or less ok, but the last two commands in <code>script</code> can be relatively slow and can delay the whole process.Also, specially the mutation tests one, which is currently always considered successful regardless the result, can make the build reach the maximum run time allowed by travis, making the whole build falsely fail.<blockquote>The reason for the <code>before_script</code> step is to determine which source code files changed, and run mutation tests on them only, in order to mitigate this.</blockquote>Other than this, the build config defines &quot;global&quot; customizations that are actually only used for some tasks:<ul><li>Running <code>npm ci</code> is not required for the <code>docker build</code> task.</li><li>Getting the <code>docker</code> service is not required by any of the other tasks.</li><li>Determining changed files is only used by the <code>mutation tests</code> task.</li></ul><h3>Making use of jobs</h3>One cool feature travis supports is the use of a job <a target="_blank" rel="noopener noreferrer" href="https://docs.travis-ci.com/user/build-matrix/">build matrix</a>, that lets you define multiple jobs that run in parallel as part of the same build, and make each one of them do unrelated tasks.Using this feature it is possible to improve the <code>.travis.yml</code> file, making it look like this:<pre><pre><code class="language-yaml">dist: bionic

language: node_js

branches:
 only:
 - /.*/

cache:
 directories:
 - node_modules

node_js:
 - &#x27;12.16.3&#x27;

jobs:
 fast_finish: true
 allow_failures:
 - name: &#x27;Mutation tests&#x27;
 include:

 - name: &#x27;Lint&#x27;
 install: npm ci
 script: npm run lint

 - name: &#x27;Unit tests&#x27;
 install: npm ci
 script: npm run test:ci
 after_success:
 - node_modules/.bin/ocular coverage/clover.xml

 - name: &#x27;Mutation tests&#x27;
 install: npm ci
 before_script:
 - echo &quot;Building commit range ${TRAVIS_COMMIT_RANGE}&quot;
 - export MUTATION_FILES=$(git diff ${TRAVIS_COMMIT_RANGE:-origin/main} --name-only | grep -E &#x27;src\/(.*).(ts|tsx)$&#x27; | paste -sd &quot;,&quot;)
 script: npm run mutate:ci

 - name: &#x27;Build docker image&#x27;
 services:
 - docker
 install: skip
 script: docker build -t project:test .

 - name: &#x27;Publish release&#x27;
 if: tag IS present
 install: skip
 script: skip
 before_deploy: npm run build ${TRAVIS_TAG#?}
 deploy:
 - provider: releases
 api_key:
 secure: &lt;key&gt;
 file: &quot;./dist/project_${TRAVIS_TAG#?}_dist.zip&quot;
 skip_cleanup: true
 on:
 tags: true
</code></pre></pre>This approach introduces all these benefits:<ul><li>Now linting, testing, mutation testing and building the docker image run in parallel, making the build finish earlier.</li><li>The mutation testing step is allowed to fail, and thanks to the <code>fast_finish</code> flag, travis won&#x27;t even wait for it to finish, considering the build successful once the rest have passed.</li><li>Configs that are only meaningful for certain jobs are defined in the job itself, instead of globally.</li><li>The &quot;publish release&quot; job only runs conditionally if the build is for a tag, thanks to the <code>if: tag IS present</code> condition. (More info about <a target="_blank" rel="noopener noreferrer" href="https://docs.travis-ci.com/user/conditional-builds-stages-jobs/">conditional jobs</a> in travis).</li><li>As soon as any of the jobs fails, the whole build fails and you don&#x27;t have to continue waiting. Before this, if the third task failed, you still had to wait for the previous two to finish.</li><li>Each job can have a name, making it more clear in travis UI what tasks are being run:</li></ul><img src="/assets/img/travis-multiple-jobs/travis-jobs.png" alt="Travis CI jobs"/><h3>Conclusion</h3>The main purpose of this article was to showcase that travis still has a lot to give. Github Actions are definitely cool, but there&#x27;s usually no silver bullet for everything, and a combination with other tools might be more suitable.Also, this allows people using travis to improve their builds without having to change everything. Then, once you have split your tasks, if you still want to move to Github Actions, it should be easier.<blockquote>If you are curious, this is the project in which I use this travis configuration: <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlinkio/shlink-web-client</a>.</blockquote>

Running several steps of a Travis CI build in parallel

Asynchronous and non-blocking runtimes are pretty usual in many programming languages, as well as long-lived web apps that stay in memory and are capable of dispatching multiple HTTP requests without having to be fully bootstrapped every time. This has not been traditionally the case with PHP apps. However, many projects are starting to get some adoption, that bring this long-lived approach to the PHP world.The problem with this is that PHP developers are not used to this, and they tend to continue acting as they have always done. Also, libraries might have made some assumptions, making them not work as expected when used under this context.Some of these projects are <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk">swoole</a>, <a target="_blank" rel="noopener noreferrer" href="https://reactphp.org/">ReactPHP</a> or <a target="_blank" rel="noopener noreferrer" href="https://amphp.org/">Amphp</a>, to give some examples.In this article, I will be focusing on the first one, and explain how to approach some of the pain points I have found after using it for a year and a half on an <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink">existing PHP project</a>.<h3>Introducing Swoole</h3>The main difference swoole has compared to other similar projects is that it&#x27;s written as a native PHP extension, and needs to be installed with <code>pecl</code>.This is obviously a bit cumbersome, but the benefit is that, since it&#x27;s written in C++, it has a better handling of memory allocation, which is very important to achieve what it tries to do.What swoole does is running a main process which bootstraps the PHP application once, and then keeps it in memory so that it can continue dispatching requests without having to load resources every time. This makes it really fast.Those HTTP requests are handled by a set of child processes called &quot;web workers&quot;. Swoole forks the contents in memory for the main process to each one of the workers, so each worker has a &quot;copy&quot; of the application in memory.Each worker is independent, but each one of them can only handle one request at a time, so you have to increase the number of workers if you expect a lot of concurrent traffic.Other than web workers, swoole also has another set of child processes called &quot;task workers&quot;, which can be used to delegate long tasks at runtime to prevent web workers from getting blocked.Here you can see a diagram of swoole&#x27;s architecture.<img src="https://www.swoole.co.uk/images/how-swoole-works.png" alt="How Swoole works"/><blockquote>More information can be found in <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/how-it-works">Swoole&#x27;s website</a>.</blockquote><h3>Problems to solve</h3>Now that we know how swoole works, let&#x27;s see some of the challanges I have faced in th past, which are usually not that ovbious when you are used to think in terms of &quot;blocking&quot; and &quot;short-lived&quot;.<h4>Database connections are kept open</h4>One of the first things you find out is that, since the application stays in memory, any database connection that has been opened by a worker (either web or task worker), will be kept open and reused in subsequent executions or HTTP requests served by the app.This, which is in theory good, has a side effect, which is not obvious during development: the opened connection will eventually expire.Since most of the popular database abstraction layers in PHP are coming from the times in which PHP was just a bootstrap-on-every-request technology, they never cared too much about expired database connections, as they would be opened again on next request.However, when using swoole and the like, you have to implement some mechanism which takes care of reconnecting or gracefully recovering when this happens.<blockquote>I wrote an article explaining how I solved this on a middleware-based app served with swoole and using doctrine: <a href="/2019/11/04/how-to-properly-handle-a-doctrine-entity-manager-on-an-expressive-application-served-with-swoole/">How to properly handle a doctrine entity manager on an expressive application served with swoole</a></blockquote>Obviously, swoole (and also others) provides some tools to work with databases (like this <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/docs/modules/swoole-async-mysql-client">MySQL client</a>), which I&#x27;m sure do not have this problem (although, I haven&#x27;t personally tried this), but they are more limited than other well adopted libraries.<h4>In-memory caches don&#x27;t behave as expected</h4>It is a frequent practice to use local in-memory caches, like APCu, to improve applications performance in production.Usually APCu is faster than other options, like redis or memcached, so it&#x27;s the choice when you only have one instance of your app running (meaning, you don&#x27;t have a cluster behind a load balancer).However, even if you are running your app on just one server with swoole, because of the web worker system explained earlier, you actually have several copies of your app in memory, and each one of them will have a separated and not shared cache when using APCu.This can lead to inconsistencies which are hard to debug, so you need to have it in mind.Also, since the information that is being loaded is kept in memory anyway, you will not notice such a big improvement by using APCu or other in-memory caches.<blockquote>Notice that this problem does not happen when using distributed/centralized caching solutions, such as redis or memcached, as in that case, the caching technology is shared by all workers.</blockquote><h4>No need to optimize file including/requiring</h4>Many projects have complex configuration systems which are spread into several files. While this make it more maintainable, it also has a performance impact when all those files need to be loaded on every request.Because of that, there are libraries like <a target="_blank" rel="noopener noreferrer" href="https://github.com/laminas/laminas-config-aggregator">laminas-config-aggregator</a> that can dynamically load config from many sources and, if some conditions are met, they can merge the result in a single bigger file that can be used in later requests.In the case of swoole, as all the files will be kept in memory once loaded for the first time, there&#x27;s no need for this kind of optimization.<h4>Class autoloading happens once</h4>This point is very related with the previous one.Composer offers a lot of optimization options so that class autoloading is faster in production. While you usually will still want to use some of those so that the first time a class is being loaded it is still reasonably fast, any subsequent hits will just use the class which is already in memory.One of the composer flags you will probably don&#x27;t want use with swoole is <code>--apcu-autoloader</code>, because of the reasons mentioned two points above this one.<h4>Services should be truly stateless</h4>This is something that you should always do anyway, but it&#x27;s specially important when a service instance is kept in memory between HTTP requests.Sometimes it&#x27;s easy to end up having some small piece of internal state on a service, where we save something as a side effect of a method call, to end up using it when another one of the public methods is invoked.While this could seem harmless when the whole app is bootstrapped on every request, it can have very dangerous side effects when the app is served with swoole, because the instance will persist until another request is dispatched by the same web worker.Just imagine you saved some user data on a service, and it ends up being served to another user on the next request. That won&#x27;t look good.Because of this, make sure your services are truly stateless, and if for any reason that&#x27;s not possible, at least remember to have some mechanism in place that flushes the data just before finishing the request.<h4>Hot reloading for dev envs is not that good</h4>One thing we usually take for granted is that we can modify a PHP file and just by making a new request, the new code will be executed. However, when the code is kept in memory, it&#x27;s not that straightforward.When you are developing a project which is served with swoole, you will have to restart the server every time to get your changes applied.Swoole has a mechanism to do a light server reload, which is faster. You can find the docs in <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/docs/modules/swoole-server-reload">it&#x27;s website</a>.The problem with this is that you need to implement the mechanism to invoke that when a file is changed, or use some library that does it for you.Also, I have noticed that even calling that and seeing that the server is reloaded, does not always have the expected effect, and you have to still end up doing a hard reload. It&#x27;s something I don&#x27;t fully understand and I still need to investigate a bit further.<h3>Conclusion</h3>As it always happens when new technologies appear, it takes some time to get used to them, but I strongly suggest you to give a try to swoole or any other non-blocking runtime, because it makes your apps super fast.Just remember to have all of the above in mind ;-)That said, here you can find some libraries that will make your life easier when trying to serve existing projects with swoole, without having to couple with it and being able to continue using your favourite framework:<ul><li>IDE helper: <a target="_blank" rel="noopener noreferrer" href="https://github.com/swoft-cloud/swoole-ide-helper">https://github.com/swoft-cloud/swoole-ide-helper</a></li><li>Mezzio (formerly Zend Expressive): <a target="_blank" rel="noopener noreferrer" href="https://github.com/mezzio/mezzio-swoole">https://github.com/mezzio/mezzio-swoole</a></li><li>Symfony: <a target="_blank" rel="noopener noreferrer" href="https://github.com/k911/swoole-bundle">https://github.com/k911/swoole-bundle</a></li><li>Laravel: <a target="_blank" rel="noopener noreferrer" href="https://github.com/swooletw/laravel-swoole">https://github.com/swooletw/laravel-swoole</a></li></ul>Also, here you can find some more articles talking about this topic:<ul><li><a target="_blank" rel="noopener noreferrer" href="https://www.zend.com/blog/why-you-should-use-asynchronous-php">https://www.zend.com/blog/why-you-should-use-asynchronous-php</a></li><li><a target="_blank" rel="noopener noreferrer" href="https://samsonasik.wordpress.com/2020/04/05/using-swoole-in-mezzio-application-with-sdebug/">https://samsonasik.wordpress.com/2020/04/05/using-swoole-in-mezzio-application-with-sdebug/</a></li></ul>

Considerations when working with async PHP runtimes like swoole

Some of you probably know that I have a pet project in which I like to work from time to time. This project is a self-hosted URL shortener called <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">Shlink</a>.Shlink is built using expressive as the base framework for the HTTP-dispatching task. A while ago, an expressive module was released to officially support serving expressive apps with <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/">swoole</a>, and I decided to include it on Shlink.<h3>How does swoole work?</h3>Swoole is an asynchronous non-blocking I/O framework which works in a similar way as node.js, but for PHP apps.It has been conceived in a way that the app stays in memory between requests, removing the bootstrapping step and making apps to be served much faster.Swoole can serve as many simultaneous requests as you tell it to. For this it makes use of workers. Each worker can serve one request at a time, and every worker has a separate instance of the app loaded in memory (this is something I <a target="_blank" rel="noopener noreferrer" href="https://twitter.com/dominikzogg/status/1189403552399134720">learned recently</a>).Because of this, you have to sometimes change a bit the way you design your code. For example, if a service needs to be stateful (it shouldn&#x27;t, but shit happens), you will have to remember that the state will persist between requests, and you will want to somehow reset it to avoid unexpected side effects.<h3>How does this affect the EntityManager?</h3>One of the main services that keeps this internal state in Shlink is the doctrine <a target="_blank" rel="noopener noreferrer" href="https://www.doctrine-project.org/projects/doctrine-orm/en/current/tutorials/getting-started.html#obtaining-the-entitymanager">EntityManager</a>.Due to the fact that it was originally designed to be used on a classical web server + CGI context, assuming it would be recreated on every request, there are a few considerations to take into account:<ul><li>It implements the unit of work pattern. It internally keeps track of all the entities that have been created/changed until you decide to flush it.</li><li>If something fails, the <code>EntityManager</code> can get closed, which makes it unusable after that moment.</li><li>If the database connection expires, the <code>EntityManager</code> will not gracefully recover. Instead, it will fail, since it&#x27;s not aware of the connection not being usable, getting yourself in previous point.</li></ul><h3>How to workaround it</h3>I learned about these issues by try and error, by using it on the app and getting the failures. Now, I want to share how I workaround those issues.The first one I faced was the one making the app unusable after the <code>EntityManager</code> was closed. In order to solve this I created a decorated instance (naming, an object implementing <code>EntityManagerInterface</code> which in turn was wrapping an actual <code>EntityManager</code>), which was making sure the wrapped instance was transparently recreated if at some point it was closed.However, this caused some side effects, and I finally realized that it was better to make the <code>EntityManager</code> to have to be reopened from the outside, and only once per request (this is closer to how it would behave on a classical set-up).That&#x27;s how I came up with the <code>ReopeningEntityManager</code> Shlink is currently using (as of v1.20.0):<pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Doctrine;

use Doctrine\ORM\Decorator\EntityManagerDecorator;

class ReopeningEntityManager extends EntityManagerDecorator
{
 /** @var callable */
 private $createEm;

 public function __construct(callable $createEm)
 {
 parent::__construct($createEm());
 $this-&gt;createEm = $createEm;
 }

 public function open(): void
 {
 if (! $this-&gt;wrapped-&gt;isOpen()) {
 $this-&gt;wrapped = ($this-&gt;createEm)();
 }
 }
}
</code></pre></pre><blockquote>You can find the actual code in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Doctrine/ReopeningEntityManager.php">shlink-common</a>.</blockquote>It just expects a factory that creates the actual <code>EntityManager</code>, and exposes an <code>open</code> public method which re-creates the wrapped instance if closed.This method is called from a <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-15/">PSR-15</a> middleware which needs to be registered early in the middleware pipeline. It makes sure the method is called before the next middleware on the stack is invoked:<pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Middleware;

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;
use Shlinkio\Shlink\Common\Doctrine\ReopeningEntityManager;

class CloseDbConnectionMiddleware implements MiddlewareInterface
{
 /** @var ReopeningEntityManager */
 private $em;

 public function __construct(ReopeningEntityManager $em)
 {
 $this-&gt;em = $em;
 }

 public function process(ServerRequestInterface $request, RequestHandlerInterface $handler): ResponseInterface
 {
 $this-&gt;em-&gt;open();

 try {
 return $handler-&gt;handle($request);
 } finally {
 $this-&gt;em-&gt;getConnection()-&gt;close();
 $this-&gt;em-&gt;clear();
 }
 }
}
</code></pre></pre><blockquote>Again, the code can be found in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Middleware/CloseDbConnectionMiddleware.php">shlink-common</a></blockquote>This middleware also does two more things (this time, after calling the next middleware on the stack, and making sure it does it even if an exception is thrown).<ul><li>It first closes the database connection. By doing it so, we make sure the <code>EntityManager</code> will reconnect the next time it needs to interact with the database, and therefore, we avoid the &quot;expired connection&quot; issue.</li><li>Then it clears the <code>EntityManager</code> itself. This avoids memory leaks, and also prevents that any orphan/non-flushed entity &quot;transcends&quot; to the next request, solving the first problem.</li></ul><h3>Further considerations</h3>This approach has been working well so far, but it&#x27;s important to assume that the <code>EntityManager</code> was not really designed for this non-blocking kind of context.You need to know that the worker making use of it will not be able to serve any request until the <code>EntityManager</code> finishes its job. Because of this you might want to increase the number of swoole workers.Also, closing the DB connection on every request is not the most optimal approach. It would be perfect to have a connection pool that was able to recreate connections when expired, and to keep them open as long as possible for faster operations. However, the <code>EntityManager</code> was not designed for that, so it&#x27;s better to close it.You also need a <code>callable</code> to pass to the <code>ReopeningEntityManager</code>. The way you solve this depends on your implementation and the dependencies you use.In my case, in Shlink I make use of the <code>ServiceManager</code>, so I have a delegator factory which takes the actual factory and passes it to the <code>ReopeningEntityManager</code>:<pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Doctrine;

use Psr\Container\ContainerInterface;

class ReopeningEntityManagerDelegator
{
 public function __invoke(ContainerInterface $container, string $name, callable $createEm): ReopeningEntityManager
 {
 return new ReopeningEntityManager($createEm);
 }
}
</code></pre></pre><blockquote>You can find the actual code in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Doctrine/ReopeningEntityManagerDelegator.php">shlink-common</a>.</blockquote><h3>Conclusion</h3>I wanted to write this article as a recap for myself too, since I had to ask and investigate many times, and I couldn&#x27;t find a single source of truth explaining everything.I hope that, if you ended up here, this article served you well.

How to properly handle a doctrine entity manager on an expressive application served with swoole

For a long time, I have been trying to include tests in every project in which I&#x27;ve worked on.There are several types of automated tests (or what should actually be called <a target="_blank" rel="noopener noreferrer" href="https://abstracta.us/insights/guide-continuous-testing/automated-checks">automated checks</a>). From unit tests, integration and functional tests, to end-to-end tests.Each one of them differs from the rest by the scope they try to cover. From small units of code which are tested completely detached from external dependencies, to a whole app which is tested as a black box like if an end consumer was using it.I&#x27;m not going to deeply discuss the semantics behind this, or where an integration test ends and an end-to-end test starts, because it&#x27;s quite frequently a blurry line.I want to focus on unit tests, and a frequently used practice (even in my own tests), which I have read (and felt) several times, it might not be the best approach.<blockquote>This article is not focused on any particular technology, since I have found this in a lot of different projects.</blockquote><h3>What&#x27;s a unit test</h3>According to <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Unit_testing">Wikipedia</a>, unit testing is a &quot;method by which individual units of source code [...] are tested to determine whether they are fit for use&quot;.While this might look like a clear definition, it&#x27;s actually quite open because, what&#x27;s a &quot;unit&quot; in the context of unit testing?Very often, in object oriented programming projects, people take a class as a unit to test, and create a unit test for every class. In other paradigms, like functional programming, you might consider a function as the unit to be tested, or a set of functions which you have decided to put together in a module.This is not really important for the purpose of this article. What&#x27;s really important is to take into account that, in order to properly &quot;unit test&quot; a &quot;unit of source code&quot;, you have to do it by isolating it from any dependency of that unit.One of the better ways to achieve this is by following the <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Injection Principle</a>. Without it you will probably end up trying to hack yourself in order to detach a &quot;unit&quot; from its dependencies.<blockquote>You can find other articles about <a href="/tag/dependency-injection/">Dependency Injection</a> in my blog.</blockquote><h3>Replacing dependencies</h3>A very frequent practice when unit testing a unit of source code is to replace its dependencies with <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Test_double">test doubles</a>, in order to properly isolate every unit when testing it.Test doubles are basically objects compatible with the dependencies of the subject under test, that are created just for the purpose of the test and have some useful capabilities, like capturing how they have been called, or what should they return when called.When you use one of those test doubles to assert how they were called as part of the test itself, you are using them as &quot;spies&quot;.<h3>Over-trusting spies</h3>All this introduction and contextualization had a single purpose.I find myself (and others) in the situation in which you prepare a test double to behave in a certain way, then you call your subject under test, and then assert that your expectations on how the test double was called are correct, by spying on it.This sometimes feels wrong, since it seems you are testing the test double/s instead of the subject under test.I have read many times that, in a unit test, we should not actually know the internal implementation of the subject under test, but only test that, based on certain input, the expected output is returned (so, trust its public API).However, there are cases in which some function/method does not really have any output, and we can only verify everything is going right by spying on the test doubles.So, at the end of the day, I have mixed feelings about this practice.<h3>Conclusion</h3>Sadly, I cannot state a clear conclusion. This is actually an open article, and I would love to get your point of view in the form of a <a href="/#comments-box">comment</a>.It would be great to see how others face this, if you use other kinds of test doubles instead of spies, or if you are just ok with the implications.

Is spying on mocks a bad practice for unit tests?

There&#x27;s no doubt that having tests in a project allows you to find potential bugs earlier and more easily.Lots of OSS projects require a minimum code coverage in order to accept new pull requests from contributors, and proprietary projects also tend to have some sort of continuous integration workflow which requires certain metrics to be fulfilled in order to get builds passing.However, the code coverage can lead to a false sense of security, which makes you think that if certain class has a 100% code coverage, it is also 100% bug-free.This is not always true, since you could be calling a method and yet not being properly testing its output or its real behavior. The code coverage will mark it as covered, but you might introduce a bug and still have a green test.This is where mutation testing comes in.<h3>Mutation testing</h3>According to <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Mutation_testing">wikipedia</a>, mutation testing involves applying small modifications to a software. These modifications are called mutations.Then, if you pass your tests and any of them fails because of this &quot;mutation&quot;, then you have &quot;killed the mutant&quot;. This is what should happen in code which is covered by tests. If tests keep passing after applying this modification, then your tests covering that part of the project are not so good, and you should improve them to really test what they are covering.So in other words, mutation testing is a quality assurance practice for your tests.In PHP projects, there used to be one library which purpose was working with the mutation testing approach, <a target="_blank" rel="noopener noreferrer" href="https://github.com/humbug/humbug">humbug/humbug</a>. However, it is mostly abandoned now, and it has motivated the appearance of another project, <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/">infection/infection</a>.<h3>Infection</h3>Infection is a mutation testing framework for PHP.It is easy to start using it in your project, since you just need to install it (<code>composer require infection/infection --dev</code>), and create a configuration file. Then, it uses your existing test suite in order to apply mutations to your code and check your tests.It currently integrates with <a target="_blank" rel="noopener noreferrer" href="https://github.com/sebastianbergmann/phpunit/">phpunit</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/phpspec/phpspec">phpspec</a>, but I suppose there will be compatibility with other testing frameworks in the future.<h4>Configuring infection</h4>It consumes an <code>infection.json</code> configuration file which could have a structure like this:<pre><pre><code class="language-json">{
 &quot;source&quot;: {
 &quot;directories&quot;: [
 &quot;module/*/src&quot;
 ]
 },
 &quot;timeout&quot;: 10,
 &quot;logs&quot;: {
 &quot;text&quot;: &quot;build/infection/infection-log.txt&quot;,
 &quot;summary&quot;: &quot;build/infection/summary-log.txt&quot;,
 &quot;debug&quot;: &quot;build/infection/debug-log.txt&quot;
 },
 &quot;tmpDir&quot;: &quot;build/infection/temp&quot;,
 &quot;phpUnit&quot;: {
 &quot;configDir&quot;: &quot;.&quot;
 }
}
</code></pre></pre>This is the configuration of one of my company&#x27;s projects. A modular application where every module is located inside the <code>module</code> directory and has its own <code>src</code> folder with source files.This config file basically defines where the sources are located, where infection&#x27;s logs should be placed, a temporary folder which will be used instead of using system&#x27;s default tmp folder, and where is the <code>phpunit.xml[.dist]</code> file.That&#x27;s all you need to get infection working.<blockquote>You can find detailed information of all the configuration options <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Configuration">here</a></blockquote><h4>Running infection</h4>Now that we have the configuration file, we just need to run <code>vendor/bin/infection</code>, and it will execute our test suite and then apply mutations to our source files.Then it will display the result, telling how many mutants have been killed.Infection supports lots of mutators, like replacing <code>++</code> by <code>--</code>, <code>true</code> by <code>false</code>, changing method visibilities, return values, etc. You can find the whole list <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/mutators.html">here</a>.The command line tool supports a list of modifiers, some of which are very interesting. This is how I would recommend you to run it if you want to address the whole project.<code>vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2</code>.<ul><li><code>--threads</code>: It runs tests and mutators in parallel processes, so you can take advantage of all the cores in your machine. Set a value that makes sense your you. If you have 8 cores, maybe you want to set it with value 8.</li><li><code>--min-msi</code>: This flag makes the process fail if the number of killed mutants is below certain percentage. In this case we require 70% of mutants to be killed. Adapt the value to your needs, and try to gradually improve your tests and increase this value (MSI stands for Mutation Score Indicator). This flag is really useful in continuous integration environments.</li><li><code>--only-covered</code>: This makes infection address only covered code. We know mutants won&#x27;t be killed in uncovered code, so I think it makes no sense to apply mutators there. I prefer to separate the required code coverage metric from the required mutation score indicator metric.</li><li><code>--log-verbosity</code>: By default, infection logs all the mutations it has applied to your code. This could lead to a log with thousands of lines if the project is big, so I prefer to set the log verbosity to 2, which makes it log only non-killed mutants.</li></ul>And that&#x27;s mainly it. You can now automate this task in your continuous integration pipeline, or just run it locally, but at least you will discover some parts both in your source files and your tests that can be improved.<h3>Troubleshooting</h3>While this could be enough for an introduction to infection, the title of this article says something about big projects, so I&#x27;m going to explain how we integrated infection in one of my company&#x27;s biggest projects and how we addressed some inherent problems.<h4>Already unstable</h4>The first problem while working with infection is that, at the moment of writing this article, it&#x27;s not in a stable version yet. That means you could find bugs.We found one in which infection wasn&#x27;t properly parsing our phpunit config file and it was excluding almost the whole project from the code coverage report. This caused all mutations to be skipped.We reported it and they fixed it very fast, but they haven&#x27;t tagged a version including the fix yet, so we require one specific commit from master while we wait for v0.8 to be released.<h4>Time consuming</h4>Another problem is that infection consumes a lot of memory and CPU, and takes a lot of time to be run when you have a lot of classes and tests.For example, our project has around 1500 unit tests, which take about 1&#x27;5min-2min to be run. Our build process also passes some coding style checks and functional repository tests. It usually takes about 3 minutes.The first time we run infection for the whole project, it took more than 16 minutes. That would make a build which is taking 3 minutes to take almost 20 minutes from now on, which is not an option because our build is run a lot of times every day.Our solution was making the build run infection only on changed sources, taking advantage of infection&#x27;s <code>--filter</code> flag, which lets you pass a comma-separated list of files and it gets applied only to them.The way in which we find which files have changed from previous build is by making a <code>git diff</code> between a commit from previous successful build and the new commit after updating the branch in which the build is being run (the commit identifiers are defined by jenkins&#x27; GIT plugin as environment variables).If it is the first time the build is run for this branch, we make the diff with <code>origin/develop</code> instead.The result is a bash script like this:<pre><pre><code class="language-bash"># Get files which have changed from latest processed commit, including only those inside sources
# If there&#x27;s no previous commit, diff with develop
INFECTION_FILTER=$(git diff ${GIT_PREVIOUS_SUCCESSFUL_COMMIT:-origin/develop} $GIT_COMMIT --name-only | grep /src/ | paste -sd &quot;,&quot; -)

# Check mutations over those files, if any, and require a 70% MSI
if [ -n &quot;$INFECTION_FILTER&quot; ]; then
 vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
else
 echo &quot;No source files affected. Infection skipped.&quot;
fi
</code></pre></pre>As you can see, we perform a diff by including only file names, then we exclude those which do not contain &quot;/src/&quot; as part of the name, and then we convert them into a comma-separated list.With this approach, the build usually takes 1 extra minute in the worst case scenario, which can be assumed.However, I&#x27;m sure the performance of this project will be improved as it grows and gets more stable.<h4>Test run duplication</h4>While previous approach filters the files where mutations are applied, it is still running all tests beforehand, which makes all tests to be run twice on every build.<ul><li>First, our own execution, which generates a code coverage report to be published later, and makes the build fail if tests do not pass.</li><li>Then, infection runs tests again, on its own &quot;conditions&quot; (they take your phpunit config file and generate a new one, which has some custom configuration entries).</li></ul><strike>If it were possible to pass infection an already generated code coverage, we could prevent this duplication, but, as far as I know, it does not support that.</strike> **It does now**.Another solution would be being able to run only tests which affect changed sources, instead of running all tests suites, but phpunit does not allow to filter by a list of files, only a specific file or folder.<strike>This is the only problem for which we have not yet found a solution, so If you know any workaround, a comment will be very welcome :-)</strike>Update 2018-02-22Following what Maks suggested in this <a href="/#comment-3762900802">comment</a>, we have changed the process a little bit, and using a more recent commit from <code>dev-master</code>, we can take advantage of a feature which allows an existing code coverage to be passed to infection.Now, our phpunit execution includes these two flags: <code>--coverage-xml=build/coverage-xml --log-junit=build/phpunit.junit.xml</code>And then, infection is executed with <code>--coverage=build</code>.That&#x27;s it, no need to run tests twice anymore :-)<h4>Xdebug vs phpdbg</h4>Since infection needs access to a code coverage report, you need some tool which is capable of generating it.People usually use xdebug, since it is the most extended and feature-rich debugging tool.However, there&#x27;s a simpler tool, which is designed to debug php console executions only, and comes bundled with php. It is <a target="_blank" rel="noopener noreferrer" href="https://github.com/krakjoe/phpdbg">phpdbg</a>.In our project, it usually takes 30%-50% less time to generate a code coverage report than xdebug. The result is not 100% the same, but it is very similar, and it&#x27;s worth it.So, instead of running infection like this:<pre><pre><code>vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
</code></pre></pre>We have to do it like this<pre><pre><code>phpdbg -qrr vendor/bin/infection --threads=4 --min-msi=70 --only-covered --log-verbosity=2 --filter=&quot;${INFECTION_FILTER}&quot;
</code></pre></pre>It is explained in <a target="_blank" rel="noopener noreferrer" href="https://infection.github.io/guide/usage.html#Running-with-phpdbg">infection&#x27;s docs</a>.We have even started to use it to run our own phpunit execution.<h3>Conclusion</h3>We have seen how to use infection, and how to integrate it in a continuous integration pipeline, even for big projects.I think it is a very useful tool, and the team behind it is doing a great job. It is also not hard to start using it, since you don&#x27;t really need to write new code, just add a config file and run the CLI tool.Finally, I recommend you to read this article, from infection&#x27;s author. It explains how it internally works, and how to use it in a development workflow <a target="_blank" rel="noopener noreferrer" href="https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1">https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1</a>.

Mutation testing with infection in big PHP projects

At some point, any enterprise project will probably need a message queue.A message queue is used to publish information (usually known as messages) that a different &quot;node&quot; (usually known as worker) will consume in order to perform a specific action.It is frequently used in web applications to pass information to background workers that consume the queue and perform long tasks, but it is also an important part when applying concepts like <a target="_blank" rel="noopener noreferrer" href="https://martinfowler.com/eaaDev/EventSourcing.html">Event Sourcing</a> to your architecture.It is also really important when working with microservices, since it is a way to enable data exchange between each service.Whatever the case, it is very likely that you will eventually need to decide which message queue to use.<h3>Why AWS Simple Queue Service</h3>In my company we had to decide this some time ago.We started testing different services, seeing what their strengths and weaknesses were.If I remember correctly, we tested <a target="_blank" rel="noopener noreferrer" href="https://www.rabbitmq.com/">RabbitMQ</a>, <a target="_blank" rel="noopener noreferrer" href="https://kr.github.io/beanstalkd/">Beanstalkd</a>, <a target="_blank" rel="noopener noreferrer" href="https://redis.io/">Redis</a> and <a target="_blank" rel="noopener noreferrer" href="https://aws.amazon.com/sqs/">AWS Simple Queue Service</a> (SQS from now on).The conclusion was that RabbitMQ is the one with the more features, by far. However, it is also the most complicated to configure and interact with.Also, we would have to maintain the RabbitMQ clusters ourselves (the same happens with Beanstalkd).Since we work with AWS, we decided to give SQS a try, and its simplicity convinced us.It is super simple to create a new queue, and using one of their SDKs, you can interact with it in minutes.Also, the price is absurdly cheap. It is virtually free for &quot;small&quot; amounts of requests (and when I say small, I mean that performing 1 request to their API every 20 seconds during 1 year costs about $0.30 at the end of the year).It is by far cheaper than maintaining your own instances with a different service, and I&#x27;m not counting the cost in time you will have to invest in maintaining those servers.<h3>The &quot;small&quot; problem with SQS</h3>One of the most interesting features with RabbitMQ is that you can open one socket with the queue server, and once a new message gets in, the worker is &quot;notified&quot; and the message can be consumed. This way, you have a process which is always running, listening for a queue, and processing messages as soon as possible.This is usually the desired behavior. You don&#x27;t want any delay on processing messages (or the delay should be as short as possible).On the other hand, SQS bills by API request. This means that the worker has to manually check if there&#x27;s any message in the queue, in small intervals.Our first approach was configuring cron jobs in our workers, so that they could check if there were any message in the queue, and process all of them, if any.This was problematic, since it applies a delay of up to one minute (you can&#x27;t configure a cron to be run with an interval smaller than that), which is not desirable, particularly when the job takes 5-10 seconds to complete. Adding an extra 600% of waiting time can affect the user experience, or have other side effects, depending on the task.<h3>The long polling to the rescue</h3>The first solution that came to our mind was making the workers run requests on an infinite loop, but that&#x27;s not a very clean approach, and will result in our workers being at a 100% of CPU usage all the time.We started investigating and found out about a feature in SQS called <a target="_blank" rel="noopener noreferrer" href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html">long polling</a>. It lets you tell the queue to wait up to 20 seconds before returning a response, if the message queue is empty, and if during that time a new message enters the queue, return a response immediately.Using it, you can make a worker continuously perform requests at 20 second intervals, but if a message is in the queue, the response will be returned immediately.The final behavior is very similar to a socket, in which you can process a message as soon as it enters the queue, and the worker does not consume any resource while it is waiting for a response.<h3>Configure a queue with long polling</h3>There are two ways of configuring the long polling. You can do it in AWS, and make the queue always have it, or you can do it client-side, and add a <code>WaitTimeSeconds</code> param with the time in seconds (with a maximum value of 20 seconds).The second option is very easy to implement when using any of the official SDKs, and is more flexible, since you can decide when to wait for a message and when to fail immediately when the queue is empty.The first one is very easy too. Just set a value for the &quot;Receive Message Wait Time&quot; param, when configuring the queue.<img src="/assets/img/sqs-socket/receive-message-wait-time.png" alt="SQS Receive Message Wait Time"/>That should be all, but there&#x27;s one important consideration.If you have several workers consuming the same queue with long polling and one message enters the queue, all the workers will receive it until one of them deletes it. This is usually problematic, since it could lead to one task being processed more than once.In order to prevent this, you should set a small value in the &quot;Default Visibility Timeout&quot; param, which tells SQS to deliver the message just to one consumer, and wait that amount of time before delivering it to the rest.This way, you can make one of the workers receive the message, delete it from the queue, and then process it, and that will prevent other workers from receiving it.Usually, setting a value of 3-5 seconds should be more than enough for the first worker to remove it from the queue. However, if you expect your workers to need a longer time, just increase the value.Remember, that value will delay delivering the message to all the workers but the first that gets it.<h3>Conclusion</h3>We have seen an efficient way of working with SQS, the most simple, yet powerful message queue.If you have more complex needs, it might end being too simple, but if not, it is possible to tweak it enough to use it on large applications and services.

How to simulate a socket connection with AWS Simple Queue Service

Earlier last week I found a <a target="_blank" rel="noopener noreferrer" href="https://github.com/WyriHaximus/awesome-phpstorm">github repository</a> which collects different resources related with PhpStorm. Plugins, themes, utilities...I found it very interesting, because I think <a target="_blank" rel="noopener noreferrer" href="https://www.jetbrains.com/phpstorm/">PhpStorm</a> is the best PHP IDE by far, and I&#x27;ve been using it on an almost daily basis for the last 4 years.The thing is that finding that repository gave me the idea of writing a post explaining the list of third party plugins I use and why, in case somebody is starting to use the IDE and wants some ideas on where to start.If you want to get any of them, just search for it inside PhpStorm. It&#x27;s the easiest way to install them.<h3>.env</h3>If you work with env files in your development process, either with <a target="_blank" rel="noopener noreferrer" href="https://github.com/vlucas/phpdotenv">vlucas/phpdotenv</a> (which comes bundled with laravel) or with <a target="_blank" rel="noopener noreferrer" href="https://symfony.com/doc/current/components/dotenv.html">symfony/dotenv</a>, this extension is very handy.It autocompletes environment variables defined in the .env file and the docker-compose.yml file when using the <code>env()</code> or <code>getenv()</code> functions.It also displays the value defined for those env variables inside the file.<img src="/assets/img/phpstorm-plugins/dot-env.png" alt=".env"/><blockquote>As a tip I will say that the first time I installed this plugin it didn&#x27;t work for me, because I had the .env files associated with the ini file type. Make sure you associate them with the .env type instead to get it working.</blockquote>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/9525--env-files-support">https://plugins.jetbrains.com/plugin/9525--env-files-support</a><h3>.ignore</h3>This plugin detects ignored files for many types of resources. Git, docker, npm and many more.It adds syntax highlight and autocompletion to those files (.gitignore, .hgignore...), displaying which of the ignored files do not exist anymore.It also sets a light grey font color in the project explorer for ignored files, so it is easier to &quot;ignore&quot; them.Finally, it displays a message at the top of the editor when you have opened an ignored file, so that you know it is not going to affect the corresponding tool.Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/7495--ignore">https://plugins.jetbrains.com/plugin/7495--ignore</a><h3>BashSupport</h3>If you have to work with shell scripts in any of your projects, this plugin is very useful.It adds syntax highlight, autocompletion, quickfixes and refactoring for <code>bash</code> and <code>sh</code>.It also allows you to add a missing shebang or even run the script right inside PhpStorm by right-clicking and selecting &quot;Run&quot;.<img src="/assets/img/phpstorm-plugins/bash-support.png" alt="BashSupport"/>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/4230-bashsupport">https://plugins.jetbrains.com/plugin/4230-bashsupport</a><h3>CodeGlance</h3>This one os more cool than useful.It adds a minimap inside any file editor, similar to the one that was made popular by Sublime text.It can be used to get an idea of the size of the file just at a glance.<img src="/assets/img/phpstorm-plugins/code-glance.png" alt="CodeGlance"/>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/7275-codeglance">https://plugins.jetbrains.com/plugin/7275-codeglance</a><h3>PHP Annotations</h3>This one is essential if you work with annotations of any kind (doctrine ORM, jms serializer, and such).Since PHP does not natively support annotations, everybody that needs that feature uses the package <a target="_blank" rel="noopener noreferrer" href="https://github.com/doctrine/annotations">doctrine/annotations</a>.It works really great, but since it works at comment level, it is very easy to break something by accident.This plugin adds a lot of features:<ul><li>Mark use statements as used when imported for an annotation.</li><li>Refactor or jump to annotation classes.</li><li>Annotation properties autocompletion (and even values, for well known projects like doctrine/orm).</li><li>Auto alias imports.</li><li>Syntax highlight and error management.</li></ul>This plugin almost suppresses the possibilities of breaking your code when running an automated use statements optimization or remove part of the annotation by mistake, things that cannot be caught by a unit test.<img src="/assets/img/phpstorm-plugins/annotations.png" alt="PHP Annotations"/>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/7320-php-annotations">https://plugins.jetbrains.com/plugin/7320-php-annotations</a><h3>PHP composer.json support</h3>Everybody uses composer for modern PHP projects, so this plugin is a must.It autocompletes composer.json blocks, and validates it, but making sure to never suggest a block that&#x27;s already in the file.It also displays warnings for common mistakes, like using the dev-master constraint for a dependency.It is very useful when you don&#x27;t remember the name of a package, since it autocompletes package names too.<img src="/assets/img/phpstorm-plugins/composer.png" alt="PHP composer.json support"/>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/7631-php-composer-json-support">https://plugins.jetbrains.com/plugin/7631-php-composer-json-support</a><h3>Php Inspections (EA Extended)</h3>If I had to uninstall all my plugins but one, this would be the one that I would keep for sure. Probably the best plugin and the one I use the most on any project.It adds a huge amount of new inspections to PhpStorm that will greatly improve your code.To name some:<ul><li>Warn about non-annotated thrown exceptions.</li><li>Warn about known PHP security issues.</li><li>Suggest better or more readable language constructions.</li><li>Suggest about more efficient ways of perform known tasks.</li><li>Warn about ugly code smells.</li><li>Warn about suspicious code blocks.</li></ul>And things like that. It is impossible to list all the things this plugin is able to inspect. I recommend you to install it and test it, because it really improves your productivity.Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/7622-php-inspections-ea-extended-">https://plugins.jetbrains.com/plugin/7622-php-inspections-ea-extended-</a><h3>PHPUnit Enhancement</h3>I discovered this plugin in the list I mentioned at the beginning of this article, so I have been using it for a short time.I still have to tweak it to fit my needs, but it is better than anything anyway.It mainly tells PhpStorm the type of mocked objects (either by PHPUnit&#x27;s mocks or by PHPSpec&#x27;s prophecies). This way, when you try to mock a method, you get autocompletion, and also, If later on you want to refactor that method or find their usages, it will find the mock too.The problem is that, for example, you might create a prophecy of a method that expects more than one argument or has a specific type hint, and you pass it something like <code>Argument::cetera()</code>. That makes it mark an error because the method expects something else.Apart from that, I think it&#x27;s worth it.Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/9674-phpunit-enhancement">https://plugins.jetbrains.com/plugin/9674-phpunit-enhancement</a><h3>Swagger plugin</h3>When you work on an API that needs to be documented, swagger is a really good tool.This plugin adds autocompletion and inspections on swagger docs, either yaml or JSON, so that you don&#x27;t need to wait for external tools to validate your files.<img src="/assets/img/phpstorm-plugins/swagger.png" alt="Swagger plugin"/>Link: <a target="_blank" rel="noopener noreferrer" href="https://plugins.jetbrains.com/plugin/8347-swagger-plugin">https://plugins.jetbrains.com/plugin/8347-swagger-plugin</a><h3>Conclusion</h3>That&#x27;s all the third party plugins I use at the moment of writing this article.If in the future I add any other, I&#x27;ll update this list.If you know of any cool plugin that&#x27;s not listed here, just add a comment to the article so that other can be aware of its existence.

The PhpStorm plugins of my choice

In a project I&#x27;m working on, we recently needed to add some kind of encryption system that allowed us to store sensitive information in a secure manner, but being able to access to it at runtime in order to pass it to third party services.Securely storing your own app passwords is easy. You just hash it, and the password can be verified without having to decrypt it, but if you need to &quot;decrypt&quot; it, the solution is not that easy.We started by building our own solution based on <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-crypt/">zend/crypt</a>, which is an excellent solution and works like a charm, but you still need to store the master key somehow, and that&#x27;s the big problem.After some research, someone pointed us to <a target="_blank" rel="noopener noreferrer" href="https://www.vaultproject.io/">vault project</a>, a self-hosted service which manages encrypted key-value pairs, in a more secure manner, where the master key is securely handled. I recommend you read the docs in order to see all its features.Vault supports different storage backends, each one with its own pros and cons. Some are meant to be used in development environments (like in memory and local file) and others are more suitable for production, with support for high availability (like DynamoDB or <a target="_blank" rel="noopener noreferrer" href="https://www.consul.io/">consul</a>, another hashicorp&#x27;s service).The official documentation indirectly recommends using vault with consul, which is a service with more features than just storing data. Indeed, it is a service discovery tool widely used this days in container-based infrastructures.For this reason, we started using both vault and consul in development, in order to get used to both services, but in the end we found it problematic because of different reasons:<ul><li>The consul server needs to get started before vault, otherwise vault fails on start-up. We saw this problem happening in development very frequently, which was annoying.</li><li>We talked with an expert sysadmin that was using consul in production environments. He told us it was a magnificent service, but he warn us about the possibility of having random data loss for no apparent reason. 
When using consul as a service discovery tool, that is not a big problem. Just recreating the consul cluster, all the data is retrieved again from running services. In our case it isn&#x27;t that simple. We were going to store sensitive and important data in consul, and loosing it could be a big problem.</li><li>I usually prefer to maintain as less services as possible in production. I see it from the point of view of a developer. If there is a cloud service that does the job, I prefer to use it rather than maintaining my own server. Be serverless as much as possible.</li></ul><h3>Finding an alternative</h3>Ok, consul is discarded. We need another storage alternative.Looking at vault&#x27;s documentation we saw there are a couple <a target="_blank" rel="noopener noreferrer" href="https://www.vaultproject.io/docs/configuration/storage/dynamodb.html">storage backends</a> that support high availability.We finally decided to go with <a target="_blank" rel="noopener noreferrer" href="https://aws.amazon.com/dynamodb/">AWS DynamoDB</a>. It is much less prone to data loss due to redundant storage under it, easier to buck-up and restore, and much cheaper than having dedicated servers. Also, the project&#x27;s infrastructure is already in AWS, so communication should be fast.Using DynamoDB we now only need to manage vault servers.<h3>Vault&#x27;s high availability</h3>Another of the vault&#x27;s feature is the possibility to configure several servers in high availability mode, as long as the storage supports it.This was a must for us, since getting vault down would make the service to not properly work and produce inconsistent data.The way vault manages high availability is very interesting. The first server that&#x27;s unsealed is set as active, and the rest are set in standby.If a request is performed to one of the servers in standby, they just redirect the request to the active server.If the active server is down, then one of the standby servers takes the leadership and is set as active.The communication between servers does not happen via network communication, but on the shared storage. The active server creates a lock file, and this way, the rest of the servers know they have to standby.This makes every server to work in an independent manner, and makes it easy to add more servers to the cluster if needed.The only problem is how do I make my app dynamically read from the active server, but be able to read from others if the active is down.Do I have to manually manage the load balancing? Do I have to implement some kind of strategy that tries to communicate to each server until one returns a response? That&#x27;s odd.Vault&#x27;s documentation recommends using another consul&#x27;s feature to solve this problem, some kind of DNS that makes a cluster name to resolve to the active instances IP address. This way you just need to perform requests to the cluster name.That&#x27;s good, but we weren&#x27;t going to use consul, right?<h3>AWS internal load balancer to the rescue</h3>After some investigation we saw contradictory opinions on behalf of hashicorp about using a load balancer with vault. Sometimes they recommend not to use it, sometimes they say they are not against nor in favor. Because of the way high availability works in vault, if the balancer redirects to a standby instance and it redirects to the balancer again, you could enter an infinite redirection loop.But what if you could make the load balancer consider standby instances out of service?Gladly, vault&#x27;s API has an <a target="_blank" rel="noopener noreferrer" href="https://www.vaultproject.io/api/system/health.html">endpoint</a> that performs a health check returning a status 200 only on unsealed active instances. If the instance is not active, it returns a 429 status. This endpoint doesn&#x27;t require authentication.Using this endpoint for the balancer&#x27;s health check, the balancer will never redirect to a standby instance.Also, if the active instance has any problem and goes down, as soon as a standby instance takes leadership, the load balancer will notice it, and mark it as InService, starting to redirect requests to that instance.Pretty nice. We can now configure our app to perform requests to the balancer and the problem is solved.<h3>Configuring the AWS load balancer</h3>When configuring the load balancer for the vault instances, we just need to take into account a few things.First, the balancer should be a classic load balancer, and it has to be configured as internal. This makes the balancer name resolve to the private IP address, and therefore, you prevent anyone accessing vault from the outside.It should also be configured to forward HTTP traffic on port 8200 (vault works on this port)<img src="%7B%7Bsite.url%7D%7D/assets/img/vault-with-load-balancer/balancer-first-step.png" alt="Load balancer"/>Then, when you are asked to configure the health check, use this params.<img src="%7B%7Bsite.url%7D%7D/assets/img/vault-with-load-balancer/health-check.png" alt="Health check"/>Configure the interval and healthy threshold according to your needs. Since for me it is a critical service, I reduced the interval to 15 seconds, and set the healthy threshold to 2. This way, when tha active instances goes down, it wont take more than 30 seconds to recover the service.Finally add the instances to the balancer.If everything is properly configured, after some seconds, in the instances tab of the balancer, you should see that only one of the instances is InService.<img src="%7B%7Bsite.url%7D%7D/assets/img/vault-with-load-balancer/instances-status.png" alt="Instances status"/>Now the balancer and the vault instances are ready to work!

Take advantage of vault project's high availability with an AWS internal load balancer

Recently I have been testing one service and its clustering capabilities, in order to see if it fits in a project I&#x27;m working on.I decided the easiest way to do this was by creating a couple docker containers and setting up a cluster between them. It should be an easy task in theory.My first approach was using a docker-compose file and linking the containers between themselves:<pre><pre><code class="language-yaml">version: &#x27;3&#x27;

services:
 test_1:
 container_name: test_1
 image: some:image
 links:
 - test_2
 - test_3

 test_2:
 container_name: test_2
 image: some:image
 links:
 - test_1
 - test_3

 test_3:
 container_name: test_3
 image: some:image
 links:
 - test_1
 - test_2
</code></pre></pre>My surprise when I run <code>docker-compose up -d</code> was that I received this message:<pre><pre><code class="language-bash">ERROR: Circular dependency between test_2 and test_3 and test_1
</code></pre></pre>It makes sense, since docker-compose checks links in order to run containers in the correct order. It is not possible in this case because every container depends on the other two.<h3>Fixing the problem with networks</h3>By default docker assigns random (sort of...) IP addresses to containers. By using links you make an entry to be added to the hosts file of a container, mapping the name of another container with its IP address. This way you don&#x27;t need to know its IP address, you can access it over the network by just using its name.I needed all containers to have a &quot;known&quot; network identifier in order to get the cluster working, because this service consumes a configuration on start-up that&#x27;s not dynamic.The solution was making the containers have a static known IP, this way they can be run without links and still communicate between themselves.Gladly, docker-compose supports networks since version 2, and I was able to define this configuration:<pre><pre><code class="language-yaml">version: &#x27;3&#x27;

services:
 test_1:
 container_name: test_1
 image: some:image
 networks:
 testing_net:
 ipv4_address: 172.28.1.1

 test_2:
 container_name: test_2
 image: some:image
 networks:
 testing_net:
 ipv4_address: 172.28.1.2

 test_3:
 container_name: test_3
 image: some:image
 networks:
 testing_net:
 ipv4_address: 172.28.1.3

networks:
 testing_net:
 ipam:
 driver: default
 config:
 - subnet: 172.28.0.0/16
</code></pre></pre>Using the <code>ipam</code> you can define a specific CIDR block for the new network, and then, attaching every container to that network, you can specify its IP address on that range.Now, test_1 is always run with the IP address 172.28.1.1, test_2 with 172.28.1.2 and test_3 with 172.28.1.3, and I&#x27;m able to hardcode those addresses in the config files.And that&#x27;s all!

Set specific IP addresses to docker containers created with docker-compose

Docker is, without any doubt, the trending tool these days. Everybody wants to use it, because it is very useful, allowing to easily generate development environments for any kind of application.A couple months ago I started working with docker myself (it has taken me a while, I know), and now I can&#x27;t imagine working without it.I started using it at work, but now I&#x27;m migrating all of my OSS projects too.<h3>The first stopper</h3>Regardless docker is very cool, there is a problem when you start using it.I had my development environment perfectly configured, and all my tools properly integrated. I was able, among other things, to run any project tests from within PhpStorm, just by right-clicking any test class, test method or phpunit.xml file, and selecting the &quot;Run&quot; option.However, once you start using docker in the project, it is not that simple.<h3>Integration with PhpStorm</h3>Gladly, since PhpStorm is a very powerful tool, it supports integration with docker, which can be then used, among other things, to automatically run tests inside a container.<blockquote>From this point, I&#x27;m going to explain the process, assuming you have already installed docker in your system. If not, you need to install at least <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/engine/installation/">docker engine</a> and <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/compose/install/">docker compose</a>.</blockquote>The first thing you need to do is configure the connection with docker engine. Go to Settings -&gt; Build, Execution, Deployment -&gt; Docker.Click the green plus button and a new docker connection will be created. The only value you should need to change is the API URL:<ul><li>Linux: unix:///var/run/docker.sock</li><li>Windows and Mac: <a target="_blank" rel="noopener noreferrer" href="http://127.0.0.1:2376">http://127.0.0.1:2376</a> (The URL of the docker machine. If you are using the deprecated docker toolbox with boot2docker, you have to make sure which is the IP address of the virtual machine)</li></ul><img src="/assets/img/phpstorm-docker/phpstorm-docker.png" alt="Docker in PhpStorm"/>Apply changes and we are done here.<h3>Create the remote interpreter</h3>Now let&#x27;s create a remote interpreter. It will be used so that tools that need a php runtime are executed inside a docker container.Go to Settings -&gt; Languages &amp; Frameworks -&gt; PHP. In here, you select the default interpreter for your project.<img src="/assets/img/phpstorm-docker/interpreter.png" alt="Interpreter"/>Click the button with the three dots, which will open the dialog to create a new interpreter.<img src="/assets/img/phpstorm-docker/interpreter-dialog.png" alt="Interpreter dialog"/>In that dialog you will likely have a local interpreter. Click the green plus button, at the top-left corner, and select Remote.That will open the dialog to create a new remote interpreter. One of the options is to use a docker image from which PhpStorm will create a docker container that should contain a php interpreter.<img src="/assets/img/phpstorm-docker/docker-interpreter.png" alt="Docker iterpreter"/>In this dialog, just select the server (this is the connection with the docker engine, the one we created in the first step), then select the docker image you want to use. The dropdown will be filled with all the images in your machine.Finally select the interpreter path. Usually, <a target="_blank" rel="noopener noreferrer" href="https://hub.docker.com/_/php/">PHP-based docker images</a> have the php interpreter in the PATH, so just setting <code>php</code> should be enough.Save everything and we are done configuring the interpreter.<h3>Create the PHPUnit config</h3>Now that the interpreter is created, we have to define a phpunit configuration that uses it.Go to Settings -&gt; Languages &amp; Frameworks -&gt; PHP -&gt; PHPUnit, Use the green plus button at the top-left corner, select By Remote interpreter and a dialog window will be prompt.It only contains a dropdown where you have to select the remote interpreter we created in previous step.Select it and you&#x27;ll see a configuration similar to this.<img src="/assets/img/phpstorm-docker/phpunit.png" alt="PHPUnit configuration"/>As you can see, PhpStorm mounts the project dir as a volume inside the container, in the <code>/opt/project</code> dir, and based on that path, selects the path to composer&#x27;s autoloader script and the phpunit.xml script.If any of those params is not correct in your project, you can (and should) change it.And that&#x27;s it!Now, you can right-click any test class, test method or phpunit.xml file and select &quot;Run&quot;. PhpStorm will automatically connect with docker engine, set up a temporary container and run the tests inside of it.You&#x27;ll know it is working because of this:<img src="/assets/img/phpstorm-docker/tests-in-container.png" alt="Tests inside container"/><h3>Conclusion</h3>PhpStorm&#x27;s team is always working to add integrations with the latest development tools, making it the most powerful IDE for PHP.Now go, configure your environment and start using docker. It&#x27;s amazing!

Run PHPUnit tests inside a docker container from PhpStorm

<a target="_blank" rel="noopener noreferrer" href="https://letsencrypt.org/">Let&#x27;s Encrypt</a> is the latest internet trending. Its a relatively young Certification Authority which intends to provide free and automated SSL certificates so that everybody can set encryption to their sites and applications.It is a <a target="_blank" rel="noopener noreferrer" href="https://www.linuxfoundation.org/">Linux Foundation</a> project sponsored by most of the biggest Internet companies, such as Google, Facebook, Mozilla or Cisco.They provide tools to automatically generate, setup and renew SSL certificates, but those tools are still in early stages, and not all the integrations are easy.In this article I&#x27;m going to explain how to setup one of this certificates in an Amazon Web Services Elastic Load Balancer (ELB from now on).<h3>Generate the certificate</h3>There are very good tutorials out there on how to generate a certificate, so I&#x27;m not going to dedicate much to this.I usually follow the instructions from <a target="_blank" rel="noopener noreferrer" href="https://certbot.eff.org/">certbot</a>.In there, you have to choose the web server of your choice, in this case &quot;None of the above&quot;, and your operating system, for example Ubuntu 16.04 (the only difference with older Ubuntu versions is that you can install the CLI tool from the standard repositories, otherwise there is a previous step where you have to download the <code>certbot-auto</code> script and run it, but that&#x27;s it).Once you have installed the <code>letsencrypt</code> tool (or <code>certbot-auto</code>), you have to generate the certificate with the webroot plugin. This plugin just generates the certificate by providing the document root for each domain that will be included in the certificate.<pre><pre><code>letsencrypt certonly --webroot -w /var/www/mydomain -d mydomain.com -d www.mydomain.com -w /var/www/cool-site -d cool-site.com -d www.cool-site.com
</code></pre></pre>This will create a certificate valid for mydomain.com, <a target="_blank" rel="noopener noreferrer" href="http://www.mydomain.com">www.mydomain.com</a>, cool-site.com and <a target="_blank" rel="noopener noreferrer" href="http://www.cool-site.com">www.cool-site.com</a>.All of the domains get verified from let&#x27;s encrypt side by putting a temporary hidden file inside provided web root folders.In this case, the first two domains will be checked in the /var/www/mydomain folder, and the last two in /var/www/cool-site. If those domains are really served from there, the certificate will get properly created.The problem with this in an Elastic Load Balancer is that you are running this tool in one server, but you will have more than one balanced server. This could make the previous process fail if the wrong server responds to the let&#x27;s encrypt check request.To prevent this you will have to leave just one server temporarily. Don&#x27;t worry, it only takes a few seconds. Once the certificate is created you can add the rest of the servers again.<blockquote>There are other plugins which simplify deploying certificates in standalone web servers.</blockquote><blockquote>For example, the apache plugin automatically sets the configuration in the apache virtual host of your choice.</blockquote>If everything worked, you should now have 4 files in the <code>/etc/letsencrypt/live/mydomain.com</code> folder.<pre><pre><code>/etc/letsencrypt/live/mydomain.com
 |-- cert.pem
 |-- chain.pem
 |-- fullchain.pem
 |-- privkey.pem
</code></pre></pre><h3>Setup the certificate in an ELB</h3>Now that you have your certificate files, you need to upload them to the ELB.Go to the AWS web console, select the EC2 service and go to Load Balancers. Once there select the load balancer of your choice and go to the Listeners tab.<img src="/assets/img/load-balancers-listeners.png" alt="Load Balancer listeners"/>Now you have to click on the &quot;Change&quot; link, in the SSL Certificate column of the HTTPS protocol, in order to set the new certificate.<img src="/assets/img/load-balancers-set-certificate.png" alt="Load Balancer set SSL Certificate"/>In this window, select the third option, Upload a new SSL certificate to AWS... and set the name of your choice for it.Then, in the Private key field you have to place the contents of the <code>cert.pem</code> file, and in the Public key certificate field, the contents of the <code>fullchain.pem</code> file. You can leave the Certificate chain field empty.Save, and you are done. Your ELB is now serving HTTPS request by using the previously generated Let&#x27;s Encrypt certificate.<h3>Renewal</h3>Renewing this certificate is easy. Just run the command <code>letsencrypt renew</code> (or <code>certbot-auto renew</code>) in the same server used to create the certificate in the first place.That will generate the new certificate in the same location, and you can repeat the previous step with it.The main problem with Let&#x27;s Encrypt certificates is that they are valid for three months only. Auto renewal is a feature that they want to implement at some point, but it&#x27;s not available yet.By the moment, the best solution for standalone servers is creating a cron job that automatically runs the <code>letsencrypt renew</code> command, but for an ELB that won&#x27;t fix the problem.You will probably need to create some kind of script that automatically uploads the certificate to the ELB after being renewed, by using the AWS REST API or the CLI interface.<a target="_blank" rel="noopener noreferrer" href="http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-update-ssl-cert.html#us-update-lb-SSLcert-cli">This documentation</a> should be helpful. Pay special attention to the To replace an SSL certificate with a certificate uploaded to IAM section.

Alejandro Celaya | Blog

Software development, agile methodologies and open source projects.

category Archive — "tools"

— 2021 —

— 2020 —

— 2019 —

— 2018 —

— 2017 —

— 2016 —