<p>I remember many years ago, when I released my first open source project, that every time I had a new release ready, I was never sure what should be the new version number.</p><p>Versioning software is not as straightforward as it might look when you don&#x27;t have a lot of experience.</p><p>In this article I&#x27;ll try to explain the <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">SemVer</a> approach to version software, and some things you might want to take into account on top of that.</p><h3>Semantic Versioning</h3><p>The first thing you need is a solid set of rules that help you decide which should your next version be.</p><p>I remember thinking <em>&quot;is this version big enough as to jump from 1.0 to 2.0?&quot;</em> or <em>&quot;I have seen projects using three numbers for the version, X.Y.Z, should I do the same?&quot;</em></p><p>Those questions are easier to answer if you stick to some standard that tells you what to do, and that will help others more easily understand your intention.</p><p>My recommendation is to use <a target="_blank" rel="noopener noreferrer" href="https://semver.org/">Semantic Versioning</a>, as it defines a solid starting point, and it&#x27;s quite adopted by the community.</p><p>Semantic Versioning states that you should use three numbers to identify versions, <code>MAJOR.MINOR.PATCH</code>, and follow these rules in order to decide which one to increase:</p><ul><li><code>PATCH</code>: the new release includes only bug fixes. People are encouraged to update to this release and no changes are needed in their code bases.</li><li><code>MINOR</code>: the new release includes new backwards-compatible features. People can update to this version without changing their code bases, and make use of the new features afterwards if they wish.</li><li><code>MAJOR</code>: the new release includes <a target="_blank" rel="noopener noreferrer" href="https://en.wikipedia.org/wiki/Backward_compatibility">backward compatibility</a> breaks. Changes in the code base will be needed when updating to this version.</li></ul><blockquote><p>These rules are easier to apply to libraries and frameworks than it is for apps and services. More about this later.</p></blockquote><h3>Try to keep backwards-compatibility</h3><p>Ok, we have some rules now. However, I have seen very frequently people mistaking what <em>can</em> be done with what <em>should</em> be done.</p><p>Yes, if you introduce a breaking change, you can update the major version and that&#x27;s it. However, if you do it too frequently, people won&#x27;t be happy, and you will end up with a fragmented user base, that stop updating to newer versions.</p><p>I remember using a library to deploy stuff that had a new major version every couple of months, changing things like &quot;The <code>host</code> function is now called <code>server</code>&quot;. Yes, ok, but why do I have to change my stuff so frequently.</p><p>It&#x27;s usually better to try to implement things in a backwards-compatible way, and keep track of the things you want to change and do it all at a time.</p><p>You can also deprecate things, and document the way you want those things to be done moving forward. That way you give people the chance to start changing their code and simplify a hypothetical upcoming major release.</p><h3>Provide clear upgrade paths</h3><p>As mentioned above, too frequent major version changes are not desirable, but sooner or later you will want to release a new major version for different reasons.</p><ul><li>Getting rid of that feature which is hard to maintain, introduces lots of bugs, and provides little value.</li><li>Supporting something new which is very hard to implement without breaking backwards compatibility.</li><li>Removing all those deprecated methods you have been dragging for months.</li><li>Fix design issues, where something was architected in the wrong way, and only spotted after some time.</li></ul><p>At the time this happens you will want your users to upgrade to the newer version, as you don&#x27;t want people to report bugs that only affect older versions, or to expect you to backport fixes to older versions because they haven&#x27;t updated to the new one.</p><p>The best way to achieve this is to provide clear and simple upgrade paths:</p><ul><li><p>Prepare your releases for the things to come: If you want to introduce new ways of doing things, do it before the major release, and deprecate the older ones. For example, you can have a new method that gets called by an older one, while you mark the old one as deprecated.</p><p>People will be more willing to update and change their code afterwards. By the time the major release is published, their code will be ready to upgrade.</p></li><li><p>Include a CHANGELOG file in your project, explaining the changes introduced in every version. A very good way to document a change log is the <a target="_blank" rel="noopener noreferrer" href="https://keepachangelog.com/">Keep a changelog</a> initiative.</p><p>You can also use some other platform to document the change log, like GitHub releases.</p></li><li><p>Include documentation on how to upgrade to a major release: This is probably the most important one. Document all the steps that need to be followed and all the required changes to update from a release to the next major one.</p><p>A good way to do this is by including an UPGRADE file in your project. You can see <a target="_blank" rel="noopener noreferrer" href="https://github.com/acmailer/acmailer/blob/main/UPGRADE.md">this example</a>.</p></li></ul><h3>Versioning libraries vs apps</h3><p>Semantic Versioning is a good standard to version software, but it plays much better with libraries and frameworks that are installed using some kind of package manager, than it does with apps and services that are not directly depended on.</p><p>For example, how do you identify a backward compatibility break on a web user interface?</p><p>With this kind of software there are more grey areas, and you will end up using your intuition.</p><p>For example, not long ago I released a <a href="/2020/12/24/released-shlink-web-client-v3-0-0/">new major version</a> for a web app I maintain, and it was completely compatible with the previous one. However, it introduced a larger amount of features than usual, and big UI changes, so I decided it was worth the bump.</p><p>Because of this there are projects that use different approaches to version apps. For example:</p><ul><li>Firefox and Chrome have 6-week cycles, and after each one of them they release a new major version no matter what.</li><li>Jetbrains publishes a few releases of their IDEs every year (I think 3) which always start with the year they were released on, like <code>2020.3.0</code>, and they increase the patch number for bugfixes.</li><li>Ubuntu releases 2 versions every year, one in April and one in October, and they are always identified by the year and month they were released on, like <code>19.04</code> or <code>20.10</code>.</li></ul><h3>Special versions</h3><p>There&#x27;s also a set of special versions that can be used to publish the so-called <strong>pre-release</strong> versions. These <a target="_blank" rel="noopener noreferrer" href="https://semver.org/#spec-item-9">are covered</a> by Semantic Versioning, but sometimes they are not so easy to understand.</p><p>It is also very likely that you never have to use these, as only relatively big projects will need this level of granularity, but it&#x27;s still good to know what&#x27;s their purpose and what do they imply.</p><h4>0.x.y</h4><p>It is possible to release a version of your software where the major number is <code>0</code>. When this happens, it means your software is still in its early stages.</p><p>It does not mean it&#x27;s necessarily unstable, but it means you are still introducing frequent changes on it, and therefore, it might be a bit more fragile than a version starting with a number greater than <code>0</code>.</p><h4>Alpha</h4><p>The intention of an alpha release is that a subset of people con test some new functionality, but you don&#x27;t want people to use it in production</p><p>Alpha versions are on their very early stages. Their public API is very likely to change, and they probably contain bugs.</p><p>Semantic Versioning states that alpha versions are identified by this pattern: <code>MAJOR.MINOR.PATCH-alpha.NUMBER</code>, for example <code>2.0.0-alpha.1</code>.</p><h4>Beta</h4><p>Similar to alpha versions, beta versions are also prone to contain bugs, but their API will in theory not change. That means that you could potentially adapt your software to start using a beta version, and when the corresponding stable release is published, you shouldn&#x27;t need that many changes.</p><p>Semantic Versioning defines a pattern similar to the one for alpha versions: <code>MAJOR.MINOR.PATCH-beta.NUMBER</code>, for example <code>2.0.0-beta.1</code>.</p><h4>Release candidate</h4><p>Release candidates (RC) are pre-release versions that you will want to publish for the general user-base to test an upcoming release. You want to publish a RC so that it&#x27;s battle tested by as many people as possible, in order to find possible bugs and edge cases.</p><p>They should not contain many bugs, but some could be found. The public API should not change anymore.</p><p>According to Semantic Versioning, the pattern to identify a RC is <code>MAJOR.MINOR.PATCH-rc.NUMBER</code>, for example <code>2.0.0-rc.1</code>.</p><h3>Conclusion</h3><p>We have seen some approaches and considerations to properly version your software.</p><p>Versioning is not so straightforward when you don&#x27;t have a lot of experience, but as you have seen, you just need to know a small set of rules and use a bit your intuition.</p><p>With those tools, you will be able to properly version your software.</p>

What to take into account when versioning software

<p>Two days ago I released the first major version in almost 2 years for <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlink-web-client</a>, a web UI for <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">shlink</a>, my self-hosted URL shortener.</p><p>This new version introduces several improvements to the project, and I thought it was worth writing a post to explain all of them in depth.</p><h3>New design</h3><p>The project has been using the same design almost since the beginning. It was clean, but it laked a bit of contrast and consistency in many sections.</p><p>Now the main background is light-gray, with components composed in white cards with a slight box shadow, that makes them easier to spot inside the app.</p><p>Also, the main area has now a max-width that prevents components from stretching too much on wide screens.</p><div class="row"><div class="col-md-6  "><img alt="Before" src="/assets/img/shlink-web-client-3/design-before.png" class="gallery-item"/><p class="text-center"><small>Before</small></p></div><div class="col-md-6  "><img alt="After" src="/assets/img/shlink-web-client-3/design-after.png" class="gallery-item"/><p class="text-center"><small>After</small></p></div></div><h3>Overview page</h3><p>Up until now, after connecting to a Shlink server, you went directly to the list of short URLs.</p><p>With this new version, you now see an overview page first, which displays some general stats, a quick short URL creation form, and the latest 5 short URLs.</p><p>From there, you can navigate to the full list of short URLs, with filtering capabilities, or go to the advanced short URL creation form.</p><div class="row"><div class="col-md-8 col-md-offset-2 "><img alt="Overview" src="/assets/img/shlink-web-client-3/overview.png" class="gallery-item"/><p class="text-center"><small>Overview</small></p></div></div><h3>Improved short URL creation form</h3><p>One of the sections with more room for improvement was the short URL creation form. It had all the inputs mixed, displaying just the &quot;long url&quot; one, with the option to un-collapsing the rest at will.</p><p>The new design groups the inputs by context, which makes it more clear. It also displays the inputs all the time, since the &quot;quick form&quot; is now located in the overview page.</p><div class="row"><div class="col-md-6  "><img alt="Before" src="/assets/img/shlink-web-client-3/creation-form-before.png" class="gallery-item"/><p class="text-center"><small>Before</small></p></div><div class="col-md-6  "><img alt="After" src="/assets/img/shlink-web-client-3/creation-form-after.png" class="gallery-item"/><p class="text-center"><small>After</small></p></div></div><h3>Enhanced visits page</h3><p>The visits page had a problem, it was trying to display too much information in a single place.</p><ul><li>Visits over time</li><li>Visits from every platform</li><li>Visits from every location</li><li>A list of visits as a table</li></ul><p>The new approach splits those in subsections with their own sub-route each (<code>/by-location</code>, <code>/by-context</code>, etc). This helps to render a smaller amount of charts at once, since they are a bit CPU greedy.</p><p>Also, only the visits from the last 30 days are loaded by default, making it load way faster for URLs with a lot of visits.</p><div class="row"><div class="col-md-6  "><img alt="Before" src="/assets/img/shlink-web-client-3/visits-before.png" class="gallery-item"/><p class="text-center"><small>Before</small></p></div><div class="col-md-6  "><img alt="After" src="/assets/img/shlink-web-client-3/visits-after.png" class="gallery-item"/><p class="text-center"><small>After</small></p></div></div><h3>Outer sections</h3><p>The app has some sections which are not related to a specific server, like creating/editing servers, or the settings page.</p><p>These sections have also been adapted to the new style, and they also have a max-width for wide screens.</p><div class="row"><div class="col-md-6  "><img alt="Before" src="/assets/img/shlink-web-client-3/create-server-before.png" class="gallery-item"/><p class="text-center"><small>Before</small></p></div><div class="col-md-6  "><img alt="After" src="/assets/img/shlink-web-client-3/create-server-after.png" class="gallery-item"/><p class="text-center"><small>After</small></p></div></div><h3>Date range selector</h3><p>A new component allows selecting date ranges either by relative time ranges (last 30 days, last 90 days...) or by selecting absolute start and end dates.</p><p>This component is used to filter both short URL lists and visits lists.</p><div class="row"><div class="col-md-6  col-md-offset-3"><img alt="Date range selector" src="/assets/img/shlink-web-client-3/date-range-selector.png" class=""/><p class="text-center"><small>Date range selector</small></p></div></div><h3>Domain selector</h3><p>Shlink has had multi-domain support for some versions, but changing the domain from the web client required to set the value manually, which is a bit error-prone.</p><p>This version introduces a new component which allows selecting from domains already used in the past (which is what you will want in most of the cases), preventing accidental typos.</p><p>It also allows setting new values in case you want to use a new domain that you never used before. This domain will then appear in the list the next time.</p><div class="row"><div class="col-md-6  "><img alt="Existing domains" src="/assets/img/shlink-web-client-3/domains-existing.png" class=""/><p class="text-center"><small>Existing domains</small></p></div><div class="col-md-6  "><img alt="New domain" src="/assets/img/shlink-web-client-3/domains-new.png" class=""/><p class="text-center"><small>New domain</small></p></div></div><h3>Other improvements</h3><p>The changes described so far were the major ones. However, v3.0.0 includes a few other minor improvements.</p><ul><li>Result and loading messages have been standardized: They are now more consistent and use the same set of components, which also ensures consistency in the future.</li><li>Error messages are now more descriptive: Instead of showing generic error messages when API calls fail, the actual error message is now always displayed, so that it&#x27;s easier to know what failed.</li><li>Dropped support for Shlink v1: All major versions come with some breaking changes that improve code maintainability. This one drops Shlink v1 support, which allows removing some conditional paths here and there.</li></ul><p>Other than the last point, Shlink v3.0.0 is mostly backwards compatible, so it&#x27;s safe to upgrade if you are already using Shlink v2.x</p><h3>Next steps</h3><p>Of course this doesn&#x27;t end here, and I have a lot of ideas to improve it even further.</p><ul><li>Dark theme: It&#x27;s what the cool kids do these days, and I have already done some experiments around it.</li><li>Shlink dashboard: Next gen web app for Shlink. You can read the whole reasoning in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client/issues/338">this issue</a>.</li><li>Filtering in lists added to URL: Many sections allow filtering lists, but the state is saved in memory, which means it gets lost if the page is refreshed, and it cannot be bookmarked. The intention is to make all filtering params to become part of the URL.</li><li>Exporting visits in CSV: It&#x27;s cool to see charts full of info, but sometimes admins need to feed other services with the data, and consuming the original API is not always an option. Exporting visits to a CSV file will be supported soon by shlink-web-client.</li></ul>

Released shlink-web-client v3.0.0

<p>Let&#x27;s not deny that <a target="_blank" rel="noopener noreferrer" href="https://github.com/features/actions">GitHub actions</a> are hitting hard in the tech community, and many projects are transitioning from other continuous integration systems like <a target="_blank" rel="noopener noreferrer" href="https://circleci.com/">Circle CI</a> or <a target="_blank" rel="noopener noreferrer" href="https://travis-ci.org/">Travis CI</a>.</p><p>Personally I have been using Travis a lot during the past years, for all my OSS projects, and I have gotten very used to it.</p><p>However, people seem to be &quot;abandoning&quot; travis in favor of GitHub Actions, for different reasons. &quot;It&#x27;s faster&quot;, &quot;I can run smaller tasks in parallel&quot;, etc.</p><p>In my case, I have moved a few very specific tasks there, where travis was becoming a bottleneck, but I still have mixed feelings about the very-verbose yaml-based config needed to configure a few GitHub Actions, compared to the more concise travis one.</p><p>In this article I&#x27;ll explain how I improved a travis build to run several smaller tasks in parallel jobs, without having to completely move away from it.</p><h3>Initial state</h3><p>This project&#x27;s travis config is relatively simple. It is used to build a JS project for the web browser, and it has to run the next tasks:</p><ul><li>Lint the code, to make sure it fulfils the project&#x27;s coding standards.</li><li>Run unit tests.</li><li>Run mutation tests.</li><li>Build the docker image, to make sure none of the new changes broke it.</li><li>Conditionally publish a release on GitHub, with a dist file attached to it, if the build is being run for a git tag.</li></ul><pre><pre><code class="language-yaml">dist: bionic

language: node_js

branches:
  only:
    - /.*/

cache:
  directories:
    - node_modules

services:
  - docker

node_js:
  - &#x27;12.16.3&#x27;

install: npm ci

before_script:
  - echo &quot;Building commit range ${TRAVIS_COMMIT_RANGE}&quot;
  - export MUTATION_FILES=$(git diff ${TRAVIS_COMMIT_RANGE:-origin/main} --name-only | grep -E &#x27;src\/(.*).(ts|tsx)$&#x27; | paste -sd &quot;,&quot;)

script:
  - npm run lint
  - npm run test:ci
  - npm run mutate:ci
  - docker build -t project:test .

after_success:
  - node_modules/.bin/ocular coverage/clover.xml

before_deploy: npm run build ${TRAVIS_TAG#?}

deploy:
  - provider: releases
    api_key:
      secure: &lt;key&gt;
    file: &quot;./dist/project_${TRAVIS_TAG#?}_dist.zip&quot;
    skip_cleanup: true
    on:
      tags: true
</code></pre></pre><h3>Problems with this approach</h3><p>This is more or less ok, but the last two commands in <code>script</code> can be relatively slow and can delay the whole process.</p><p>Also, specially the mutation tests one, which is currently always considered successful regardless the result, can make the build reach the maximum run time allowed by travis, making the whole build falsely fail.</p><blockquote><p>The reason for the <code>before_script</code> step is to determine which source code files changed, and run mutation tests on them only, in order to mitigate this.</p></blockquote><p>Other than this, the build config defines &quot;global&quot; customizations that are actually only used for some tasks:</p><ul><li>Running <code>npm ci</code> is not required for the <code>docker build</code> task.</li><li>Getting the <code>docker</code> service is not required by any of the other tasks.</li><li>Determining changed files is only used by the <code>mutation tests</code> task.</li></ul><h3>Making use of jobs</h3><p>One cool feature travis supports is the use of a job <a target="_blank" rel="noopener noreferrer" href="https://docs.travis-ci.com/user/build-matrix/">build matrix</a>, that lets you define multiple jobs that run in parallel as part of the same build, and make each one of them do unrelated tasks.</p><p>Using this feature it is possible to improve the <code>.travis.yml</code> file, making it look like this:</p><pre><pre><code class="language-yaml">dist: bionic

language: node_js

branches:
  only:
    - /.*/

cache:
  directories:
    - node_modules

node_js:
  - &#x27;12.16.3&#x27;

jobs:
  fast_finish: true
  allow_failures:
    - name: &#x27;Mutation tests&#x27;
  include:

    - name: &#x27;Lint&#x27;
      install: npm ci
      script: npm run lint

    - name: &#x27;Unit tests&#x27;
      install: npm ci
      script: npm run test:ci
      after_success:
        - node_modules/.bin/ocular coverage/clover.xml

    - name: &#x27;Mutation tests&#x27;
      install: npm ci
      before_script:
        - echo &quot;Building commit range ${TRAVIS_COMMIT_RANGE}&quot;
        - export MUTATION_FILES=$(git diff ${TRAVIS_COMMIT_RANGE:-origin/main} --name-only | grep -E &#x27;src\/(.*).(ts|tsx)$&#x27; | paste -sd &quot;,&quot;)
      script: npm run mutate:ci

    - name: &#x27;Build docker image&#x27;
      services:
        - docker
      install: skip
      script: docker build -t project:test .

    - name: &#x27;Publish release&#x27;
      if: tag IS present
      install: skip
      script: skip
      before_deploy: npm run build ${TRAVIS_TAG#?}
      deploy:
        - provider: releases
          api_key:
            secure: &lt;key&gt;
          file: &quot;./dist/project_${TRAVIS_TAG#?}_dist.zip&quot;
          skip_cleanup: true
          on:
            tags: true
</code></pre></pre><p>This approach introduces all these benefits:</p><ul><li>Now linting, testing, mutation testing and building the docker image run in parallel, making the build finish earlier.</li><li>The mutation testing step is allowed to fail, and thanks to the <code>fast_finish</code> flag, travis won&#x27;t even wait for it to finish, considering the build successful once the rest have passed.</li><li>Configs that are only meaningful for certain jobs are defined in the job itself, instead of globally.</li><li>The &quot;publish release&quot; job only runs conditionally if the build is for a tag, thanks to the <code>if: tag IS present</code> condition. (More info about <a target="_blank" rel="noopener noreferrer" href="https://docs.travis-ci.com/user/conditional-builds-stages-jobs/">conditional jobs</a> in travis).</li><li>As soon as any of the jobs fails, the whole build fails and you don&#x27;t have to continue waiting. Before this, if the third task failed, you still had to wait for the previous two to finish.</li><li>Each job can have a name, making it more clear in travis UI what tasks are being run:</li></ul><p><img src="/assets/img/travis-multiple-jobs/travis-jobs.png" alt="Travis CI jobs"/></p><h3>Conclusion</h3><p>The main purpose of this article was to showcase that travis still has a lot to give. Github Actions are definitely cool, but there&#x27;s usually no silver bullet for everything, and a combination with other tools might be more suitable.</p><p>Also, this allows people using travis to improve their builds without having to change everything. Then, once you have split your tasks, if you still want to move to Github Actions, it should be easier.</p><blockquote><p>If you are curious, this is the project in which I use this travis configuration: <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-web-client">shlinkio/shlink-web-client</a>.</p></blockquote>

Running several steps of a Travis CI build in parallel

<p>Asynchronous and non-blocking runtimes are pretty usual in many programming languages, as well as long-lived web apps that stay in memory and are capable of dispatching multiple HTTP requests without having to be fully bootstrapped every time. This has not been traditionally the case with PHP apps. </p><p>However, many projects are starting to get some adoption, that bring this long-lived approach to the PHP world.</p><p>The problem with this is that PHP developers are not used to this, and they tend to continue acting as they have always done. Also, libraries might have made some assumptions, making them not work as expected when used under this context.</p><p>Some of these projects are <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk">swoole</a>, <a target="_blank" rel="noopener noreferrer" href="https://reactphp.org/">ReactPHP</a> or <a target="_blank" rel="noopener noreferrer" href="https://amphp.org/">Amphp</a>, to give some examples.</p><p>In this article, I will be focusing on the first one, and explain how to approach some of the pain points I have found after using it for a year and a half on an <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink">existing PHP project</a>.</p><h3>Introducing Swoole</h3><p>The main difference swoole has compared to other similar projects is that it&#x27;s written as a native PHP extension, and needs to be installed with <code>pecl</code>.</p><p>This is obviously a bit cumbersome, but the benefit is that, since it&#x27;s written in C++, it has a better handling of memory allocation, which is very important to achieve what it tries to do.</p><p>What swoole does is running a main process which bootstraps the PHP application once, and then keeps it in memory so that it can continue dispatching requests without having to load resources every time. This makes it really fast.</p><p>Those HTTP requests are handled by a set of child processes called &quot;web workers&quot;. Swoole forks the contents in memory for the main process to each one of the workers, so each worker has a &quot;copy&quot; of the application in memory.</p><p>Each worker is independent, but each one of them can only handle one request at a time, so you have to increase the number of workers if you expect a lot of concurrent traffic.</p><p>Other than web workers, swoole also has another set of child processes called &quot;task workers&quot;, which can be used to delegate long tasks at runtime to prevent web workers from getting blocked.</p><p>Here you can see a diagram of swoole&#x27;s architecture.</p><p><img src="https://www.swoole.co.uk/images/how-swoole-works.png" alt="How Swoole works"/></p><blockquote><p>More information can be found in <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/how-it-works">Swoole&#x27;s website</a>.</p></blockquote><h3>Problems to solve</h3><p>Now that we know how swoole works, let&#x27;s see some of the challanges I have faced in th past, which are usually not that ovbious when you are used to think in terms of &quot;blocking&quot; and &quot;short-lived&quot;.</p><h4>Database connections are kept open</h4><p>One of the first things you find out is that, since the application stays in memory, any database connection that has been opened by a worker (either web or task worker), will be kept open and reused in subsequent executions or HTTP requests served by the app.</p><p>This, which is in theory good, has a side effect, which is not obvious during development: <strong>the opened connection will eventually expire</strong>.</p><p>Since most of the popular database abstraction layers in PHP are coming from the times in which PHP was just a bootstrap-on-every-request technology, they never cared too much about expired database connections, as they would be opened again on next request.</p><p>However, when using swoole and the like, you have to implement some mechanism which takes care of reconnecting or gracefully recovering when this happens.</p><blockquote><p>I wrote an article explaining how I solved this on a middleware-based app served with swoole and using doctrine: <a href="/2019/11/04/how-to-properly-handle-a-doctrine-entity-manager-on-an-expressive-application-served-with-swoole/">How to properly handle a doctrine entity manager on an expressive application served with swoole</a></p></blockquote><p>Obviously, swoole (and also others) provides some tools to work with databases (like this <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/docs/modules/swoole-async-mysql-client">MySQL client</a>), which I&#x27;m sure do not have this problem (although, I haven&#x27;t personally tried this), but they are more limited than other well adopted libraries.</p><h4>In-memory caches don&#x27;t behave as expected</h4><p>It is a frequent practice to use local in-memory caches, like APCu, to improve applications performance in production.</p><p>Usually APCu is faster than other options, like redis or memcached, so it&#x27;s the choice when you only have one instance of your app running (meaning, you don&#x27;t have a cluster behind a load balancer).</p><p>However, even if you are running your app on just one server with swoole, because of the web worker system explained earlier, you actually have several copies of your app in memory, and each one of them will have a separated and not shared cache when using APCu.</p><p>This can lead to inconsistencies which are hard to debug, so you need to have it in mind.</p><p>Also, since the information that is being loaded is kept in memory anyway, you will not notice such a big improvement by using APCu or other in-memory caches.</p><blockquote><p>Notice that this problem does not happen when using distributed/centralized caching solutions, such as redis or memcached, as in that case, the caching technology is shared by all workers.</p></blockquote><h4>No need to optimize file including/requiring</h4><p>Many projects have complex configuration systems which are spread into several files. While this make it more maintainable, it also has a performance impact when all those files need to be loaded on every request.</p><p>Because of that, there are libraries like <a target="_blank" rel="noopener noreferrer" href="https://github.com/laminas/laminas-config-aggregator">laminas-config-aggregator</a> that can dynamically load config from many sources and, if some conditions are met, they can merge the result in a single bigger file that can be used in later requests.</p><p>In the case of swoole, as all the files will be kept in memory once loaded for the first time, there&#x27;s no need for this kind of optimization.</p><h4>Class autoloading happens once</h4><p>This point is very related with the previous one.</p><p>Composer offers a lot of optimization options so that class autoloading is faster in production. While you usually will still want to use some of those so that the first time a class is being loaded it is still reasonably fast, any subsequent hits will just use the class which is already in memory.</p><p>One of the composer flags you will probably don&#x27;t want use with swoole is <code>--apcu-autoloader</code>, because of the reasons mentioned two points above this one.</p><h4>Services should be truly stateless</h4><p>This is something that you should always do anyway, but it&#x27;s specially important when a service instance is kept in memory between HTTP requests.</p><p>Sometimes it&#x27;s easy to end up having some small piece of internal state on a service, where we save something as a side effect of a method call, to end up using it when another one of the public methods is invoked.</p><p>While this could seem harmless when the whole app is bootstrapped on every request, it can have very dangerous side effects when the app is served with swoole, because the instance will persist until another request is dispatched by the same web worker.</p><p>Just imagine you saved some user data on a service, and it ends up being served to another user on the next request. That won&#x27;t look good.</p><p>Because of this, make sure your services are truly stateless, and if for any reason that&#x27;s not possible, at least remember to have some mechanism in place that flushes the data just before finishing the request.</p><h4>Hot reloading for dev envs is not that good</h4><p>One thing we usually take for granted is that we can modify a PHP file and just by making a new request, the new code will be executed. However, when the code is kept in memory, it&#x27;s not that straightforward.</p><p>When you are developing a project which is served with swoole, you will have to restart the server every time to get your changes applied.</p><p>Swoole has a mechanism to do a light server reload, which is faster. You can find the docs in <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/docs/modules/swoole-server-reload">it&#x27;s website</a>.</p><p>The problem with this is that you need to implement the mechanism to invoke that when a file is changed, or use some library that does it for you.</p><p>Also, I have noticed that even calling that and seeing that the server is reloaded, does not always have the expected effect, and you have to still end up doing a hard reload. It&#x27;s something I don&#x27;t fully understand and I still need to investigate a bit further.</p><h3>Conclusion</h3><p>As it always happens when new technologies appear, it takes some time to get used to them, but I strongly suggest you to give a try to swoole or any other non-blocking runtime, because it makes your apps super fast.</p><p>Just remember to have all of the above in mind ;-)</p><p>That said, here you can find some libraries that will make your life easier when trying to serve existing projects with swoole, without having to couple with it and being able to continue using your favourite framework:</p><ul><li>IDE helper: <a target="_blank" rel="noopener noreferrer" href="https://github.com/swoft-cloud/swoole-ide-helper">https://github.com/swoft-cloud/swoole-ide-helper</a></li><li>Mezzio (formerly Zend Expressive): <a target="_blank" rel="noopener noreferrer" href="https://github.com/mezzio/mezzio-swoole">https://github.com/mezzio/mezzio-swoole</a></li><li>Symfony: <a target="_blank" rel="noopener noreferrer" href="https://github.com/k911/swoole-bundle">https://github.com/k911/swoole-bundle</a></li><li>Laravel: <a target="_blank" rel="noopener noreferrer" href="https://github.com/swooletw/laravel-swoole">https://github.com/swooletw/laravel-swoole</a></li></ul><p>Also, here you can find some more articles talking about this topic:</p><ul><li><a target="_blank" rel="noopener noreferrer" href="https://www.zend.com/blog/why-you-should-use-asynchronous-php">https://www.zend.com/blog/why-you-should-use-asynchronous-php</a></li><li><a target="_blank" rel="noopener noreferrer" href="https://samsonasik.wordpress.com/2020/04/05/using-swoole-in-mezzio-application-with-sdebug/">https://samsonasik.wordpress.com/2020/04/05/using-swoole-in-mezzio-application-with-sdebug/</a></li></ul>

Considerations when working with async PHP runtimes like swoole

<p>Some of you probably know that I have a pet project in which I like to work from time to time. This project is a self-hosted URL shortener called <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">Shlink</a>.</p><p>Shlink is built using expressive as the base framework for the HTTP-dispatching task. A while ago, an expressive module was released to officially support serving expressive apps with <a target="_blank" rel="noopener noreferrer" href="https://www.swoole.co.uk/">swoole</a>, and I decided to include it on Shlink.</p><h3>How does swoole work?</h3><p>Swoole is an asynchronous non-blocking I/O framework which works in a similar way as node.js, but for PHP apps.</p><p>It has been conceived in a way that the app stays in memory between requests, removing the bootstrapping step and making apps to be served much faster.</p><p>Swoole can serve as many simultaneous requests as you tell it to. For this it makes use of workers. Each worker can serve one request at a time, and every worker has a separate instance of the app loaded in memory (this is something I <a target="_blank" rel="noopener noreferrer" href="https://twitter.com/dominikzogg/status/1189403552399134720">learned recently</a>).</p><p>Because of this, you have to sometimes change a bit the way you design your code. For example, if a service needs to be stateful (it shouldn&#x27;t, but shit happens), you will have to remember that the state will persist between requests, and you will want to somehow reset it to avoid unexpected side effects.</p><h3>How does this affect the EntityManager?</h3><p>One of the main services that keeps this internal state in Shlink is the doctrine <a target="_blank" rel="noopener noreferrer" href="https://www.doctrine-project.org/projects/doctrine-orm/en/current/tutorials/getting-started.html#obtaining-the-entitymanager">EntityManager</a>.</p><p>Due to the fact that it was originally designed to be used on a classical web server + CGI context, assuming it would be recreated on every request, there are a few considerations to take into account:</p><ul><li>It implements the unit of work pattern. It internally keeps track of all the entities that have been created/changed until you decide to flush it.</li><li>If something fails, the <code>EntityManager</code> can get closed, which makes it unusable after that moment.</li><li>If the database connection expires, the <code>EntityManager</code> will not gracefully recover. Instead, it will fail, since it&#x27;s not aware of the connection not being usable, getting yourself in previous point.</li></ul><h3>How to workaround it</h3><p>I learned about these issues by try and error, by using it on the app and getting the failures. Now, I want to share how I workaround those issues.</p><p>The first one I faced was the one making the app unusable after the <code>EntityManager</code> was closed. In order to solve this I created a decorated instance (naming, an object implementing <code>EntityManagerInterface</code> which in turn was wrapping an actual <code>EntityManager</code>), which was making sure the wrapped instance was transparently recreated if at some point it was closed.</p><p>However, this caused some side effects, and I finally realized that it was better to make the <code>EntityManager</code> to have to be reopened from the outside, and only once per request (this is closer to how it would behave on a classical set-up).</p><p>That&#x27;s how I came up with the <code>ReopeningEntityManager</code> Shlink is currently using (as of v1.20.0):</p><pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Doctrine;

use Doctrine\ORM\Decorator\EntityManagerDecorator;

class ReopeningEntityManager extends EntityManagerDecorator
{
    /** @var callable */
    private $createEm;

    public function __construct(callable $createEm)
    {
        parent::__construct($createEm());
        $this-&gt;createEm = $createEm;
    }

    public function open(): void
    {
        if (! $this-&gt;wrapped-&gt;isOpen()) {
            $this-&gt;wrapped = ($this-&gt;createEm)();
        }
    }
}
</code></pre></pre><blockquote><p>You can find the actual code in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Doctrine/ReopeningEntityManager.php">shlink-common</a>.</p></blockquote><p>It just expects a factory that creates the actual <code>EntityManager</code>, and exposes an <code>open</code> public method which re-creates the wrapped instance if closed.</p><p>This method is called from a <a target="_blank" rel="noopener noreferrer" href="https://www.php-fig.org/psr/psr-15/">PSR-15</a> middleware which needs to be registered early in the middleware pipeline. It makes sure the method is called before the next middleware on the stack is invoked:</p><pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Middleware;

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;
use Shlinkio\Shlink\Common\Doctrine\ReopeningEntityManager;

class CloseDbConnectionMiddleware implements MiddlewareInterface
{
    /** @var ReopeningEntityManager */
    private $em;

    public function __construct(ReopeningEntityManager $em)
    {
        $this-&gt;em = $em;
    }

    public function process(ServerRequestInterface $request, RequestHandlerInterface $handler): ResponseInterface
    {
        $this-&gt;em-&gt;open();

        try {
            return $handler-&gt;handle($request);
        } finally {
            $this-&gt;em-&gt;getConnection()-&gt;close();
            $this-&gt;em-&gt;clear();
        }
    }
}
</code></pre></pre><blockquote><p>Again, the code can be found in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Middleware/CloseDbConnectionMiddleware.php">shlink-common</a></p></blockquote><p>This middleware also does two more things (this time, after calling the next middleware on the stack, and making sure it does it even if an exception is thrown).</p><ul><li>It first closes the database connection. By doing it so, we make sure the <code>EntityManager</code> will reconnect the next time it needs to interact with the database, and therefore, we avoid the &quot;expired connection&quot; issue.</li><li>Then it clears the <code>EntityManager</code> itself. This avoids memory leaks, and also prevents that any orphan/non-flushed entity &quot;transcends&quot; to the next request, solving the first problem.</li></ul><h3>Further considerations</h3><p>This approach has been working well so far, but it&#x27;s important to assume that the <code>EntityManager</code> was not really designed for this non-blocking kind of context.</p><p>You need to know that the worker making use of it will not be able to serve any request until the <code>EntityManager</code> finishes its job. Because of this you might want to increase the number of swoole workers.</p><p>Also, closing the DB connection on every request is not the most optimal approach. It would be perfect to have a connection pool that was able to recreate connections when expired, and to keep them open as long as possible for faster operations. However, the <code>EntityManager</code> was not designed for that, so it&#x27;s better to close it.</p><p>You also need a <code>callable</code> to pass to the <code>ReopeningEntityManager</code>. The way you solve this depends on your implementation and the dependencies you use.</p><p>In my case, in Shlink I make use of the <code>ServiceManager</code>, so I have a delegator factory which takes the actual factory and passes it to the <code>ReopeningEntityManager</code>:</p><pre><pre><code class="language-php">&lt;?php

declare(strict_types=1);

namespace Shlinkio\Shlink\Common\Doctrine;

use Psr\Container\ContainerInterface;

class ReopeningEntityManagerDelegator
{
    public function __invoke(ContainerInterface $container, string $name, callable $createEm): ReopeningEntityManager
    {
        return new ReopeningEntityManager($createEm);
    }
}
</code></pre></pre><blockquote><p>You can find the actual code in <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink-common/blob/master/src/Doctrine/ReopeningEntityManagerDelegator.php">shlink-common</a>.</p></blockquote><h3>Conclusion</h3><p>I wanted to write this article as a recap for myself too, since I had to ask and investigate many times, and I couldn&#x27;t find a single source of truth explaining everything.</p><p>I hope that, if you ended up here, this article served you well.</p>

How to properly handle a doctrine entity manager on an expressive application served with swoole

<p>The day Zend Expressive 2 was released I was super excited. I have been using it a lot for both professional and personal projects, so I&#x27;m quite used to it.</p><p>Since I&#x27;ve been using it in many projects, being able to update all of them to version 2 was a challenge, but I can say, I have succeed :-)</p><h3>The projects</h3><p>I have used expressive for two small websites, <a target="_blank" rel="noopener noreferrer" href="https://www.alejandrocelaya.com">https://www.alejandrocelaya.com</a> and <a target="_blank" rel="noopener noreferrer" href="https://shlink.io">https://shlink.io</a>.</p><p>The code can be found <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/alejandrocelaya.com">here</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink.io">here</a>.</p><p>When I migrated my website to expressive, I had to create a custom router for backward compatibility reasons, because none of the provided implementations supports optional params at the beginning of the path, and I was using them.</p><p>That&#x27;s why I created <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/expressive-slim-router">this</a> implementation based on Slim framework router (Slim 2, not 3). Since expressive 2 depends on the expressive router 2, I had to release a new version of this component too.</p><p>The next project to migrate was <a target="_blank" rel="noopener noreferrer" href="https://github.com/shlinkio/shlink">Shlink</a>, a little OSS project of my own. Like its website, it is based on expressive too.</p><p>This is a little bit bigger application, but not too big anyway.</p><p>Shlink is mainly a REST API, but has two or three endpoints that are meant to be consumed by humans in a web browser. For this reason I created a component that was responsible of handling errors in a different way, depending on the request&#x27;s <code>Accept</code> header.</p><p>Since the concept of error handling has completely changed in expressive 2, that library needed to be updated too: <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/ze-content-based-error-handler">https://github.com/acelaya/ze-content-based-error-handler</a></p><p>And I left the two most important projects for the end.</p><p>First, my company&#x27;s website. Another small project, not too hard to migrate.</p><p>Finally, my company&#x27;s main product. A big and modular REST API, with about 100 actions/middlewares I wanted to migrate to <a target="_blank" rel="noopener noreferrer" href="https://github.com/http-interop/http-middleware">http-interop middleware</a>. It wasn&#x27;t difficult, but it took me a while to fix all tests xD.</p><p>In the end I migrated <strong>5</strong> projects and <strong>2</strong> libraries, and I am quite proud of the result.</p><h3>The process</h3><p>When I decided to migrate the first project, I started by following this article from the official documentation: <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-expressive/reference/migration/to-v2/">https://docs.zendframework.com/zend-expressive/reference/migration/to-v2/</a>.</p><p>It has a very detailed explanation with everything you need to know. Indeed, the best achievement of the Zend Framework team has been the number of articles, tools and documentation they provide to make this migration easier.</p><p>They have also spent a lot of time on making as many features as possible, backward compatible, to the point that almost everything keeps working after updating the components.</p><p>My conclusion after reading the article was that the only two things I needed to fix was the error handler and the middlewares signature, to use the single-pass approach instead of the old double-pass. However, the second one is not really necessary, since double-pass middleware keeps working with expressive 2, but I wanted to change it.</p><h3>Error handler</h3><p>At first, when one reads the docs, it seems that changing the error handler is going to be hard.</p><p>They explain the idea behind the new approach, why they have changed it, and how you should implement it.</p><p>The fact is that after reading this you realize expressive 2 and stratigility 2 come with a basic implementation that makes migrating from the old error handler to the new approach as easy as registering two services and one middleware. Done.</p><p>Now, instead of having a component that gets called when the middleware stack gets exhausted or an exception is thrown, like in expressive 1, the recommendation is to register a middleware at the outermost layer of the stack, that is responsible of catching any exception thrown by any inner middleware. Basically, what you do is invoke the next middleware inside a try/catch, and return an error response if the catch block is reached.</p><p>Stratigility 2 comes with a <code>Zend\Stratigility\Middleware\ErrorHandler</code> middleware that already does this. Just pipe it at the beginning of the stack and register the service using the provided <code>Zend\Expressive\Middleware\ErrorHandlerFactory</code> factory.</p><p>In expressive 1, the error handler was responsible of returning a response object. Now, this new ErrorHandler middleware delegates this responsibility into a service called ErrorResponseGenerator. Stratigility comes with an implementation, the <code>Zend\Stratigility\Middleware\ErrorResponseGenerator</code> that returns plain text errors, like the old stratigility ErrorHandler used to do.</p><p>On the other hand, expressive comes with two implementations, the <code>Zend\Expressive\Middleware\ErrorResponseGenerator</code>, which gets a template renderer injected, and acts like the old <code>TemplatedErrorHandler</code>, and also, a <code>Zend\Expressive\Middleware\WhoopsErrorResponseGenerator</code>, which is meant to be used in development, and acts like the old <code>WhoopsErrorHandler</code>.</p><p>In other words, in order to migrate the error handler, you just need to do three things:</p><ul><li>Remove the old <code>Zend\Expressive\ErrorHandler</code> service</li><li>Register both <code>Zend\Stratigility\Middleware\ErrorHandler</code> and <code>Zend\Expressive\Middleware\ErrorResponseGenerator</code>, with whichever implementation you want.</li><li>Pipe the <code>Zend\Stratigility\Middleware\ErrorHandler</code> middleware as the outermost middleware of your stack.</li></ul><p>After doing this, everything should work.</p><h3>Error middleware</h3><p>Expressive 1 had a feature called error middleware.</p><p>These middlewares were not exactly the same as regular middleware. Instead, they received a fourth <code>error</code> param, and were piped at the end of the stack, so they were invoked only if the latest regular middleware called the next one, or any other middleware called next with an <code>error</code> parameter, which had the effect of bypassing any other regular middleware and jumping directly to the first error middleware.</p><p>I never liked this feature, because it wasn&#x27;t clear.</p><ul><li>What if I pipe error middleware between regular middleware, with a higher priority? Does it get invoked?</li><li>Why error middleware doesn&#x27;t get invoked in case of status 404 or 405? Isn&#x27;t that an error? Apparently not.</li><li>What is an &quot;error&quot; in the scope of middleware then? An exception only?</li></ul><p>Because I was never sure how to use error middleware, I decided to reduce it to the point that only one of the projects had one error middleware that was responsible of logging exceptions in a file.</p><p>Gladly, the new error handling approach, the <code>Zend\Stratigility\Middleware\ErrorHandler</code>, allows &quot;error&quot; listeners to be attached, so that when an exception is caught, the error can be processed by any number of listeners.</p><p>The signature of the listeners is the same as the error middlewares used to have. Invokable objects with three parameters, an error, a request and a response.</p><p>This allowed me to register that error middleware as a listener, and just remove the return statement, which is no longer used.</p><h3>Single-pass middleware</h3><p>One of the most important new features in expressive 2, and the one I most wanted to use, is the support for interop middleware. It is what has been proposed to end up as the <a target="_blank" rel="noopener noreferrer" href="https://github.com/php-fig/fig-standards/blob/master/proposed/http-middleware/middleware.md">psr-15</a> standard for server middleware.</p><p>The main difference with middleware in expressive 1 is that middlewares no longer get the response object (which is less error prone), and that the last argument is no longer a <code>callable</code>, but an object (the <strong>delegate</strong>) which invokes the next middleware and returns the response generated by it.</p><p>Migrating middlewares to this approach is very easy. Remove the second argument and replace <code>$out($request, $response)</code> by <code>$delegate-&gt;process($request)</code>. Done.</p><p>The hardest part here is migrating tests.</p><p>Since expressive 1 middlewares are <code>callables</code>, I used to use <code>Closures</code> in tests. Now, the last argument is type hinted to <code>DelegateInterface</code>, so you have to explicitly mock it.</p><p>I have used <a target="_blank" rel="noopener noreferrer" href="https://github.com/phpspec/prophecy">prophecy</a>, which comes with PHPUnit, and it works like a charm. It&#x27;s not very difficult, but it could take a while if the application is big.</p><p>Of course, If you don&#x27;t test your code, you won&#x27;t have this problem, but I hope it is not your case ;-)</p><p>Tests allowed me to be sure all middlewares and actions were still working, before testing the apps manually.</p><h3>Router</h3><p>Changes in the router shouldn&#x27;t affect you, unless you have implemented your own router, like me.</p><p>However, the change is very small. This is what I had to change: <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya/expressive-slim-router/commit/46bb000cb4389a6be6bff1335f875d7c6d8f5ca9">feature/v3</a></p><p>You could also be affected by this if you rely on the <code>RouteResult</code> in any of your middlewares and used to call <code>RouteResult::fromRouteMatch()</code> in tests. Just replace it by <code>RouteResult::fromRoute()</code> and that&#x27;s it.</p><h3>Programmatic approach</h3><p>Another of the changes in expressive 2 is that now they promote the programmatic approach to configure routes and pipe middlewares, because newcomers find it easier to understand.</p><p>However, the config-driven approach still works, and you can still use it if you want.</p><p>That&#x27;s my case indeed. I like having my modules provide different routes, instead of defining all of them in a single config file. It better fits my workflow and my way of thinking.</p><p>Something similar happens with middleware, I prefer defining it in a config file with array notation. However, I have to confess I have moved all the middleware config to a single file in all projects. </p><p>I have found that, as the project grows, since middleware needs to be piped in a specific order, defining it in different files makes it harder to maintain and much more error prone. This is probably one of the reasons of promoting the programmatic approach.</p><h3>Conclusion</h3><p>My conclusion after migrating all these projects is that expressive is a great framework, and the Zend Framework team is expending a lot of time to make it scalable, maintainable an usable.</p><p>The more I use it, the more I see it is the best option for projects of any size. A framework where everything is perfectly explained, with a really good documentation, very loosely coupled with any other component, and a framework where you always have the last word.</p><p>No black magic or obscure conventions. But at the same time, you can quickly prototype any application without writing a lot of boilerplate code.</p><p>Thanks to all of this, the migration between major versions has been a breeze.</p>

My thoughts after migrating some projects to Zend Expressive 2

<p>It has been a long time since I first realized that handling file uploads in non-<strong>POST</strong> requests (like <strong>PUT</strong>) wasn&#x27;t an easy task.</p><p>One could assume the <code>$_FILES</code> array should be populated regardless the HTTP verb, but actually, PHP doesn&#x27;t do it on its own.</p><p>After a long time wanting to find a solution to this problem, I&#x27;ve finally dedicated the time to get something functional, that allows file uploads to be transparently handled regardless the HTTP verb (it works the same way in <strong>POST</strong>, <strong>PUT</strong> and <strong>PATCH</strong> requests).</p><p>Since nowadays I try to work with psr-7/middleware based applications, I have created a <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-expressive/">Zend Expressive</a> app that registers a middleware capable of parsing a <code>multipart/form-data</code> request body, populating the request&#x27;s uploaded files array and parsed body array.</p><p>This way, you can call <code>$request-&gt;getUploadedFiles()</code> or <code>$request-&gt;getParsedBody()</code> in any <strong>PUT</strong> or <strong>PATCH</strong> action, the same way you would do in a <strong>POST</strong> action.</p><p>You can find the example application here: <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/put-patch-file-uploads">https://github.com/acelaya-blog/put-patch-file-uploads</a></p><h3>The example</h3><p>When you clone the application, enter the project directory and run <code>composer install &amp;&amp; composer serve</code>. Then you should be able to access <a target="_blank" rel="noopener noreferrer" href="http://localhost:8080">http://localhost:8080</a> and see something like this:</p><p><img src="%7B%7Bsite.url%7D%7D/assets/img/put-patch-uploads/app-screenshot.png" alt="Example application"/></p><p>This is a simple form which submit event is captured via javascript in order to get it sent using the selected HTTP verb.</p><p>The server then dumps the request&#x27;s uploaded files and parsed body using <a target="_blank" rel="noopener noreferrer" href="http://symfony.com/doc/current/components/var_dumper.html">symfony/var-dumper</a>, and the result is appended to the bottom of the page.</p><p>Using the browser&#x27;s console you should be able to see the actual request. Regardless the selected HTTP method, the result should be exactly the same.</p><p>Also, all the uploaded files will be stored in the <code>data/files</code> folder.</p><p>This is done by the <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/put-patch-file-uploads/blob/master/src/App/Action/UploadAction.php">UploadAction</a> class, which is dispatched when the <strong>/upload</strong> route is resolved with <strong>POST</strong>, <strong>PUT</strong> or <strong>PATCH</strong> methods.</p><h3>Let&#x27;s see how it works</h3><p>If we wouldn&#x27;t have done anything, this example would only work in <strong>POST</strong> requests. <strong>PUT</strong> and <strong>PATCH</strong> requests would always print an empty array of files and parsed body.</p><p>The class responsible of doing the magic is the <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/put-patch-file-uploads/blob/master/src/App/Middleware/MultipartRequestBodyParser.php">MultipartRequestBodyParser</a>. Let&#x27;s analyze it.</p><p>The first couple of lines is simple. This middleware should only be executed when the request uses one of the HTTP verbs that allows body, but <strong>POST</strong>, which is automatically parsed by PHP. Also, the content type of the request should be <code>multipart/form-data</code></p><pre><pre><code class="language-php">&lt;?php
namespace App\Middleware;

use App\File\UploadedFile;
use Fig\Http\Message\RequestMethodInterface;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;

class MultipartRequestBodyParser implements RequestMethodInterface
{
    public function __invoke(ServerRequestInterface $request, ResponseInterface $response, callable $next = null)
    {
        // Find content type
        $contentTypeParts = explode(&#x27;; boundary=&#x27;, $request-&gt;getHeaderLine(&#x27;content-type&#x27;));
        if (count($contentTypeParts) === 1) {
            $contentTypeParts[] = &#x27;&#x27;;
        }

        // Apply this middleware only to PUT and PATCH requests when content type is multipart/form-data
        list($contentType, $boundary) = $contentTypeParts;
        if ($contentType !== &#x27;multipart/form-data&#x27;
            || ! in_array($request-&gt;getMethod(), [self::METHOD_PUT, self::METHOD_PATCH], true)
        ) {
            return $next($request, $response);
        }
        
        // [...]
    }
}
</code></pre></pre><p>First, we get the content type, and a boundary, that will be used later to identify every body part.</p><p>If either the request method or content type are not correct, we just call the next middleware.</p><p>But let&#x27;s see now what happens when those conditions are met:</p><pre><pre><code class="language-php">class MultipartRequestBodyParser implements RequestMethodInterface
{
    public function __invoke(ServerRequestInterface $request, ResponseInterface $response, callable $next = null)
    {
        // Find content type
        // [...]

        // Apply this middleware only to PUT and PATCH requests when content type is multipart/form-data
        // [...]
        
        // Explode parts
        $parts = explode(&#x27;--&#x27; . $boundary, (string) $request-&gt;getBody());
        // Discard first and last part, which are inconsistencies from previous explode
        $parts = array_slice($parts, 1, count($parts) - 2);

        $bodyParams = [];
        $files = [];
        foreach ($parts as $part) {
            $this-&gt;processPart($files, $bodyParams, $part);
        }

        return $next(
            $request-&gt;withUploadedFiles($files)
                    -&gt;withParsedBody($bodyParams),
            $response
        );
    }
}
</code></pre></pre><p>Using the boundary (the identifier used in multipart requests to separate each part), we explode the body in order to get every separated part, and then iterate them in order to get them processed.</p><p>The <code>processPart</code> protected method is responsible of finding out the type of the part, and appending it to the <code>$files</code> array or the <code>$bodyParams</code> array. Both of those arrays are passed by reference.</p><p>Once both arrays have been populated, the next middleware is invoked, but the request now includes this information.</p><p>Let&#x27;s see the <code>processPart</code> method implementation:</p><pre><pre><code class="language-php">class MultipartRequestBodyParser implements RequestMethodInterface
{
    public function __invoke(ServerRequestInterface $request, ResponseInterface $response, callable $next = null)
    {
        // [...]
    }
    
    protected function processPart(array &amp;$files, array &amp;$bodyParams, $part)
    {
        // Separate part headers from part body
        $part = ltrim($part, &quot;\r\n&quot;);
        list($partRawHeaders, $partBody) = explode(&quot;\r\n\r\n&quot;, $part, 2);

        // Cast headers into associative array
        $partRawHeaders = explode(&quot;\r\n&quot;, $partRawHeaders);
        $partHeaders = array_reduce($partRawHeaders, function (array $headers, $header) {
            list($name, $value) = explode(&#x27;:&#x27;, $header);
            $headers[strtolower($name)] = ltrim($value, &#x27; &#x27;);
            return $headers;
        }, []);

        // Ignore any part without content disposition
        if (! isset($partHeaders[&#x27;content-disposition&#x27;])) {
            return;
        }

        // Parse content disposition, in order to find out the nature of each field
        $contentDisposition = $partHeaders[&#x27;content-disposition&#x27;];
        preg_match(
            &#x27;/^(.+); *name=&quot;([^&quot;]+)&quot;(; *filename=&quot;([^&quot;]+)&quot;)?/&#x27;,
            $contentDisposition,
            $matches
        );
        $name = $matches[2];
        $filename = isset($matches[4]) ? $matches[4] : null;

        // Check if current part is a properly uploaded file, a not uploaded file or another field
        if ($filename !== null) {
            // If file was correctly uploaded, write into temp dir and create an UploadedFile instance
            $tempFile = tempnam(ini_get(&#x27;upload_tmp_dir&#x27;), &#x27;php&#x27;);
            file_put_contents($tempFile, $partBody);

            $this-&gt;addFile($files, $name, new UploadedFile(
                $tempFile,
                strlen($partBody),
                UPLOAD_ERR_OK,
                $filename,
                isset($partHeaders[&#x27;content-type&#x27;]) ? $partHeaders[&#x27;content-type&#x27;] : null
            ));
        } elseif (strpos($contentDisposition, &#x27;filename&#x27;) !== false) {
            $this-&gt;addFile($files, $name, new UploadedFile(
                null,
                0,
                UPLOAD_ERR_NO_FILE
            ));
        } else {
            $bodyParams[$name] = substr($partBody, 0, -2);
        }
    }
}
</code></pre></pre><p>This is the most complex method. </p><p>Using a couple of explodes, an <code>array_reduce</code> and a regular expression, this method separates the headers from the body of every part, and then, depending on the information present in the <strong>content-disposition</strong> header of the part, it determines if it belongs to a properly uploaded file, a file element that has not been uploaded or a regular body parameter.</p><p>When a properly uploaded file is found, it is written in the directory configured in the ini <strong>upload_tmp_dir</strong> option, using the same file pattern used by PHP when storing files uploaded to a POST request. </p><p>Finally, it appends the parsed field to the <code>$bodyParams</code> array or the <code>$files</code> array.</p><p>There&#x27;s only one thing left to see. When an uploaded file is found, the <code>addFile</code> protected method is called. Let&#x27;s see it:</p><pre><pre><code class="language-php">class MultipartRequestBodyParser implements RequestMethodInterface
{
    public function __invoke(ServerRequestInterface $request, ResponseInterface $response, callable $next = null)
    {
        // [...]
    }
    
    protected function processPart(array &amp;$files, array &amp;$bodyParams, $part)
    {
        // [...]
    }
    
    protected function addFile(array &amp;$files, $name, UploadedFile $newFile)
    {
        $isArray = false;

        // If name has array notation, append it as array
        if (strpos($name, &#x27;[]&#x27;) === strlen($name) - 2) {
            $name = substr($name, 0, -2);
            $isArray = true;
        }

        if (! isset($files[$name])) {
            $files[$name] = $isArray ? [$newFile] : $newFile;
            return;
        }

        $files[$name] = $isArray ? array_merge($files[$name], [$newFile]) : $newFile;
    }
}
</code></pre></pre><p>Since files can be uploaded as arrays, we have to take it into account when generating the <code>$files</code> array.</p><p>If the name of the array uses the array notation (including brackets at the end, like <code>someFile[]</code> instead of <code>someFile</code>), we have to make sure the value of that file element is an array of <code>UploadedFileInterface</code> objects, and any part that is a file and uses that same name is appended to the same array under the same name.</p><p>And that&#x27;s it. If you have made some tests with the example app, you have already seen how it works.</p><h3>Considerations</h3><p>Regardless this works, it is just an experiment, and I wouldn&#x27;t recommend you to do this in your project, unless it is essential for the application to be able to upload files in a <strong>PUT</strong> or <strong>PATCH</strong> request.</p><p>If you can, I would rather change the endpoint, so that it works with the <strong>POST</strong> method.</p><p>These are the main reasons:</p><ul><li>The <code>UploadedFile</code> object implementation included in <a target="_blank" rel="noopener noreferrer" href="https://docs.zendframework.com/zend-diactoros/">zend/diactoros</a> (and thus, in zend expressive), calls PHP&#x27;s <code>move_uploaded_file</code> when the <code>$file-&gt;moveTo()</code> method is called (probably, other implementations do the same).<br/>
This throws an exception in <strong>PUT</strong> and <strong>PATCH</strong> requests, since PHP doesn&#x27;t consider those files to have been uploaded.<br/>
In order to get this working, in this example I have included a new <a target="_blank" rel="noopener noreferrer" href="https://github.com/acelaya-blog/put-patch-file-uploads/blob/master/src/App/File/UploadedFile.php">UploadedFile</a> implementation, which extend&#x27;s from diactoros&#x27; implementation, but always moves the file using the resource, without checking if the file is really an uploaded file.<br/>
While this solution works, it has some security concerns, and could be exploited by a malicious attacker.</li><li>This solution implies loading the whole body into memory, in order to parse files and temporarily save them in disk, which is a much less optimized process than having them already in disk, like in POST requests.</li><li>Generated temporary files are not deleted at the end of the request, like in <strong>POST</strong> requests. However, we could use <code>tmpfile</code> instead of <code>tempnam</code>, or delete the files after calling <code>$next</code>, if they already exist and have not been handled by other middlewares.</li></ul><p>Apart from that, I hope you learned reading the article as much as I did writing it.</p><p>Now I can say I know better how multipart requests work.</p>

Managing PUT requests with file uploads in psr-7 and middleware PHP applications

<p>Docker is, without any doubt, the trending tool these days. Everybody wants to use it, because it is very useful, allowing to easily generate development environments for any kind of application.</p><p>A couple months ago I started working with docker myself (it has taken me a while, I know), and now I can&#x27;t imagine working without it.</p><p>I started using it at work, but now I&#x27;m migrating all of my OSS projects too.</p><h3>The first stopper</h3><p>Regardless docker is very cool, there is a problem when you start using it.</p><p>I had my development environment perfectly configured, and all my tools properly integrated. I was able, among other things, to run any project tests from within PhpStorm, just by right-clicking any test class, test method or phpunit.xml file, and selecting the &quot;Run&quot; option.</p><p>However, once you start using docker in the project, it is not that simple.</p><h3>Integration with PhpStorm</h3><p>Gladly, since PhpStorm is a very powerful tool, it supports integration with docker, which can be then used, among other things, to automatically run tests inside a container.</p><blockquote><p>From this point, I&#x27;m going to explain the process, assuming you have already installed docker in your system. If not, you need to install at least <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/engine/installation/">docker engine</a> and <a target="_blank" rel="noopener noreferrer" href="https://docs.docker.com/compose/install/">docker compose</a>.</p></blockquote><p>The first thing you need to do is configure the connection with docker engine. Go to <strong>Settings</strong> -&gt; <strong>Build, Execution, Deployment</strong> -&gt; <strong>Docker</strong>.</p><p>Click the green <em>plus</em> button and a new docker connection will be created. The only value you should need to change is the API URL:</p><ul><li><strong>Linux</strong>: unix:///var/run/docker.sock</li><li><strong>Windows and Mac</strong>: <a target="_blank" rel="noopener noreferrer" href="http://127.0.0.1:2376">http://127.0.0.1:2376</a> (The URL of the docker machine. If you are using the deprecated docker toolbox with boot2docker, you have to make sure which is the IP address of the virtual machine)</li></ul><p><img src="/assets/img/phpstorm-docker/phpstorm-docker.png" alt="Docker in PhpStorm"/></p><p>Apply changes and we are done here.</p><h3>Create the remote interpreter</h3><p>Now let&#x27;s create a remote interpreter. It will be used so that tools that need a php runtime are executed inside a docker container.</p><p>Go to <strong>Settings</strong> -&gt; <strong>Languages &amp; Frameworks</strong> -&gt; <strong>PHP</strong>. In here, you select the default interpreter for your project.</p><p><img src="/assets/img/phpstorm-docker/interpreter.png" alt="Interpreter"/></p><p>Click the button with the three dots, which will open the dialog to create a new interpreter.</p><p><img src="/assets/img/phpstorm-docker/interpreter-dialog.png" alt="Interpreter dialog"/></p><p>In that dialog you will likely have a local interpreter. Click the green <em>plus</em> button, at the top-left corner, and select <strong>Remote</strong>.</p><p>That will open the dialog to create a new remote interpreter. One of the options is to use a docker image from which PhpStorm will create a docker container that should contain a php interpreter.</p><p><img src="/assets/img/phpstorm-docker/docker-interpreter.png" alt="Docker iterpreter"/></p><p>In this dialog, just select the server (this is the connection with the docker engine, the one we created in the first step), then select the docker image you want to use. The dropdown will be filled with all the images in your machine.</p><p>Finally select the interpreter path. Usually, <a target="_blank" rel="noopener noreferrer" href="https://hub.docker.com/_/php/">PHP-based docker images</a> have the php interpreter in the PATH, so just setting <code>php</code> should be enough.</p><p>Save everything and we are done configuring the interpreter.</p><h3>Create the PHPUnit config</h3><p>Now that the interpreter is created, we have to define a phpunit configuration that uses it.</p><p>Go to <strong>Settings</strong> -&gt; <strong>Languages &amp; Frameworks</strong> -&gt; <strong>PHP</strong> -&gt; <strong>PHPUnit</strong>, Use the green <em>plus</em> button at the top-left corner, select <strong>By Remote interpreter</strong> and a dialog window will be prompt.</p><p>It only contains a dropdown where you have to select the remote interpreter we created in previous step.</p><p>Select it and you&#x27;ll see a configuration similar to this.</p><p><img src="/assets/img/phpstorm-docker/phpunit.png" alt="PHPUnit configuration"/></p><p>As you can see, PhpStorm mounts the project dir as a volume inside the container, in the <code>/opt/project</code> dir, and based on that path, selects the path to composer&#x27;s autoloader script and the phpunit.xml script.</p><p>If any of those params is not correct in your project, you can (and should) change it.</p><p>And that&#x27;s it!</p><p>Now, you can right-click any test class, test method or phpunit.xml file and select &quot;Run&quot;. PhpStorm will automatically connect with docker engine, set up a temporary container and run the tests inside of it.</p><p>You&#x27;ll know it is working because of this:</p><p><img src="/assets/img/phpstorm-docker/tests-in-container.png" alt="Tests inside container"/></p><h3>Conclusion</h3><p>PhpStorm&#x27;s team is always working to add integrations with the latest development tools, making it the most powerful IDE for PHP.</p><p>Now go, configure your environment and start using docker. It&#x27;s amazing!</p>