Software Patterns & Best Practice: What if the authors are wrong?

I take the following view:

Software patterns and “best practice” are just the opinions of people who managed to get published.

Who says they are right?
Why are they right?
Are they right?

Because in my considerable experience there is no such thing as “best practice”:

You do the best you can, with what you have, under the circumstances at the time.

As for software patterns…. pfft!
I’ve tried reading “Design Patterns: Elements of Reusable Object-Oriented Software” and “Patterns of Enterprise Application Architecture” and never made it more than a quarter of the way through. (I think I made it further in War and Peace when I was a teenager.)
Then there’s SOLID and REST APIs.
(REST was a f**king thesis for goodness sake!)

Some people blindly follow the so-call best practices of software but forget they are just the opinions of a handful – literally a handful – of people who manage to get noticed or get a book deal.
And in no way do they reflect reality.

Reality is “average” people try to figure out the best, fastest and easiest way to implement software for business requirements that won’t get them fired.

It’s 2019 and today we’re all supposed to be doing DevOps, Agile, writing unit tests, performing automated testing, using non-SQL databases, in the Cloud, following OWASP Top 10, logging and auditing everything and making no mistakes.
(There are major companies in my city with names on the top of skyscrapers asking university students to teach them about DevOps pipelines.)

The reality is it’s no better than 15 years ago.
We still haven’t solved the “low hanging fruit” problems.

Every day brings a new data breach. Automated or unit testing is not in the budget. Agile just means standing in a circle telling your team what your working on each morning (if your team lead gets their shit together). Many businesses are still running Exchange, Active Directory and databases on-site. Windows 7 is still live in Enterprise. Testing is more of a pain-in-the-ass than ever, Selenium still sucks, and SQL databases still rules.
(Also jQuery isn’t going anywhere, and if Google or Facebook were really ‘that good’, why are there so many versions of React and Angular where they’re still trying to get it right?)

Plus there is still someone running a PC under their desk in a small-to-mid sized business with the all important Excel file/Access database/ASP website that the business lives-or-dies on.

Just because the most vocal people in the industry say “it should be” doesn’t make it so.
Businesses still pays the bills, and the priorities of businesses are not the “best practice” of software development.
Businesses don’t give a shit about “software best practice”.
That doesn’t make it right – but it is reality.

Right now I can’t recite to you a single definition of SOLID, or tell you definitively how any software pattern is supposed to work.
But I’ve dedicated over 50,000 hours of my life to creating software and solving business problems with software, and I can tell you this for certain: it doesn’t matter.

What really matters is simple:

  • Keep solutions and code simple.
  • Always think about security before anything else. (Then actually make your fucking systems secure!)
  • Make software usable and understandable for users.
  • Create readable code for yourself and your fellow software developers (if it can be read by a junior developer you’ve done your job).
  • Write documentation. Code is NOT self-documenting – it requires documentation and comments.
  • Make sure the software can be maintained and changed easily.
  • Log changes and events. It doesn’t matter how – just make sure you start logging from day 1.
  • Watch and monitor the systems you build.
  • Remember: You are not the user. How you think it will be used is almost certainly wrong or not enough.
  • If you find yourself fighting your tools or swearing at your screen, you’re doing it wrong.
  • Bonus: Command-line requires you to remember command, and more manual work. GUIs do not. Create GUIs. Command-line sucks.

And any estimate you give, multiply it by 3 first.

Finally, consider this:

The only times I’ve ever discussed or had to describe software patterns is in highly technical job interviews.

What does that tell you?
I talk to a lot of developers, on a lot of topics, so why haven’t we had “software pattern” conversations in passing?
I would welcome it! If only to learn how other developers have implemented different software patterns (successfully or otherwise).

In reality I think we all work with different patters, we just don’t think about it at the time, and frankly who gives a shit – we’re doing our job. Do we really need to say “hey, I just implemented the observer patter”? (I had to look that up to give you an example).

I’ve spent almost 20 years reading and and modifying other people’s code. I’m an expert debugger in my field (CSS is my favourite, and raw JavaScript is my friend).
I don’t care what patterns people use (because what’s the “right” or “wrong” pattern anyway?)
What I care about is being able to quickly and easily read and understand code when I have to fix a problem that is causing difficulties for a customer or employer.

Software patterns and “best practice” don’t mean shit.

What matters are:

  • Business continuity
  • Security
  • Usability
  • Readability and Maintenance
  • Extensibility

Everything else is a side issue.

The thing about Micro-services

You might decrease the complexity of a monolithic application (I’m not actually sure that’s even a valid statement, that a “monolith application” is “complex”) but you conversely increase the complexity by adding in all the network hops, latency, asynchronicity, and having to debug strange timeout errors.

It doesn’t matter what you call your architecture, you’re still going to encounter an equal number of problems to think about, debug and solve.

WordPress: Changing File Upload and Processing Sizes

[This is a post for self-reference]

Up front I’m going to say I got exactly the help I need from here: https://www.wpbeginner.com/wp-tutorials/how-to-increase-the-maximum-file-upload-size-in-wordpress/.
The author of that post deserves all my credit.

I’m creating this post to summarise it in my own words and to give it a little educational context.


When allowing file uploads on a website there are 3 big things you need to think about:

  • The size of each file.
  • The total size of all files in an upload.
  • The time it takes to process the upload.

Actually, there are a couple of things to consider that are worthy of mention but I won’t cover in this post:

  • Asynchronous uploads (AJAX as we used to call them) can change the figure of an upload, because instead of uploading multiple files at one, you upload one file at a time (even if they are in parallel) so the “total size of all files” number will be different.
  • Tell users about the limits they face before they upload. Also, when uploading images a bit of guidance around things like “27MB images are a stupid side for most sites because 27MB isn’t going to be better than 7MB, and the site will trim the size down to a couple of hundred KB at most anyway.”
  • Trap, log and respond to any errors. This is on my mind because WordPress/PHP seems to like gobbling errors and not telling anyone about it (I’m in the process of hunting down how to handle this and will be a topic of another post).

 

There are 3 main ways in PHP to affect file upload sizes and times:

  • @set_init() in a PHP script itself.
  • php.ini file
  • .hataccess file.

Be aware, as a PHP/Linux novice I reckon there are other defaults and hierarchies that come into play.
This article is spawned by a situation where I was changing php.ini without realising .htaccess had a value overriding it.

 

How do you implement these values?

This is where I rip from the wpbeginner site I mentioned above for quick reference:

1. PHP script file (e.g. functions.php in WordPress themes)

@ini_set( 'upload_max_size' , '64M' );
@ini_set( 'post_max_size', '64M');
@ini_set( 'max_execution_time', '300' );

(I haven’t tried this approach yet, but I have used @ini_set() in other instances (e.g. ‘memory_limit’) and it’s worked well.

2. php.ini

upload_max_filesize = 64M
post_max_size = 64M
max_execution_time = 300

I’ve used this approach in development and it did the trick.

3 .htaccess

php_value upload_max_filesize 64M
php_value post_max_size 64M
php_value max_execution_time 300

Before writing this post I had to fix a production issue and after trying php.ini I got lucky and discovered .htaccess was where the value was coming from.
So the moral: .htaccess overrides php.ini.

Also, I did not need to restart apache for this change to take affect.

 

DISCLAIMER: I haven’t actually read in-depth how all this works. I’m a typical developer who expects other people’s shit to work, and when it doesn’t I’m usually in the middle of my own concerns so it’s a case of “search and scan by keyword until someone else’s solution works for me”.
Such is this age of software development.

 

Need to know what your current settings are?

It’s helpful in debugging and verification to know what your actual PHP settings are so you know what your starting point is (and  you have a number to search your files for).

A good way is to call phpinfo();.

For example, create a file in your site (I call mine info.php) and add the following PHP code:

<?php
phpinfo();
?>

You should see something like the following when you run it. This screenshot shows some output from the section titled “Core”, and the 3 columns are (from left to right): “Directive”, “Local Value” and “Master Value”. I can say from experience the “Local Value” is the one that matters.

phpinfo() output

Install XML RPC for PHP 7.2 on Ubuntu Linux

Background

One of the components in a web application solution we’re developing uses xmlrpc_encode_request() in PHP.

It was executing fine in most of our systems systems – development and multiple test servers – but while trying to run it for the first time in my development environment I received the following error:

Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request()

So off to Google I go and, find a few answers, have a couple of mis-starts, then finally get it working.

 

Solution

The solution was very simple.

  1. Install the xmlrpc extension.
    sudo apt-get install php-xmlrpc
  2. Restart Apache web server
    sudo service apache2 restart

As a sanity check, create a PHP file (e.g. info.php) and add the following:

<?php
echo 'XMLRPC is ', extension_loaded('xmlrpc') ? 'loaded' : 'not loaded';
?>

When you run the file it you should see “XMLRPC is loaded” if all is good.

If you’re still having trouble, try updating your php.ini file to include:

extension=xmlrpc.so;

and/or

extension=php_xmlrpc.so;

Unfortunately, if you still have trouble then I apologise and suggest hitting Google/Bing/DuckDuckGo/StackOverflow again and playing around with search terms that meet your specific Linux/OS/PHP versions.

 

More Background

I started with a search for “Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request” then “lamp Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request”, and had these results:

But they were aimed at PHP 5.x (by the way, if you’re still on PHP 5.x then update! It’s no longer supported and you’re now a security risk).

Then I refined my search to “apt-get php 7.2 xmlrpc” and had:

And I found the helpful suggestion for the info PHP file (you could also just run phpinfo(); in a file) at:

Upgade Cloud9 Workspace to PHP 7.2

Updated: 14 April 2019

According to this announcement, the old Cloud9 service (c9.io) is soon to be no more, with all functionality ending 30 June 2019, and complete shutdown 30 November 2019.
What is the future of C9? It will be 100% within Amazon Web Services as AWS Cloud9.
I – like many others – am not a fan of AWS. In fact, I think AWS is one of the worst development services I’ve used, and Cloud9 proves it. They took an incredibly powerful and easy to use service and… complicated it. To the point I spent an hour of my Sunday afternoon swearing while trying to setup a new AWS Cloud9 WordPress environment.

As such, I am unsure at this time if I will continue to work with AWS Cloud9 or find an alternate cloud IDE. I use IDEs in a professional capacity and I’m happy to pay for convenience and ease-of-development experience.

Anyway, I won’t be attempting any C9.io workspace upgrades in future. I will, however, create a new post when I decide which development environment to use.

 

Updated: 03 March 2019
After my original environment upgrade I started to see CPU maxing on page loads. I thought it was code changes, but it wasn’t. I run WordPress in my environment and even /wp-admin was causing a 5-20x increase in page load times with 100% CPU spikes.

I happened to find a copy of my environment that was still the original PHP 5.5.9 version and have been using that since a week after this original post. CPU has been fine.

Today I cloned my working environment, did some more searching, and tested a PHP upgrade again. The section “Updated Solution (03 March 2019)” below contains the results of that upgrade. So far, so good. The upgrade process is not that different to the original.

However, /phpmyadmin shows an error “The mbstring extension is missing. Please check your PHP configuration.” I’m not surprised because I my script excluded that (see notes within the 03 March section).
I will work on this next, but I wanted to publish updated results before that happens.


Updated Solution (03 March 2019)

Today’s results are modified from [and thanks to] https://www.joeferguson.me/install-and-configure-php-7-1-on-cloud-9-workspaces/.

Run the following commands, but note:

sudo add-apt-repository ppa:ondrej/php -y
sudo apt-get update -y

#sudo apt-get install php7.2-curl php7.2-cli php7.2-dev php7.2-gd php7.2-intl php7.2-mcrypt php7.2-json php7.2-mysql php7.2-opcache php7.2-bcmath php7.2-mbstring php7.2-soap php7.2-xml php7.2-zip -y
sudo apt-get install php7.2-curl php7.2-cli php7.2-dev php7.2-gd php7.2-intl php7.2-json php7.2-mysql php7.2-opcache php7.2-bcmath php7.2-mbstring php7.2-soap php7.2-xml php7.2-zip -y

sudo mv /etc/apache2/envvars /etc/apache2/envvars.bak
sudo apt-get remove libapache2-mod-php5 -y
sudo apt-get install libapache2-mod-php7.2 -y
sudo cp /etc/apache2/envvars.bak /etc/apache2/envvars

sudo a2dismod php5
sudo a2enmod php7.2

sudo service apache2 restart

 

 

 

 


Original Post Solutions

Cloud9 is a great online IDE (or it was before the Amazon AWS team got their hands on it).

The problem is the default PHP version in my workspace was 5.6, well out of date and unsupported.

It turns out, upgrading PHP version’s isn’t a simple matter. After much search in found this solution on StackOverflow that worked for me:

https://stackoverflow.com/a/51028239/115704

sudo add-apt-repository ppa:ondrej/php -y
sudo apt-get update -y

sudo apt-get install php7.2 php-pear php7.2-curl php7.2-dev php7.2-gd php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml -y

sudo mv /etc/apache2/envvars /etc/apache2/envvars.bak
sudo apt-get remove libapache2-mod-php5 -y
sudo apt-get install libapache2-mod-php7.2 -y
sudo cp /etc/apache2/envvars.bak /etc/apache2/envvars

sudo a2dismod php5
sudo a2enmod php7.2

sudo service apache2
sudo service apache2 restart

Git: branch commit is in the future

I was just checking some branch commits from one of my developers and noticed the commits where 22 hours in the future.

It turns out they’d changed their system time for testing and forgot to revert it before pushing.

It makes sense when I think about it.

Because git is a distributed system, it would use the local system time when committing.
Git doesn’t track track off a server time.
That means you can be totally disconnected from other systems and still make commits – in that case the only time it knows is the system time.

That doesn’t make it any less confusing or counter-intuitive in an age of servers, synchronisation and centralisation.

Visually hidden field with ‘required’ attribute causes error: “An invalid form control with name=’xxxxx’ is not focusable.”

Situation

I asked one of my developers to comment out a field in a web page as part of a requirements change (I prefer to comment out initially before deleting so we are sure everything works fine with the change).

The developer added a style="display: none" to a surrounding div to hide a <select> field.

However, this in turn caused a JavaScript console error of An invalid form control with name='xxxxx' is not focusable when trying to submit the form.

Solution

In helping my developer I did a Google search for “an invalid form control with name=” is not focusable” and one of the first results lead to https://stackoverflow.com/questions/22148080/an-invalid-form-control-with-name-is-not-focusable.

On that page I immediately saw “Adding a novalidate attribute to the form will help” and that made me to immediately remember there is a “required” attribute still on the field that was hidden.

Experience and instinct kicked in.

We removed the “required” attribute and the form submission worked.

My guess is when the form was submitted the native browser-based validation did not work correctly because the specified field is “visually” hidden so the native validation message could not be shown, and that triggered the error in the console.

RegEx: Find occurrences in code, except when commented

Background

I’ve created a debug helper function in my PHP/WordPress development called TraceIt($value, $type, $param1).
TraceIt does a bit more than just echo out a value. It will format echo out based on value type or specified type, can be environmentally switched to not echo, and can write to a log file.

It’s also easier to find where actual tracing is being used versus the legitimate echo output.

The problem is, sometimes I forget to comment a trace, which is no good, especially when trying to perform a redirect.
And while I usually include a string value in the output that I can search for, sometimes in the heat of the debugging moment I forget that.

 

Solution

The easiest [and some may argue “painful”] way to find occurrences of forgotten TraceIt calls is regex.

What I needed was a way to find all instances of “TraceIt” but not “//TraceIt” (already commented).

And here it is:

^(?!([ \t]*\/\/[ \t]*TraceIt)|[ \t]*\/\/).*((^TraceIt)|( TraceIt)|(\tTraceIt)|(;TraceIt))

You can see it in action, with an explanation of each component, at https://regexr.com/46nbq.

The only problem with this is it also picks up instances of TraceIt on a line that already has //TraceIt earlier in the line.  But I can live with those rare occurences for now.

Oh, and don’t forget to use the /gim flags (global, case insensitive and multi-line).

 

The regex will find occurrences like this:

coding is here; TraceIt("aaaa");
TraceIt("aaaa");
another line of code;
TraceIt('bbb'); more code
TraceIt('ccc');
TraceIt('bbb');
some more code; TraceIt("adsfadfa");

But not like this:

//TraceIt("aaaa");
//TraceIt("bbb"); TraceIt("ccc")
// TraceIt("ddd");
// TraceIt("ddd");
// TraceIt("eee");

another line of code;

some code; //TraceIt('fff');
    //     // TraceIt("gggg");

platform_browser_dynamic_1.platformBrowserDynamic is not a function

I received the following error today in a project I’m working on (the relevant part is in bold):

MainModule.bundle.js:27136 Uncaught TypeError: platform_browser_dynamic_1.platformBrowserDynamic is not a function
   at Object.__decorate (MainModule.bundle.js:27136)
   at __webpack_require__ (Polyfills.bundle.js:55)
   at webpackJsonpCallback (Polyfills.bundle.js:26)
   at MainModule.bundle.js:1

You can see my response to a similar problem at https://github.com/angular/angular/issues/10732#issuecomment-417173741.

And my response here as well, for completeness:

I just had the same issue in a project I’m working on. We’re using Angular 5.0.2 and Webpack on Windows in an IIS/ASP.NET project, with our primary browser as Chrome.
My issue randomly appeared after rebooting my PC and loading up my dev environment again (no code or configuration changes).
I managed to resolve the issue after a couple of attempts at starting the project (in Visual Studio) by also browsing to the the app URL Microsoft Edge. It loaded find. I then did a hard refresh (Ctrl+F5) in Chrome and it came good too.
Note: we manually start and leave running Webpack in separate command window, but start the actual application from withing Visual Studio.
My guess is Webpack crapped out behind the scenes and needed some time and a hard refresh in the browser to get the correctly compiled code to the browser again.