Software Patterns & Best Practice: What if the authors are wrong?

I take the following view:

Software patterns and “best practice” are just the opinions of people who managed to get published.

Who says they are right?
Why are they right?
Are they right?

Because in my considerable experience there is no such thing as “best practice”:

You do the best you can, with what you have, under the circumstances at the time.

As for software patterns…. pfft!
I’ve tried reading “Design Patterns: Elements of Reusable Object-Oriented Software” and “Patterns of Enterprise Application Architecture” and never made it more than a quarter of the way through. (I think I made it further in War and Peace when I was a teenager.)
Then there’s SOLID and REST APIs.
(REST was a f**king thesis for goodness sake!)

Some people blindly follow the so-call best practices of software but forget they are just the opinions of a handful – literally a handful – of people who manage to get noticed or get a book deal.
And in no way do they reflect reality.

Reality is “average” people try to figure out the best, fastest and easiest way to implement software for business requirements that won’t get them fired.

It’s 2019 and today we’re all supposed to be doing DevOps, Agile, writing unit tests, performing automated testing, using non-SQL databases, in the Cloud, following OWASP Top 10, logging and auditing everything and making no mistakes.
(There are major companies in my city with names on the top of skyscrapers asking university students to teach them about DevOps pipelines.)

The reality is it’s no better than 15 years ago.
We still haven’t solved the “low hanging fruit” problems.

Every day brings a new data breach. Automated or unit testing is not in the budget. Agile just means standing in a circle telling your team what your working on each morning (if your team lead gets their shit together). Many businesses are still running Exchange, Active Directory and databases on-site. Windows 7 is still live in Enterprise. Testing is more of a pain-in-the-ass than ever, Selenium still sucks, and SQL databases still rules.
(Also jQuery isn’t going anywhere, and if Google or Facebook were really ‘that good’, why are there so many versions of React and Angular where they’re still trying to get it right?)

Plus there is still someone running a PC under their desk in a small-to-mid sized business with the all important Excel file/Access database/ASP website that the business lives-or-dies on.

Just because the most vocal people in the industry say “it should be” doesn’t make it so.
Businesses still pays the bills, and the priorities of businesses are not the “best practice” of software development.
Businesses don’t give a shit about “software best practice”.
That doesn’t make it right – but it is reality.

Right now I can’t recite to you a single definition of SOLID, or tell you definitively how any software pattern is supposed to work.
But I’ve dedicated over 50,000 hours of my life to creating software and solving business problems with software, and I can tell you this for certain: it doesn’t matter.

What really matters is simple:

  • Keep solutions and code simple.
  • Always think about security before anything else. (Then actually make your fucking systems secure!)
  • Make software usable and understandable for users.
  • Create readable code for yourself and your fellow software developers (if it can be read by a junior developer you’ve done your job).
  • Write documentation. Code is NOT self-documenting – it requires documentation and comments.
  • Make sure the software can be maintained and changed easily.
  • Log changes and events. It doesn’t matter how – just make sure you start logging from day 1.
  • Watch and monitor the systems you build.
  • Remember: You are not the user. How you think it will be used is almost certainly wrong or not enough.
  • If you find yourself fighting your tools or swearing at your screen, you’re doing it wrong.
  • Bonus: Command-line requires you to remember command, and more manual work. GUIs do not. Create GUIs. Command-line sucks.

And any estimate you give, multiply it by 3 first.

Finally, consider this:

The only times I’ve ever discussed or had to describe software patterns is in highly technical job interviews.

What does that tell you?
I talk to a lot of developers, on a lot of topics, so why haven’t we had “software pattern” conversations in passing?
I would welcome it! If only to learn how other developers have implemented different software patterns (successfully or otherwise).

In reality I think we all work with different patters, we just don’t think about it at the time, and frankly who gives a shit – we’re doing our job. Do we really need to say “hey, I just implemented the observer patter”? (I had to look that up to give you an example).

I’ve spent almost 20 years reading and and modifying other people’s code. I’m an expert debugger in my field (CSS is my favourite, and raw JavaScript is my friend).
I don’t care what patterns people use (because what’s the “right” or “wrong” pattern anyway?)
What I care about is being able to quickly and easily read and understand code when I have to fix a problem that is causing difficulties for a customer or employer.

Software patterns and “best practice” don’t mean shit.

What matters are:

  • Business continuity
  • Security
  • Usability
  • Readability and Maintenance
  • Extensibility

Everything else is a side issue.

Uninstall/remove Spotify on Windows 10

This is mainly a post for my own reference.

I had trouble uninstalling the Windows 10 Spotify application from an old system.

Going into Control Panel\Programs\Programs and Features and trying to uninstall simply wouldn’t do anything.

A quick search lead me to https://community.spotify.com/t5/Desktop-Windows/How-to-permanently-uninstall-Spotify-on-Windows-10/td-p/1697157 which had the answer.

Here are the steps you need to take:

  1. Open Windows Task Manager and kill the “Spotify Web Connector” process (It will be something like SpotifyWebConnector.exe or SpotifyWebHelper.exe – sorry, I forgot to note the exact name before writing this post, but it will be obvious in the process list).
  2. Disable your network/internet access (turn off/disconnect your wireless or pull out your network cable).

Now you should be able to uninstall Spotify. Just be patient give it a few seconds when you first try – there’s going to be a bit of a delay before the confirmation dialog comes up.

Don’t forget reconnect your internet access.

The thing about Micro-services

You might decrease the complexity of a monolithic application (I’m not actually sure that’s even a valid statement, that a “monolith application” is “complex”) but you conversely increase the complexity by adding in all the network hops, latency, asynchronicity, and having to debug strange timeout errors.

It doesn’t matter what you call your architecture, you’re still going to encounter an equal number of problems to think about, debug and solve.

Being able to identify a problem doesn’t mean I have an answer

I’m a big proponent of asking questions. But one questions I hate is:

What’s the solution?

It’s a loaded question, intended to deflect.

The scenario where we most hear this is when someone says “X is a problem” to a creator, leader or manager of X.
Then the leader immediately replies: “Great. What’s the solution? How do we fix it?”

Similarly, some leaders will say: “Don’t just raise problems. Suggest solutions.”

This is a pure bullshit tactic by leaders to deflect the problem back on the person reporting it.

I look at it like this:

Just because a person can see a problem, doesn’t mean they have the vision to understand the cause or how to solve it.

Likewise, they probably do not have the vision to create the thing that caused the problem in the first place.

Humans are good at spotting problems. We’re not that great at finding solutions.

Yes, there are times when the person identifying the problem can offer a solution. These are times where their expertise is what allows them to see the problem in the first place.

But working in business, many times I’ve heard someone raise a concern that is immediately quashed by the “how do we solve it” question, the underlying subtext being “if don’t have an answer, don’t ask the question”.

And with a quiet flurry, we launched today!

I’ve just worked 15 days straight, with 12 hours being my minimum work day.
Some days were closer to 14-16 hours.
That’s a minimum 180 working hours in 2 weeks.
(And since I came back from a holiday on the 4th January, I’ve had a total of 3 days without working.)

Those are not figures to be proud of. Working massive hours as a software developer is not a badge of honour. It’s a bad of stupidity (usually).

But in this case it’s been worth it. The time has been spent on a last minute push as a new university year starts and we complete features for the rollout of a new Intranet for one of the local colleges.

It’s a project 2 years and 1 week in the making since first discussion.
And about 18 months in development.

The first 3 months were binned after the requested technology was deemed non-viable.
They specified SharePoint. It would never achieve what they wanted in the long run.

So I said – almost as a joke – “what about WordPress?”
The IT Managers said “evaluate it and let me know”. So I did. And we decided it was a better choice.
That was late November 2017.

I probably could have finished this project 9 months ago with the base WordPress and a collection of existing plug-ins.
And we would have spent the next 12 months fighting fires, hacking together fixes and losing a lot of sleep over security.

Instead, what started as a project evolved into a product and platform – one we plan on taking well into the future.

Over the course of 2018 – while also working full-time until September – I developed the web application platform designed with security, usability, customisation and extensibility in mind. And, of course, ongoing maintenance and support.
During that time I leant WordPress and PHP development. I also went through nearly 2 full evolution builds of the core architecture – effectively jamming 2-3 years of product life into about 3 months prior to initial release.

I don’t believe in traditional MVP (Minimum Viable Product), especially when security is involved.
Most small-team MVP projects are a “hack”.

Through most of the last 12 months of the project I have been at the helm of design, architecture and development. My business partner has been project manager and [a very excellent] chief tester. And our 2 developers progressively took on a growing share of the programming work as other work finished.

The weekend gone was one of last minute development, testing and bug fixes. Nothing that phased us – just the sort of things at usually come up or get left to the time of “launch”.
And in future projects we will even reduce that happening.

Yesterday (Sunday) afternoon I reset and our production environment (which was setup a couple of months ago). It was ready before dinner. We were at that point effectively live.

This morning I sent a message to the project sponsor telling him they can start the 2 week process of creating content.

And that was it.
No stress. No anxiety. No last minute deployments. Just a simple message and we were running.

 

The story doesn’t end here though. In different ways ways it’s just the start and the middle.

Today, already, there were a couple of inevitable bugs that come from a first-time “real life” use of new software.

We also still have a number of tasks to complete and progressively roll in over the coming 2 weeks.

Then the final phase of the project needs to be run to complete development for another set of workflows and some different types of user.

But, the system is running. And we are proud creators.

 

The product itself – outside of this particular project – already has a massive backlog of work planned for this year alone.

PWA (Progressive Web Application) optimisation, push notification services, a more responsive SPA-like (Single Page Application) interface, and an accompanying full API. Not to mention plenty of design and UX enhancements.

In a few weeks, as the semester starts and this phase of the project completes, I’ll start setting up fully-functional demo sites people can play with. We don’t have a proper DevOps pipeline yet, but I can still setup a new demo site in under 15 minutes, and most of that is waiting for files to transfer.

And then comes the process of marketing, sales and finding new customers with business needs we can solve.

 

Regardless of all that, here’s something I can happily take to bed tonight:

We designed, developed and delivered a working product that solves some very real business problems.

WordPress: Changing File Upload and Processing Sizes

[This is a post for self-reference]

Up front I’m going to say I got exactly the help I need from here: https://www.wpbeginner.com/wp-tutorials/how-to-increase-the-maximum-file-upload-size-in-wordpress/.
The author of that post deserves all my credit.

I’m creating this post to summarise it in my own words and to give it a little educational context.


When allowing file uploads on a website there are 3 big things you need to think about:

  • The size of each file.
  • The total size of all files in an upload.
  • The time it takes to process the upload.

Actually, there are a couple of things to consider that are worthy of mention but I won’t cover in this post:

  • Asynchronous uploads (AJAX as we used to call them) can change the figure of an upload, because instead of uploading multiple files at one, you upload one file at a time (even if they are in parallel) so the “total size of all files” number will be different.
  • Tell users about the limits they face before they upload. Also, when uploading images a bit of guidance around things like “27MB images are a stupid side for most sites because 27MB isn’t going to be better than 7MB, and the site will trim the size down to a couple of hundred KB at most anyway.”
  • Trap, log and respond to any errors. This is on my mind because WordPress/PHP seems to like gobbling errors and not telling anyone about it (I’m in the process of hunting down how to handle this and will be a topic of another post).

 

There are 3 main ways in PHP to affect file upload sizes and times:

  • @set_init() in a PHP script itself.
  • php.ini file
  • .hataccess file.

Be aware, as a PHP/Linux novice I reckon there are other defaults and hierarchies that come into play.
This article is spawned by a situation where I was changing php.ini without realising .htaccess had a value overriding it.

 

How do you implement these values?

This is where I rip from the wpbeginner site I mentioned above for quick reference:

1. PHP script file (e.g. functions.php in WordPress themes)

@ini_set( 'upload_max_size' , '64M' );
@ini_set( 'post_max_size', '64M');
@ini_set( 'max_execution_time', '300' );

(I haven’t tried this approach yet, but I have used @ini_set() in other instances (e.g. ‘memory_limit’) and it’s worked well.

2. php.ini

upload_max_filesize = 64M
post_max_size = 64M
max_execution_time = 300

I’ve used this approach in development and it did the trick.

3 .htaccess

php_value upload_max_filesize 64M
php_value post_max_size 64M
php_value max_execution_time 300

Before writing this post I had to fix a production issue and after trying php.ini I got lucky and discovered .htaccess was where the value was coming from.
So the moral: .htaccess overrides php.ini.

Also, I did not need to restart apache for this change to take affect.

 

DISCLAIMER: I haven’t actually read in-depth how all this works. I’m a typical developer who expects other people’s shit to work, and when it doesn’t I’m usually in the middle of my own concerns so it’s a case of “search and scan by keyword until someone else’s solution works for me”.
Such is this age of software development.

 

Need to know what your current settings are?

It’s helpful in debugging and verification to know what your actual PHP settings are so you know what your starting point is (and  you have a number to search your files for).

A good way is to call phpinfo();.

For example, create a file in your site (I call mine info.php) and add the following PHP code:

<?php
phpinfo();
?>

You should see something like the following when you run it. This screenshot shows some output from the section titled “Core”, and the 3 columns are (from left to right): “Directive”, “Local Value” and “Master Value”. I can say from experience the “Local Value” is the one that matters.

phpinfo() output

Change a WordPress site URL (when copying or migrating a WordPress site)

This is a post for quick reference. But given 25% (or more) of the Interwebs run on WordPress I want to give it some love too.

Background

Moving (migrating) or copying a WordPress site has some hidden complexity.

While it is easy, there are 3 things you need to do:

  1. Copy the whole website structure, starting with the folder than contains the wp-config.php and other files (it also contains the  “wp-admin”, “wp-content” and “wp-includes” folders).
  2. Change the URL of the site as defined in WordPress (ordinarily set during setup, or within the Settings area).
  3. Update all the saved URLs within the database.

Step 3 is the real gotacha. When you add a link to another page within your site, or insert media, or set a navigation item, it saves the full URL of that item in the database.

For example, if you have a site “https://MySite.com” but want to change it to “https://MySite.com” (or as a software developer, you may want to create a copy of the production site at “https://test.MySite.com”) then after you change the actual URL of the site, all the internal links and images will still be pointing to the original MySite.com.

 

Solution

Many sites have covered how to change a WordPress site URL before, including the official Codex at https://codex.wordpress.org/Changing_The_Site_URL.

I like to keep things simple, so this post is my process:

  1. Backup your site database if you can (I connect directly to the database and run backup scripts, or use phpmyadmin and export).
  2. Update your site URLs by adding the following lines to the top of your active theme’s functions.php file:
    update_option( 'siteurl', 'https://MyNewSite.com' );
    update_option( 'home', 'https://MyNewSite.com' );
    
    1. Browser to your new site (via the IP or URL).
    2. Browse to it again and make sure it’s OK.
    3. Remove the lines from functions.php
    4. Browser to the site again one more time to make sure you can access it now you’ve removed the lines from functions.php.
  3. Upload the interconnect/it “Database Search and Replace Script” script to our site, found at https://interconnectit.com/products/search-and-replace-for-wordpress-databases/.
    1. Install it in a “not easily guessed” sub-directory of the site. This is for security reasons (so other people can’t scan and execute the scripts).
    2. Most fields are filled in. In the “search for…” enter your old URL or IP address, including “http” or “https” (e.g. “https://MySite.com” – you should be using https by now). I don’t end with a trailing forward slash.
      In the “replace with…” field enter the new URL or IP Address (e.g. “https://MyNewSite.com”). Again, not trailing forward slash.
    3. Make sure you do “dry run” to start. This should tell you if you have any problems.
    4. If all is good, do a “live run” and update your database.
    5. At the end, delete the “Database Search and Replace Script” from your site. This is important. If you leave it there bad people may find it and execute it and break your site.

Microsoft All-in-One Media Keyboard: How to Print Screen

I have a backup “Microsoft All-in-One Media Keyboard” I need to use when my primary Ergonomic Wireless Desktop keyboard decides to die (possibly “concussion” induced at 11pm when technology decides to become my Nemesis).

The only problem is it does not come with a Print Screen button.

But thanks to some wonderful soul on the Interwebs we have an answer:

Print full screen: Fn + Windows Key + Space
Print current window: Fn + Alt + Space

Source: Microsoft Answers

Install XML RPC for PHP 7.2 on Ubuntu Linux

Background

One of the components in a web application solution we’re developing uses xmlrpc_encode_request() in PHP.

It was executing fine in most of our systems systems – development and multiple test servers – but while trying to run it for the first time in my development environment I received the following error:

Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request()

So off to Google I go and, find a few answers, have a couple of mis-starts, then finally get it working.

 

Solution

The solution was very simple.

  1. Install the xmlrpc extension.
    sudo apt-get install php-xmlrpc
  2. Restart Apache web server
    sudo service apache2 restart

As a sanity check, create a PHP file (e.g. info.php) and add the following:

<?php
echo 'XMLRPC is ', extension_loaded('xmlrpc') ? 'loaded' : 'not loaded';
?>

When you run the file it you should see “XMLRPC is loaded” if all is good.

If you’re still having trouble, try updating your php.ini file to include:

extension=xmlrpc.so;

and/or

extension=php_xmlrpc.so;

Unfortunately, if you still have trouble then I apologise and suggest hitting Google/Bing/DuckDuckGo/StackOverflow again and playing around with search terms that meet your specific Linux/OS/PHP versions.

 

More Background

I started with a search for “Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request” then “lamp Fatal error: Uncaught Error: Call to undefined function xmlrpc_encode_request”, and had these results:

But they were aimed at PHP 5.x (by the way, if you’re still on PHP 5.x then update! It’s no longer supported and you’re now a security risk).

Then I refined my search to “apt-get php 7.2 xmlrpc” and had:

And I found the helpful suggestion for the info PHP file (you could also just run phpinfo(); in a file) at:

Upgade Cloud9 Workspace to PHP 7.2

Updated: 14 April 2019

According to this announcement, the old Cloud9 service (c9.io) is soon to be no more, with all functionality ending 30 June 2019, and complete shutdown 30 November 2019.
What is the future of C9? It will be 100% within Amazon Web Services as AWS Cloud9.
I – like many others – am not a fan of AWS. In fact, I think AWS is one of the worst development services I’ve used, and Cloud9 proves it. They took an incredibly powerful and easy to use service and… complicated it. To the point I spent an hour of my Sunday afternoon swearing while trying to setup a new AWS Cloud9 WordPress environment.

As such, I am unsure at this time if I will continue to work with AWS Cloud9 or find an alternate cloud IDE. I use IDEs in a professional capacity and I’m happy to pay for convenience and ease-of-development experience.

Anyway, I won’t be attempting any C9.io workspace upgrades in future. I will, however, create a new post when I decide which development environment to use.

 

Updated: 03 March 2019
After my original environment upgrade I started to see CPU maxing on page loads. I thought it was code changes, but it wasn’t. I run WordPress in my environment and even /wp-admin was causing a 5-20x increase in page load times with 100% CPU spikes.

I happened to find a copy of my environment that was still the original PHP 5.5.9 version and have been using that since a week after this original post. CPU has been fine.

Today I cloned my working environment, did some more searching, and tested a PHP upgrade again. The section “Updated Solution (03 March 2019)” below contains the results of that upgrade. So far, so good. The upgrade process is not that different to the original.

However, /phpmyadmin shows an error “The mbstring extension is missing. Please check your PHP configuration.” I’m not surprised because I my script excluded that (see notes within the 03 March section).
I will work on this next, but I wanted to publish updated results before that happens.


Updated Solution (03 March 2019)

Today’s results are modified from [and thanks to] https://www.joeferguson.me/install-and-configure-php-7-1-on-cloud-9-workspaces/.

Run the following commands, but note:

sudo add-apt-repository ppa:ondrej/php -y
sudo apt-get update -y

#sudo apt-get install php7.2-curl php7.2-cli php7.2-dev php7.2-gd php7.2-intl php7.2-mcrypt php7.2-json php7.2-mysql php7.2-opcache php7.2-bcmath php7.2-mbstring php7.2-soap php7.2-xml php7.2-zip -y
sudo apt-get install php7.2-curl php7.2-cli php7.2-dev php7.2-gd php7.2-intl php7.2-json php7.2-mysql php7.2-opcache php7.2-bcmath php7.2-mbstring php7.2-soap php7.2-xml php7.2-zip -y

sudo mv /etc/apache2/envvars /etc/apache2/envvars.bak
sudo apt-get remove libapache2-mod-php5 -y
sudo apt-get install libapache2-mod-php7.2 -y
sudo cp /etc/apache2/envvars.bak /etc/apache2/envvars

sudo a2dismod php5
sudo a2enmod php7.2

sudo service apache2 restart

 

 

 

 


Original Post Solutions

Cloud9 is a great online IDE (or it was before the Amazon AWS team got their hands on it).

The problem is the default PHP version in my workspace was 5.6, well out of date and unsupported.

It turns out, upgrading PHP version’s isn’t a simple matter. After much search in found this solution on StackOverflow that worked for me:

https://stackoverflow.com/a/51028239/115704

sudo add-apt-repository ppa:ondrej/php -y
sudo apt-get update -y

sudo apt-get install php7.2 php-pear php7.2-curl php7.2-dev php7.2-gd php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml -y

sudo mv /etc/apache2/envvars /etc/apache2/envvars.bak
sudo apt-get remove libapache2-mod-php5 -y
sudo apt-get install libapache2-mod-php7.2 -y
sudo cp /etc/apache2/envvars.bak /etc/apache2/envvars

sudo a2dismod php5
sudo a2enmod php7.2

sudo service apache2
sudo service apache2 restart