The thing about micro-services

You might decrease the complexity of a monolithic application (I’m not actually sure that’s even a valid statement, that a “monolith application” is “complex”) but conversely you increase the complexity by adding in all the network hops, latency, asynchronicity, and having to debug strange timeout errors.

It doesn’t matter what architecture you user, you’re still going to encounter an equal number of problems to think about, debug and solve.

WordPress: Changing File Upload and Processing Sizes

[This is a post for self-reference]

Up front I’m going to say I got exactly the help I need from here: https://www.wpbeginner.com/wp-tutorials/how-to-increase-the-maximum-file-upload-size-in-wordpress/.
The author of that post deserves all my credit.

I’m creating this post to summarise it in my own words and to give it a little educational context.


When allowing file uploads on a website there are 3 big things you need to think about:

  • The size of each file.
  • The total size of all files in an upload.
  • The time it takes to process the upload.

Actually, there are a couple of things to consider that are worthy of mention but I won’t cover in this post:

  • Asynchronous uploads (AJAX as we used to call them) can change the figure of an upload, because instead of uploading multiple files at one, you upload one file at a time (even if they are in parallel) so the “total size of all files” number will be different.
  • Tell users about the limits they face before they upload. Also, when uploading images a bit of guidance around things like “27MB images are a stupid side for most sites because 27MB isn’t going to be better than 7MB, and the site will trim the size down to a couple of hundred KB at most anyway.”
  • Trap, log and respond to any errors. This is on my mind because WordPress/PHP seems to like gobbling errors and not telling anyone about it (I’m in the process of hunting down how to handle this and will be a topic of another post).

 

There are 3 main ways in PHP to affect file upload sizes and times:

  • @set_init() in a PHP script itself.
  • php.ini file
  • .hataccess file.

Be aware, as a PHP/Linux novice I reckon there are other defaults and hierarchies that come into play.
This article is spawned by a situation where I was changing php.ini without realising .htaccess had a value overriding it.

 

How do you implement these values?

This is where I rip from the wpbeginner site I mentioned above for quick reference:

1. PHP script file (e.g. functions.php in WordPress themes)

@ini_set( 'upload_max_size' , '64M' );
@ini_set( 'post_max_size', '64M');
@ini_set( 'max_execution_time', '300' );

(I haven’t tried this approach yet, but I have used @ini_set() in other instances (e.g. ‘memory_limit’) and it’s worked well.

2. php.ini

upload_max_filesize = 64M
post_max_size = 64M
max_execution_time = 300

I’ve used this approach in development and it did the trick.

3 .htaccess

php_value upload_max_filesize 64M
php_value post_max_size 64M
php_value max_execution_time 300

Before writing this post I had to fix a production issue and after trying php.ini I got lucky and discovered .htaccess was where the value was coming from.
So the moral: .htaccess overrides php.ini.

Also, I did not need to restart apache for this change to take affect.

 

DISCLAIMER: I haven’t actually read in-depth how all this works. I’m a typical developer who expects other people’s shit to work, and when it doesn’t I’m usually in the middle of my own concerns so it’s a case of “search and scan by keyword until someone else’s solution works for me”.
Such is this age of software development.

 

Need to know what your current settings are?

It’s helpful in debugging and verification to know what your actual PHP settings are so you know what your starting point is (and  you have a number to search your files for).

A good way is to call phpinfo();.

For example, create a file in your site (I call mine info.php) and add the following PHP code:

<?php
phpinfo();
?>

You should see something like the following when you run it. This screenshot shows some output from the section titled “Core”, and the 3 columns are (from left to right): “Directive”, “Local Value” and “Master Value”. I can say from experience the “Local Value” is the one that matters.

phpinfo() output

Bad UX: Microsoft Films & TV

This is a UX failure I experienced.

My default media player in Windows 10 is Microsoft’s “Films & TV” app. So most .mp4 or .mkv files I have or produce I play through this app.

The problem is, if my focus is on the app and I press the backspace button (which can happen if I’m working and switching between multiple screens), I go from a playing video to the following:

So, where the UX fails and I become lost is: how do I get back to the file I was just playing?

There is nothing on the screen that indicates this possibility.
And exploring the few extra options (including a back arrow that only appears after moving between “Explore”, “Purchase” and “Personal”) yields nothing.
In the end I have to go back to Windows Explorer and re-execute the file.
(Now I think about it, the application doesn’t even have a typical “File” menu to find and open a file directly.

I call this a fail because I cannot simply “go back” to the file I was playing.
I would expect this from a default operating system application in 2019.

Software Development Estimation (and breaking through The Planning Fallacy)

Everyone involved in software development sucks at estimating the time to complete tasks.
Developers. Analysts. Even project managers.

I recently heard a very really simple solution for this problem on a podcast that was discussing The Planning Fallacy.
The solution:

Multiply your estimates by Pi (3.14)

I’ve tried this a few times over the last week and it actually works.
My estimates are finally starting to match reality. And this includes such simple things as writing and responding to emails.

There’s only one problem – that’s jot really a problem – is:

Managers and customers are not going to like hearing the truth

Estimates are going to sound long and expensive, and you will be pushed to reduce them again.

Don’t.
Don’t back down.
Stick to your guns.

The real problem with estimation is not that we suck at it.
The problem is we are forced to cut corners and revise to unrealistic numbers that “sound good” to the customer’s ears.
It never works, and always blow out, and then the shit hits the fan, and stress increases as projects run over time and developers are coerced into working stupid overtime.

If we just do the right thing in the first place and communicate what’s real, everyone knows where they stand and everyone will be happy in the long run.

A Git Strategy

Background

Over the years, I’ve worked on a number of different projects with different Git branching strategies. I’ve also wasted many hours sitting in meeting arguing over what should be a relative non-issue.

The beauty of being the bloke in charge is sometimes you get to skip all the bullshit and just dictate the approach.
Or to be correct in this case, sometimes the approach is dictated for you.

Working remotely with developers who were essentially new to Git and were already facing a lot of learning and change, I found their actions have driven my Git process decisions.
Which is good:

You want to make business and development processes as easy as possible for everyone.

After all, complexity is the enemy of productivity.

I started with GitFlow, which has worked well in the past, but ended up simplifying if further.

 

The Strategy

The Git “Pilot”

One person is designated responsible for reviewing and managing a repository.
All developers should be capable of doing this role – and the role may rotate between developers – but at any point in time one person is the designated “pilot” in control and responsible for the repository.

Workflow

Normal feature development

  • A working branch is created from develop.
  • When work is complete it is merged back into develop via a Pull Request.
  • A release is created.
    • A version number is generated an applied at the start of this process.
    • A new release branch named “release/release_{version}” is created from master.
    • develop is merged into the release branch.
  • The release goes through test environments.
    • Note: Only 1 feature release branch is tested at a time. This reduced management complexity.
    • If a fix is required:
      • A working branch is created from the release branch.
      • The working branch is PR merged back into the release branch.
    • If a production hotfix occurs during this process, develop is merged back into the release branch.
  • A release completes testing and is published to production.
    • Assuming any production hotfixes have already been merged into develop and the release branch.
    • The release branch is merged into master.
    • The release branch is merged into develop (to ensure any fixes are available for continued development).

Hotfixing production issues

  • A working branch is created from master.
  • When work is complete, a release branch is created from master (same versioning and naming process at normal feature development) and the feature branch is merged into the release branch.
  • The release goes through testing.
  • When testing is complete:
    • The release branch is merged back into master.
    • The release branch is merged into develop.
      develop then needs to be merged back into any working branches, and feature release branch currently in test.

Version Numbers

  • Use the format {yyyy}.{mm}.{dd}.{HHmm}
    • {yyyy}: The current 4-digit year at UTC.
    • {mm}: The current 2-digit month at UTC.
    • {dd}: The current 2-digit day at UTC.
    • {HHmm}: The current 24-hour hour and minute at UTC.
    • Example: 2018.02.22.2158
  • Version number is applied as soon as a release is generated.
  • I don’t care about identifying alpha’s, betas, fixes, or otherwise. A release is a release. Use release notes to document details of what is going into a release.

Core Branches

Required:

  • master: The current production code.
  • develop: The current code being developed and tested.
  • release: Each release starts in its own branch.

Optional:

  • feature: Optional node for developer to create working branches under.

Working Branches

Developers can optionally create working branches (known as “feature” branches in GitFlow) under a feature node (e.g. feature/jason_my_work), or directly off the root – at the same level as master, develop, and release.

This “either or” approach came about because my developers were forgetting to create working branches under feature.
In reality this is not a problem in any way and, if anything, made them easier to find and access, so I’m happy for them to stay there.
Good repository maintenance will handle any clutter issues.

Branch Maintenance

After merging working branches back into develop, keep the branch for a “period of time” – generally through testing – we’re happy all code is included and complete.
Periodically delete old working branches.

Release Notes

Each release should contain notes identifying:

  • The type of release – e.g. scheduled feature release, production hotfix, security patching, etc.
  • A summary of changes included in the release.
  • Identify any breaking changes.

Saving yourself with Git (Pro Tip!)

I don’t fully trust source control systems. Not even Git. That’s because I’m an old fella in the industry and I’ve been bitten too many times to fully trust systems.

Point in hand. I just had a merge from ‘develop’ royally screw up some code in such a way the effort to find and fix the change is greater than the effort to just blow away my working branch and pull again from remote.

Pro Tip!

When merging from another branch like ‘develop’ into your working branch, first commit and push your changes to remote.
That way, when your merge screws something up without you realising until after you’ve re-compiled and tested the merge – you do re-compile and test your merges right? – you can reset or blow away the branch (even delete your local branch and pull again if need be) without losing your hard-won changes.

Tip for Junior Programmers: Tracing Flow Through Code

Learning how code “hangs together” is crucial for all programmers. And it’s not that difficult.

Most software has an entry point, though it varies depending on the language or framework. Find and understand that entry point.

From there just follow the code. In procedural and object-oriented languages you’ll mainly be dealing with IF statements, SWITCH statements, calls to functions and calls to objects with properties and methods.

Watch what is called. Look at the parameters being passed into methods or class constructors.

Look at how variables are manipulated. Consider the scopes of variables and class properties – are the private to a function or class, or are the public? Are function parameters passed by value or by reference?

And think about namespaces – from the variables and functions that sit in the global namespace through to those nested under specific namespaces.

Functions will sometimes seem to be in a global namespace but are actually implicitly at a deeper namespace level thanks to using or include statements in the file (e.g. in C# and VB.NET).

PRO TIP: Make Find All (usually Ctrl+F) your best friend. If you come across a function or variable and don’t know where it’s defined or what it’s used for, any decent IDE with allow you to perform a “Find All” and list out all occurrences of it. Use that to trace the use of the variable/function.

But the most important thing you can do as a programmer is: take the time to read and understand the code. The more years of experience you have the easier and more intuitive it becomes, but always it is important. Learn how to follow the flows, intuit the structure and find where functionality lives.

This is the true heart of most programming activity.

What is a predicate in computer/software programming?

I’m a very practical developer. My learning style has always been to learn how to do something, usually by example, now what it’s called.

As such, I know that I know how to write code with predicates, but I might still fail a multiple-choice test to describe what a predicate is.

So I looked it up again. And it’s really quite simple:

A predicate is a statement (or function) that returns either true or false.

You pass some data into the predicate. The predicate logic then performs a check on the data. Then the function/statement return a true or false result to the caller.

The 2 clearest resources with descriptions and examples I’ve found so far are this StackOverflow answer and this other StackOverflow answer (which is C# oriented).

A common example is a function or short comparison statement you write and pass into a function on a list that then performs a mapping or reduction on the list (e.g. to find a subset of items in the list that contain the value(s) your predicate function or statement is checking for).

For example, if I have a list or array of People objects and each object has an Age property, I can write a predicate …

(Person p) => (p.Age >= 18)

… that I pass into a select() function on the list to return only those people who are age 18 or older.

For example:

var australianDrinkingAgePeople = fullList.Select(Person p => p.Age >= 18);

In this case, it’s a shortcut for writing your own big for loops and if statements over the list (although there’s nothing to say that the select() statement doesn’t implement that logic behind the scenes).

SQL SERVER WHERE clause returns match when field contains trailing space

It’s nice to know you can still discover new things after 20 years in the industry.

Last week while investigating a defect of some data import work I’d done in SQL SERVER, one of my initial steps was to query the database for the imported records. For example:

SELECT * FROM [MyTable] WHERE [SomeField] = ‘SomeValue’;

That’s a standard, simple query. Nothing tricky about it.

However, what I had already discovered from inspecting the value returned to the application that uses it, is my value actually had a trailing space (i.e. ‘SomeValue ‘).

What surprised me was when I ran the query above it still returned the record.

That is: [SomeField] actually equals ‘SomeValue ‘ (with a trailing space) in the database record, yet my comparison to ‘SomeValue’ (no trailing space) still returned the record.

It turns out this is a quirk of SQL SERVER’s implementation of SQL.
It also turns out in all my years working with databases I must have been cleansing my data enough to never encounter this (in this instance I was generating SQL statements in Excel and was explicitly told not to touch import values [some of which included an incorrect trailing space]).

Check out this StackExchange question for more information.

Recommended Software Development Tools

From years of experience and use (and a few recommendations from other people), I recommend the following tools:

IDEs/Editors

  • Visual Studio Code – Windows/Linux/Mac. It might be from
  • Atom – Windows/Linux/Max
  • Cloud9 – Browser-based IDE. The original version was better. Now owned by Amazon AWS and they’ve made a royal mess of the licensing, usage and documentation. Still good under the covers though.

GUI Git Managers

Command-line has its place but for the complicities of Git-based source control (and being a visual person), I prefer a good GUI for Git management.

File Compare and Merge

Web Proxy for Debugging

  • Telerik Fiddler – Windows (Beta for Max/Linux)
  • Charles Proxy – Windows/Mac/Linux
  • OWASP ZAP – Windows/Max/Linux (I’ve not used this, but OWASP does good work in security and have seen others recommend it)

General Development Helpers

  • JSON formatter – Great for inspecting large sets of JSON (and now XML and YAML)
  • RegEx Testers (enter a regex expression, test it, find out what it does):
  • Can I Use – The best resource to determine what CSS and JavaScript feature the major browsers support.