The F5 Test: Your project onboarding probably sucks
A thing that really bothers me when onboarding on new projects is going through often lengthy setup and onboarding documents. It's not that onboarding documents are bad, or a mistake, they are a tool, and like all tool they have appropriate uses, and require proper maintenance, or they will become more trouble than they are worth.
Specifically an onboarding document is a tool that allows you to have a new team member self-serve a portion of their onboarding to a project without taking up the time of other team members. It's meant to make ramping up new team members cheaper by reducing the burden on existing staff. Therefore, the more that a new team member needs to seek the directions from existing team members, the less it's doing its job.
The Ways Things Go Awry #
I'll go into details on a few common issues, but if you want to skip to the conclusion just jump to The F5 Test.
As is my preference, I've led with the most important information, aside from the final section.
The Decision Tree #
The decision tree often takes the form of something like the following:
Step X Install Db Explorer
You'll want to install a tool that allows you to view the state of actors in the database Some popular options are X, Y, Z
Typically, when this arises you'll find either the team all use one of the listed applications, or most on the team don't actually use any of the listed options.
Or as another example:
Step X (Optional) Go through the setup for <Other Project>
If you need to run/run against/also debug <Other Project> see <Linked wiki page>
This one crops up in microservice projects pretty often. Some services don't yet or won't have stubs in the project. This can also happen on tools projects where they are in a different repository, or have a different setup from the project it is a tool for.
So what's the issue there?
Things like this I see all the time, and they seem innocent at first since, most likely, you're capturing important project knowledge that would otherwise be just tribal wisdom. The issue is the audience.
When a person is starting on a project they are the least equipped to make informed choices and then there are only two ways for a person to handle these.
- Ideally they just ask someone on the team, which is good, but defeats the purpose of having self-serve onboarding documents.
- They guess, which might be harmless, but also might accidentally sabotage a new team member before they even started.
An example from my past where this went awry was a simple line in Halo onboarding docs I was linked that simply stated:
If you need to run the editor, follow the <main team project setup>
The project I was on was a tool that integrated with the editor in a roundabout way, but didn't actually need the runtime of editor or the full game repository (which was several TB in size), since the tool lived in its own git repo.
Since the person who was most in the know was unavailable, I erred on the side of following the steps and went through the multi-day process of pulling the main game repository and getting all the dependencies installed and projects compiled only to be informed afterward that it wasn't needed, and I wasted the better part of a week and most of my dev machine's hard drive space for basically nothing.
The first order of business after that experience was to remove that line from the project onboarding docs.
Fixing the Decision Tree #
Apart from keeping documents up to date, because again your onboarding doc is a tool that needs maintenance, going through your documents and removing decision points with a strict lens for what sets up a new team member for success is your best course of action. Any tribal knowledge, optional toolsets or related project setups, that those decision points encapsulated should be moved to their own page if they are still relevant, or removed if not.
When it comes to "you need a tool for x," you can just leave it as is, but I would recommend selecting one good option, ideally one that is used by most of the team and writing the documents assuming that is the chosen option, including any tutorials or how-to guides going forward. The consistency will be worth it.
The Installation Catalogue #
NPM, Nuget, Cargo and linux package managers have really spoiled us, but for Windows-dominated shops there is invariably a section of onboarding where you're going to get a bunch of links to sites where you need to download the right version of a bunch of software.
Project runs on Java? What version? Amazon's JDK, or OpenJDK? What's even the difference?
The game team flavour of this is you need a specific version of Unreal or Unity, but which? Is the version listed in the wiki page correct? Who is responsible for updating the wiki when it changes?
This is worse on windows, where, lacking the support for command line installations, you end up having to click to download, click to open your downloads, click to run, click through the installer, and repeat and repeat. For those of us with well-developed repetitive strain injuries this is an unpleasant process in addition to be error-prone as you may link, innocently to a tool you use, but it might have since ceased to be free, or the version up for download contains changes that break your teams workflow, or otherwise cause the tool to no longer serve its purpose.
The other version of this I've seen is where there is a share drive, or a folder in version control that contains a glut of installers that you're simply told to install everything. It's less likely to end up with unsuited or incompatible software issues but no less unpleasant of an experience.
Use a tool! #
For Linux and MacOs the obvious solution for this is to wrap the installation steps in a shell script. Any Linux distro worth any of your time is going to have a well-supported package manager, additional options like AppImages and flatpak packages that can install from the command line and on Mac HomeBrew has existed for ages, and has most everything you'd need.
On Windows there is Chocolatey that can fill the same role as HomeBrew.
While using these tools might be intimidating for people who are not familiar with commandline interfaces, you don't actually need team members to interact with their command lines to see the benefit.
All you need is to have all the software and their versions live in a script in the form of installation commands for your package manager, and replace the installation steps in your onboarding docs with an instruction to run the one script that installs everything.
Usually what I have in each repo is something like this:
# developer_setup_ubuntu.sh
#!/bin/sh
( sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-8.0 &&\
# keep chaining everything that you need to install ...
# It even works with non package manager things, though you may need
# to run source on your bash_profile for things like nvm
dotnet tool install --global GitVersion.Tool --version 5.* &&\
)\
|| echo "Failed to install go paste the contents of this window into <team chat>"If I need to support Mac, I make one that is the same but uses brew install <package>, or choco install <package>
for windows. If you need to share versions you can make a common script that stores version stamps in variables in a way
that can be shared if the D.R.Y.ness of your setup script matters that much to you.
The added side-benefit is that if you design the script right, any time what you need for the project changes, you can re-run the script to catch any new tools needed.
Another potential upside is that if you ever need to set up a build machine, you now have a script that installs everything you need to build your project, so you can just run it as a prebuild step. (It's another article entirely, but I am constantly amazed by how few companies I have worked for have reliable automated builds which is a thing that I care a lot about getting set up)
Default Data or Configuration Files That Live... Somewhere #
I see this one less often but does it ever aggravate me when I see it.
If there is some configuration file(s) needed in order to run your project on a local developer machine, it should be checked into the repository along with the project itself.
The fix for it is fairly straightforward but the two projects I worked on where this wasn't the case was largely because the field values in their local config required secrets that granted access to a dev database, which I replaced with a by creating a local database for each developer as part of the setup script.
The other instance was I've come across this was purely the result of no standard ports and endpoints for each project so depending on what projects each developer was running, they each had slightly different values and didn't want to fight over having the file checked out.
In this case I standardized the ports to resolve the issue entirely, but the alternative would be to add an override
layer that is just <config_file_name>.user.json that is loaded after the main config files or whatever the file is and
add *.user.json to .gitignore.
Staggered Contact Points #
If your organization has ant sort of credentialing or permissions that need to be granted to get set up on a project you will invariably have a point in the onboarding process where someone needs to grant accessed, create accounts, add to share drives, ect.
Ideally these are taken care of before the new team member joins, but team specific repository permissions, cloud dashboard permissions, that change project to project, team to team sometimes can't be taken care of ahead of time.
But for the love of all things just and pure, condense all the "Ask X for permissions for Y" and "Check that you have access to A, if not send and message to B" into one step, ideally the first one, so the quantity of dependencies of this sort is easy to assess.
The F5 Test #
So what is the F5 Test, and why does it warrant the title of the article, but also is the last section?
The F5 test is a heuristic I use to rate the setup process of a project. It is so named for the default keybinding in Visual Studio to start debugging.
The test is as follows: I should be able to pull the project from source control, open it in the editor the rest of the
team uses, and hit F5, or one of these two buttons:
and something reasonable should happen, ie I should be running the project, with a debugger attached, and ready to work.
Like so:
Every step I have to do before the test to get it to work (pulling the repository, opening the project and hitting f5) needs to be either a) justified or b) remedied.
I allow for only one "install dev tools" step, ideally a script per the advice in The Installation Catalogue, but that's it.
I shouldn't have to do any of the following:
- Write my own run configuration file to get the application to boot
- Pull some other project I am not working on to get the application to boot
- Mess around with existing run configurations
- Change the default configuration files that are checked into the repository
- Ask someone else on the team for their configuration files
- Manually apply some default data into a database or manually set up entities
- Add my username to some whitelist file
- Have a specific SKU of game console devkit reachable over network or usb
These are all actual f5 test failures I've encountered.
The pushback I get for this test has been universally the same: Each team member only onboards once. While this is debatable, since if a team member gets a new dev machine, has a hard-drive die, receives a breaking operating system update that requires a reformat of their dev machine or any other calamity, they'll have to go through the setup process again, even under ideal circumstances, you're usually adding team members because of lack of developer capacity, which is to say, you only run into the issues the f5 test is meant to uncover when you can least afford the downtime.