We have all heard the common social media refrain, “Pics or it didn’t happen.” Do you ever recall an occasion on which you’ve lamented, “I can’t remember; I didn’t write it down!”
In business, documentation may seem a nuisance and not a particularly good use of time. Allow me to share my personal journey toward embracing documentation and realizing its benefits, and hopefully you will similarly come to appreciate how documentation can serve your organization!
For much of my childhood, I thought I was going to be a scientist. I began participating in science fairs in second grade and did them for near-consecutive years leading up to my high school graduation. As I matured, so did the rigor of my approach to my science projects. I invested myself more in the upfront research to better frame the context and purpose of the experiments. I increased the complexity of the experiments themselves in both their objectives and in the details of the procedures. I loved observing the results as they would unfold.
Me at age 13, genetically manipulating E. coli in my parents’ kitchen at our collective peril.
What it took me a long time to learn to love was the practice of documentation. Before college, this practice was not as stringently enforced. However, my college-level biology and chemistry lab work enforced this discipline much more strictly, which at first I practiced begrudgingly. However, I gradually developed a tolerance and, eventually, an appreciation for documentation. It served me well.
Marketing and Ecommerce Test & Learn Documentation
Fast forward some years later after an unforeseen academic diversion into Economics followed by a career diversion into Marketing, and I found myself deeply invested in the practice of A/B testing ad variations and web experiences. In keeping with my learned habits, I dutifully recorded every aspect of what I was doing with each experiment from conception through conclusion. I had taken a job with a company whose leadership hailed from graduate mechanical engineering programs, and we immediately saw eye-to-eye on the scientific method, leveraging statistical methods to assess outcomes, and – you guessed it – documentation practices. We ran thousands of experiments over a period of years and maintained an organized library of our documentation.
Business Analysis Documentation
We subsequently decided that many of our exploratory deep dive data analyses – not necessarily predicated on a formally conducted experiment -- warranted similar documentation, and these reports also became invaluable to the organization. We began keeping records of what data we analyzed on what audience segments and what we could definitively conclude; this helped us to not duplicate work and to verify facts that we may not have been able to recall without rehashing the entire exercise. These documents also helped us in a training and onboarding capacity whenever we would reorganize teams or experience some kind of turnover. They got new team members up to speed on what we have tried in the past – plus why we tested, how we executed, and what we learned.
Documentation Practices in Other Organizations
Since transitioning to an agency, I have observed a variety of record-keeping practices used at various companies. Most clients believe in the value of documentation, but often their teams are too lean or moving too quickly to add this step to their regular routine of getting their work done. As a result, their “history” has a lot of holes in it. This is especially problematic when a lot of activity pre-dates the existing team members’ onboarding. Beware the overreliance upon certain long-tenured team members being your resident “historian” in lieu of documentation! If they leave and there isn’t any concrete record of their “recollections,” you have lost a valuable resource.
As one example of a consequence of lacking documentation, I witnessed a client whose new leadership insisted upon testing a concept that has failed on numerous past occasions, based only on others’ verbal accounts. Their long-tenured team members who were cautioning them against repeating this failed experiment lacked documentation to verify what exactly happened in previous attempts. In another instance, a client’s overburdened marketing leader who had left the company had hastily written up spotty summaries of some marketing channels’ “failures,” but the documentation lacked citation of any data to support these conclusions. To make matters worse, upon performing a deep dive analysis, the data apparently was not tracked (or perhaps wasn’t retained long enough) to validate these conclusions. The lack of documentation was ultimately very costly to the organizations in these real-world examples.
Operationalizing Documentation in Your Organization
Take the time to document your experiments, log your analyses, and keep records of any major campaign launches or changes. It may sound like a prohibitively large time investment for you and your employees, but it need not be that way. This can be done expeditiously, and the documents can save you and your employees a lot of time (and money, ultimately) otherwise spent on rehashing work done previously or repeating failures from the past. Here are some tips to get your organization on track to robust, yet easy, documentation discipline:
1. Templatization: Create a template for 3 different types of documentation: Experiments (A/B Tests; Multivariable Tests); Business Analysis; Campaigns (or Projects)
2. Training: Train your team members on what constitutes complete record-keeping. Teach them that if they document what is going on as it is happening, the cumulative process is far less onerous and time-consuming.
3. Morale: Create fanfare around these documents. Use them as the foundation on which people can socialize their findings from tests, analyses, or campaign/project initiatives, perhaps in a dedicated forum for presenting summaries derived from the documents.
4. Organization: Designate someone in your company to be responsible for maintaining a centralized repository of documents that is easily searchable. Bear in mind, if these documents contain sensitive data or performance metrics, you will want to ensure that your storage method is secure.
I hope that reading this has inspired you to embrace documentation within your organization. You will be very pleased with the benefits on many future occasions!
If you have any questions on testing, analyses, and documentation practices surrounding these areas, please contact us. The AB Growth Marketing team is always happy to be of help!