Takeaways from Write the Docs + Open Help Cincinnati

This weekend I attended Write the Docs + Open Help Cincinnati. Write the Docs + Open Help is an event that brings together documentation contributors to share and learn from one another. The weekend included a day of talks and day of unconference sessions and lightning talks. It also included three days of documentation sprints led by Open Help, which I was unable to attend.

I work as the main contributor and maintainer of Cypress’ documentation. My background is in web design and more recently web development. Over the last 5 years working at startups, I also took on the role as the main documentarian. I like writing docs - a statement that is rarely uttered in the world of developers. 🙈

I’ve been following the Write the Docs community on Slack, but was never able to dedicate copius amounts of time to fully immerse myself. This weekend’s event gave me a chance to really get to know the community and focus on Cypress’ goals for our documentation.

Cypress’ documentation has been praised by many and described as lacking by a few. My main goals were to learn:

1. How we can continue delivering quality documentation to our users
2. How to identify where our documentation needs improvement
3. How to encourage contributions to our documentation, since all of it is open source.  

Here are a few of my takeaways from the weekend –

Learn who your contributors are

Since one of my objectives was to determine how to encourage contributions to our documentation, I perked up during Adam Michael Wood’s talk titled “Docs are an Engineering Product”.

Documentation is highly valued, but often overlooked within the open source community. It’s also very natural as a documentation maintainer to feel that you need to make editing your documentation as uncomplicated as possible. I have often felt anguish over the build process required for contributors to get our docs running to see their changes.

Adam’s talk made me think differently and ask - what is the motivation behind the people who come to make contributions to our docs? The answer is that contributing helps them achieve their career goals. The people looking to contribute likely fit under 2 categories:

1. They are already developers and looking to fill up their GitHub OSS shower tiles
2. They are working towards becoming developers

What do either of these groups get from contributing to documentation that is disconnected from the developer experience?

New contributors want to learn how to do this stuff. They don’t want to edit Wiki pages.
— Adam Michael Wood

This helped me realize that I shouldn’t agonize over our build process, and that I should embrace it instead. We should catalog how to get our documentation up and running for those who are new to npm, new to build processes, new to GitHub, or new to writing Markdown.

Markdown isn’t that great

This one was a bit of an eye opener for me. For developers, Markdown has become the go-to markup language for text-based documents. What’s wrong with Markdown?

This blog explains why in much more detail, but the gist is that there are several flavors of Markdown. There’s also nothing inherent about Markdown that satisfies all that is needed for documentation.

After this was pointed out, I realized that I already know this. Cypress’ uses ‘Markdown’ for our documentation. But, in reality, it is a frankenstein of GitHub flavored Markdown combined with built in and custom Hexo helpers. It started out simple enough: we wanted to have callout sections for notes in our documentation, so we created a helper for that!

{% note success Benefits %}
- Guaranteed to work in production
- Test coverage around server endpoints
- Great for traditional server-side HTML rendering
{% endnote %}

Now we have many more helpers that serve the needs of adding icons, reusing snippets of text, linking videos, linking issues and more.

I don’t feel like anyone in the community was able to offer an amazing alternative today. But it did give me a fresh perspective on the problem space of technical writers. I’m hopeful that a tool will be created to address this one day.

Analytics - you should be gathering them

A good deal of time was spent discussing gathering analytics on how your documentation is used. Particularly David Payne’s talk on how “Analytics Can Change Your World!”. Luckily Cypress’ documentation has had Google Analytics set up since day 1, so the data is there.

But what do you do with this data? I was able to determine some questions that I hope to answer over the next year.

Questions analytics can help answer

• What browser/OS do your users use?
• How many page views per product?
• Review pages with high traffic - are users benefitting? Can we help them here?
• Review pages with zero traffic - why don’t they have any traffic? - is it hard to get to? is the content relevant?
• How long is the user staying on a page?
• What is the path of the user through online help?
• How much multimedia is consumed?
• Is your help localized? Are your users consuming it?
• Time on page - is being on the page for a long time a good thing or bad thing?
• You need to define what metrics you care about - what question am I trying to answer?
• Error pages are different from marketing pages - The page that tells you how to recover from your server crash (for example) - should be least visited piece of content, but it NEEDS to be there.
• What is the 2nd - 3rd page that they go to?
• Were your users searches any “good”? What percentage of searches result in ‘no results’?
• What are search queries telling you? What they search for most should be most the accessible information - OR this should be the thing that is easier to USE in the product itself.
• Do people hit the first page and then immediately search?
• How do you get feedback on if content is any good - page ratings?

Automation can likely help

There was a lot of discussion on process and tooling throughout the weekend.

Cypress’ core product is automated testing. So we are not shy about automating processes. We already run Cypress tests on our documentation. For example, we have a test that ensures every H1 and H2 level heading on the page is displayed within our Table of Contents as links.

const API_PATH = '/api/introduction/api'

context('Table of Contents', () => {
  beforeEach(() => {
    cy.visit(`${API_PATH}.html`)
  })

  it('displays toc', () => {
    cy.get('.sidebar-link').each((linkElement) => {
      cy.log(linkElement[0].innerText)
      cy.request(linkElement[0].href).its('body').then((body) => {
        const $body = Cypress.$(body)

        const $h1s = $body.find('.article h1').not('.article-title')
        const $h2s = $body.find('.article h2')

        const $h1links = $body.find('.toc-level-1>.toc-link')
        const $h2links = $body.find('.toc-level-2>.toc-link')

        $h1s.each((i, el) => {
          const $h1 = Cypress.$(el)
          const $link = $h1links.eq(i)

          expect($link.text()).to.eq($h1.text())
          expect($link.attr('href')).to.eq(`#${$h1.attr('id')}`)
        })
        $h2s.each((i, el) => {
          const $h2 = Cypress.$(el)
          const $link = $h2links.eq(i)

          expect($link.text()).to.eq($h2.text())
          expect($link.attr('href')).to.eq(`#${$h2.attr('id')}`)
        })
      })
    })
  })
})

We also have a node script that checks every link in our documentation to ensure there are no dead links. Depending on whether the url is defined as within the same document, an internal link to another doc, or an outside link, it checks to see that it exists within our own documentation or does not 404 as an outside link.

So, we have some really cool automated stuff, but there are definitely improvements that can be made. I felt a bit embarrassed sitting around a table full of writers and realizing we have nothing that automatically spell checks our documentation. 🤦‍ Such a simple thing that we should be doing.

There are some other things I aim to look into automating also, like using Markdown Magic to automatically build Table of Contents for our repo’s Readmes and syncing content across our 100+ repos to normalize their Readme content.

Conclusion

It’s been a goal of Cypress from day one to make our documentation as important as the code we write for every feature. I’m excited to keep learning from the community, dive in, and continue improving our documentation.