Prior to the release of WordPress 5.0, I conducted a survey asking WordPress users with Gutenberg experience to offer their perspective on the new editor. The survey was conducted as part of a broader user-research effort designed to understand more about how content publishers view page-building and site-building tools, and how the new content editor matches expectations. This research was intended to explore the behaviors and opinions of people who had already tried Gutenberg. Updates to the new editor based on this research are being tracked on Github and MakeWordPress.org.

Survey Details

Participants were asked seven open-ended questions about their experience with Gutenberg. The survey was intentionally designed as open-ended to gather a breadth of answers and to avoid biasing ourselves with expected responses. The goal was to make it feel like an unmoderated questionnaire more than a survey with finite choices.

Out of 161 initial survey participants, 4 did not qualify based on screening criteria. The study required experience working with WordPress and at least some experience with Gutenberg.

Participants

The survey participants came from a variety of backgrounds. Respondents self-identified their roles:

How would you describe your role at your current job?

• Developer: 44
• Engineer: 42
• C-Level: 19
• Designer: 15
• Other: 10
• All Roles: 6
• Project Owner / Manager: 6
• Support: 5
• Director: 2
• Architect: 2

Each participant needed to have at least “a little” familiarity with Gutenberg:

How much experience do you have working with Gutenberg? (161 responses)

• A moderate amount: 50.9%
• A lot: 26.1%
• A little: 21.1%
• None at all: 1.9%
• A great deal: 0.00%

Survey Questions

The following questions were asked:

• What lessons, if any, did you learn from Gutenberg phase 1?
• What do you feel needs further refinement in Gutenberg?
• What areas of focus should be considered for phase 2 of Gutenberg?
• Do you have any worries about Gutenberg? If so, what?
• Do you have any hopes for Gutenberg and its future? If so, what are they?
• Who do you feel are the most important WordPress users to consider for the next iteration of Gutenberg?
• Would you like to add anything else about Gutenberg?

A Well-Received Concept

Survey responses show that overall, the promise of the new editor was generating excitement. The open-ended format allowed participants to respond as though it was a post-project retrospective.

The overall concept and interaction design received praise. Participants specifically responded to the added “freedom” afforded content creators who benefit from a visual editor due to limited knowledge of coding. As one participant said:

“Gutenberg is good. It is new and easy to use. So far, the biggest problem is backward compatibility.”

Other participants said:

“Its current iteration is quite an amazing experience and offers a lot of potential. Gutenberg has taken great strides to alleviate the cognitive load for users to distinguish what is the editor experience and what is ultimately display. Simultaneously, it affords great potential in its core APIs for developers to hook into. My hope is that Gutenberg has a bright future.”

“Great work! I can see how much hard work has been put into it. I don’t think any great change happens without heated discussions and disagreements. Know that although some people have not been diplomatic that no one should take it personally. Let the best ideas win even when it means killing something that a lot of work went into if there is a better solution. Keep iterating and convince those that doubted but provided no ideas.”

Some expressed enthusiasm that new audiences considering WordPress will have a modern editing experience.

“It’s amazing. Change is something we all face in life at different stages. That sometimes is not expected, but it happens. We have to find the opportunities in there, rather than keep complaining about it. Gutenberg is the change in the WordPress ecosystem. It is resetting the competition, bringing new opportunities and making the WordPress CMS future proof.”

Areas for Improvement

This survey was conducted prior to the launch of Gutenberg as the core editing experience in WordPress 5.0; a current survey might yield different responses to what could be improved. The survey asked:

What do you feel needs further refinement in Gutenberg?

• The overall UX and UI needs to be improved: 18
• Requested features for current and future functionality: 17
• Better accessibility and usability: 15
• Compatibility and improved extensibility: 11
• Fixing bugs and minor issues with Gutenberg: 10
• Refinement and added functionality of blocks: 10
• More documentation for users and developers: 7
• Miscellaneous comments that did not fall into a specific category: 6
• Focusing on typical workflows and improving them: 4
• Improved communication: 4

Criticism coalesced around a few themes. In particular, while participants acknowledged Gutenberg’s potential, they also felt that a lack of refinement detracted from successful early adoption. This issue was exacerbated by a perceived lack of communication and visibility into features and release timelines.

Communication & Visibility

Some participants acknowledged that gaining 100% consensus is not feasible, especially within an open-source project with a community as large and far-reaching as WordPress’s. As one person said, “Shipping new features in core is hard.” Still, it became clear that a feeling of not being heard contributed to some backlash.

“Discussion within and the feedback loop from the community was sought way too late. The community was not part of this project.”

Visibility concerns created anxiety and fear for some, particularly those who worried what the impact would be for their projects or clients. Participants often requested more documentation so they could help their clients with the editorial transition.

“Training hundreds of content producers on a new editor will be quite the task, especially since it’s still very buggy and changing rapidly.”

Participants felt that poor communication decreased overall adoption.

“Communication is super important. I think Gutenberg is great, but I think a lot of people aren’t giving it a chance because they feel like it’s being forced on them or rushed. Those aren’t technical issues, they’re miscommunications.”

Participants wanted to understand the roadmap and key project dates. Uncertainty and “unknowns” caused anxiety. This anxiety manifested in survey responses.

Accessibility & Usability

Accessibility and usability were subjects of signification discussion and concern within the community of active WordPress users at the time of the survey. These community conversations were echoed in survey responses, with participants raising concerns about accessibility and the need to center future iterations around it. One participant noted that:

“Accessibility has to be considered throughout the life of a project from the start and not ignored.”

Another said:

“The Gutenberg user experience is not inclusive of all users; it is overly complex and does not feel stable in the context of normal use. These issues are exacerbated for users who rely on assistive technology.”

Many were concerned that Gutenberg was leaving behind people with disabilities and by not making the new editor accessible to all, causing a decline in WordPress use.

The Accessibility team is chronicling their work if you would like to follow their updates.

Refinement

Respondents sought additional refinement to the new editor. Considering this survey was conducted a week before WordCamp US, which was closely timed with the official launch of Gutenberg as the new editor in WordPress 5.0, there was heightened concern about the final product. It came as no surprise that participants often focused on bugs or small interactions that were clunky. When specifically asked, “what needs refinement?” a few people answered:

“Little bugs. Lots of small bugs add up to big frustrations.”

“I hope that as it evolves it becomes a refined, intuitive experience that leaves me undistracted while enabling powerful editing modalities.”

“I think phase 2 should be a hardening and cleanup phase. Don’t bring [G]utenberg to any other areas of WordPress until the foundation settles. I think there’s a lot about how developers use WordPress that the [G]utenberg core team doesn’t understand, and they’re going to hear all about it after phase 1 officially rolls out.”

Other comments focused on inconsistencies in the interface. Participants felt like the “UX is still clunky” and that “the UI can improve to be more intuitive.”

Generally, it seemed participants believed the new editor is headed in the right direction but needed more finesse.

“The overall design of Gutenberg is extremely poor. While I love ‘clean’ and ‘minimalist’ design, Gutenberg has taken this to extremes and hides way too much functionality under hover states, menus, toolbars & toggles. I feel that a lot of Gutenberg’s poor usability is because of the terrible design and their lack of listening to advice on how to improve it.”

Scaling

Participants saw beyond Gutenberg’s flaws to it’s greater potential. Many suggested new blocks for the core editor and imagined expanding the paradigm to other areas of WordPress could make the overall experience better. Menus, the customizer, widgets, and templates were often cited as fertile ground for future iterations:

“Not sure if this is part of the focus, but I’d love to see Gutenberg be used for content curation on landing/archive pages.”

“Replace “widgets” with blocks. Ability to build header and footer with Gutenberg.”

Conclusion and Next Steps

At the time of writing, WordPress 5.1 has already been released, bringing several updates and refinements. You can follow a weekly digest, and you can follow accessibility updates on Make WordPress.

Another survey is recommended to gauge changes in perception following the official launch of the new editor. This test is scheduled to be conducted close to the one-year anniversary of the WordPress 5.0 release.

Speaking of surveys, I’m currently conducting research to learn more about how writers and editors use their smartphones when writing, publishing, and updating a website. If you currently write and publish for the web we’d love to hear from you!