New testing feature: Swatches!

Today we've added a new beta testing feature that will allow us to test new additions and changes to the site in a more simplified and organized way.

For many years on Ravelry, we'd release new features or site changes and work together with users in the For the Love of Ravelry forum to fix bugs and tweak things. As the site grew, we tried other methods of testing, including a special Swatcher Secret Hideout group we used for several years. Chatting in the forums was how we built Ravelry at the very beginning and they have historically been our main source of feedback. This has left a lot of people—most of the Ravelry community—out of the process. Prior to launching the current design, we reached beyond our forums and invited varied groups of Ravelers in batches to test the site over a two-month rolling period. Our process for beta testing and gathering feedback is one of the things we have been reflecting on and discussing as a team since our redesign was in progress, and we know that improving this process will help us to serve our community better.

Our new beta testing feature will allow us to invite broader groups of Ravelers to participate in testing and share feedback and bug reports in specific channels, rather than in our forums, so that we can make sure each new feature works well and is polished before we release it sitewide.

Swatches!

Swatches are new Ravelry features that select groups of Ravelers can opt in to use before they're available to everyone. While swatching a feature, you can provide feedback and/or bug reports through its feedback link on the Swatches page. New swatches will be released to a subsample of Ravelers, called a Swatcher Group. This is for three reasons:

1. It’s important to us that we hear feedback from a wide range of Ravelers. We’ll make sure that the feedback is representative of the entire community, and focus on varied groups of Ravelers depending on the Swatch.

2. We can be sure to preview new features with Ravelers whose feedback is important to us for a specific Swatch.

3. It will allow us to reasonably manage the feedback. We can test a Swatch with a subsample of Ravelers and collect statistically significant data.

A green notification dot will appear in the new Swatches section of your Profile drop-down menu when new Swatches are available to you; you can click through to the Swatches section to see a brief description and image preview of the new feature before choosing to toggle it on to try it out. Once you toggle a Swatch on, you'll see a box that lets you know what type of feedback and/or bug reports we’re looking for, and a link to submit that feedback if you so choose.

The Swatcher will allow us to run multiple kinds of tests on new features: remote usability, concepting, bug reporting, QA, etc. We’re excited about all the different context-specific possibilities for testing that the Swatcher will afford.

With the implementation of the Swatches feature, we have retired the Swatcher Secret Hideout group and Beta Testing Volunteer preference in your settings, as we will be using different criteria to invite people to participate in Swatches. If you are not interested in Swatching at all, we have a Swatches opt-out feature in your Ravelry settings; you can check that box if you would not like to be selected for Swatcher Groups. As with all new Ravelry features, the Swatcher and any settings related to it are only available in the current site.

Our first Swatch called Practice Swatch: Sherbert, and it is available now so that you can get a preview of how the Swatches page works. Practice Swatch: Sherbert will put a sheep illustration (her name is Sherbert!) at the bottom right of the Swatches page. Normally we'll run Swatches for a finite amount of time before deciding whether to make it into a permanent feature, but Sherbert will be in the Swatches section forever so that Ravelers can try it out at any time.

As we get data from running Swatches, we may make some adjustments to the functionality of the Swatcher, but wanted to share our new process with you right away. We're looking forward to continuing to improve Ravelry with you!