Evolving the Query Experience
Learning how to code Flux is key to creating performant queries in InfluxData's time series database. How could we introduce new users to Flux and entice them to run more queries? I led the design of a centralized way to query data while learning Flux.
I was the sole Product Designer, focused on discovery, user research, design, and facilitating design exercises with the broader team.
I worked with the Senior Product Manager, Engineering Managers, front-end engineers, back-end engineers, and a technical docs writer. We also brought in members from Sales and a Developer Advocate.
Project Background
InfluxData technology is built to handle the massive volumes of time-stamped data produced by IoT devices, applications, networks, containers and computers.
Flux is a functional data scripting language for querying, processing, writing, analyzing, and acting on data from InfluxDB.
Bboxx develops and manufactures products to provide affordable, clean solar energy to off-grid communities in the developing world.
InfluxDB helps them continuously monitoring their geographically dispersed 85,000 solar rooftop units providing insights into customer-usage patterns and anomaly detection.
Multiple Ways to Query Data in InfluxDB
InfluxData had three tools to query data that were created by different teams while the platform was growing. This led to inconsistencies in the experience and there was no one-stop shop for users to query their data, create tasks, and learn how to write in Flux.
Users could built their tasks in a simple script editor. There was no way to browse the data to input into the query, add Flux functions, or visualize this data from this tool. Because of this, it required users to be proficient in Flux to create a task.
Due to multiple, inconsistent ways to query data in the platform, there was confusion around the purpose of each tool and a soft adoption of all features. Every experience lacked sufficient tools to teach new users Flux. This impacted query run rates, platform adoption, and revenue.
Preview of Before vs. After
In Data Explorer, users could select their data from a metric selector, but couldn't see the underlying Flux script in the same view. They would have to switch to the Flux Editor, and if the user wanted to go back to the metric selector to select different data after writing Flux, they would have to start the query over, making it difficult to learn and iterate on queries.
In the new Script Editor, users can now see their data selection and the corresponding Flux code in the same view. This makes it easier to iterate on queries, while also teaching new users how their data selections translate into Flux.
Research & Discovery
At the start of the project, I conducted user testing with eight users of various backgrounds and understanding of Flux (from beginner to advanced). I made sure to involve the Product Manager and engineers in the planning and testing process so they would feel more connected to our users and understand the output of those conversations. Here were the key insights that informed the solution:
Users wanted to be able to save and reuse queries and code snippets
A majority preferred querying their data in the Data Explorer tool and didn’t understand the purpose and goal of Notebooks
Novice users didn’t want to spend too much time learning the language or relying on other team members to train them on it
Advanced users who already knew how to code in Flux wanted more query options
Users didn’t like switching between the visual builder and script editor in the Data Explorer
I led the team through the step-by-step process of querying data within each tool to figure out where the gaps were. At each step we stopped and asked ourselves the following:
Will the user know what to do?
If the user does the right thing, will they know that they 1) did the right thing, and 2) are making progress toward their goal?
The project team and I evaluated other query experiences, listing out the pros and cons of each tool. I also did a detailed visual analysis of Microsoft Azure and Snowflake’s script editing experiences, finding commonalities in the layout between the two experiences. I think it’s important not to re-invent the wheel when designing and to take inspiration from other products because of Jakob’s law, which is the idea that users prefer your site to work the same way as all the other sites they already know.
.png){kind=logo}
I developed user personas based on the notes from user research and sales calls. There were three personas we wanted to consider while building the new experience, but to decrease the scope of the initial launch, we prioritized building features for Nora, "The Flux Novice". We felt like the Flux Novice user features would also benefit the experience of the Flux Enthusiast and Evaluator personas.
.png)
.png)
Novice flux users need one main tool to learn how to write flux queries so they’re able to perform the necessary analysis and tasks for their business.
After a clear understanding of the challenge, it was important that the team and I had a list of prioritized requirements for the project to use as guidance during the design process.
1. Introduce users to scripting language early on
2. Make it easy for users to save and reuse work they’ve already done
3. Teach users foundational flux principles
4. Make it easy for users to discover query building in the platform
5. Focus on getting users to build a query and understand the results
6. Provide quick and clear feedback
Selecting Data in a Schema Browser
It’s important that a user can browse their data in an organized way. The columnar view in Data Explorer and Notebooks made it difficult to understand the hierarchy of data, which we heard from a few users during testing. I decided to implement a tree navigation browser in which users can view their fields and tags after selecting a bucket and measurement. This is a common way to present data in other script editing tools, so it was important that we match users' expectations.
Automatically Write Flux
As a way to teach users how their data selection translates to Flux code, I designed a feature that synced the code editor with the data selected in the Schema Browser. Instead of hiding the code from users, we wanted to introduce users to it early on and entice them to learn more. This helps new users learn Flux and makes it quicker for all users to build queries.
Refine Queries With Query Options & Flux Functions
There are additional actions users can take to pivot, group, and aggregate their data. I grouped these actions along with the Flux Library which included a complete list of Flux functions that users could insert into the script editor.
.png)
Create Automated Tasks in the Script Editor
Since we wanted the new Script Editor to be the one-stop shop for users to query their data, we planned to imbed the Task parameters and settings as an option within the Script Editor. This mitigates the need for users to go to a separate page to create a Task. Automated Tasks are a large revenue driver, so we wanted to make it as easy as possible. Unfortunately, due to scope and changing priorities, this feature hasn't been implemented yet.
Save & Reopen Scripts
An important addition to the query experience was the ability to save the query and open it back up later. During user research we saw that users were copy and pasting their queries into their own code editors to save them. We included actions for creating a new Script, opening and saving scripts, and editing the script details (edit name, edit description, copy invocable URL, and delete script).
Feature Onboarding & Flux Education
Since I was catering the design to new users and users who were previously using the other tools, it was important that we created a onboarding experience to the new Script Editor. The walkthrough helps users feel more comfortable starting a query and learning about the different features. Additionally, I included hover-over tool tips for several features to educate users ongoing.
In-app Survey & Feedback
Since we launched this as an optional “beta-like” feature, iterating on it over time, I included an easy way for users to provide feedback in a link at the top of the page. After the user submitted their survey, they had the option to sign up for a user testing session via a Calendly link. This helped source user testers.
The new Script Editor is now helping users quickly build queries and learn Flux
The new experiences provides better features for users to learn how to write Flux, enabling novice users to become advanced users
Out of the in-app feedback survey respondents, 73% rated it “Easy” or “Very Easy” to create a query using the new script editor.
Even though the script editor is still in its MVP state, 30% of users are sticking with the new experience after switching from the old Data Explorer.
Speed up the discovery and decision process by incorporating aspects of the traditional 1-week design sprint
Gather more analytics and incorporate analytics & event tagging requirements in feature tickets
Test small and often - you don’t have to wait until something is complete to user test
