Background

The Azure Data Factory service allows Data Engineers & Data Scientists the ability to create data pipelines that ingest, prepare, transform & publish their data on a scheduled frequency. Once pipelines are set up and running, users manage and monitor them to identify issues and take action.

As Lead Designer I was responsible for producing numerous interaction wire-frames, prototypes and visual designs on a iterative basis with
user testing to bring the Manage & Monitor product to Private & Public Preview to General Availability.

Project Scope

I worked very closely with my Project Manager and Design Research counterparts to define new scenarios and features that we would bring into the product. We constantly talked with our customers and ran numerous user research sessions to better understand how our users were using the current product and what areas they would like to see enhancements and new features.

One area that we discovered was a huge pain point was the lack of monitoring which activities were failing and at what frequency. Because data pipelines can transform data on a particular schedule and then be consumed by another pipeline running a separate transformation it was important to see where failures would occur during the process. Not only was it important to see where delays or failures were happening, it was vital to see what up and down stream effects they would have on other pipelines in the Factory.

Because the product lacked this level of management and monitoring of the pipeline and its activities I was tasked to develop a UX that would give this level of control to our users.

PROCESS

I worked closely with my Project Manager to break down the fundamental tasks a user would want to do around this feature and help define several concrete scenarios. We started with whiteboard brainstorm sessions to unpack key "I can" statements. I then moved to paper wire-frame prototypes, allowing us to collaborate and iterate at a much quicker rate.

Once scenarios were defined and we iterated on wire-frame click troughs, I began to develop several high res visual designs that we would begin to test with our customers during usability sessions run by our Design Researcher. 

Final Designs

From talking to customers we learned what resonated most was the ability at a glance to understand where problems were in their Factory diagram view. They wanted to be able to understand quickly where those problems were and how they could drill down deeper to figure out how to fix and what impacts they had on other Pipelines.

Once designs were finalized I red-lined and worked with the development team to implement. For this project I designed in Adobe Illustrator but have since moved to Sketch and use the Zeplin plugin to generate red-lines which has been extremely accurate and a huge time saver. 

Outcome

After the calendar feature was introduced into the product we received a great deal of feedback from our customers as how useful it was to them. They were able to incorporate it into their daily workflow and began to ask for other new features that allowed them to gain additional insights from their data.