Lessons Learned from Using an Online Platform to Conduct Large-Scale, Online Controlled Security Experiments with Software Developers

Abstract

Security and privacy researchers are increasingly conducting controlled experiments focusing on IT professionals, such as software developers and system administrators. These professionals are typically more difficult to recruit than general end-users. In order to allow for distributed recruitment of IT professionals for security user studies, we designed Developer Observatory, a browser-based virtual laboratory platform that enables controlled programming experiments while retaining most of the observational power of lab studies. The Developer Observatory can be used to conduct large-scale, reliable online programming studies with reasonable external validity.

We report on our experiences and lessons learned from two controlled programming experiments (n>200) conducted using Developer Observatory.

Reference

@inproceedings{DBLP:conf/uss/StranskyANWKR0G17,
 author = {Christian Stransky and
Yasemin Acar and
Duc Cuong Nguyen and
Dominik Wermke and
Doowon Kim and
Elissa M. Redmiles and
Michael Backes and
Simson L. Garfinkel and
Michelle L. Mazurek and
Sascha Fahl},
 bibsource = {dblp computer science bibliography, https://dblp.org},
 biburl = {https://dblp.org/rec/conf/uss/StranskyANWKR0G17.bib},
 booktitle = {10th USENIX Workshop on Cyber Security Experimentation and Test,
CSET 2017, Vancouver, BC, Canada, August 14, 2017},
 editor = {José M. Fernandez and
Mathias Payer},
 publisher = {USENIX Association},
 title = {Lessons Learned from Using an Online Platform to Conduct Large-Scale,
Online Controlled Security Experiments with Software Developers},
 url = {https://www.usenix.org/conference/cset17/workshop-program/presentation/stransky},
 year = {2017}
}