In this week’s EconTalk podcast (around minute 55,) Russ Roberts reiterates an interesting idea: that economists should film a constant video feed of themselves and their computer screen while they’re doing research. That way, all the discarded dead ends and fruitless attempts are documented along with the brilliant paper-worthy successes.
The reason for taking this video is that null results can actually be just as valuable as positive results, but are far less rewarding – to the extent that most researchers don’t find them worth documenting well and thus a lot of potentially valuable information falls through the cracks.
Russ makes the suggestion in the context of economics research, but I think it would be valuable in my field – mechanical engineering – and don’t see why it wouldn’t be useful in other fields as well.
The problem is that the raw video feed would have a very low signal to noise ratio – a lot of useless typing for a few interesting attempted starts and stops. (The other problem is the huge amount of video data itself, but I’m going to cop out and invoke Moore’s Law for that one.)
I recently learned about a project that might present a solution to the signal-to-noise problem. The GDS group at Ames has developed software that ties together in time and importance automatically generated data (here, the video feed) with other inputs (such as files generated by the researcher, their notes, comments etc.) The application here (watch the whole thing, but the especially relevant part is at 24:00) is tying the video feed of a submarine to comments by those viewing the feed and actually in the submarine. It doesn’t seem like too much of a leap to extend it to much more boring videos of scientists at their computers.
Will researchers be willing to have a running video feed of themselves at work? Probably not. But I believe a way to archive the ‘failures’ in research rather than having them ignored and forgotten in favor of ‘successes’ would be a huge win for human knowledge.