Misleading Models

In so many parts of our lives, our focus becomes a tunnel towards just ‘making it work’ at whatever cost. For now, I’m going to avoid the snake filled bog that is the exploration of that idea in the realm of morality.  But in engineering and science, dogged focus on a solution can be incredibly valuable but also very dangerous. Of course, it can lead to rousing success. However, the solution can end up built on hidden assumptions, forgotten in the dash to a solution. If they are undocumented, they are a giant trap for someone who tries to use your result. There are many examples, but I’ll go with a personal one:

The danger was particularly salient for me today. I’ve been trying to get QuIRK-E (my eddy-current model) to match the measurements of a paper that experimentally investigated eddy-current forces.

It’s so easy to go down the rabbit hole: ‘well, it should work if I just add this term and tune the constants. Oh, that didn’t work, but got me closer, let’s add this other term.’ If I hadn’t been forcing myself to write down every step I took and why I took it, I could easily have gotten lost in the warren. I had to say to myself ‘stop. You’ve completely removed the model from physical reality, just to exactly replicate this ONE SITUATION.’

It’s so easy for this to happen:

very close

The model (in green) looks like it really captures real life (blue), doesn’t it? Let’s zoom out a bit.

not so close

Whoops.

Now, if you’re only going to use the model in the smaller first region and are explicit about that fact (aha! The value of documenting assumptions raises its head) then it’s a fine model. But claiming that the model ‘captures the system’ is just untrue.

I worry that this happens all the time in many areas in science, engineering, and  the rest of life as well. Someone develops a model or heuristic that works very well for a certain area or set of data. But the specialized purpose of the model is lost, either because they don’t record the fact or acknowledge it in the first place.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s