I’ve been looking at a lot of data lately. Visions of data are dancing in my head. Why? Because often, an oceanographic sensor will just crap out. And I must find that bad data and banish them from my analysis.
Every sensor has the potential for a problem. Water velocity, temperature, oxygen, nitrate and conductivity have all been known to rebel. THE STRUGGLE IS REAL. Even though they are super useful for computing seawater salinity and density when combined with a with temperature-sensing thermistor, conductivity cells were the worst offenders. Maybe it’s because they are delicate glass tubes that are just trying to measure how well electricity conducts through seawater? Or maybe it’s because they are just jerks. Nonetheless, after looking at literally 1000’s of data records I’ve gone ahead and categorized some of the most common ways I saw conductivity go wild so you don’t have to. YOU’RE WELCOME SCIENCE.
THE DAILY GRIND.
THE FLATLINE.
Thanks for the entertaining classification of conductivity failure modes! I’ve seen these all during my career; now I know what to call them (other than the unprintable things I call them at the time).
What do you do with that data? Do you discard it? Or do you have a way of correcting for it?
Glad I could help out ;)
In these cases, there is no way to correct for it. It usually has to be some type of systemic error to correct.
this is hilarious! I want to see more unicorn barf.
Do the jellyfish live?!?!? D-:
Until they reach the pump, which is a spinning propellor of death.
This is Beautiful. We don’t talk enough about failures.
Just apply the discipline of Probability and Statistics to your data. Better yet, do it on a device called a Computer so you don’t drive yourself crazy looking through all the data. You will be fine – provided your confidence goals are reasonably set – and met in the end :-)
OMG DAVID! WHAT IS THIS COMPUTER YOU SPEAK OF? PROBABILITY AND STATISTICS! NO ONE HAS EVER THOUGHT OF THIS EVER.
Thanks for mansplaining.
This could not have come at a better time as I think I had something like “attack of the Cnidaria” in the data I was looking at yesterday and I was going goggle-eyed trying to make sure it wasn’t a real phenomenon before excluding it. And no, @David, statistics were not good enough to remove the bad data points, I had to do it by hand.