Big data

Disruption of the beneficial kind.

Is big data a lot of fuss over nothing or a genuine revolution or a problem?

So what of we now have more data than we ever imagined. There has always been records and information. It is used sometimes but numbers don’t really run our lives. Surely we do that.

It is true, we have always had data but not until very recently have we been able to capture, store and process vast amounts of it. Never before have databases talked a similar language or sensors ping records automatically into clouds or processing capacity seemed infinite.

Big data means we can aggregate pattern from whole populations of subjects so we can leave behind the complexities and assumptions of sample statistics. Information from a whole population in time and space is a natural scientists nirvana. This opportunity alone transforms inference.

And for every subject, we have repeated measures. We can follow what happens to them in time, and, of course, we can do this in space. Actions and measurements happen at a location, they are geo-referenced. The GPS responder on the cow can tell us exactly where it went to find forage and the satellite can tell us what was on the ground to make that place so attractive.

For the time being, big data is more about capturing what people do and mining that information for ways to make a profit. This is always a core use for information and the new bigness just makes new options available. It is more of the same, just bigger.

Where big data is genuinely disruptive is for activities with uncertain outcomes or with risk or with both. The ability to bound uncertainty without the constraints of sample size and randomisation errors is unprecedented. When big data feeds into models and artificial intelligence algorithms we have the capability to predict outcomes for our most complex problems. We can test multiple hypotheses at the same time and review just about any feasible scenario. In anything to do with the environment, from policy to production, this is a huge disruption.

Not everything about big data is so positive.

Nirvana for the ecologist sounds more like the death of civil liberties. If all this information on our whereabouts, choices and actions is collected, then who gets to use it and for what purpose? The farmer might say the same thing about information on his cows.

Data attributed to individuals is at least an invasion of privacy and once in ‘big data’ personal details are at risk. The line between data and control is a fine one easily crossed.

Caution over big data is prudent. But for agriculture and environmental issues, Afterbefore believes that the opportunity to revolutionise inference is the one thing that will transform how the environment is managed, shared and has its values restored.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s