Author 
Topic: Information derived from a data set (Read 2527 times) 

davkrs
Newbie
Posts: 6


Information derived from a data set
« on: Dec 26^{th}, 2013, 1:25pm » 
Quote Modify

In this blog post, http://www.evanmiller.org/smalldata.html, the author claims that amount of information that can be derived from a data set is proportional to square root of the size of the data set (in the best case). Is this true. Is there a proof behind this claim. Did you ever notice in your research/job/work/life. What are your thoughts P.S. Though this question, and blog post linked to this question, are more applied math than pure math, I posted it here because I wasn't sure which category this applied math question should be posted in. Let me know if this question should be moved to a different category

« Last Edit: Dec 26^{th}, 2013, 1:28pm by davkrs » 
IP Logged 



towr
wu::riddles Moderator Uberpuzzler
Some people are average, some are just mean.
Gender:
Posts: 13729


Re: Information derived from a data set
« Reply #1 on: Dec 26^{th}, 2013, 10:44pm » 
Quote Modify

He seems to be speaking from the viewpoint of regression analysis only. From basic statistics we also know that e.g. the estimation of the variance (i.e. how certain we are about the true value of a statistical variable) is inversely proportional to the squareroot of the size of the data (for approximately normally distributed data). So you get less additional certainty for every extra amount of data. However, if you want to analyze rare events, then you need to filter a huge amount of data just to find enough of them to be able to say anything about them. And therefore you need to collect a huge amount of data (especially if you don't know beforehand what rare events you want to analyze  which brings it's own troubles, because you can always find something on a finishing expedition, even if it turns out to be an old shoe). And as he says, if you deal with a lot of parameters, you need a lot of data to estimate them all. And some data is so noisy, you need a lot to average out the noise; it's often not neatly normally distributed data.


IP Logged 
Wikipedia, Google, Mathworld, Integer sequence DB



