{"id":2307,"date":"2022-03-29T15:39:17","date_gmt":"2022-03-29T15:39:17","guid":{"rendered":"http:\/\/optimumsportsperformance.com\/blog\/?p=2307"},"modified":"2022-03-29T15:39:17","modified_gmt":"2022-03-29T15:39:17","slug":"if-the-examples-we-see-in-textbooks-dont-represent-the-real-world-what-about-the-sports-science-papers-we-read","status":"publish","type":"post","link":"https:\/\/optimumsportsperformance.com\/blog\/if-the-examples-we-see-in-textbooks-dont-represent-the-real-world-what-about-the-sports-science-papers-we-read\/","title":{"rendered":"If the examples we see in textbooks don&#8217;t represent the real world, what about the sports science papers we read?"},"content":{"rendered":"<p>I enjoyed this <span style=\"color: #0000ff;\"><strong><a style=\"color: #0000ff;\" href=\"https:\/\/statmodeling.stat.columbia.edu\/2022\/03\/27\/the-examples-you-see-in-the-textbooks-are-not-representative-of-the-sorts-of-problems-you-see-in-the-real-world\/\">short article<\/a><\/strong><\/span> by Andrew Gelman on <strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" href=\"https:\/\/statmodeling.stat.columbia.edu\/\">the blog he keeps<\/a><\/span><\/strong> with several of his colleagues. The main point, which I agree 100% with, is that the examples in our stats and data analysis textbooks never seem to match what we see in the real world. The examples always seem to work! The data is clean and looks perfectly manicured for the analysis. I get it! The idea is to convey the concept of how different analytical approaches work. The rub is that once you get to the real world and look at your data you end up being like, <em>\u201cUh. Wait\u2026.what is this? What do I do now?!\u201d<\/em><\/p>\n<p>The blog got me thinking about something else, though. Something that really frustrates me. If the examples we see in textbooks don\u2019t reflect the data problems we face in the real world, what about the examples we read about in applied sport science research? How much do those examples reflect what we see in the real world?<\/p>\n<p>At the risk of upsetting some colleagues, I\u2019ll go ahead and say it:<\/p>\n<blockquote><p><em>I\u2019m not convinced that the research we read in publications completely represents the real world either!<\/em><\/p><\/blockquote>\n<p>How can that be? This is applied science! Isn\u2019t the real word <strong><u>THE<\/u><\/strong> research?<\/p>\n<p>Well, yes and no. Yes, the data was collected in the real world, with real athletes, and in a sport setting. But often, reading a paper, looking at the aim and the conclusions, and then parsing through the methods section to see how they <em>handled<\/em> the data leaves me scratching my head.<\/p>\n<p>My entire day revolves around looking at data and I can tell you; the real world is very messy.<\/p>\n<ul>\n<li>How things get collected<\/li>\n<li>How things get saved<\/li>\n<li>How things get loaded<\/li>\n<li>How things get logged<\/li>\n<\/ul>\n<p>There are potential hiccups all along the way, no matter how stringent you are in trying to keep sound data collection practices!<\/p>\n<p>In research, the problem is that the data is often <em>touched up<\/em> in some manner to create an analysis sufficient for publication. Missing values need to be handled a certain way (sometimes those rows get dropped, sometimes values get imputed), class imbalance can be an issue, errant values and outliers from technology flub-ups are a very real thing, data entry issues arise, etc. These things are all problematic and, if not identified prior to analysis, can be a major issue with the findings. I do believe that most people recognize these problems and would agree with me that they are very real issues. However, it is less about knowing that there are problems but rather, figuring out what to do about them. Often the methods sections gloss over these details (I get it, word counts for journals can be a pain in the butt) and simply produce a result that, on paper at least, seems overly optimistic. As I read through the results section, without details about data processing, I frequently say to myself, <em>\u201cNo way. There is no way this effect is as real as they report. I can\u2019t reproduce this without knowing how they cleaned up their data to observe this effect.\u201d<\/em><\/p>\n<p>Maybe someone should post a paper about how crappy their data is in the applied setting? Maybe we should just be more transparent about the data cleaning processes we go through so that we aren\u2019t overly bullish on our findings and more realistic about the things we can say with confidence in the applied setting?<\/p>\n<p>Does anyone else feel this way? Maybe I\u2019m wrong and I\u2019m being pessimistic, and this isn\u2019t as big of an issue as I believe it to be? Is the data we see in publication truly representative of the real world?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I enjoyed this short article by Andrew Gelman on the blog he keeps with several of his colleagues. The main point, which I agree 100% with, is that the examples in our stats and data analysis textbooks never seem to match what we see in the real world. The examples always seem to work! The [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43,42,27],"tags":[],"class_list":["post-2307","post","type-post","status-publish","format-standard","hentry","category-sports-analytics","category-sports-science","category-strength-and-conditioning"],"_links":{"self":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2307","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/comments?post=2307"}],"version-history":[{"count":1,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2307\/revisions"}],"predecessor-version":[{"id":2308,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2307\/revisions\/2308"}],"wp:attachment":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/media?parent=2307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/categories?post=2307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/tags?post=2307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}