{"id":23259,"date":"2022-02-22T16:19:45","date_gmt":"2022-02-22T21:19:45","guid":{"rendered":"https:\/\/www.3pillarglobal.com\/?p=23259"},"modified":"2025-05-14T17:28:41","modified_gmt":"2025-05-14T17:28:41","slug":"data-models-gone-bad-can-spoil-your-predictions","status":"publish","type":"blog","link":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/","title":{"rendered":"Data Models Gone Bad Can Spoil Your Predictions"},"content":{"rendered":"\n<h2>Artificial Intelligence and Machine Learning Require Ongoing Tuning<\/h2>\nWe recently discussed how important it is to <a href=\"https:\/\/www.3pillarglobal.com\/insights\/navigating-data-bias\/\">choose the correct set of data<\/a> from which to drive predictions and inferences. It\u2019s also important that the data not be skewed in any particular direction\u2014 or else it will generate skewed predictions. In this post, we discuss the number-crunching processes downstream from data acquisition.\n<br><br>\nA data model represents what was learned by a machine learning algorithm. As data flows downstream through your various business processes, the data models supporting your algorithms that direct your systems or people can fall out of alignment. This occurs when the premises and assumptions surrounding the data models change enough to erode the accuracy of the analytics. Or\u2026the data models take the algorithms out of meta-stable equilibrium and into turbulent predictive behavior.\n<br><br>\nIt\u2019s not a good thing when data models have gone bad, spoiling your predictions and inferences!\n<br><br>\n<h2>Artificial Intelligence and Machine Learning: Not Set and Forget Tools<\/h2>\nUnderstanding why this happens requires realizing that machine learning algorithms and artificial intelligence are not \u201cset and forget\u201d tools. Like any other software system, they require tuning to maintain optimal performance. Also, it is important to understand the boundaries of the data values going into the algorithms so you aren\u2019t pushing them out of their \u201cflight envelope.\u201d\n<br><br>\nTuning includes the normal maintenance engineering tasks you would apply to any software deployment\u2014upgrades, patches, and adaptations\u2014in reaction to infrastructure changes. As you maintain, be sure to validate the algorithms are still \u201cin tune\u201d and that data feeds adhere to the original assumptions and conditions.\n<br><br>\nThe process is similar to what Formula One race car drivers go through. They know the techniques and wisdom (data model) gathered about the nuances of one track do not apply to another track. They also contend with races that require a change to the car type\u2014Indy, NASCAR, or an off-road vehicle; all these things impact their driving technique. So as premises and assumptions change from race-to-race and from car-to-car, drivers adapt their internal algorithms that direct them on how to best navigate each track.\n<br><br>\n<h2>Factors Causing Data Models to Spin-out<\/h2>\nHere\u2019s a rundown of various factors that can cause data models to derail:\n<br><br>\n<ul>\n \t<li><em><strong>Concept Drift<\/strong><\/em> occurs due to statistical changes in the interrelationships of interdependent variables. For instance, economic &lt;If this, then that&gt; statements can change due to differences in unemployment. This has happened during Covid with a record number of people simply resigning and dropping out of searching for a job\u2014mainly because they consider the risks of their job to be unacceptable, plus government subsidies are more economically beneficial than actual work.<\/li>\n \t<li><em><strong>Spec Creep<\/strong><\/em> (also known as Changing Requirements and related to Concept Drift) happens when a data science team designs a model to perform a focused task, but another internal team (such as marketing) decides to modify the model\u2019s capabilities. Perhaps there was a change in how the marketing team manages product promotion campaigns. The accuracy of the outputs is affected by changes in functional requirements rather than by causal changes in data.<\/li>\n \t<li><em><strong>User Persona Change<\/strong><\/em> is another type of Spec Creep that takes place because of a change in the user base. New users representing a different target segment might require new sources of data to draw similar inferences as previous target segments. For example, as people from a younger generation enter an older age range that an algorithm was tuned for, they bring different behavioral characteristics that might require a completely different set of data to base predictions upon.<\/li>\n \t<li><em><strong>Data Drift<\/strong><\/em> can dramatically change the accuracy of algorithms when the data models are trained on data from a different era or different environmental situation. Take irrigation algorithms\u2026just because they worked for a specific geography with a particular economic set point in the past, they may not do so in the future due to changes in the climate, economic considerations, water pricing, or usage regulations.<\/li>\n \t<li><em><strong>Feature Drift<\/strong><\/em> relates to changes associated with machine learning model inputs, which are not static; change is inevitable, and the models depend heavily on the data they point to. Some models can handle minor input changes, but as distributions divert from what the model saw in training, performance on the task will degrade.<\/li>\n \t<li><em><strong>Prediction Drift<\/strong><\/em> is a change in the actuals within a production model, which decays over time. Monitoring for prediction drift allows you to measure model quality. As actuals change, models that make predictions can be right one day and wrong the next, which causes a regression in model performance.<\/li>\n<\/ul>\nBy watching out for these factors, you can make changes to your algorithms before the integrity of your data output is impacted.\n<br><br>\n<h2>Tips for Keeping Analytic Algorithms Healthy<\/h2>\nKeeping the algorithms that drive your analytics healthy and clear is vital to the mission of converting raw data into information that enables your end-users to augment their decisions on how to make smarter products and how to run the business more efficiently. Here\u2019s a set of high-level tips on how to deal with the factors that cause data models supporting your algorithms to fall out of alignment:\n<br><br>\n<ul>\n \t<li>Develop and maintain a static model of your algorithms as a baseline for future comparisons.<\/li>\n \t<li>Always test the edge cases and watch for sudden changes in model behavior.<\/li>\n \t<li>Set up a recurring schedule to retrain and update the models.<\/li>\n \t<li>Implement processes to detect changes to concepts, specs, user personas, data, features and predictions.<\/li>\n \t<li>Measure the changes you discover and prioritize the need to react; not all changes will warrant an adjustment to your models.<\/li>\n \t<li>Create new models as necessary to resolve any changes.<\/li>\n<\/ul>\nOnce you go through this process, test to make sure your new models work. You will then have a new static model to use as a new baseline as you continuously review your algorithms and look for changes and drifts.\n<br><br>\n<h2>Moving Up Maslow\u2019s Pyramid<\/h2>\nIn this article, the process of monitoring and adapting algorithms maps to the middle part of Maslow\u2019s pyramid\u2014cognitive needs. Abraham Maslow was an American psychologist who developed a hierarchy of needs to explain human motivation as shown below:\n<br><br>\n<a href=\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Expanded_Maslows_Needs.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-23755\" style=\"max-width: 480px;margin: 0 auto;\" src=\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Expanded_Maslows_Needs.jpg\" alt=\"Expanded Maslow's Needs Model\" width=\"593\" height=\"547\" \/><\/a>\n<br><br>\nIn our next post, we\u2019ll discuss bringing your data analysis up to the top of the pyramid\u2014actualization and transcendence. Once you know your algorithms align with the results you are looking for in relation to the data you are crunching, you want to make sure the resulting inferences are actionable. That\u2019s when you gain the ability to augment the decision-making of your end-users so they can make smarter products and more efficient business processes.\n<br><br>\nWe&#8217;d be pleased to work with you on your next data analysis project. <a href=\"\/contact\/\">Contact 3Pillar today<\/a> to learn how we can help.\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer is-style-small\"><\/div>\n\n\n<div class=\"wp-block-heading\">\n<h2 class=\"wp-block-heading\">About the author<\/h2>\n<\/div>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<div class=\"wp-bootstrap-blocks-row row\">\n\t\n\n<div class=\"col-12 col-lg-6\">\n\t\t\t\n\n    <div  class=\"custom-block card-profile-block card card-profile card-image-offset card-image-offset-right card-image-offset-sm card-border-left text-bg-light-cyan\">\n        <div class=\"card-body\">\n            <div class=\"row flex-md-row-reverse\">\n                <div class=\"col-md-5\">\n                                            <img loading=\"lazy\" decoding=\"async\" width=\"532\" height=\"532\" src=\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2024\/07\/leadership-henry-martinez.jpg\" class=\"img-fluid\" alt=\"Henry Martinez portrait\" data-aos=\"none\" srcset=\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2024\/07\/leadership-henry-martinez.jpg 532w, https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2024\/07\/leadership-henry-martinez-300x300.jpg 300w, https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2024\/07\/leadership-henry-martinez-172x172.jpg 172w\" sizes=\"auto, (max-width: 532px) 100vw, 532px\" \/>                                    <\/div>\n\n                <div class=\"col-md-7\">\n                    <h3 class=\"card-title\">Henry Martinez<\/h3>\n                    \n                                            <p class=\"card-text\">Senior Director, Global Head of Solutions, Engineering &#038; Architecture<\/p>\n                                        \n                    <a href=\"https:\/\/www.3pillarglobal.com\/?post_type=leadership&#038;p=1638\" class=\"link-arrow link-arrow-dark\">\n                        <span>Read bio<\/span>\n                    <\/a>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n\n\t<\/div>\n\n\n\n<div class=\"col-12 col-lg-6\">\n\t\t\t\t<\/div>\n\n<\/div>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer is-style-small\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Artificial Intelligence and Machine Learning Require Ongoing Tuning We recently discussed how important it is to choose the correct set of data from which to drive predictions and inferences. It\u2019s also important that the data not be skewed in any particular direction\u2014 or else it will generate skewed predictions. In this post, we discuss the [&hellip;]<\/p>\n","protected":false},"featured_media":30768,"template":"","industry-types":[],"service-types":[48],"topics":[29,28],"class_list":["post-23259","blog","type-blog","status-publish","has-post-thumbnail","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Models Gone Bad Can Spoil Your Predictions - 3Pillar<\/title>\n<meta name=\"description\" content=\"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Models Gone Bad Can Spoil Your Predictions - 3Pillar\" \/>\n<meta property=\"og:description\" content=\"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/\" \/>\n<meta property=\"og:site_name\" content=\"3Pillar\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-14T17:28:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1208\" \/>\n\t<meta property=\"og:image:height\" content=\"680\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/\",\"url\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/\",\"name\":\"Data Models Gone Bad Can Spoil Your Predictions - 3Pillar\",\"isPartOf\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png\",\"datePublished\":\"2022-02-22T21:19:45+00:00\",\"dateModified\":\"2025-05-14T17:28:41+00:00\",\"description\":\"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage\",\"url\":\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png\",\"contentUrl\":\"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png\",\"width\":1208,\"height\":680},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.3pillarglobal.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Models Gone Bad Can Spoil Your Predictions\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.3pillarglobal.com\/#website\",\"url\":\"https:\/\/www.3pillarglobal.com\/\",\"name\":\"3Pillar\",\"description\":\"Together we create incredible\",\"publisher\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.3pillarglobal.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.3pillarglobal.com\/#organization\",\"name\":\"3Pillar\",\"url\":\"https:\/\/www.3pillarglobal.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.3pillarglobal.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/qa-www.3pillarglobal.com\/wp-content\/uploads\/2024\/08\/3pillar-organization-logo.png\",\"contentUrl\":\"https:\/\/qa-www.3pillarglobal.com\/wp-content\/uploads\/2024\/08\/3pillar-organization-logo.png\",\"width\":696,\"height\":696,\"caption\":\"3Pillar\"},\"image\":{\"@id\":\"https:\/\/www.3pillarglobal.com\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Models Gone Bad Can Spoil Your Predictions - 3Pillar","description":"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/","og_locale":"en_US","og_type":"article","og_title":"Data Models Gone Bad Can Spoil Your Predictions - 3Pillar","og_description":"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.","og_url":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/","og_site_name":"3Pillar","article_modified_time":"2025-05-14T17:28:41+00:00","og_image":[{"width":1208,"height":680,"url":"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/","url":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/","name":"Data Models Gone Bad Can Spoil Your Predictions - 3Pillar","isPartOf":{"@id":"https:\/\/www.3pillarglobal.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage"},"image":{"@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage"},"thumbnailUrl":"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png","datePublished":"2022-02-22T21:19:45+00:00","dateModified":"2025-05-14T17:28:41+00:00","description":"Data models supporting your algorithms that direct your systems or people can fall out of alignment. Learn why this happens and what to do when it does.","breadcrumb":{"@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#primaryimage","url":"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png","contentUrl":"https:\/\/www.3pillarglobal.com\/wp-content\/uploads\/2022\/02\/Data-Models-Gone-Bad_1208x680.png","width":1208,"height":680},{"@type":"BreadcrumbList","@id":"https:\/\/www.3pillarglobal.com\/insights\/blog\/data-models-gone-bad-can-spoil-your-predictions\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.3pillarglobal.com\/"},{"@type":"ListItem","position":2,"name":"Data Models Gone Bad Can Spoil Your Predictions"}]},{"@type":"WebSite","@id":"https:\/\/www.3pillarglobal.com\/#website","url":"https:\/\/www.3pillarglobal.com\/","name":"3Pillar","description":"Together we create incredible","publisher":{"@id":"https:\/\/www.3pillarglobal.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.3pillarglobal.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.3pillarglobal.com\/#organization","name":"3Pillar","url":"https:\/\/www.3pillarglobal.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.3pillarglobal.com\/#\/schema\/logo\/image\/","url":"https:\/\/qa-www.3pillarglobal.com\/wp-content\/uploads\/2024\/08\/3pillar-organization-logo.png","contentUrl":"https:\/\/qa-www.3pillarglobal.com\/wp-content\/uploads\/2024\/08\/3pillar-organization-logo.png","width":696,"height":696,"caption":"3Pillar"},"image":{"@id":"https:\/\/www.3pillarglobal.com\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/blog\/23259","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/types\/blog"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/media\/30768"}],"wp:attachment":[{"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/media?parent=23259"}],"wp:term":[{"taxonomy":"industry-types","embeddable":true,"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/industry-types?post=23259"},{"taxonomy":"service-types","embeddable":true,"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/service-types?post=23259"},{"taxonomy":"topics","embeddable":true,"href":"https:\/\/www.3pillarglobal.com\/wp-json\/wp\/v2\/topics?post=23259"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}