Professional Grade is a term that was popularized in the early 2000’s with the advertising tagline “GMC: We Are Professional Grade.” Today, it is used to distinguish from general-use (or Consumer Grade) products and to communicate that a product will work better or longer in a more stressful environment where it is used more frequently or by…well, professionals.
Most of the Artificial Intelligence we employ today is powered by some sort of machine learning model developed by a human, a Data Scientist. These models are typically built using a training data set. This training data is a critical factor in determining the “intelligence” level of the resulting Artificial Intelligence application. It’s a simple fact: better training data produces better models. An axiom of this is: more and richer training data will produce more robust models. Let’s discuss two important components data scientists must consider when searching for the perfect training dataset: relevance and specification.
Finding a dataset with relevant content is the first critical step in a data science project. If, for example, you wish to build a chatbot to answer customer support questions, you need training data that contains sample customer support questions. And not just any set of questions; if the set of questions is too narrow or too perfect (yes, too perfect!), the resulting model will not have the necessary variability to learn from and will not perform well in the wild. In Data Science terms, we would say the resulting model is not very robust. It is best to use data that consists of a broad range of well-formed and poorly-formed questions.
Locating high-quality and relevant training data is so critical to model development today that dedicated websites source unique data assets that can be leveraged for training data. For example, a big search provider like Google offers Data Search, a search solely dedicated to finding data sources. Also, each of the Big-3 cloud vendors have their own data sharing platforms that provide unique and useful data sets: Google Cloud, AWS, Azure.
While locating relevant data to the project is a critical first step, the next important step is the process of specification where much of the “intelligence” comes from in Artificial Intelligence. Most training data is found in a…
Continue reading: https://towardsdatascience.com/is-your-ai-professional-grade-f402c6bc7213?source=rss—-7f60cf5620c9—4