In my studies about research and how to conduct it, "theory" has often been emphasized. The basic idea was that a good theory would function as a guide and a plan. I was in a group of men that served as data analysis coaches in a statistics lab. In that time and place, there were no computers available. Our lab was filled with big, clunky typewriter-sized calculators. A statistical analysis was carried out by doing addition, subtraction, multiplication and division on our calculations.
One day, two earnest men came in with big sheets covered with small, handwritten numbers. They were grad students at Georgetown University and they had "sacrificed" one lab rat a day for 30 or so days. Each rat body was analyzed in many ways and the men had written down the readings on a large number of variables from each body. They wanted to know what we could find in the data. We listened and then rejected the analysis plan as a "fishing trip." That was our term for a more or less hopeless search through a mishmash of numbers. As on a fishing trip, maybe you will catch something but maybe not.
One of the most helpful books I have read is "Too Big to Know" by David Weinberger. The book explores the situation of today when all sorts of voices and opinions get expressed by people with all sorts of levels of knowledge, with many different agendas and motivations. "Too Big to Know" refers to the world, knowledge and the internet. Weinberger is a technologist at Harvard. He recently came out with a new book "Everyday Chaos". I haven't read too far in it but it seems headed in the direction of the death of, or at least less attention to, theory.
He describes an ad with a woman's picture on the right (A) and another version of the same ad but with the picture on the left (B). In this case, one arrangement harvested much more response. That's good. A theory-based researcher might well want to know why the difference in responses. The point is that artificial intelligence (AI) and machine learning (ML) and deep learning (DL) are methods using complex machines that blindly find results, but that run through such totally enormous sets of data that no human in a human lifetime can duplicate their work. The machines find a pattern but they don't "understand" and they don't "explain".
