When I learned econometrics, I was told there is the formal treatment of the topic, and of course you need to know that, but equally important to the practitioner is the "art of econometrics." It wasn't something you could learn from books, it was a different kind of knowledge, perhaps best described (?) as "tacit knowledge" that allows you to make the right choices -- there are more judgment calls than you might imagine involved in actually specifying a model and testing an hypothesis -- and arrive at the empirical specification that will shed the most light on the question of interest. This is from Daniel Little:
Tacit knowledge, by Daniel Little: Scientist and philosopher Michael Polanyi introduced the idea of "tacit knowledge" in his 1958 book, Personal Knowledge: Towards a Post-Critical Philosophy (Google Books link). The book was presented as a critique of the positivist conception of scientific knowledge and the idea of knowledge as a system of logical statements. Polanyi was trained as a physician before World War I; worked as a research chemist between the wars; and found his voice as a philosopher of science subsequently. (Michael Polanyi is the brother of Karl Polanyi, discussed here.)
This is an interesting concept, and one that captures an important dimension of knowledge that is absent in most philosophical treatments of epistemology ("knowledge is a system of true justified beliefs"). The simple idea is that there are domains of knowledge that are not represented propositionally or as a system of statements, but are rather somehow embodied in the knower's cognitive system in a non-propositional form. This aspect of knowledge is more analogous to "knowing how" than "knowing that". Polanyi gives the example of a physician in training learning to "read" an x-ray. What is first perceived simply as an unintelligible alternation of light and dark areas, eventually is perceived by the experienced radiologist as a picture of a lung with a tumor. So the physician has somehow acquired a set of perceptual and conceptual skills that cannot be precisely codified but that permit him/her to gain a much more knowledgeable understanding of the patient's hidden disease than the novice. (This seems to be part of what Malcolm Gladwell is getting at in Blink: The Power of Thinking Without Thinking.)
It is unremarkable to observe that many aspects of skilled performance depend on "knowledge" that cannot be articulated as a set of statements or rules (link). The basketball guard's ability to weave through defenders and find his way to the basket reflects a complex set of representations of the court, the defenders, and probable behaviors of others that can't be codified. So it seems fairly straightforward to conclude that human cognition incorporates representations and knowledge that do not take the form of explicit systems of statements. Rather, these areas of knowledge are more "intuitive"; they are more akin to "body knowledge." But they are nonetheless cognitive; they depend on experience, they can be criticized and corrected, and they are representational of the world. (Talk to a skilled athlete about a complex task like finding a shot on goal in hockey or beating the defender to the basket, and you will be struck by the degree of intuition, gestalt, and realism that is invoked. And the same is true if you talk to an experienced labor organizer or a police detective.)
What is striking about Polanyi's position in Personal Knowledge is that he shows that these forms of practical knowledge do not pertain solely to physical skills like wine-making, playing basketball, or piloting a tug boat. Instead, they extend deeply into the enterprise of scientific knowledge itself. An experimental chemist or physicist has an uncodified ability to interpret instruments, evaluate complex devices, or recognize unexpected results that is the result of experience and training and cannot be reduced to a recipe or algorithm. And the difference between a gifted sociologist and a pedestrian one is his/her ability to probe a set of social facts for the underlying patterns, mechanisms, or questions that they suggest. These are ideas that return again in post-positivist philosophers and historians of science like Thomas Kuhn, Norwood Hanson, and Peter Galison; each of these scholars emphasizes the tight relationship that exists in scientific education and imagination between scientific practices, scientific instruments, and scientific knowledge. Galison's Einstein's Clocks, Poincare's Maps: Empires of Time is particularly interesting.
This recognition opens up a number of areas for empirical and theoretical research. The cognitive psychologist can ask a series of questions about how this kind of knowledge is represented in the knower's cognitive system. Does the basketball guard have a "cognitive map" of the court that he updates regularly? (This is a subject for non-human cognitive psychology as well as human cognition.) And the phenomenological sociologist like Erving Goffman can probe in detail the forms of tacit knowledge that the skilled intellectual performer is making use of -- the lawyer or accountant, or the research scientist (link).
This line of thought converges to some extent with arguments that Hubert Dreyfus advanced in What computers can't do: A critique of artificial reason in 1972. (Here is his update of his position; What Computers Still Can't Do: A Critique of Artificial Reason.) Dreyfus was fundamentally critical of artificial intelligence research in the 1960s, and the strategy of attempting to codify expert knowledge in the form of a set of rules that could then be implemented as computational algorithms. His position was a phenomenological one; basically, he took issue with the idea that cognitive competences like chess-playing, problem-solving, or pattern recognition could be reduced to a set of precise and separate rules and statements. Instead, there is a holistic aspect in ordinary practical knowledge that cannot be reduced to a set of discrete algorithms.
Of special interest for UnderstandingSociety is the question of whether ordinary people have "tacit social knowledge" of the social world they inhabit (link, link, link). What is involved in the competence a person demonstrates when he/she successfully navigates a formal dinner or a contentious union-hall argument? Can the knowledge that the competent social participant has about expected behavior from particular individuals be represented as a sum of propositional beliefs? Or, more plausibly, is this a good example of tacit knowledge, more akin to a rough map of a terrain than a codified set of statements? What about failures of tacit knowledge -- clumsiness (link)? And what is the system of social cognition through which ordinary social knowers gain these cognitive competences and update them through subsequent experience? All of this makes me think that we need richer models of mental life and competence than we currently possess (link).