ISCTE - INSTITUTO UNIVERSITÁRIO DE LISBOA

ESCOLA DE SOCIOLOGIA E POLÍTICAS PÚBLICA

FINAL COURSE ESSAY

SOCIOLOGY AND EPISTEMOLOGY

How to make sociology more scientific?

Critical commentary on the AI answer

Chiara Maraucci José Luís Casanova

[email protected] [email protected]

Academic Year 2023/2024

Contents

I. How do we make sociology more scientific? ChatGPT’s answer .................... 1

ChatGPT’s References ............................................................................................... 3

II. Critical commentary on the text by ChatGPT ............................................... 4

Formal mistake ........................................................................................................... 4

The relation between science and theory ................................................................... 4
Interpreting reality: Realism and its evolution ........................................................... 4
The limits of objective observation ............................................................................ 5
The issue of reliability and replicability .................................................................... 5
The power of controversy .......................................................................................... 5
The importance of collective trust.............................................................................. 6
References .................................................................................................................. 7
 I. How do we make sociology more scientific? ChatGPT’s answer

The Scientific Nature of Sociology: Approaches and Scientific Method

Sociology, as a discipline, strives to employ scientific methods to understand and analyze social
phenomena. Its scientific nature can be enriched through various approaches.

Adherence to the Scientific Method

The scientific method is fundamental for sociological research, involving the development and
verification of theories based on empirical evidence. This rigorous approach establishes a solid
foundation for the discipline.

Use of Tried and True Methods

Sociologists can employ established methods such as experiments, surveys, and textual analysis
to ensure the validity of results. Utilizing these consolidated approaches contributes to the reliability
of sociological investigations.

Data Collection and Analysis

Emphasis on accurate and impartial data collection is crucial for sociology. This, coupled with
systematic analysis of social data, follows scientific practices, ensuring a robust foundation for result
interpretation.

Theoretical Development and Hypothesis Testing

Careful formulation and testing of hypotheses contribute to the systematic advancement of

sociological knowledge. This process is essential for theoretical development and validation of
research hypotheses.

Incorporation of Multidisciplinary Approaches

Drawing on methodologies from other scientific disciplines enriches the scientific foundation of
sociology, allowing a more comprehensive understanding of social phenomena. Interdisciplinarity
enhances the sociological perspective.

Peer Review Process

Implementing a peer review process ensures scrutiny, validation, and improvement of research
methodologies and results. This contributes to maintaining a high standard in the sociological
discipline.

1
 Quantitative and Qualitative Balance

Striking a balance between quantitative and qualitative methodologies enriches sociological

studies, providing a comprehensive and in-depth view of complex social issues.

Transparent Research Practices

Transparency in research practices is fundamental. Open sharing of research methods, data, and
results promotes study replicability, strengthening the scientific credibility of sociology.

Longitudinal Studies

Inclusion of longitudinal studies contributes to establishing causal relationships and provides a

deeper understanding of social trends over time. This temporal approach amplifies the scope of
sociological research.

Ethical Considerations

Maintaining ethical standards in research design and data collection is essential to ensure the
reliability and integrity of sociological studies.

In conclusion, enhancing the scientific nature of sociology requires commitment to rigorous

methodologies, interdisciplinary collaboration, transparency, and ethical considerations. Embracing
these principles, sociologists can contribute to the continuous evolution of sociology as a scientific
discipline.

2
ChatGPT’s References

https://courses.lumenlearning.com/wm-introductiontosociology/chapter/the-scientific-method/

https://opentextbc.ca/introductiontosociology/chapter/chapter2-sociological-research/#section2.1.

https://openstax.org/books/introduction-sociology-3e/pages/2-1-approaches-to-sociological-
research#:~:text=The%20scientific%20method%20involves%20developing,critical%2C%20skeptic
al%2C%20and%20logical

https://rwu.pressbooks.pub/rothschildsintrotosociology/chapter/approaches-to-sociological-
research/
https://socialsci.libretexts.org/Bookshelves/Sociology/A_Career_in_Sociology_(Kennedy)/01%3A_
Careers_in_Sociology/1.03%3A_The_Scientific_Method_in_Practice

https://www.cliffsnotes.com/study-guides/sociology/sociological-research-methods/sociological-
research-designs-methods

https://www.jaypeedigital.com/eReader/chapter/9789351526162/ch1

https://www.yourarticlelibrary.com/sociology/the-scientific-nature-of-sociology-447-words/8488

3
 II. Critical commentary on the text by ChatGPT

Formal mistake
The first remark that seems to be necessary after a first reading of ChatGPT's answer concerns the
formal aspect. The text generated is nothing more than a list of points to be considered, almost like a
shopping list. Its characteristics don't correspond at all to those of an academic paper (as requested).
We cannot even assume that ChatGPT doesn't know how an academic paper should be written: if we
try to ask it, it answers correctly, listing all the steps to be followed and the sections to be included in
it. However, essential elements are missing from its answer: first of all, a review of the existing
literature and the names of the authors or works from which the concepts are derived.

The relation between science and theory

With regard to the references chosen, it is interesting to note that ChatGPT uses sources that are
all more or less identical, both in terms of words and images they contain. In terms of content, the
suggested sources tend to trivialise the issue. The algorithm has simply realised that the aim is to
think about a more scientific sociology and has therefore done nothing more than find articles on the
web explaining the application of the scientific method in sociology. This is not wrong, but it is
reductive. One cannot imagine a scientific sociology by simply thinking of making its methods of
investigation the same as those used in science. There is no social science without theory. Theory
should guide research and not be subordinated to specific methodological rules of how research
should be conducted (Danermark, Ekström, Karlsson, 2019).

Interpreting reality: Realism and its evolution

The algorithm answers automatically, without going through the theoretical debate behind the
question, such as that between Constructivism and Realism. The advice of artificial intelligence
suggests a mathematical interpretation of reality, as if it were made up of a sum of components to be
broken down and analysed objectively and impersonally. It ignores the possibility that the data
collected can be challenged by different interpretations. ChatGPT doesn't seem to ask the underlying
epistemological question: "How can we gain absolute knowledge of reality?". The analysis stops at a
realist interpretation of reality, without considering the possibility that there is a part of it that we
cannot access, as suggested by critical realism, which argues that reality has an objective existence,
but our knowledge of it is conceptually mediated. Facts are dependent on theory but not determined
by it, which means that all knowledge is fallible and open to adjustment (Danermark, Ekström,
Karlsson, 2019).

4
 The limits of objective observation
According to the sources recommended by ChatGPT, the only essential condition for accurate
research is an objective, logical and systematic observation of the empirical world. This would mean
that, in order for a theory to be considered valid, it is enough for it to follow perfectly all the steps of
the "recipe" (one of the references uses this example) drawn up by the researcher. As we know,
however, philosophy of science has made progress on this point over time. Karl Popper, in particular,
showed that observation alone is not sufficient to explain the success of science. As Kent Staley notes:
«For Popper, to engage in science is to pursue knowledge by considering only falsifiable theories»
(Staley, 2014). The introduction of the concept of falsification helped to define the boundary between
science and pseudoscience, although its limitations have also been overcome over time, for example
with the emergence of Imre Lakatos's “sophisticated falsificationism” (Lakatos, 1978).

The issue of reliability and replicability

Sources tend to take it for granted that adherence to already codified standard procedures
guarantees not only the reliability of a study, but also its replicability and the possibility of maximising
its results. Thomas Gieryn also speaks of the existence of stable boundaries and reference points that
delineate a kind of “cultural cartography” of science, which represents the legacy of Walden Thoreau.
What Gieryn stresses, however, is that behind these cartographic representations there is no "real
science", but there are too many "real sciences", and even when all these sciences are added up, they
still do not allow the sociologist to know a priori what science will look like on the next occasion of
its mapping (Gieryn, 1998). Gieryn also questions the labels usually used to delimit science (such as
“quantitative” and “qualitative”, terms used by ChatGPT).
«Real science and its boundaries on cultural maps are supple and pliable things,
but not so elastic that they may stretch endlessly in every direction».
(Gieryn, 1998)
The power of controversy
Four of ChatGPT's suggested references contain the same sentence: «Sociologists deliberately
apply scientific logic to maintain as much objectivity, focus and consistency as possible. The scientific
method provides a shared basis for discussion and analysis (Merton, 1963)». Another source also
states: «Using a scientific research design or plan is a recipe for other researchers to test and
substantiate someone's work and findings». This would imply that for sociologists the scientific
method is a guarantee of accuracy, and therefore anything that deviates from proven scientific logic
(to be used as a “test”) is unreliable. If this were true, we would have to deny the validity of all those

5
discoveries that have been made over time precisely by questioning assumptions that were previously
considered incontestable. The post-Merton and Kuhn-inspired sociology of science, on the other
hand, has shown that controversy is an important test and source of evidence. Collins and Pinch use
the controversy between Louis Pasteur and Félix Pouchet over the theory of spontaneous generation
to show how disagreement and challenges to existing theories are engines of scientific progress.
«It is hard to see how Pasteur would have brought about the changes in our ideas
of the nature of germs if he had been constrained by the sterile model of behaviour
which counts, for many, as the model of scientific method».
(Collins and Pinch, 1993)
The importance of collective trust
In the articles to which ChatGPT refers, little attention is paid to the collective dimension in which
the sociologist's work is embedded. The only times the sources refer to the scientific community are
at the preliminary stage of reviewing the existing literature and at the final stage of publishing the
results. Once again, it seems to be taken for granted that if a sociologist respects the scientific method,
his work will be directly and easily accepted by the expert community. In reality, two essential
concepts have to be taken into account: trust and witnessing. The problem is well summed up by the
latin motto of the Royal Society in the XVII century: nullius in verba, which emphasises the need not
to accept uncritically what others say and encourages direct observation and personal experience. As
Shapin and Schaffer show, scientific observation is permeated by power dynamics, authority and
rhetoric (Shapin and Schaffer, 1989). A credible witness is needed to trust an observation: empirical
evidence depends on social trust. Shapin coined the term “virtual witnessing” for this idea. There are
real spaces, which Shapin calls “experimental spaces” (or places for witnessing), where the scientific
community works together to conduct experiments, exchange ideas and share knowledge. Shapin
emphasises that it is precisely the sharing of such spaces that fosters mutual trust between experts and
the creation of their social reputation (Shapin, 1984). Therefore, the sociologist’s work doesn't end
with the publication of his results. In order for a study to be considered reliable, it must be able to
stand up to constant comparison with the rest of the community.

6
References

Collins H., Pinch T., (1998). The Golem: What You Should Know about Science, Cambridge
University Press.

Danermark B., Ekström M., Karlsson J. C. (2019), Explaining Society: Critical Realism in the Social
Sciences, London: Routledge

Gieryn T. F., (1998), Cultural Boundaries of Science: Credibility on the Line, Chicago: University of
Chicago Press

Lakatos I., (1978), The methodology of scientific research programmes, New York: Cambridge
University Press

Merton R., 1968 [1949], Social Theory and Social Structure, New York: Free Press

Shapin S., (1984), Pump and circumstance: Robert Boyle’s literary technology, Social Studies of
Science

Shapin S., Schaffer S., (1989), Leviathan and the Air-Pump, Princeton University Press

Staley K., (2014), An Introduction to the Philosophy of Science, Cambridge University Press