top of page

How to Critique a Scientific Paper

I realized after I started writing this that I wouldn't be able to address and include everything that I wish, so this will be part one.


In veterinary medicine, we use evidence based medicine to aid in how we practice. During my time in this career I have been part of many journal clubs, each with their own way of analyzing a scientific paper. I have written articles for submission to journals and undergone revision after revision on my Master's thesis.


When reading a scientific paper, we end up being the peers that review the publications. So here I'll add my critique of a scientific paper.


The paper I'm going to go through today was recently published by MDPI Animals where MDPI is an open access publisher that has been under some scrutiny for being on a list of "predatory publishers".


Predatory Publishers

Most of the scientific community would know that a journal's reputation for sound, peer-reviewed science is a combination of its Impact Factor and how much does this work contribute to the scientific database of information within your field of study.


Predatory Journals

As a scientist you want your research to have the most impact in your field of study. You want the research groups in your field to see you as an expert in your field. It is extremely important that your research prior to publishing has gone through the wringer, so that at the time of publication it is the best it can be with the knowledge that we have on the subject. At this stage, I do not know what the impact would be if you can submit to a more reputable publisher, but you don't.


So for those of you who are budding scientists aim high.


I understand that not everyone will get to publish in the journal of Nature or Science. But, you will want to find the highest impact journal in your field and aim there!


Top publications Life Sciences Google Scholar

So for those of you in Animal Behaviour, you may wish to aim for that journal.


Top publications Animal Behavior & Ethology

Aiming high when you are a scientist and researcher allows you to apply for grants and funding for additional research based on the previous impact that your research has contributed.


I forget now where I read it, but... good research will be published in reputable journals, you just have to be patient to go through the process.


Is our scientific community looking for that instant gratification?


Johnson, A.C. and Wynne, C.D., 2024. Comparison of the Efficacy and Welfare of Different Training Methods in Stopping Chasing Behavior in Dogs. Animals, 14(18), p.2632. Link


Title: The title of the paper should be concise but also informative. If you are working on a particular species, you may include the scientific name of said species. For example, the domestic dog is not Canine... it is Canis lupus ssp familiaris or just Canis familiaris - depending on what taxonomist you ask.


If you are studying companion dog welfare, ensure that the topic of welfare is clearly defined within the introduction section of your paper. The journal Animals is peer-reviewed, but I would like to know whether that includes diplomats of the European College of Animal Welfare and Behavioural Medicine (ECAWBM).


Part of the Five Freedoms of Animal Welfare is Freedom from pain and Freedom from fear and distress. As animal caregivers and guardians we want to limit the amount of pain, fear and distress in our pet dogs in order for them to have adequate welfare.


We have said this previously and repeat it yet again, the dog training industry is an unregulated industry. Those who are certified with the CCDPT would be required to follow the code of ethics put forth by the CCDPT. This includes "To understand and promote Least Intrusive, Minimally Aversive (LIMA) training and behavior work." and "Shall work to minimize the use of aversive stimuli and maximize the effective use of positive reinforcement to modify animal behavior through a Least Intrusive, Minimally Aversive (LIMA) based approach."


Abstract: This tends to be a summary of the work and should include your hypotheses and objectives of your research.


Introduction: The introduction lays the ground work of what is currently in the literature, what is missing from the literature and then you follow this up with your objectives and hypotheses.


A scientific study should have a hypothesis (or hypotheses) that is testable.


So for example, if your Null hypothesis is: If companion dogs are trained using electronic shock collars, then their welfare will not be negatively affected compared to dogs trained with alternative methods. First we must know what poor welfare means - as I said above, minimizing pain, fear and distress. A measurement of pain in a dog could be yelping. A measurement of fear could be distance increasing behaviour such as avoidance. A measurement of distress could be whining.


What it shouldn't be is acute changes in cortisol. What it shouldn't be is fecal levels of cortisol... see below.



Materials and Methods: When we use live animals in research, we balance the need for scientific rigor, thus reducing the amount of Type I and Type II errors in our research, and the ethical use of sentient beings.


If you take eight dogs, and two of these dogs did not respond to the treatment of shock, then they were excluded from the statistics of the study based on animal ethics. That is 25% of your treatment group that did not respond to the treatment. In case you cannot do math, that is 2 over 8 = 1/4 ---- one quarter of the treatment group did not respond. So a 75% efficacy rate just to train the shock collar. It's better than a pass, especially if we want to say that electronic shock collars stop chasing behaviour. What we can see is that Group A had an high incidence of yelping compared to Group B. As above, yelping is very much associated with negative emotional affect, be it pain, startling, or fear. It is very much a vocalization of distress. That alone suggests that the study was unable to be conducted on all eight participants because it didn't work for two, and evoked pain, fear and distress significantly compared to Group B with no shock.


I do not even want to start on why the positive reinforcement group did not work. I will leave that to my colleagues in the dog training industry. I will focus on just the experimental design and allocation of what is deemed negative welfare indicators.


When you wish to have the strongest results for your scientific study, your experimental design needs to control for everything except the treatment. When we wish to test two treatment groups against a control group, then all else stays the same. For even more control over individual variation (since we are testing owned dogs, not laboratory beagles), you may wish to perform a crossover design, where each dog becomes it's own matched pair. You will have a stronger experimental design and thus, the results would become more meaningful. If intra-group variability and inter-group variability is too high, it becomes less meaningful.


Lets talk corticosteroids. Acute increases in corticosteroids from the adrenal glands are adaptive. Stress is neither good nor bad - it just is. Chronic elevations of corticosteroids are maladaptive. But we need cortisol to live! Have I said this previously in a blog post? If not, here it is again. An Addisonian crisis is a severe lack of cortisol in a stressful situation. When there is a lack of cortisol, you will see electrolyte derangements and hypoglycemia, low blood albumin levels and the body goes into shock. We need cortisol. An increase in cortisol during an acute stressor is not and indicator of poor welfare.


That being said, salivary cortisol is a more accurate measure of acute stress than fecal cortisol (Link). So if you did not wish to acquire a blood sample, I suspect that the dog's humans would be able to collect saliva samples easily from the cheek pouches of their dogs.


For those not in the scientific community, we are going away from using cortisol as a measure of stress and welfare, because it is not an accurate measurement of negative stress. If you are going to measure cortisol in your study, understand its limitations, and perhaps have an endocrinologist on your research team.


In fact, even if you exclude training with food or training with electronic shock, you will see an increase in cortisol levels with chase! Chase as part of the domestic dogs' predation sequence is a normal behaviour. Chase also is activation of part of the autonomic nervous system called the sympathetic nervous system. The sympathetic nervous system, known commonly as the Fight or Flight system, which is adaptive, comes on board during both good and bad scenarios. Take that excitement and anticipation that you feel prior to an event you are looking forward to.


Most interesting from the graphic below is the lack of variability in group C! The lure was only moved slowly, and the dogs' cortisol levels didn't vary as much as the other two groups.


Figure 4 - fecal cortisol levels

Vocalizations come in many forms with dogs. Barking for example, is one of the distance increasing behaviours that dogs will perform with several different underlying affects, fear, anxiety, and territorial behaviours. We also see barking as a response to frustration, or lack of frustration tolerance. We see barking as an anticipatory behaviour prior to something exciting, which would be considered a positive affect. Some dog owners would be able to tell the difference between their dog's barks, while others may not. A discussion of dog vocalization would be lacking if we didn't include Miklósi one of the leading authorities in dog ethology and evolution. See the below further readings for more on dog vocalizations.


Whining tends to be a care soliciting vocalization. While it is an attention-seeking behaviour, it is normally directed towards the attachment figure. It can be heard during anticipation, and may have previously been reinforced. For example, the dog is ready to chase a lure and may whine in anticipation of this exciting activity. Dogs may whine as a sign of discomfort. Most certainly as a veterinarian we observe this during hospital stays. While whining may be associated with negative emotional affect, it isn't always. Knowing when and the context of the whining will assist us on determining the motivation.


Yelping would be considered a vocalization of negative affect. Pain, startle and fear would be causative reasons for yelping. Does anyone disagree with that?


Where I am getting to is that the dog vocal repertoire is vast and placing Barking, Whining and Yelping into the same emotional category to determine welfare or stress is inherently flawed. As I mentioned above, Group A was the shock group, and they experienced higher levels of yelping in that group. If yelping is a sign of distress, and distress is a sign of poor welfare, then would it not suggest that the cause of the yelping is leading to poor welfare?


Before I move on to another section, on another day... chasing behaviour is a normal dog behaviour. In my experience, it is aggression towards the familiar people in the home or aggression towards familiar dogs or animals in the home that are the top reasons for humane euthanasia due to problem behaviours, not that of chasing a squirrel or prey species. To state that one must use a fast acting highly aversive punishment to prevent euthanasia of the dog lacks the understanding of the primary reasons for euthanasia due to behaviours incompatible with human cohabitation.



Citations: When you are publishing a scientific article, you can pull from online sources. By scientific merit, anecdotes are not data. And blog posts, just like this one you are reading, are not the gold standard of scientific literature. I would never expect a reputable scientist to use my blog post as reason to do or not do something.


All of the writings in these blog posts are my own opinions.



Many in the modern and humane dog training community are what are called 'crossover' trainers. They once used the old methods of punishment, alpha-rolling, dominance and intimidation in order to achieve these results. Now, they know better. When you know better, do better.


It was once thought that animals did not feel pain. Now it is standard practice to ensure that pain control is used during the castration of boar piglets.


It is the perfect time for all of us to say "We're not going back".



 

Further reading:


Bennett, A. and Hayssen, V., 2010. Measuring cortisol in hair and saliva from dogs: coat color and pigment differences. Domestic animal endocrinology, 39(3), pp.171-180. Link


Bergamasco, L., Osella, M.C., Savarino, P., Larosa, G., Ozella, L., Manassero, M., Badino, P., Odore, R., Barbero, R. and Re, G., 2010. Heart rate variability and saliva cortisol assessment in shelter dog: Human–animal interaction effects. Applied animal behaviour science, 125(1-2), pp.56-68. Link


Hitchcock, M., Workman, M.K., Guthrie, A.P., Ruple, A. and Feuerbacher, E.N., 2024. Factors associated with behavioral euthanasia in pet dogs. Frontiers in Veterinary Science, 11, p.1387076. Link


Pongrácz, P., Molnár, C. and Miklósi, Á., 2010. Barking in family dogs: an ethological approach. The Veterinary Journal, 183(2), pp.141-147. Link


Pongrácz, P., Molnár, C., Miklósi, A. and Csányi, V., 2005. Human listeners are able to classify dog (Canis familiaris) barks recorded in different situations. Journal of comparative psychology, 119(2), p.136. Link


Siracusa, C., Provoost, L. and Reisner, I.R., 2017. Dog-and owner-related risk factors for consideration of euthanasia or rehoming before a referral behavioral consultation and for euthanizing or rehoming the dog after the consultation. Journal of veterinary behavior, 22, pp.46-56. Link


Todd, Z., 2018. Barriers to the adoption of humane dog training methods. Journal of Veterinary Behavior, 25, pp.28-34. Link



7 views0 comments

Recent Posts

See All

Comments


bottom of page