By David Clinton

I have my opinions about which academic research is worth funding with public money and which isn’t. I also understand if you couldn’t care less about what I think. But I expect we’ll all share similar feelings about research that’s actually been retracted by the academic journals where it was published.

Globally, millions of academic papers are published each year. Many – perhaps most – were funded by universities, charitable organizations, or governments. It’s estimated that hundreds of thousands of those papers contain serious errors, irreproducible results, or straight-up plagiarized or false content.

Not only are those papers useless, but they clog up the system and slow down the real business of science. Keeping up with the serious literature coming out in your field is hard enough, but when genuine breakthroughs are buried under thick layers of trash, there’s no hope.

Society doesn’t need those papers and taxpayers shouldn’t have to pay for their creation. The trick, however, is figuring out how to identify likely trash before we approve a grant proposal.

I just discovered a fantastic tool that can help. The good people behind the Retraction Watch site also provide a large dataset currently containing full descriptions and metadata for more than 60,000 retracted papers. The records include publication authors, titles, and subjects; reasons for the retractions; and any institutions with which the papers were associated.

Using that information, I can tell you that 798 of those 60,000 papers have an obvious Canadian connection. Around half of those papers were retracted in the last five years – so the dataset is still timely.

There’s no single Canadian institution that’s responsible for a disproportionate number of clunkers. The data contains papers associated with 168 Canadian university faculties and 400 hospital departments. University of Toronto overall has 26 references, University of British Columbia has 18, and McMaster and University of Ottawa both have nine. Research associated with various departments of Toronto’s Sick Children’s Hospital combined account for 27 retractions.

To be sure, just because your paper shows up on the list doesn’t mean you’ve done anything wrong. For example, while 20 of the retractions were from the Journal of Obstetrics and Gynaecology Canada, those were all pulled because they were out of date. That’s perfectly reasonable.

I focused on Canadian retractions identified by designations like Falsification (38 papers), Plagiarism (41), Results Not Reproducible (21), and Unreliable (130). It’s worth noting that some of those papers could have been flagged for more than one issue.

Of the 798 Canadian retractions, 218 were flagged for issues of serious concern. Here are the subjects that have been the heaviest targets for concerns about quality:

You many have noticed that the total of those counts comes to far more than 218. That’s because many papers touch on multiple topics.

For those of you keeping track at home, there were 1,263 individual authors involved in those 218 questionable papers. None of them had more than five such papers and only a very small handful showed up in four or five cases. Although there would likely be value in looking a bit more closely at their publishing histories.

This is just about as deep as I’m going to dig into this data right now. But the papers I’ve identified are probably just the tip of the iceberg when it comes to lousy (and expensive) research. So we’ve got an interest in identifying potentially problematic disciplines or institutions. And, thanks to Retraction Watch, we now have the tools.

Kyle Briggs over at CanInnovate has been thinking and writing about these issues for years. He suggests that stemming the crippling flow of bad research will require a serious realigning of the incentives that currently power the academic world.

That, according to Briggs, is most likely to happen by forcing funding agencies to enforce open data requirements – and that includes providing access to the programming code used by the original researchers. It’ll also be critical to truly open up access to research to allow meaningful crowd-sourced review.

Those would be excellent first steps.

By David Clinton

The fact that millions of Canadians lack primary healthcare providers is a big deal. The grand promise of universal healthcare rings hollow for families forced to spend six hours waiting in a hospital emergency room for a simple ear infection diagnosis.

Just how big a deal is it? Statistics Canada data from 2021 ranks provinces by their ability to provide primary health providers. As you can see from the chart, New Brunswick and Ontario were doing the best, with doctors for nearly 90 percent of their residents. Quebec, able to find providers for just 78.4 percent of their population, landed at the bottom. But even just 10-15 percent without proper coverage is a serious systemic failure.

Since healthcare is administered by the provinces, it makes sense to assume that provincial policies will influence results. So comparing access to primary care practice results over time might help us understand what’s working and what isn’t.

To that end, I pulled Statistics Canada data tracking total employment in offices of physicians (NAICS code 621111) by province. The data covers all employees (including nurses, office managers, and receptionists) in all non-hospital medical offices providing services that don’t include mental health.

I originally searched unsuccessfully for usable data specific to doctors. But as it turns out, such data would have included surgeons and other hospital-based specialties when I’m really looking for general (family) care providers. So I think what we got will actually act as a better proxy for primary care access.

Do keep in mind that staffing levels in the sector represent just one of many statistical signals we could use to understand the healthcare universe. And it’s just a proxy that’s not necessarily a perfect map to reality.

In any case, I adjusted the numbers by provincial populations so they’d make statistical sense. The chart below contains ratios representing how many residents there are per worker between 2010 and 2023:

You might notice that PEI is missing from that chart. That’s because the reported numbers fell below Statistics Canada’s privacy threshold for most of the covered years.

Alberta, with a ratio of just 282:1 is the current champion, while Newfoundland (438:1) has the worst record. But changes over time are where things get interesting. BC’s performance declined by more than 11 percent. And Quebec improved by more than 40 percent!

As you can see for yourself in that chart, Quebec’s most dramatic growth took place between 2016 and 2019. What was going on around that time? Well, both Bill 10 and Bill 20 were introduced in 2015.

• Bill 10 restructured the healthcare system by reducing the number of administrative regions and centralizing governance to streamline services and improve efficiency.
• Bill 20 established patient quotas for doctors, mandating a minimum number of patients they were required to see. Physicians who did not meet these quotas faced penalties, such as reduced compensation.

I don’t need to speak French to assume that those measures must have inspired an awful lot of anger and push back from within the medical profession. But the results speak for themselves.

Or do they? You see there’s something else about Quebec we can’t ignore: Chaoulli v. Quebec (2005). That’s the Supreme Court of Canada case where the Canada Health Act’s ban on private delivery of healthcare was ruled unconstitutional (for Quebec, at any rate).

As a direct result of that decision, there are now more than 50 procedures that can be performed in private surgical clinics in the province. The number of private clinics nearly doubled between 2014 and 2023.

Predictably, wait times for surgeries fell significantly over that time. But the numbers of non-hospital employees would probably have climbed at the same rate. That could possibly go further to explain Quebec’s steady and consistent improvements in our data.

What about the other provinces? There have been structural changes to delivery policies in recent years, but they’re mostly too new to have produced a measurable impact. But here’s a brief overview of what’s being tried:

• This Toronto Star piece describes efforts in both Ontario and BC involving plans among some smaller municipalities to build and manage family health practices and pay their doctors as employees. The idea is that many doctors will prefer to avoid the headaches of starting and running their own businesses and would prefer instead to work for someone else. The obvious goal is to attract new doctors to underserved communities. It’s still way too soon to know whether they’ll be successful in the desperate race for the shrinking pool of family physicians.
• Both Ontario and Alberta have championed Family Health Groups (FHGs), where physicians receive additional incentives for providing comprehensive care. Ontario’s Family Health Networks (FHN) and Family Health Organizations (FHO) also compensate physicians based on the number and demographics of enrolled patients.
• British Columbia and Nova Scotia have implemented variations of a Longitudinal Family Physician (LFP) Payment Model. LFPs compensate family physicians based on factors like time spent with patients, patient panel size, and the complexity of care. They claim to promote team-based, patient-partnered care.