cytocentric visionaries tb p1Cytocentric Visionaries: Tim Bushnell

Part Two: What are the Key Points for Reproducibility in Flow Cytometry? Layers of Variability


Alicia Henn, PhD MBA

Dr. Tim Bushnell is the Director of the Research Core Facilities at the University of Rochester Medical Center. He is a globally-recognized expert in flow cytometry, serving on the Executive Committee of the International Society for the Advancement of Cytometry (ISAC). He also served on the board of the Great Lakes International Imaging and Flow Cytometry Association, founded the Western New York Flow User’s Group, and is co-founder of ExCyte, a company that provides master classes in flow cytometry to academics and industry.

Here, Dr. Alicia Henn, Chief Scientific Officer of BioSpherix, talks with Dr. Bushnell about flow cytometry and scientific reproducibility.

Alicia Henn

As a cytometrist, you have been trying to draw attention to scientific reproducibility far longer than it has been in the scientific mainstream. Do you think that this technique lends itself to variability more than others?

Tim Bushnell

Every technique has its reproducibility pitfalls. In flow, the classic example is the concept of compensation. If you don’t correctly do compensation, which is correcting for the spectral spillover of the different fluorochromes in your flow cytometry panel, you will have erroneous data.

The first level of reproducibility in flow is the quality control (QC) on the instruments. The second level is training the investigator in how to generate reproducible data: from titering their antibodies to optimizing voltages.

People have all sorts of myths about compensation that get passed from lab to lab. People don’t always stop and think about what they’re doing. Compensation is one of those areas in flow cytometry that people may just wave their hands and hope for the best. That leads to erroneous data. We have an obligation to help educate people about the best practices.


Do we need better reproducibility for how the data is acquired or how it’s interpreted?


It’s the whole process. Setting up the right controls for compensation, collecting the data properly on the instrument, and then proper analysis.


So reproducibility of compensation hits on everything: materials, data acquisition, machine set-up, and experimental designJust in that one process you have to control multiple sources of variability and error?


Absolutely. The one variable that I can control best is how the machines are maintained. We do a lot of QC to make sure those machines are consistent. But the investigator has to build in their controls. Most people don’t.

Say if I’m going to do a ten color flow panel, I may have to run 30 tubes of controls before I run my single tube of sample. But, if I don’t have those controls, I can’t interpret the data properly. People don’t want to do or don’t understand that process, but if you do go through that process you can make your data very consistent and reproducible.


Isn’t that a lot of cells and antibodies (and time and money) used up in controls?


I get questions all the time, “Do I really need to run control X, Y, or Z?” My answer has always been “Well, if you want your data to be interpretable, yes.”

We had an investigator who moved to another institution. We worked with them and set up the other machine so the data coming off both instruments looked identical. That took time, but we knew how to do it. They have now ten years of consistent data. In that case, they were good at doing all the other pieces of the puzzle, like controls.


Do you think that getting far away instruments to match is something that’s available on a larger scale to address flow variability between labs?


The techniques are there. There are discussions on how to get work flows published. The bottom line is that if researchers don’t demand it, core facilities may not do it.

I’m fortunate that we have a large team, but I know colleagues who are the only person in their core. Their job is to get the machines up and running in the morning and then sort cells all day. They don’t have the bandwidth to do more than the minimum.

In that respect, ISAC’s shared resource laboratory task force just published a paper outlining some of the best practices. I think it’s a good read for people, not just in core facilities, but outside, to say: “These are the things that we need our core facilities to do.”


You hit on a key point there. It takes resources to change. If you could get flow cores across the world to change one thing, what would it be?


I think that QC of the instruments is essential. Within that, my colleagues do a great job at building QC programs. Then it comes down to the education of the end users, but you run into money and time pressures. The end users just want to do their experiments and get their data out.

Recently, I was called in to review some data for an investigator. They had sent sorted cells for RNA-seq. The RNA data was all over the place. The first thing I looked at was the QC. The machine had failed on multiple occasions, so they couldn’t be confident in what they were sorting. They had wasted $30,000 to $40,000 on RNA-seq because of bad QC. That cost a lot of money.

QC is not sexy, let’s face it. Sitting down at a machine, cleaning the machine, and running QC, and looking at it is not fun, but it’s critically essential. It’s like changing the oil in your car. If I invest a half a million dollars in a flow cytometer and I don’t have the time to do QC, I’m not going to get good data out. My research community is not going to be able to publish, and they’re not going to get grants.

I think when we’re going to see this ultimately get done is when NIH says they are not going to fund you unless you’re addressing reproducibility and rigor.

The institution administration also needs to recognize that they need these resources and funding needs to come from an institutional pot. NIH is not going to fund research if they don’t know that there’s going to be good reproducibility.


Do you think we need technical experts that are more experienced to do our flow experiments for us?


No, I think flow cytometry is still a technology that is approachable, but I think it needs to be done in a partnership with experts in the field.

Recently a certain flow cytometry data analysis package had a problem with one of their algorithms and it was brought to light through the Purdue Cytometry email list. An end user who isn’t on that mailing list may not be aware of the issues with the software.

Now we’ve just added another layer of complexity to reproducibility, which is the analysis side. Taking advantage of the technical application scientists at your institution, which is what they live and breathe for, improves reproducibility.

You’ve got to work with them and you’ve got to understand what you need to take back. You can’t just go buy 12 antibodies off the shelf now and expect good results.


There’s another layer, the materials. What should researchers know about buying antibodies for better reproducibility?


There was a call to arms in Nature in 2015 that said we waste about 350 million dollars a year just in protein binding reagents. With every technology you, as an end user, don’t necessarily get into the nuances because you have enough demands.

To get a PhD now, you have to identify a gene, you have to use CRISPR to knock that gene out, and you have to do RNA-seq experiments to see what the gene is doing. Then you’ll do your western blotting, your proteomics, or flow cytometry to look at the cells.

You don’t have time as an end researcher to get into all of these details of reproducibility for all of these methods, but that’s what is being demanded by NIH right now.


So there is investigator-level, funder-level and institutional-level awareness of cytometric reproducibility needed. What about publishers of journals? Are they trying to improve the quality of flow data?


ISAC with their Journal of Cytometry A has been doing two things. They’ve implemented the MIFlowCyt standard, which is the minimal information needed for a flow cytometry experiment, and they highlight papers that are MIFlowCyt compliant. Investigators have to provide additional data on reagents, on the experiment, on how the instrument was run.

The other area that they’ve moved into, with support of the Wallace Coulter Foundation, is a flow repository. It is a place where you can put your flow cytometry data so that when you publish, anyone can look at your data. That’s part of the requirement for the MIFlowCyt standard. Now we have a database where anyone can look at the original flow data and say “Do I agree with these conclusions?”


That makes sense for better transparency. Let me ask you, if science is going to be fighting for funding even more in the future, how important is it to make sure that the flow experiments that you’re doing are reproducible?


It is absolutely critical. If you’re going to build a model to test a drug on and it’s based upon the cells that you isolate, you want to make sure that you know what those cells are doing.

Think about personalized medicine. If we don’t have all the tools so that data is consistent, personalized medicine is going to be a flash in the pan.

Doxorubicin is a very powerful drug in breast cancer. A fraction of patients develop severe heart conditions on it. Other patients can bathe in the stuff. It kills the breast cancer, and it doesn’t affect them at all. What are the genetic differences between those two patient populations?

When we get to the point where we can routinely screen patients to build a personalized profile for how we treat them, the data that we generate is going to have to be spot on. Robustness and rigor in how we generate our data is essential.

Thank you Dr Bushnell for your time and your expertise.

For the latest in the field of Flow Cytometry check out ExCyte’s Blog.

For more interviews with experts that are putting the cell first read the Cytocentric Visionary series on the Cytocentric Blog.


alicia author iconAbout the Author

Alicia D Henn, PhD, MBA

Alicia Henn has been the Chief Scientific Officer of BioSpherix, Ltd for two years. Previously, she was a researcher at the Center for Biodefense Immune Modeling in Rochester, NY. Alicia obtained her PhD in molecular pharmacology and cancer therapeutics from Roswell Park Cancer Institute in Buffalo, NY and her MBA from the Simon School at University of Rochester in Rochester, NY.